[discussion] Static Perl compilation without PAR + make optional / drop more dependencies

Hi! I'm building yet-another-attempt-of-latex-in-the-browser: https://vadimkantorov.github.io/busytext/busytex.html. So far I managed to build xetex+bibtex8.
I was thinking to somehow try to "compile" biber into WebAssembly. I was considering two options:

perlcc to generate C code
WebPerl to replace PAR packing

What would be your thoughts on feasibility of these two variants? Have you tried using perlcc on biber? Are there any "nasty" required dependencies?
Does biber require popen / signals / web requests for normal functioning? If not, it probably can be made working in WASM.

I would be interested in helping to look into this. perlcc could work as it's an established system. For WebPerl, biber does use http requests to fetch external data sources and this might be an issue I suppose. Another potential issue is the btparse library that is used by Text::BibTeX as this is very old and I don't know how this would play with modern frameworks like this.  I do some minor signal messing about in biber to contain some segfaults from libbtparse for example.

Cool :) I'll post here my building attempts

As part of prep work, I'm working on compiling cperl (a small Perl compiler) statically with all modules in order to run tlmgr from C code (a smaller Perl codebase than biber). If that works, that could be an alternative way to WebPerl (WebPerl does same for regular Perl), and it would also run natively.
If you're interested to also work on this, feel free to slide into https://github.com/perl11/cperl/issues/423#issuecomment-706597439

An update:
I wanted to see if one can get a fully static build (Emscripten has a dynamic linking support, but it's new; and a static build is valuable on its own for embedding in other programs).

I first checked if cperl + perlcc are able to produce a fully static build. At the moment they cannot, the advancement is blocked: https://github.com/perl11/cperl/issues/423
I failed to install RPerl that might also work - CPAN installs was taking too much time and producing too many errors on my system
https://metacpan.org/pod/distribution/App-Staticperl/staticperl.pod may work, needs to be checked (it configures the main perl distro for a static build)
WebPerl approach of getting a static build - probably similar to staticperl, may work outside of WebAssembly context as well

@plk What is the minimum set of biber's dependencies for a minimial local-only working setup? (without downloading stuff from internet, without running external programs) Full list from Build.pl: autovivification Class::Accessor Data::Dump Data::Compare Data::Uniqid DateTime::Format::Builder DateTime::Calendar::Julian File::Slurper IPC::Cmd IPC::Run3 List::AllUtils List::MoreUtils List::MoreUtils::XS Mozilla::CA Regexp::Common Log::Log4perl Unicode::Collate Unicode::Normalize Unicode::LineBreak Unicode::GCString" # Encode::Locale Encode::EUCJPASCII Encode::JIS2K Encode::HanExtra Parse::RecDescent PerlIO::utf8_strict XML::LibXML XML::LibXML::Simple XML::LibXSLT XML::Writer Sort::Key Storable Text::CSV Text::CSV_XS Text::Roman IO::String URI Text::BibTeX LWP::UserAgent LWP::Protocol::https Business::ISBN Business::ISSN Business::ISMN Lingua::Translit

That is the minimum dependencies you have listed there ...

E.g. Mozilla::CA or LWP should not be needed in theory if consulting internet is disabled, right?
Same for IPC. Does biber require running external programs for minimal functioning?

Well, I'd have to see whether things would work without them as their integration isn't really that modular as it was never designed to be used without them.

Okay! In the meanwhile I'll try to build it with all dependencies. but for sure it'd be more robust for these cross-compilation scenarios if there is a mode "no-calling-external-programs + no-downloading-from-internet" that enables running even if LWP/IPC are not installed/present

Do you know how to skip tests during ./Build installdeps? Doing CPAN_OPTS=-T ./Build installdeps does not help

That is the minimum dependencies you have listed there ...

Why does the https://github.com/plk/biber/blob/dev/dist/linux_x86_64/build.sh PAR run command contain much fewer dependencies? namely,  only
  --module=Pod::Simple::TranscodeSmart \
  --module=Pod::Simple::TranscodeDumb \
  --module=List::MoreUtils::XS \
  --module=List::SomeUtils::XS \
  --module=List::MoreUtils::PP \
  --module=HTTP::Status \
  --module=HTTP::Date \
  --module=Encode:: \
  --module=File::Find::Rule \
  --module=IO::Socket::SSL \
  --module=IO::String \
  --module=PerlIO::utf8_strict \
  --module=Text::CSV_XS \
  --module=DateTime \

Also, I'm trying to set up a Github Actions build script in https://github.com/vadimkantorov/buildbiber/blob/master/.github/workflows/build.yml
Maybe it would be good for biber to have it for some testing as well in the main repo

Do you know how to skip tests during ./Build installdeps? Doing CPAN_OPTS=-T ./Build installdeps does not help. Otherwise installing all dependencies is very slow.
It may be that CPAN_OPTS handling is broken in cpan. So a switch in ./Build or ./Build.pl to enable cpan -T instead of cpan would be nice.

Did you try ./Build installdeps --cpan_client 'cpan -T'

Did you try ./Build installdeps --cpan_client 'cpan -T'

Not yet. Will try!

That is the minimum dependencies you have listed there ...

Why does the https://github.com/plk/biber/blob/dev/dist/linux_x86_64/build.sh PAR run command contain much fewer dependencies? namely, only
  --module=Pod::Simple::TranscodeSmart \
  --module=Pod::Simple::TranscodeDumb \
  --module=List::MoreUtils::XS \
  --module=List::SomeUtils::XS \
  --module=List::MoreUtils::PP \
  --module=HTTP::Status \
  --module=HTTP::Date \
  --module=Encode:: \
  --module=File::Find::Rule \
  --module=IO::Socket::SSL \
  --module=IO::String \
  --module=PerlIO::utf8_strict \
  --module=Text::CSV_XS \
  --module=DateTime \

It does depend on the platform - it's somewhat empirical, you have to tweak the build script until the executable works in terms of module includes. This is because the dependency checking modules are quite sensitive to platforms and the idiosyncracies of particular perl builds. If in doubt, use a module line to explicitly import - it can't hurt. I usually start with a basic set from a similar OS and then add as needed which I test the binary and see that it didn't auto-detect pack a module that's needed.

Regarding testing, I haven't looked this in github as it required setting up a perl environment etc. and that's probably not trivial ...

They have an Ubuntu installation, but without a perl distribution pre-installed.
I opted for compiling one from scratch (since going forward I'd like to replace it by a custom statically-built WebAssembly one):
name: build

on: workflow_dispatch

env:
  URL: https://www.cpan.org/src/5.0/perl-5.32.0.tar.gz
  MAKEFLAGS: -j2

jobs:
  build:
    runs-on: ubuntu-20.04
    steps:
       - name: Install Perl
         run: |
           echo Downloading and compiling in [$PWD] from [$URL]

           mkdir perl
           wget -nc $URL
           tar -xf $(basename $URL) --strip-components=1 --directory=perl

           pushd perl
           bash +x ./Configure -sde -Dprefix="$PWD/../prefix"
           test -f Makefile
           make
           make install
           popd

       - name: Install Biber Dependencies Without Test
         run: |
           BEFORE=$(find ./prefix | wc -l)
           ./prefix/bin/cpan -T Module::Build Config::AutoConf ExtUtils::LibBuilder    autovivification Class::Accessor Data::Dump Data::Compare Data::Uniqid DateTime::Format::Builder DateTime::Calendar::Julian File::Slurper IPC::Cmd IPC::Run3 List::AllUtils List::MoreUtils List::MoreUtils::XS Mozilla::CA Regexp::Common Log::Log4perl Unicode::Collate Unicode::Normalize Unicode::LineBreak Unicode::GCString Encode::Locale Encode::EUCJPASCII Encode::JIS2K Encode::HanExtra Parse::RecDescent PerlIO::utf8_strict XML::LibXML XML::LibXML::Simple XML::LibXSLT XML::Writer Sort::Key Storable Text::CSV Text::CSV_XS Text::Roman IO::String URI Text::BibTeX LWP::UserAgent LWP::Protocol::https Business::ISBN Business::ISSN Business::ISMN Lingua::Translit
           AFTER=$(find ./prefix | wc -l)
           echo files before: $BEFORE after: $AFTER

Running https://github.com/vadimkantorov/buildbiber/blob/master/.github/workflows/build.yml, essentially:
# perl and cpan are built from sources

cpan -T Test::More Test::Differences File::Which    Module::Build    Config::AutoConf ExtUtils::LibBuilder    autovivification Class::Accessor Data::Dump Data::Compare Data::Uniqid DateTime::Format::Builder DateTime::Calendar::Julian File::Slurper IPC::Cmd IPC::Run3 List::AllUtils List::MoreUtils List::MoreUtils::XS Mozilla::CA Regexp::Common Log::Log4perl Unicode::Collate Unicode::Normalize Unicode::LineBreak Unicode::GCString Encode::Locale Encode::EUCJPASCII Encode::JIS2K Encode::HanExtra Parse::RecDescent PerlIO::utf8_strict XML::LibXML XML::LibXML::Simple XML::LibXSLT XML::Writer Sort::Key Storable Text::CSV Text::CSV_XS Text::Roman IO::String URI Text::BibTeX LWP::UserAgent LWP::Protocol::https Business::ISBN Business::ISSN Business::ISMN Lingua::Translit

URLBIBER=https://github.com/plk/biber/archive/v2.15.tar.gz
URLTESTFILES=https://master.dl.sourceforge.net/project/biblatex-biber/biblatex-biber/testfiles
BIBERTESTFILES="test.bib test.bcf test-dev.bcf unifont.ttf"

mkdir biber
wget $URLBIBER
tar -xf $(basename $URLBIBER) --strip-components=1 --directory biber

pushd biber
perl ./Build.PL
perl ./Build install

wget $(printf "$URLTESTFILES/%s " $BIBERTESTFILES)
perl ./Build test
perl ./bin/biber --validate-control --convert-control test

produced a lot of sortinithash mismatches. Is there a way to install the specific version of Unicode::Collate to get rid of them?
I attach the full test log log.txt

What version of perl and U::C is on the test box?

I install there perl 5.32.0 from sources in https://github.com/Perl/perl5/tree/v5.32.0
I could easily upgrade the perl version there if needed by just specifying a different version

I don't know what U::C version it installs from CPAN. If a particular U::C version is required, I can check if possible to install a specific version from CPAN

Do you know what version of Unicode::Collate is there?

I see - I think that my install is not using the latest U::C - let me update and update the test results and then let's see. It would be best to currently use perl 5.30 for the tests until I migrate all the builds/tests to 5.32

In all likelihood it's:
./prefix/bin/cpan -D Unicode::Collate
Reading '/home/vadimkantorov/.cpan/Metadata'
  Database was generated on Sat, 28 Nov 2020 14:17:03 GMT
Unicode::Collate
-------------------------------------------------------------------------
        (no description)
        S/SA/SADAHIRO/Unicode-Collate-1.29.tar.gz
        /home/vadimkantorov/perl/../prefix/lib/5.32.0/x86_64-linux/Unicode/Collate.pm
        Installed: 1.27
        CPAN:      1.29  Not up to date
        SADAHIRO Tomoyuki (SADAHIRO)
<EMAIL_ADDRESS>

If it matters, the latest release at https://github.com/Perl/perl5/releases is 5.33.4

I never use odd numbers as they tend to be considered experimental. 5.32 would be the latest stable version.

My github workflow file doesn't do PAR packing, since I didn't need it, but if it's useful for a single beginner/simple/free_CI build script, please feel free to build upon it

DEV branch is updated with requirement for U::C 1.29 and all test should have the correct sortinithashes for this now.

Can PAR output the final list of resolved module dependencies? I'm advancing with compiling WebPerl, so a full list of modules to try bundling with Perl at build time would be very useful.

Have a look at the docs for Module::ScanDeps - that's what PAR uses to resolve dependencies. The -module lines in the par script are simply there to catch things that this module can't detect (usually due to runtime includes etc.)

Instead of a binary, I produced a zipball with PAR and dumped its file listing: log.txt

I will try to have a static perl build with just these modules (for native and for wasm).
If you can make optional LWP/internet access/process launching (and corresponding module imports), it would greatly increase the chance of success.

 So probably it makes sense to start with them for a static_ext Perl build.
B
Class::XSAccessor
Clone
Compress::Raw::Bzip2
Compress::Raw::Zlib
Cwd
DBD::SQLite
DBI
Data::Dumper
DateTime
Devel::Caller
Devel::LexAlias
Digest::MD5
Encode::Byte
Encode::CN
Encode::EBCDIC
Encode::EUCJPASCII
Encode
Encode::HanExtra
Encode::JIS2K
Encode::JP
Encode::KR
Encode::Symbol
Encode::TW
Encode::Unicode
Fcntl
File::DosGlob
File::Glob
HTML::Parser
Hash::Util::FieldHash
I18N::Langinfo
IO
IPC::SysV
List::MoreUtils::XS
List::SomeUtils::XS
List::Util
MIME::Base64
Math::BigInt::FastCalc
Opcode
Posix
Package::Stash::XS
PadWalker
Params::Validate::XS
PerlIO::encoding
PerlIO::scalar
PerlIO::utf8_strict
Socket
Sort::Key
Storable
Sub::Identify
Sys::Hostname
Sys::Syslog
Text:BibTeX
Text::CSV_XS
Time::HiRes
Unicode::Collate
Unicode::LineBreak
Unicode::Normalize
Variable::Magic
XML::LibXML
XML::LibXSLT
XML::Parser
attributes
autovivification
mro
re
threads::shared
threads
PerlIO::encoding
PerlIO::mmap
PerlIO::scalar
PerlIO::via

Hi both
I wanted to make you aware that changes made in support of WebAssembly could also be of great help for Biber+Tectonic integration, which right now is difficult due to interop complexity. Most people are using this upstream by invoking Biber themselves in their IDE of choice in addition to Tectonic. Integration is being discussed in the following issue.
https://github.com/tectonic-typesetting/tectonic/issues/35
Being able to build without internet functionality could well be appreciated for this use case, in line with the compilation reproducibility goals of Tectonic.
Thanks again for your work here!

@aterenin I made some attempts at compiling Perl statically with all the dependencies. It almost worked, but I don't have time to continue this fight at the moment. If you'are interested to hear about my advancements, ping me at @vadimkantorov on Telegram or on<EMAIL_ADDRESS>:)
On the side note, I also lazily develop https://busytex.github.io - completely client-side wasm TexLive compiler and basic integration with github (~3k lines for Makefiles and JavaScript). Let know if you wish to collaborate.

Will revisit in future is there is sufficient interest.

I've retried again. Here is the list of PAR-discovered dependencies:
./shlib/x86_64-linux/libcrypto.so.3
./shlib/x86_64-linux/libexslt.so.0
./shlib/x86_64-linux/libbtparse.so
./shlib/x86_64-linux/libssl.so.3
./shlib/x86_64-linux/libxslt.so.1
./shlib/x86_64-linux/libz.so.1
./shlib/x86_64-linux/libxml2.so.2
+ ./lib/auto/I18N/Langinfo/Langinfo.so
+ ./lib/auto/Digest/MD5/MD5.so
./lib/auto/Sort/Key/Key.so
./lib/auto/Compress/Raw/Zlib/Zlib.so
./lib/auto/Compress/Raw/Bzip2/Bzip2.so
+ ./lib/auto/Encode/Encode.so
./lib/auto/Encode/Unicode/Unicode.so
./lib/auto/Encode/CN/CN.so
./lib/auto/Encode/KR/KR.so
./lib/auto/Encode/EUCJPASCII/EUCJPASCII.so
./lib/auto/Encode/Symbol/Symbol.so
./lib/auto/Encode/EBCDIC/EBCDIC.so
./lib/auto/Encode/Byte/Byte.so
./lib/auto/Encode/TW/TW.so
./lib/auto/Encode/HanExtra/HanExtra.so
./lib/auto/Encode/JP/JP.so
./lib/auto/Encode/JIS2K/JIS2K.so
./lib/auto/autovivification/autovivification.so
./lib/auto/Devel/Caller/Caller.so
./lib/auto/Devel/LexAlias/LexAlias.so
./lib/auto/XML/LibXSLT/LibXSLT.so
./lib/auto/XML/Parser/Expat/Expat.so
./lib/auto/XML/LibXML/LibXML.so
./lib/auto/Unicode/Normalize/Normalize.so
./lib/auto/Unicode/LineBreak/LineBreak.so
./lib/auto/Unicode/Collate/Collate.so
./lib/auto/IPC/SysV/SysV.so
./lib/auto/Clone/Clone.so
+ ./lib/auto/B/B.so
./lib/auto/Text/CSV_XS/CSV_XS.so
./lib/auto/Text/BibTeX/BibTeX.so
./lib/auto/PadWalker/PadWalker.so
+ ./lib/auto/PerlIO/encoding/encoding.so
+ ./lib/auto/PerlIO/scalar/scalar.so
+ ./lib/auto/PerlIO/via/via.so
./lib/auto/PerlIO/utf8_strict/utf8_strict.so
./lib/auto/PerlIO/mmap/mmap.so
./lib/auto/MIME/Base64/Base64.so
./lib/auto/HTML/Parser/Parser.so
./lib/auto/File/DosGlob/DosGlob.so
+ ./lib/auto/File/Glob/Glob.so
./lib/auto/threads/shared/shared.so
./lib/auto/threads/threads.so
./lib/auto/Time/HiRes/HiRes.so
./lib/auto/Sys/Syslog/Syslog.so
./lib/auto/Sys/Hostname/Hostname.so
+ ./lib/auto/Opcode/Opcode.so
+ ./lib/auto/Data/Dumper/Dumper.so
+ ./lib/auto/attributes/attributes.so
./lib/auto/IO/Compress/Brotli/Brotli.so
+ ./lib/auto/IO/IO.so
./lib/auto/List/MoreUtils/XS/XS.so
./lib/auto/List/SomeUtils/XS/XS.so
+ ./lib/auto/List/Util/Util.so
./lib/auto/Filter/Util/Call/Call.so
./lib/auto/DBI/DBI.so
./lib/auto/Net/SSLeay/SSLeay.so
./lib/auto/re/re.so
+ ./lib/auto/Fcntl/Fcntl.so
./lib/auto/Socket/Socket.so
./lib/auto/Sub/Identify/Identify.so
./lib/auto/DateTime/DateTime.so
+ ./lib/auto/mro/mro.so
./lib/auto/Storable/Storable.so
./lib/auto/Variable/Magic/Magic.so
./lib/auto/Math/BigInt/FastCalc/FastCalc.so
./lib/auto/Class/XSAccessor/XSAccessor.so
./lib/auto/Hash/Util/FieldHash/FieldHash.so
./lib/auto/DBD/SQLite/SQLite.so
./lib/auto/Package/Stash/XS/XS.so
+ ./lib/auto/Cwd/Cwd.so
./lib/auto/Params/Validate/XS/XS.so
./lib/auto/Params/Util/Util.so
+ ./lib/auto/POSIX/POSIX.so

@plk Btw I managed to build statically all these *.so package dependencies with a simple enough script
So there might be path towards building statically all of biber with musl - without any packer utils.
Maybe a remaining question for more portability is: does biber use backticks/system shell calls? If so, for portability it would be better to replace them with native Perl function calls or at least localize all such system shell calls in one perl source file to ease inspection and replacing them with more portable Perl code in the future.

No, no backticks. Any such thing is done with OS neutral perl modules by design.

This is great news, as I found that tlmgr.pl / install-tl.pl are using backticks all over the place...

SInce biber has to run on WIndows, Mac and Linix, a lot of effort was put in to make sure it played nicely with all of them in a non hacky way ....

@plk I succeded in compiling a fully static variant of Perl (without any .so) - https://github.com/vadimkantorov/perlpack
I'll at some point try to use it instead of PAR to compile/pack a biber

@plk for building libbtparse from source, is it https://metacpan.org/pod/LaTeX::BibTeX and https://metacpan.org/release/AMBS/Text-BibTeX-0.89/source/btparse and https://www.ctan.org/tex-archive/biblio/bibtex/utils/btOOL/?

I only use the AMBS/Text-BibTeX package which includes a modified libbtparse which I have added to over the years.

https://github.com/ambs/Text-BibTeX right? or some other place from CTAN/CPAN?

so basically https://metacpan.org/release/AMBS/Text-BibTeX-0.89/source/btparse/src ?

Exactly, that's libbtparse with additions to support biber (some bug fixes and basic UTF8 support). It's a very old library but we managed to modify it to pass through UTF8 so that we can handle it all in Perl.

@plk Where testfiles are stored now? https://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/testfiles/ now has only unifont.ttf?
I used to do
wget https://master.dl.sourceforge.net/project/biblatex-biber/biblatex-biber/testfiles/test.bib https://master.dl.sourceforge.net/project/biblatex-biber/biblatex-biber/testfiles/test.bcf https://master.dl.sourceforge.net/project/biblatex-biber/biblatex-biber/testfiles/test-dev.bcf https://master.dl.sourceforge.net/project/biblatex-biber/biblatex-biber/testfiles/unifont.ttf
./biber-linux_x86_64 --validate-control --convert-control test

but how to run tests now?

Ah, sorry, they were moved into the git repo so that they are more up to date:
https://github.com/plk/biber/tree/dev/testfiles

this is nice! curious, why unifont.ttf is remaining on sourceforge?

I forgot to delete it ...

I somehow made it work with a fully static Perl build instead of using PAR: https://github.com/busytex/busybiber/blob/main/.github/workflows/busybiber.yml
I'm embedding all Perl and data modules inside the executable. And somehow getting an error Cannot find XML::LibXSLT stylesheet. Skipping conversion
How can I check that biber can discover all the bundled data files / stylesheets? Is there some command checking for this?

Well, that error will tell you if you can find it or not. Sometimes you have to explicitly package files in the pp call to make sure they are in the package.

I made some fixes. Is this log looking correct now?
INFO - This is Biber 2.20
INFO - Logfile is 'test.blg'
INFO - Converted BibLaTeX control file 'test.bcf' to 'test.bcf.html'
INFO - Reading 'test.bcf'
INFO - Using all citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex file 'test.bib' for section 0
INFO - LaTeX decoding ...
INFO - Found BibTeX data source 'test.bib'
INFO - Datamodel validation starting
INFO - Datamodel validation complete
INFO - Overriding locale 'en-US' defaults 'variable = shifted' with 'variable = non-ignorable'
INFO - Overriding locale 'en-US' defaults 'normalization = NFD' with 'normalization = prenormalized'
INFO - Sorting list 'nty/global//global/global/global' of type 'entry' with template 'nty' and locale 'en-US'
INFO - No sort tailoring available for locale 'en-US'
INFO - Writing 'test.bbl' with encoding 'UTF-8'
INFO - Output to test.bbl

Looks good as long as there is something in the .html.

Yeah, test.bcf.html is in there

So essentially what I completed successfully is:

on Alpine
I compiled a static perl single-file binary, containining all the required module files (and overriding a few of I/O functions to let these be read by Perl when it requests them)
the binary is fully static and sort of portable, as I'm linking with musl libc and all other libraries statically

If we can reduce the number of library dependencies and Perl dependencies, this might be a PAR-less, simpler way of building. At least for some setups, this might be interesting.

To be precise, in attempt to match all the .so files included in biber, I'm currently linking a lot of stuff:
 $(printf "$PERLPREFIX/lib/perl5/site_perl/$PERLVER/x86_64-linux/auto/%s " Class/XSAccessor/XSAccessor.a Params/Util/Util.a Params/Validate/XS/XS.a XML/LibXML/LibXML.a XML/Parser/Expat/Expat.a XML/LibXSLT/LibXSLT.a DateTime/DateTime.a Text/CSV_XS/CSV_XS.a Text/BibTeX/BibTeX.a Variable/Magic/Magic.a IO/Compress/Brotli/Brotli.a IO/Compress/Brotli/libbrotlidec.a IO/Compress/Brotli/libbrotlienc.a IO/Compress/Brotli/libbrotlicommon.a HTML/Parser/Parser.a Package/Stash/XS/XS.a PadWalker/PadWalker.a Net/SSLeay/SSLeay.a List/SomeUtils/XS/XS.a List/MoreUtils/XS/XS.a Unicode/LineBreak/LineBreak.a Devel/LexAlias/LexAlias.a Devel/Caller/Caller.a Storable/Storable.a autovivification/autovivification.a DBD/SQLite/SQLite.a DBI/DBI.a Sub/Identify/Identify.a Sort/Key/Key.a PerlIO/utf8_strict/utf8_strict.a Clone/Clone.a Encode/JIS2K/JIS2K.a Encode/HanExtra/HanExtra.a Encode/EUCJPASCII/EUCJPASCII.a) \
            $(printf "$PERLPREFIX/lib/perl5/$PERLVER/x86_64-linux/auto/%s " mro/mro.a File/Glob/Glob.a File/DosGlob/DosGlob.a I18N/Langinfo/Langinfo.a SDBM_File/SDBM_File.a Compress/Raw/Zlib/Zlib.a Compress/Raw/Bzip2/Bzip2.a POSIX/POSIX.a Fcntl/Fcntl.a Data/Dumper/Dumper.a Math/BigInt/FastCalc/FastCalc.a IO/IO.a Opcode/Opcode.a Socket/Socket.a threads/shared/shared.a threads/threads.a MIME/Base64/Base64.a Time/HiRes/HiRes.a Time/Piece/Piece.a Filter/Util/Call/Call.a List/Util/Util.a Unicode/Collate/Collate.a Unicode/Normalize/Normalize.a B/B.a Devel/Peek/Peek.a Storable/Storable.a attributes/attributes.a re/re.a Hash/Util/Util.a Hash/Util/FieldHash/FieldHash.a IPC/SysV/SysV.a PerlIO/encoding/encoding.a PerlIO/mmap/mmap.a PerlIO/via/via.a Sys/Hostname/Hostname.a Sys/Syslog/Syslog.a Encode/KR/KR.a Encode/JP/JP.a Encode/Byte/Byte.a Encode/CN/CN.a Encode/Encode.a Encode/Unicode/Unicode.a Encode/Symbol/Symbol.a Encode/EBCDIC/EBCDIC.a Encode/TW/TW.a Cwd/Cwd.a Digest/MD5/MD5.a Digest/SHA/SHA.a) \
            $PERLPREFIX/lib/libbtparse.a \
            $PERLPREFIX/lib/perl5/$PERLVER/x86_64-linux/CORE/libperl.a \
            /usr/lib/libxslt.a \
            /usr/lib/libexslt.a \
            /usr/lib/libexpat.a \
            /usr/lib/libcrypto.a /usr/lib/libgcrypt.a /usr/lib/libgpg-error.a \
            /usr/lib/libxml2.a \
            /lib/libz.a \
            /usr/lib/liblzma.a \
            /usr/lib/libssl.a

It would be good to know which ones are actually not needed - then it would allow to reduce a lot the disk footprint of the final binary...


[bug] Bitget wallet is not detected

Link to minimal reproducible example
https://web3modal.com
Summary
Bitget Wallet(formerly Bitkeep) is not being detected in web3modal V3. I believe it's because current web3modal is checking window.ethereum.isBitKeep instead of window.bitkeep.ethereum.
List of related npm package versions
"@web3modal/wagmi": "^3.0.2"

This is not planned for v3, we are moving towards EIP-6963 instead. Actually the last version of v3 already supports EIP-6963 which will detect your wallet if (it's installed) automatically and it's going to be shown on top of the modal. I would highly suggest integrating the protocol.
Some major wallets already support this EIP and we have a dapp for testing it. https://github.com/WalletConnect/EIP6963


Hide arrows on shortcut container when fully expanded

Description ✏
Closes #301

Adds a simple check to render/hide the arrows.

Need to fix commit & branch


[BUG] Frame field not declared when computing embeddings on a frame view

In the example below, add_frame_field() should not have to be manually called, but, for some reason, it is not automatically happening. The embeddings data is present, the field is just not being declared...
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-video", max_samples=1)
view = dataset.to_frames(sample_frames=True, fps=1)

model = foz.load_zoo_model("mobilenet-v2-imagenet-torch")
view.compute_embeddings(model, embeddings_field="embeddings")

# Bug fix: manually add `embeddings` field to dataset's frame schema
dataset.add_frame_field("embeddings", fo.VectorField)

# Now the embeddings are available, as expected
view2 = dataset.to_frames(sample_frames=True, fps=1)
print(view2.values("embeddings"))

Hey @victor1cea, I appreciate your offer to help! I don't think select_fields() is at fault here; even if select_fields() is included, its still permissible to add a new field. For example:
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")
view = dataset.select_fields()

sample = view.first()
sample["foo"] = "bar"
sample.save()

assert "foo" in dataset.get_field_schema()

I believe I've tracked the issue down to some changes in https://github.com/voxel51/fiftyone/commit/05a5f17f43fa064940d8022061137b4ef1096771.
This one is probably a bit subtle to fix, so I better handle it. But thanks again!


Can materialize be used in production? What about the durability of this product?

Hi guys!
I saw that the team is a group of students and 2 of them began to work at Google for 1 month as UX designer :
https://www.linkedin.com/pub/alvin-wang/9b/21a/559
https://www.linkedin.com/pub/alan-chang/73/160/319/fr
As I would like to use this Framework in production, I'm wondering if they'll still have time to work on their framework. Otherwise the durability of this project seems to be compromised.
Thanks a lot!

I'm using it in prod for a personnal project. Even if not finished i think it's the best starter kit for material design actually. You'll have to code some more scss to fit your needs, but feel free to PR :)

Ok, thank you.
But will they have time to work on the Framewok in the future as the team is a group of students?

@Clemzd I don't understand the question.

I'm not talking about my production.
I'm talking about fixings bugs of the Framework, maintain the support on the different browsers. Those tasks are handled by Materialize Team, right ?

In my understanding an open source project like Materialize works like this: Maintainers might fix stuff themselves if their personal usecase is in question. But mostly they take care of community and guide PRs to merging.
If you got the initial framework for free, why wouldn't you fix bugs yourself and send a PR but instead ask for the maintainers to fix it?

@filtercake because in a professional environement, you usually do not adopt a framework unless it is stable and actively maintained, since one often maintains multiple projects and the project can move from team to team where team members are not necessarily fully familiarized with this or that framework, so you wan't it to be plug and play as much as possible to be able to upgrade it, yet not focus on the framewor, but instead on the product you are building.
IE. jQuery since years you can put it in a project and when a new version comes out just overwrite with the latest, and usually you do not have to worry about compatibility and stuff because the team made sure almost everything was tested heavily before release.

@robert-hoffmann I don't want to argue about this, that is why I phrased my statements as questions. I am totally aware what many people consider "normal" and "professional" behaviour in this regard and I think it's broken :) but let's continue that somewhere else if you're interested...

@filtercake i agree with @robert-hoffmann I will not have time to spend and fix the bugs alone because I'm working in a professional environnement.
The fact is the better css material design framework i've found so far is Materialize. I'm pretty sure new ones will come up in the next few years. But currently I think it is the best and that's why I'd like to know if the team plan to maintain it within the 2-3 next years.

@Clemzd All things said ..with over 10K watchers and 1K+ forks, i do not think you have to worry about things being maintained. Probably more with breaking changes as the project matures and adds/removes/reorganizes features.
For example i just saw a thread asking why old fonts where left in the project since they moved to google fonts.
If i where you (and i'm not) i'd either just use this project as-is in a single project (as a live test) to it's full extent and not worry too much about the rest. Or just sit things out for 6-12months and see how things evolve.
Still some feedback from the devs would be nice to get a clearer picture of what the roadmap is.

I'm using this in a production product as this seems to be the most popular Material design framework currently out there. I'm hoping that it continues to receive development and get to a really mature and full-featured 1.0. If not, I'll probably end up forking it as internal project.

@robert-hoffmann the old fonts were left in the project because we want to phase out the fon'ts slowly rather than remove the dependency immediately which would break people's sites.

Hey guys, We definitely do have less time since we left college and started working. Our intentions are to maintain and keep this project alive, but we are going to be more open towards pull requests as this would be the most efficient use of our time. Once things settle down around work we can look into working more on Materialize. We hope you guys understand.
Keep submitting issues, giving feedback, and pull requesting! We appreciate all of it.

Just my two cents, Materialize is a pretty production-ready framework. It's not the best with older browsers, but it has amazing compatibility compared to Google's Polymer framework and looks about as nice.
I use Materialize in my app Nimbus and I've only had to change my code once while upgrading the framework (when they changed the format of the toast function a bit). Materialize lets me target a fairly generous amount of web browsers and works well :)

@Clemzd is this professional environment with capital "P" or just "p"? will it remain as such in 3-4 years time or are you planning upgrade/downgrade? ;-P take it easy.
jokes aside i do appreciate the problem highlighted here, which is why i would like to propose renaming this thread before closing it. i would give it an informative title like: "can i have the cake and eat it?"
thumbs up to @filtercake and @robert-hoffmann

@rafszul i you give me some cake, i would definitely want to eat it ^^

@Dogfalo All this talk, and you saying you guys are now working and figuring out how to balance work-life and this cool project kinda reminds me when @jereisig left jquery to pursue other things.
A great reflex i think he had is moving jquery to a foundation (first dojo) and the jquery. Maybe you can reach out to him and ask him some questions on how this kind of stuff works.
I know from a project i maintained for a few years that some big companies and open-source projects also ask crazy questions like "are you the some code owner", or do you have some kind of disclaimer in place for contributors to make sure the IP is problem free, before they even dare including you code in some kind of distribution.
In any case, going the foundation route could maybe be a good idea in this case. Reach out to @jereisig, i'm sure he'll be more than happy to answer a few questions. Or if you want more media coverage, someone like paul irish would probably jump at the idea of getting his name associated with a project like this.
Anyways, just some food for thought.
Great project guys !

@acburst @Dogfalo Since you guys will have less time since you started working, dont you think you should put materialize's website on github also? so as features are added the website could grow without there being additions without reference. Thanks.

@UriahsDev Look at the repository, everything is right there, and seems to be hosted in github pages.

@carlosperate silly me! i knew that i totally forgot :3 got caught up in the conversation

OMG ! Google just released their Material Design Framework : https://github.com/google/material-design-lite.

@Clemzd  I looked through this last night. It's definitely lightweight and doesn't offer anywhere near the level of functionality that Materialize offers. I am wondering if Google will add functionality to it or not, because a lot of the missing functionality exists as part of the material elements in Polymer.

I have used Materialize in two simple websites for clients. As long as you don't have it setup to import the latest version, you'll be fine.

Bringing back what @robert-hoffmann suggested a while ago, it looks like the Materialize team is in some serious need for community management support.
We all understand that you guys can only work on this on your spare time, and I'm not trying to sound ungrateful, as clearly this is a great project we have all benefited from. But issues are piling up, pull requests are taking weeks to be reviewed, or are incorrectly reviewed, or aren't reviewed at all! and there is barely any real support offered.
Have you guys planned how to deal with this issue at all? Right now it just feels like you struggle to barely support more than a couple issues a week.

We are looking into recruiting some people to help manage issues and pull requests. @carlosperate we appreciate your contributions throughout the lifetime of this project, and think you could greatly impact this project. If you are interested in helping us out, send us an email at<EMAIL_ADDRESS>

@Dogfalo followed up this conversation from the top.
Is it possible to get a roadmap of pending features the team would like to add for the next release?
Also, it might be helpful if we find a way of prioritizing the issues. We can chip in and work on them based on priority.

If it helps, we can mark certain issues to be completed for next release and then if we have consistent contributions from the open source community, it could help out the project a lot.

I think that will be helpful. You can mark those that have high demand and you feel they are achievable. I am looking at the code, I hope to help in a couple of them.

How many people have commit access to this project? Maybe for starters it would be useful to enlarge the group with commit bit?

I found this by searching Google for "Should I use Materizlizecss in production?"
There are 1000 issues and 300 pull requests and I can't imagine searching through all thos properly to determine if someone else already asked my issue or even has a pr for it.
That's the only thing stopping me from using materialize at the moment.

I will close this issue for now. We are currently working on cleaning up old and duplicate issues.

closed without answering or indicating if there's anything wrong with it or what exactly it is a duplicate of.

This is a general question and as you can see, we are working on issue closing and fixing bugs with pull request.


Add use of priorityClasses to SDK

This PR enables the use of PriorityClasses for our RayClusters.
closes #182

Adds a new parameter dispatch_priority to ClusterConfig
If dispatch_priority is left empty, appwrapper is not affected.
If dispatch_priority is used, then appwrapper adds the field priorityClassName to the head and worker group specs.
Also adds functionality to check the validity of the appwrapper during cluster.up(). This addition is required as not all values for dispatch_priority are valid and the cluster needs to be checked for valid values before applying the appwrapper.

@asm582 Do you want to give this another look and approve the changes requested?

LGTM, priority levels for now are hardcoded in some way, in the future users may need fewer or more priority levels.

@MichaelClifford Checked this is approved, please feel free to merge


Should the token have all scopes listed in swagger description?

Hi, I have a question regarding implementation:
https://github.com/zalando/connexion/blob/master/connexion/decorators/security.py#L88
checks that access token must have any scope from the list of scopes.
Should the check be more strict, by checking that declared scopes is a subset of token's scopes?

@zd0 that is exactly what it does, no?

@zd0 I think you are right, it should check allowed_scopes <= user_scopes.

I get it now. In the logging message that was clearly the intention: https://github.com/zalando/connexion/blob/master/connexion/decorators/security.py#L89 ("one of allowed scopes")
If we are going to change that, it is a breaking change with older versions.

This is clearly a bug and needs to be fixed, see https://github.com/OAI/OpenAPI-Specification/blob/OpenAPI.next/versions/2.0.md#securityRequirementObject .

The object can have multiple security schemes declared in it which are all required (that is, there is a logical AND between the schemes).
...
If the security scheme is of type "oauth2", then the value is a list of scope names required for the execution.


Add Docker support

Add Docker support and Change *.zh-cn.resx to *.zh.resx (Msbuild does not support language-specific issues in Linux/Docker)

@Seaear - Great work, I will check it.
Thanks!

@skoruba
Add missing Chinese translation

@skoruba Fix carsh on SQL Server 2008 R2

Using ROW_NUMBER command/query works down to SQL Server 2005 :)

Why did you close this PR, @Seaear ? Is docker support not getting added?

New PR https://github.com/skoruba/IdentityServer4.Admin/pull/178
@Jaben


Probably shouldn't use git paths in tutorial

Currently, if you try to follow the tutorial, you'll get two different versions of futures, resulting in a confusing error message about futures::Future not being in scope. This is because tokio-core uses futures = "0.1", so the versions pulled are different. The same applies to tokio-tls and tokio-core, with tokio-tls pulling in the crates.io version.

Indeed! Changed.


Several power infrastructure items are more shocking

About the PR

Substations, SMESes, APCs and the AME controller are now shocking. I suggest you don't hit them.

Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl:

tweak: Due to safety cutbacks, electrical devices like substations are more dangerous. Safety protocols demand you do not hit them. We strongly advise you not to bite them either!

logically these dont make much sense but gameplay wise this is perfect for rats destroying apcs

I'm thinking it'd be nicer if they released an electricity "splash" after being attacked a few times, or just apply electrification not on the first hit; I don't see why hitting an APC once with your fist would make you go wzhzhzhzhwhzwhwzhwzhwhzzhhzhz

maybe just stun people instead of electrocuting them

Of course it's possible to reduce the level of shocking. But why are you hitting these things?

Of course it's possible to reduce the level of shocking. But why are you hitting these things?

Accidentally in harm mode.

i feel like these are uncommon enough that if you are hitting them by accident that is on you

(and if you are hitting them by accident you are probably insulated anyway)


system/dark/light theme in patterns

in registry/patterns-react it's ready to use
wip in other patterns


Use mac_ver instead of sysconfig.get_platform when determining the platform tag and document the MacOS platform tag handling

Signed-off-by: Filipe Laíns<EMAIL_ADDRESS>

I pushed a commit changing to mac_ver and adjusted the documentation.

The change looks correct, however the tests are unhappy. I'll test locally.

The test failures are all from
    assert name.group('plat') == sysconfig.get_platform().replace('-', '_').replace('.', '_')

so that's kind of a self-consistency check. Given that that check is supposed to still be true for other platforms, it looks like it'd be useful to create that as a single test function.
Testing related: adding x86_64 and arm64 CPython tags to test_interpreter_tag may be good too.


Docker Build Errors

### Description

#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     ... 5 lines matching cause stack trace ...
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445 {
#12 42.47   digest: '3755571661',
#12 42.47   [cause]: n [Error]: Dynamic server usage: Route / couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#12 42.47       at l (/app/.next/server/chunks/948.js:1:37247)
#12 42.47       at c (/app/.next/server/chunks/998.js:47:7557)
#12 42.47       at /app/.next/server/chunks/998.js:47:2457
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47       at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47       at /app/.next/server/chunks/237.js:1:13916
#12 42.47       at /app/.next/server/chunks/998.js:47:3614
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at /app/.next/server/chunks/998.js:47:4106 {
#12 42.47     description: "Route / couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#12 42.47     digest: 'DYNAMIC_SERVER_USAGE'
#12 42.47   }
#12 42.47 }
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     ... 5 lines matching cause stack trace ...
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445 {
#12 42.47   digest: '3755571661',
#12 42.47   [cause]: n [Error]: Dynamic server usage: Route / couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#12 42.47       at l (/app/.next/server/chunks/948.js:1:37247)
#12 42.47       at c (/app/.next/server/chunks/998.js:47:7557)
#12 42.47       at /app/.next/server/chunks/998.js:47:2457
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47       at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47       at /app/.next/server/chunks/237.js:1:13916
#12 42.47       at /app/.next/server/chunks/998.js:47:3614
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at /app/.next/server/chunks/998.js:47:4106 {
#12 42.47     description: "Route / couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#12 42.47     digest: 'DYNAMIC_SERVER_USAGE'
#12 42.47   }
#12 42.47 }
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     ... 5 lines matching cause stack trace ...
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445 {
#12 42.47   digest: '3755571661',
#12 42.47   [cause]: n [Error]: Dynamic server usage: Route / couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#12 42.47       at l (/app/.next/server/chunks/948.js:1:37247)
#12 42.47       at c (/app/.next/server/chunks/998.js:47:7557)
#12 42.47       at /app/.next/server/chunks/998.js:47:2457
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47       at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47       at /app/.next/server/chunks/237.js:1:13916
#12 42.47       at /app/.next/server/chunks/998.js:47:3614
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at /app/.next/server/chunks/998.js:47:4106 {
#12 42.47     description: "Route / couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#12 42.47     digest: 'DYNAMIC_SERVER_USAGE'
#12 42.47   }
#12 42.47 }
#12 42.47
#12 42.47 Error occurred prerendering page "/". Read more: https://nextjs.org/docs/messages/prerender-error
#12 42.47
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47     at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47     at /app/.next/server/chunks/237.js:1:13916
#12 42.47     at /app/.next/server/chunks/998.js:47:3614
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445
#12 42.47    Generating static pages (2/6)
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     ... 5 lines matching cause stack trace ...
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445 {
#12 42.47   digest: '3755571661',
#12 42.47   [cause]: n [Error]: Dynamic server usage: Route /_not-found couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#12 42.47       at l (/app/.next/server/chunks/948.js:1:37247)
#12 42.47       at c (/app/.next/server/chunks/998.js:47:7557)
#12 42.47       at /app/.next/server/chunks/998.js:47:2457
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47       at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47       at /app/.next/server/chunks/237.js:1:13916
#12 42.47       at /app/.next/server/chunks/998.js:47:3614
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at /app/.next/server/chunks/998.js:47:4106 {
#12 42.47     description: "Route /_not-found couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#12 42.47     digest: 'DYNAMIC_SERVER_USAGE'
#12 42.47   }
#12 42.47 }
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     ... 5 lines matching cause stack trace ...
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445 {
#12 42.47   digest: '3755571661',
#12 42.47   [cause]: n [Error]: Dynamic server usage: Route /_not-found couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#12 42.47       at l (/app/.next/server/chunks/948.js:1:37247)
#12 42.47       at c (/app/.next/server/chunks/998.js:47:7557)
#12 42.47       at /app/.next/server/chunks/998.js:47:2457
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47       at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47       at /app/.next/server/chunks/237.js:1:13916
#12 42.47       at /app/.next/server/chunks/998.js:47:3614
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at /app/.next/server/chunks/998.js:47:4106 {
#12 42.47     description: "Route /_not-found couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#12 42.47     digest: 'DYNAMIC_SERVER_USAGE'
#12 42.47   }
#12 42.47 }
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     ... 5 lines matching cause stack trace ...
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445 {
#12 42.47   digest: '3755571661',
#12 42.47   [cause]: n [Error]: Dynamic server usage: Route /_not-found couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#12 42.47       at l (/app/.next/server/chunks/948.js:1:37247)
#12 42.47       at c (/app/.next/server/chunks/998.js:47:7557)
#12 42.47       at /app/.next/server/chunks/998.js:47:2457
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47       at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47       at /app/.next/server/chunks/237.js:1:13916
#12 42.47       at /app/.next/server/chunks/998.js:47:3614
#12 42.47       at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47       at /app/.next/server/chunks/998.js:47:4106 {
#12 42.47     description: "Route /_not-found couldn't be rendered statically because it used `headers`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#12 42.47     digest: 'DYNAMIC_SERVER_USAGE'
#12 42.47   }
#12 42.47 }
#12 42.47
#12 42.47 Error occurred prerendering page "/_not-found". Read more: https://nextjs.org/docs/messages/prerender-error
#12 42.47
#12 42.47 Error: Usage of next-intl APIs in Server Components currently opts into dynamic rendering. This limitation will eventually be lifted, but as a stopgap solution, you can use the `unstable_setRequestLocale` API to enable static rendering, see https://next-intl-docs.vercel.app/docs/getting-started/app-router/with-i18n-routing#static-rendering
#12 42.47     at /app/.next/server/chunks/998.js:47:2555
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at u (/app/.next/server/chunks/998.js:47:3297)
#12 42.47     at get locale [as locale] (/app/.next/server/chunks/998.js:47:3607)
#12 42.47     at /app/.next/server/chunks/237.js:1:13916
#12 42.47     at /app/.next/server/chunks/998.js:47:3614
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4106
#12 42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
#12 42.47     at /app/.next/server/chunks/998.js:47:4445
#12 42.47    Generating static pages (4/6)
#12 42.55  ✓ Generating static pages (6/6)
#12 42.56
#12 42.56 > Export encountered errors on following paths:
#12 42.56 	/_not-found/page: /_not-found
#12 42.56 	/page: /
#12 42.64 error Command failed with exit code 1.
#12 42.64 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
#12 ERROR: process "/bin/bash -ol pipefail -c yarn run build" did not complete successfully: exit code: 1
------
 > [stage-0  8/10] RUN --mount=type=cache,id=rNUQWv1F8Zw-next/cache,target=/app/.next/cache --mount=type=cache,id=rNUQWv1F8Zw-node_modules/cache,target=/app/node_modules/.cache yarn run build:
42.47     at /app/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:185493
42.47     at /app/.next/server/chunks/998.js:47:4445
42.47    Generating static pages (4/6)
42.55  ✓ Generating static pages (6/6)
42.56
42.56 > Export encountered errors on following paths:
42.56 	/_not-found/page: /_not-found
42.56 	/page: /
42.64 error Command failed with exit code 1.
42.64 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
------
Dockerfile:24
--------------------
  22 |     # build phase
  23 |     COPY . /app/.
  24 | >>> RUN --mount=type=cache,id=rNUQWv1F8Zw-next/cache,target=/app/.next/cache --mount=type=cache,id=rNUQWv1F8Zw-node_modules/cache,target=/app/node_modules/.cache yarn run build
  25 |
  26 |
--------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c yarn run build" did not complete successfully: exit code: 1
Error: Docker build failed
ERROR: Error: child exited with code 1:

Verifications

[x] I've verified that the problem I'm experiencing isn't covered in the docs.
[X] I've searched for similar, existing issues on GitHub and Stack Overflow.
[X] I've compared my app to a working example to look for differences.

Mandatory reproduction URL
https://github.com/FullGreenGN/technosphere/
Reproduction description
Just use next-into and build docker file
Expected behaviour
Impossible to build app for Vercel

Error caused by layout in wrong folder


Using ES6 with Node 4.0.0+ as engine

Right now, our tests in TravisCI cover (and pass) older versions of Node (0.10, 0.11, 0.12, iojs). I'm considering to use some ES6 features and wanted to understand you think there would be a tremendous drawback if we remove backward compatibility.
@shernshiou @puzzlehe4d

@agraebe yes that's my plan. That's the reason I am not developing in pre4.0.0 version anymore.

Cool, how about we create a new branch for es6 development?


allow button group to be unmarshaled from JSON

Signed-off-by: Wayne Witzel III<EMAIL_ADDRESS>What this PR does / why we need it:
Allows ButtonGroup to be used by Plugins.
Which issue(s) this PR fixes

related #646

Special notes for your reviewer: Example plugin https://github.com/wwitzel3/octant-actions-plugin

I know you were interested in this functionality @ImJasonH


Update with Bloch theory

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/24065
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.3 -m "<description of version>" 523bb0fe69db9b112f64b44acbbdafc8fa08631a
git push origin v0.1.3


Booster: Optimise handling of known predicates when rewriting

When processing an "execute" request, we are rewriting from one Pattern to one or more Patterns. In Booster, we assume definedness of the initial Pattern, i.e. it must have no "bad" applications of partial functions and the constrains should not be inconsistent.
Since the introduction of the SMT solver into Booster, we have been very conservative in how we interact with it, potentially leaving a lot of performance on the table. We should optimise the communication of the constraints of the initial Pattern, aka the known predicates, aka "the path condition" to the solver:

when starting an execute request, we should check the consistency of the constraints to make sure we are not rewriting inside a vacuous state (this is done in https://github.com/runtimeverification/haskell-backend/pull/4020)
if the constraints are consistent, we will reuse the created Z3 scope for all subsequent queries, hopefully making the side condition checks very fast
when we add new constraints to the current Pattern withing the same "execute" request, we have two options:

the conservative option would be to pop the scope, add the ensured condition to the known predicates, push them ch check-sat;
the more performant option is to push the scope before checking the ensures conditions and avoid re-checking the previous know truth.

We will need to re-think how we do solver retries. Currently, we do a hard reset every time we retry, which restarts the solver. We should instead to a soft reset:

pop the last scope, which includes the predicate we are checking
reuse the previous scopes that contain the known predicates
change the timeout setting, push and check sat

The comments to this issue will outline the specific changes that need to be done to the SMT-related types and function in Booster to implement this approach.

I think that we always want the ground truth G at the top of the Z3 stack, knowing it's SAT.
Then, when we're checking a branching on P, we do:

add P, check SAT, pop
add !P, check SAT, pop

get the results back and G is still at the top of the stack.
When an ensures P comes, should we not do the following?

add P, check SAT

if SAT, drop the top of the stack (to avoid accumulation), push, and we have the (new) ground truth at the top of the stack;
if UNSAT, the branch is vacuous, do whatever is appropriate

Some notes from a first code inspection:

needs additional state above the SMT.Runner.SMTContext level

each request during a rewrite/simplification (isSat, checkPredicates) will send the path condition again.
The current path cond. needs to be checked against the one used most recently (and pushed).
additional constraints added since the last time (set diff) need to be translated and asserted to z3 (and pushed)

the state is now two-fold (or we add this to SMTContext), remembering pushed assertions, which needs more careful management
we have to refactor to remove the hardResetSolver calls
on retry, we need to re-push the last known path cond. (as a whole) and check isSat for it to seed the solver (probably
timeout prone?) before re-trying the actual request at hand.

As it turned out from lido experiments, it is a much bigger benefit to reduce the  amount of prelude and ground truth assertions than to keep them around as state in Z3. Closing


Import the cloud package from the new path

/cc @broady

Mailed https://go-review.googlesource.com/#/c/25461/ instead.


Free Reroll

Hi, I do like your module very much, it is great. Would you mind adding a choicebox for functionality of a free Reroll? Reroll and All-In seems to be working correctly, also Danger and Gamble. But I cannot choose Free Reroll, so I have to check back the deleted Success. And I cannot make the Reroll, when I got no success on the first dice-throw (there should be only the option for a free reroll).
Thank you for checking.

Leave it with me and I'll look in to it once I've upgraded the system to Foundry V12.

Leave it with me and I'll look in to it once I've upgraded the system to Foundry V12.

I founde the issue, there were only two lines in the wrong sequence. I tested locally and everything is working now :). I created a Pull request, would be perfect if you approve. Thank you!

Approved, merged and new release created.
Thank you for fixing this Karsten


m1 max 64g, environment crashes when running powerpoint for object removal

BrushNet inference, step = 0: image batch = 1, got 2 latents, starting from 0
BrushNet inference: sample torch.Size([2, 4, 192, 128]) , CL torch.Size([2, 5, 192, 128])
/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion [MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32' Abort trap: 6 (comfyui) weiweideMacBook-Pro:comfyui weiwei$ /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown   warnings.warn('resource_tracker: There appear to be %d ' (comfyui) weiweideMacBook-Pro:comfyui weiwei$ BrushNet inference: do_classifier_free_guidance is True -bash: BrushNet: command not found (comfyui) weiweideMacBook-Pro:comfyui weiwei$ BrushNet inference, step = 0: image batch = 1, got 2 latents, starting from 0 -bash: BrushNet: command not found (comfyui) weiweideMacBook-Pro:comfyui weiwei$ BrushNet inference: sample torch.Size([2, 4, 192, 128]) , CL torch.Size([2, 5, 192, 128]) -bash: syntax error near unexpected token ('
(comfyui) weiweideMacBook-Pro:comfyui weiwei$ /AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'

It's just that the environment crashes when you run the function with this parameter “object removal”, but it's fine with the other parameters.

I just added debug key to PowerPaint node. Could you please, update the nodes and run it with debug=True and then post ComfyUI output? So I can pinpoint the location of error.

Okay, I'll try that later.

image outpainting, The error log is as follows:
clip missing: ['text_projection.weight']
PowerPaint base CLIP file:  /Users/weiwei/ComfyUI/models/clip/model.fp16.safetensors
Successfully add external embeddings: P_ctxt, P_shape, P_obj. current
Successfully add trainable external embeddings: P_ctxt, P_shape, P_obj current
PowerPaint CLIP file:  /Users/weiwei/ComfyUI/models/inpaint/powerpaint/pytorch_model.bin
Requested to load SD1ClipModel
Loading 1 new model
Base model type: SD1.5
BrushNet image.shape = torch.Size([1, 1536, 1536, 3]) mask.shape = torch.Size([1, 1536, 1536])
Requested to load AutoencoderKL
Loading 1 new model
BrushNet CL: image_latents shape = torch.Size([1, 4, 192, 192]) interpolated_mask shape = torch.Size([1, 1, 192, 192])
You should add to positive prompt: "empty scene"
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
0%|                                                                                | 0/20 [00:00<?, ?it/s]BrushNet inference, step = 0: image batch = 1, got 1 latents, starting from 0
BrushNet inference: sample torch.Size([1, 4, 192, 192]) , CL torch.Size([1, 5, 192, 192])
/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
Abort trap: 6
(comfyui) weiweideMacBook-Pro:comfyui weiwei$ /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

It seems you didn't set debug flag in PowerPaint node to True.

It seems you didn't set debug flag in PowerPaint node to True. Please, do it.

Sorry I overlooked that, I'm running it now.

PowerPaint model type: SD1.5
PowerPaint model file: /Users/weiwei/ComfyUI/models/inpaint/powerpaint/powerpaint_v2.safetensors
PowerPaint model is loaded
clip missing: ['text_projection.weight']
PowerPaint base CLIP file:  /Users/weiwei/ComfyUI/models/clip/model.fp16.safetensors
Successfully add external embeddings: P_ctxt, P_shape, P_obj. current
Successfully add trainable external embeddings: P_ctxt, P_shape, P_obj current
PowerPaint CLIP file:  /Users/weiwei/ComfyUI/models/inpaint/powerpaint/pytorch_model.bin
Requested to load SD1ClipModel
Loading 1 new model
Base model type: SD1.5
BrushNet image.shape = torch.Size([1, 1536, 1536, 3]) mask.shape = torch.Size([1, 1536, 1536])
Requested to load AutoencoderKL
Loading 1 new model
BrushNet CL: image_latents shape = torch.Size([1, 4, 192, 192]) interpolated_mask shape = torch.Size([1, 1, 192, 192])
You should add to positive prompt: "empty scene"
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
0%|                                                                                | 0/20 [00:00<?, ?it/s]BrushNet inference, step = 0: image batch = 1, got 1 latents, starting from 0
BrushNet inference: sample torch.Size([1, 4, 192, 192]) , CL torch.Size([1, 5, 192, 192])
BrushNet: step = 0
BrushNet CA: attn mask
BrushNet CA: time
BrushNet CA: pre-process
BrushNet CA: down
/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
Abort trap: 6
(comfyui) weiweideMacBook-Pro:comfyui weiwei$ /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

“object removal”, The error log is as follows:
model_type EPS
Using split attention in VAE
Using split attention in VAE
using extra model: /Users/weiwei/ComfyUI/models/sams/sam_vit_h_4b8939.pth
final text_encoder_type: bert-base-uncased
/Users/weiwei/Envs/comfyui/lib/python3.10/site-packages/transformers/modeling_utils.py:1051: FutureWarning: The device argument is deprecated and will be removed in v5 of Transformers.
warnings.warn(
PowerPaint model type: SD1.5
PowerPaint model file: /Users/weiwei/ComfyUI/models/inpaint/powerpaint/powerpaint_v2.safetensors
PowerPaint model is loaded
clip missing: ['text_projection.weight']
PowerPaint base CLIP file:  /Users/weiwei/ComfyUI/models/clip/model.fp16.safetensors
Successfully add external embeddings: P_ctxt, P_shape, P_obj. current
Successfully add trainable external embeddings: P_ctxt, P_shape, P_obj current
PowerPaint CLIP file:  /Users/weiwei/ComfyUI/models/inpaint/powerpaint/pytorch_model.bin
Requested to load SD1ClipModel
Loading 1 new model
Base model type: SD1.5
BrushNet image.shape = torch.Size([1, 1536, 1024, 3]) mask.shape = torch.Size([1, 1536, 1024])
Requested to load AutoencoderKL
Loading 1 new model
BrushNet CL: image_latents shape = torch.Size([1, 4, 192, 128]) interpolated_mask shape = torch.Size([1, 1, 192, 128])
You should add to positive prompt: "empty scene blur"
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
0%|                                                                                | 0/20 [00:00<?, ?it/s]BrushNet inference: do_classifier_free_guidance is True
BrushNet inference, step = 0: image batch = 1, got 2 latents, starting from 0
BrushNet inference: sample torch.Size([2, 4, 192, 128]) , CL torch.Size([2, 5, 192, 128])
BrushNet: step = 0
BrushNet CA: attn mask
BrushNet CA: time
BrushNet CA: pre-process
BrushNet CA: down
/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
Abort trap: 6
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

There is also a case where the powerpoint node parameter is set to "object removal", which occasionally runs through, but does not produce the correct output.

Let's do this again: update the nodes and run it with debug = True.

Let's do this again: update the nodes and run it with debug = True.

ok

model_type EPS
Using split attention in VAE
Using split attention in VAE
PowerPaint model type: SD1.5
PowerPaint model file: /Users/weiwei/ComfyUI/models/inpaint/powerpaint/powerpaint_v2.safetensors
PowerPaint model is loaded
clip missing: ['text_projection.weight']
PowerPaint base CLIP file:  /Users/weiwei/ComfyUI/models/clip/model.fp16.safetensors
Successfully add external embeddings: P_ctxt, P_shape, P_obj. current
Successfully add trainable external embeddings: P_ctxt, P_shape, P_obj current
PowerPaint CLIP file:  /Users/weiwei/ComfyUI/models/inpaint/powerpaint/pytorch_model.bin
Requested to load SD1ClipModel
Loading 1 new model
Base model type: SD1.5
BrushNet image.shape = torch.Size([1, 1536, 1536, 3]) mask.shape = torch.Size([1, 1536, 1536])
Requested to load AutoencoderKL
Loading 1 new model
BrushNet CL: image_latents shape = torch.Size([1, 4, 192, 192]) interpolated_mask shape = torch.Size([1, 1, 192, 192])
You should add to positive prompt: "empty scene"
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
0%|                                                                                | 0/20 [00:00<?, ?it/s]BrushNet inference, step = 0: image batch = 1, got 1 latents, starting from 0
BrushNet inference: sample torch.Size([1, 4, 192, 192]) , CL torch.Size([1, 5, 192, 192])
BrushNet: step = 0
BrushNet CA: attn mask
BrushNet CA: time
BrushNet CA: pre-process
BrushNet CA: down
BrushNet CA (down block with XA):  <class 'ComfyUI-BrushNet.brushnet.unet_2d_blocks.CrossAttnDownBlock2D'>
XAD2: forward
XAD2: resnet hs # 0 torch.Size([1, 320, 192, 192])
XAD2: resnet temb # 0 torch.Size([1, 1280])
XAD2: attn hs # 0 torch.Size([1, 320, 192, 192])
XAD2: attn ehs # 0 torch.Size([1, 77, 768])
/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
Abort trap: 6
(comfyui) weiweideMacBook-Pro:comfyui weiwei$ /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

“object removal”, The error log is as follows:
/Users/weiwei/Envs/comfyui/lib/python3.10/site-packages/transformers/modeling_utils.py:1051: FutureWarning: The device argument is deprecated and will be removed in v5 of Transformers.
warnings.warn(
PowerPaint model type: SD1.5
PowerPaint model file: /Users/weiwei/ComfyUI/models/inpaint/powerpaint/powerpaint_v2.safetensors
PowerPaint model is loaded
clip missing: ['text_projection.weight']
PowerPaint base CLIP file:  /Users/weiwei/ComfyUI/models/clip/model.fp16.safetensors
Successfully add external embeddings: P_ctxt, P_shape, P_obj. current
Successfully add trainable external embeddings: P_ctxt, P_shape, P_obj current
PowerPaint CLIP file:  /Users/weiwei/ComfyUI/models/inpaint/powerpaint/pytorch_model.bin
Requested to load SD1ClipModel
Loading 1 new model
Base model type: SD1.5
BrushNet image.shape = torch.Size([1, 1536, 1024, 3]) mask.shape = torch.Size([1, 1536, 1024])
Requested to load AutoencoderKL
Loading 1 new model
BrushNet CL: image_latents shape = torch.Size([1, 4, 192, 128]) interpolated_mask shape = torch.Size([1, 1, 192, 128])
You should add to positive prompt: "empty scene blur"
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
0%|                                                                                | 0/20 [00:00<?, ?it/s]BrushNet inference: do_classifier_free_guidance is True
BrushNet inference, step = 0: image batch = 1, got 2 latents, starting from 0
BrushNet inference: sample torch.Size([2, 4, 192, 128]) , CL torch.Size([2, 5, 192, 128])
BrushNet: step = 0
BrushNet CA: attn mask
BrushNet CA: time
BrushNet CA: pre-process
BrushNet CA: down
BrushNet CA (down block with XA):  <class 'ComfyUI-BrushNet.brushnet.unet_2d_blocks.CrossAttnDownBlock2D'>
XAD2: forward
XAD2: resnet hs # 0 torch.Size([2, 320, 192, 128])
XAD2: resnet temb # 0 torch.Size([2, 1280])
XAD2: attn hs # 0 torch.Size([2, 320, 192, 128])
XAD2: attn ehs # 0 torch.Size([2, 77, 768])
/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:788: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
Abort trap: 6
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

This problem is related to the size of the input image and attention calculation in PowerPaint. I added an option to PowerPaint node. Update nodes and try to set save_memory to max. As far as I know Mac can't work with dtype=float16, so if the problem remains you should use CutForInpaint or resize image.

确实前面忽略了图片大小的问题，之前测试的图像尺寸是1024 × 1536。我刚刚更新之后，设置成了max，但是出的图片是黑色了

I pulled back the version, reduced the image size, and then ran the node again, and it worked fine, I apologize for wasting your time.

I rolled back to the version before your modification this afternoon and after turning the image down to 512, it came out fine. I apologize for wasting your revision time.

Thank you for assistance. It is pity that you get black image, could it be due to float16 flag? I added none option to save_memory, so it can be possible to use the old type optimization.

It could be, but when I change the parameter to float32, it runs incorrectly because the brushnet model type is only model.float16

<img width="300" alt="Screenshot 2024-06-01 00 05 56" src="https://github.com/nullquant/ComfyUI-BrushNet/assets/121092793/3faf590d-44ee-4645- a626-70531b1fb070">
I have to fall back to this version to get the image to come out properly, debug select true or false version


feat: added filter functionality.

Reviewers
Functional:
@tihuan

Changes
Added filter functionality behind filter feature flag.
Included in this PR are:

#1613 Implement core facet search module/engine.
#1609 Read datasets and collections on load.
#1602 Update collections UI to match mocks.
#1614 List basic metadata categories in left sidebar on collections view.
#1604 Add datasets view.
#1605 Update datasets table on collection detail page.

Notes
#1605 Update datasets table on collection detail page

The updates to the collection datasets table only includes UI changes and does not include a conversion to react-table.
To reduce changes to the existing collections datasets table (outside of the filter flag), we have duplicated the component hierarchy and styled the duplicated components.
Existing structure can be removed once collection datasets table is converted to react-table but this requires a review of the dataset fetch model and status checks.

Definition of Done (from ticket)

See individual tickets.

QA steps (optional)

Datasets located at /datasets.
Collections located at /collections.
Datasets table on collection detail page outside of feature flag should be reviewed.

Known Issues

Dataset downloads are not currently hooked up and will be resolved by #1724 Use dataset-metadata endpoint to populate dataset download modal.
Explorer URL is hooked up but won't be end-to-end until #1721 Add explorer_url to datasets/index is merged.
Redirect from / to /datasets is not currently hooked up and will be resolved by 1718 Remove filter feature flag.

@tihuan, thank you again for taking the time to review this PR; I have updated the code as per your review comments.
I have added added one fix to correct the display of the selected tag labels.


Simpler Core-Lightning plugin

Doesn't use gRPC at all
ILnRpcClient implementation for CLN does direct RPC with CLN instead of using gRPC to proxy the requests
The plugin forwards HTLC to gatewayd using the same authenticated HTTP POST mechanism that gateway-cli uses
The gateway's run() loop does SCID filtering to figure out whether an HTLC is relevant to us, and if it is, which federation "actor" to process it with. gRPC implementation did it inside the plugin.
If we changed LND to use this pattern, then Gateway::new() would call ILnRpcClient.subscribe_htlcs(sender) which would just send the HTLC to the Gateway.run() loop like the HTTP POST method does for CLN. So we'd have 1 HTLC subscription for the whole gateway, instead of 1 per actor (which doesn't seem to work with the gRPC library we've been using).

I think we should still keep the gRPC mechanism because it's a nice way interface for people to write their own custom gateway implementations against. Just don't use it for Core Lightning because a simpler approach works just fine.

The plugin forwards HTLC to gatewayd using the same authenticated HTTP POST mechanism that gateway-cli uses

That's not a nice architecture, I mean it works for local deployments, but now both sides have to kinda know/reach each other …

The plugin forwards HTLC to gatewayd using the same authenticated HTTP POST mechanism that gateway-cli uses

That's not a nice architecture, I mean it works for local deployments, but now both sides have to kinda know/reach each other …

Extension needs to know hostname and password for gatewayd webserver. Gatewayd doesn't need to know anything about extension. It's just the reverse of the current requirement that the address of the extension's gRPC server. No password because this connection is un-authenticated currently.


Potential issue in type inference in the CFE prevents a static method to be resolved

In the following line in the test an extension method can't be found for a receiver of a static type String. However, the extension method is resolved for a similar variable a3 of static type String a few lines above that.
https://github.com/dart-lang/co19/blob/master/LanguageFeatures/Patterns/variable_A01_t01.dart#L74

This looks like the root cause for the runtime failure in DDC for that test.

This looks like the root cause for the runtime failure in DDC for that test.


Make it work on PyPy 2.3.1/CentOS 6

I was getting this error when trying to import pybloomfiltermmap in PyPy 2.3.1 on CentOS 6.
ImportError: unable to load extension module '/usr/lib64/pypy-2.3.1/site-packages/pybloomfilter.pypy-23.so': /usr/lib64/pypy-2.3.1/site-packages/pybloomfilter.pypy-23.so: undefined symbol: __gxx_personality_v0

I didn't especially bother to figure out what was going on, or whether it was caused by PyPy or CentOS (though anecdotally it's probably CentOS or the versions of build chain there -- PyPy 2.3.1 on my mac works fine).
In any event, converting the C++ to C was straightforward and, I believe, safe. All tests are passing, and there are no errors nor warnings.

lgtm


Showing "application プロパティ****" (extra asterisks appears)

https://docs.microsoft.com/ja-jp/office/vba/api/publisher.application

Working on a fix: Bug 3290847


Update ADS to be compatible with python 3.8

ADS is currently supported only with python 3.7.
We would like ADS to be compatible with python 3.8 as well in preparation for the integration with ivadomed (which supports 3.6, 3.7 and 3.8).

Do we also need to make it 3.6-compatible?

Do we also need to make it 3.6-compatible?

I don't think so, my idea was more to "move forward" in parallel with ivadomed.
On a side note, we originally removed support of 3.6 because of FSLeyes, but FSLeyes now support 3.6 as well, so ADS would probably be ok to (see comments here)

As discussed in #405, we're goign to pause this issue and resume when making the move to IVADOMED.

Closed because we made this change with version 4.0's move to IVADOMED.


Default Javascript Popups

Is your feature request related to a problem? Please describe.
I'm always frustrated when I see default javascript alert() and prompt(), They are really lowering experience of using CirciutVerse
Describe the solution you'd like
I think adding a modal popup styled circiutverse-like will be a better solution
Additional context

Are you working on this? (Yes/No)
No

Can I work on fixing this issue. I suggest rather than prompts we could use small in-page windows or popups to replace this.

@Shivansh2407 I don't know if it will be GCI Task So, please ask on our Slack if you can do that before you start

are you still working on it @AyanChoudhary ?

No, I am no longer working on it

@YashKumarVerma are you on it ??

@Nitish145 No.

I am working on this issue.

@Nitish145 you can ask about that on #ux on Slack

@Nitish145 the jQuery dialogs will be fine or you can prefer doing something to match the theme of CircuitVerse dialogs. You can get more suggestions on Slack 😊.

@tachyons Can I work on this?

Yes

@tachyons Can I work on this?

Hey @vedant-jain03, Sorry to bother you but you know that as I was commented on the pr related to this issue, and after that, I was ill and busy with my college stuff. So, Please give me some more days to solve this if I am unable then you can start working on this...

@tachyons Can I work on this?

Hey @vedant-jain03, Sorry to bother you but you know that as I was commented on the pr related to this issue, and after that, I was ill and busy with my college stuff. So, Please give me some more days to solve this if I am unable then you can start working on this...

Absolutely fine. Let me know if you needed any help. Good luck 👍


Add support for GitHub Flavored Markdown new admonitions ("alerts")

GitHub just added support for alerts which are very very similar to admonitions.
> [!NOTE]
> Useful information that users should know, even when skimming content.

> [!TIP]
> Helpful advice for doing things better or more easily.

> [!IMPORTANT]
> Key information users need to know to achieve their goal.

> [!WARNING]
> Urgent info that needs immediate user attention to avoid problems.

> [!CAUTION]
> Advises about risks or negative outcomes of certain actions.

It would be great if the admonitions extension supports also the GitHub syntax, so one can write admonitions in a style that will also be parsed specially by GitHub and show properly.

This should already exist via https://github.com/oprypin/markdown-callouts. Did GitHub change its syntax from what is offered there? If so, you should maybe create an issue there.

I don't see any mentions there saying that the syntax is supposed to be supported by GitHub, it seems just like an alternative syntax to admonitions, so I'm not sure if that's really the right place for this issue. Maybe it is worth its own extension or something, not sure, but just saying that I don't think GitHub ever supported any other syntax for admonition-like blocks.

It was supposed to. The syntax they use used to work in GitHub. That's why he created it so that they'd render in GitHub, which the default admonitions did not.
> NOTE: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod
> nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor
> massa, nec semper lorem quam in massa.

Then GitHub recently did a pivot and now uses this new format which is used by Microsoft: https://learn.microsoft.com/en-us/contribute/content/markdown-reference#alerts-note-tip-important-caution-warning.
See relevant issue: https://github.com/oprypin/markdown-callouts/issues/13
Regardless,  Python Markdown has been generally resistant to adding new plugins and encourages people to use the extension API to create 3rd party extensions to keep the maintenance burden low on this project. This project already offers admonitions and has been there for years and is unlikely to change its syntax breaking users reliant on this format.

OK, it is probably not relevant anymore, but still I don't see anywhere that GitHub ever supported the syntax in markdown-callouts, that repo only says it uses a different syntax for graceful degradation, it doesn't say it was ever rendered specially by GitHub.

OK, closing because from what I read  here it seems it makes more sense to implement it as a separate extension (this extension being markdown-callouts or not).


Add a way to delete goals in the main branch.

Currently you cannot right click goals, (it just registers as left click). Either enable right clicking them, or add a delete button someplace inside of them. Reading the open issues, this should be fixed in some of the branches, but it is not fixed in the main branch.

Closing as duplicate of #3, fixes were made but never actually deployed to Plugin Hub.
I'll be deploying them shortly.


Translation not updated when we release it using over-the-air

Hello Team,
I have created one distribution hash key and on the behalf of that hash key i am updating my translation text.
i found that it will work for first time but foe another time it will not update the text in React Native.
Team please help me
Thanks in advance

Hi @arjundalal,
CDN caches all the translations in release for up to 1 hour, and even when new translations are released in Crowdin, CDN may return them with a delay.


Improve generated docs for commands.

Don't output a flag table if there are no flags for a command.

Don't output a Shorthand column in a flag table if there are no shorthands defined for the
command.

Attach mixc's tracing flags to the check and report commands instead of at the top since
that's where they apply.

@geeknoid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
ee4387ece62cff4e6cbb5d292e6058e16e828da3
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
a89d6da884389851850c551cb9b6f27477b91e84
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ZackButcher Ping...

@geeknoid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
0b7a9487127d67a04db02aafbe0a4314173d809d
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


How to disable for netrw?

New to Neovim and use lazy.nvim to manage my plugins, here's what I have so far to configure Precognition.
    {
        "tris203/precognition.nvim",
        config = true,
        opts = {
            disabled_fts = { "netrw" },
        },
    },

I could be making a mistake here. The above loads Precognition fine, however, it does not disable whenever I open Netrw via :Explore. Am I doing something wrong?

You don't want to have both config, and opts
Remove the config line if you have opts set

Thanks for that, it's helped clean up my configs a little bit, however the problem still persists. Is there any way to supply any debug info to make diagnosing this any easier?

I have just taken a look, an there is an issue, I will push a fix later tonight

This should now be fixed @cyrus01337
If you get any further issues let me know
Thanks for the report

Works perfectly, thanks for Precognition. Have loved it so far. :)


Feature Request: DbMigrator can handle cases where db migrations and code migrations have diverged

Currently the only supported scenarios for automatically applying migrations from code to db, whether by dbContext.Database.Migrate() or by dbMigrate.Update(), are when the code is ahead of or equal to the db.
code: A -> B -> C
  db: A -> B

code: A -> B
  db: A -> B

There is another case where code is behind the db
code: A -> B
  db: A -> B -> C

which can be handled by specifying the migration to move back to, e.g. dbMigration.Update("B") in code or Update-Database Target-Migration B in Package Manager Console. As far as I know, there is no way to automatically move back to the migration in the code, so it must be done manually or by writing code to automate.
There is one more, complex case, where the code and db have diverged
       C -> D (code)
      /
A -> B
      \
       X -> Y -> Z (db)

This can be fixed by targeting the nearest common ancestor migration, then the latest db migration. Again, no way of doing this as far as I know.
What would be nice would be to have an extension method dbMigrator.UpdateSmart() that could handle all the cases. It would be especially useful for a development team using EF because when a developer checks out any version of the code from version control he or she can sync the local db's schema with a single command.

@bricelam to take a look.

This would require #1911 and #2532

@jakingmsft Thanks for the suggestion, but after discussing this we think that the complexities of making this work reliably and consistently probably outweigh the value, especially when compared to other areas we could spent development time on instead.


KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table

Problem
https://issues.apache.org/jira/browse/KAFKA-16073
The consumer will receive 'OutOfRangeException' in the following case:
In a specific concurrent scenario, imagine sequential offsets: offset1 < offset2 < offset3. A client requests data at offset2. While a background deletion process removes segments from memory, it hasn't yet updated the LocalLogStartOffset from offset1 to offset3. Consequently, when the fetch offset (offset2) is evaluated against the stale offset1 in ReplicaManager.handleOffsetOutOfRangeError, it incorrectly triggers an OffsetOutOfRangeException. This issue arises from the out-of-sync update of localLogStartOffset, leading to incorrect handling of consumer fetch requests and potential data access errors.
Summarizing the example scenario from the JIRA comment
Let us assume each segment has one offset in this example:
log start offset 0
log end offset 10
local log start offset 4
fetch offset 6
new local log start offset 7
Deletion based on retention configs is started and eventually updating the local log start offset as 7.
There is a race condition here where the segments list is updated by removing 4, 5, and 6 offset segments in LocalLog and then updates the local-log-start-offset. But fetch offset is being served concurrently and it may throw OffsetOutOfRangeException if the inmemory segments are already removed in LocalLog and local-log-start-offset is not yet updated as 7 when it executes the code as it fails the condition because fetch offset(6) < old local-log-start-offset(4).
co-authored-by: hzh0425<EMAIL_ADDRESS>Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Unit tests are not added to assert that the as the local-log-start-offset is incremented before the segments are removed from inmemory table as it is in private method. PTAL.
cc @satishd @junrao @showuon

@junrao
Addressed the review comments. PTAL.

@kamalcph : Thanks for the updated PR. The code looks good to me. Do you know why there were 106 failed tests?

Rebased the PR with trunk, the test failures are unrelated. Ran the 11 failed tests locally, all of them got succeeded:
./gradlew cleanTest core:test --tests kafka.log.LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic tools:test --tests org.apache.kafka.tools.consumer.group.DescribeConsumerGroupTest.testDescribeExistingGroupWithNoMembers trogdor:test --tests org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated connect:mirror:test --tests org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testReplicateSourceDefault core:test --tests kafka.api.ConsumerBounceTest.testConsumptionWithBrokerFailures --tests kafka.api.PlaintextAdminIntegrationTest.testElectPreferredLeaders --tests kafka.server.DelegationTokenRequestsTest.testDelegationTokenRequests --tests kafka.zk.ZkMigrationIntegrationTest.testPartitionReassignmentInHybridMode tools:test --tests org.apache.kafka.tools.MetadataQuorumCommandTest.testDescribeQuorumStatusSuccessful --tests org.apache.kafka.tools.reassign.ReassignPartitionsIntegrationTest.testProduceAndConsumeWithReassignmentInProgress trogdor:test --tests org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated

@junrao
Please take another look, thanks!

@kamalcph : Thanks for the update. JDK 11 and Scala 2.13 didn't complete in the last test run. Could you trigger another run of the tests?

Re-triggered the tests. Tests failures are unrelated.

@junrao
Gentle reminder. PTAL.

@kamalcph : Do you want this PR to be in 3.7 and 3.6? If so, could you create a separate cherry-pick RP? Thanks.

@kamalcph : Do you want this PR to be in 3.7 and 3.6? If so, could you create a separate cherry-pick RP? Thanks.

Opened #15747 and #15748 to backport it to 3.7 and 3.6 branches. PTAL. Thanks for the review!


#3761 display running remediations in service screen

Closes #3761

Signed-off-by: Klaus Strießnig<EMAIL_ADDRESS>

@Kirdock the current implementation looks already good, but I would like to review it before merging.
Can we therefore do a pair-review session please?

sure

Updated image: