NIFI-3026: Support multiple remote target URLs

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

@markap14 I checked how Apache Flink uses SiteToSiteClient from NiFiSink. That is the same pattern Storm and Spark integration does.
I agree that third parties have been using SiteToSiteClient, but I don't think many of them re-implement SiteToSiteClient actual implementation, rather they just use the library. When they use it just to use SiteToSite protocol, the URL is what they configured by themselves. So it should be fine. It doesn't break anything.
For the case if a third party actually re-implementing a SiteToSiteClient to customize it at lower level, excuse me, the comment you concerned about was misleading. The interface hasn't been changed at all. What has changed by this PR is the behavior of one implementation of SiteToSiteClient, that is the NiFi default implementation, i.e. SocketClient and HttpClient. I didn't have to change the comment in the first place. It doesn't break any other SiteToSiteClient implementations written by third parties outside of NiFi codebase.
So, to address your concern, I just removed the comment from SiteToSiteClient.java. By doing so, the file becomes identical with the previous version. So I amended the commit to keep it intact.

@ijokarumawak it was not the comment that concerns me. What concerns me is that we have a method named getUrl() (present in both SiteToSiteClientConfig and the SiteToSiteClient.Builder), and that method returns a String that is no longer a valid URL. Given that the method name is getUrl(), it is very reasonable to expect someone to write code like the following:
final URL url = new URL(clientConfig.getUrl());
And in fact we did something very similar in the client. This will now throw an Exception because getUrl() may not be returning a valid URL. There are many reasons that may be done - not just by someone implementing a client. Also, we can't assume that "the URL is what they configured by themselves." Certainly, code can be written to pass this object around. There's no telling where it came from.
We also must keep in mind that while Apache Flink, Storm and Spark integration may be safe, we have no idea what other projects may be using the client.
So my recommendation would be this: have a method named getUrls() that returns a comma-separated list of URL's, just like getUrl() does now in this PR. Then, add a getUrl() to maintain backward compatibility but make that method return the first URL available and mark it as deprecated with documentation as to why it's deprecated and why we now recommend use of getUrls() instead.
Does that seem reasonable?

@markap14 I'm still not 100% sure if it addresses your concern.. but I've updated the code.
RemoteProcessGroup has getTargetUri method that returns a String. I wondered if we should do the same thing with this, by adding getTargetUris, but I left it as it is. Because if we do, we may want to change RemoteProcessGroupDTO as well, it will change NiFi Web REST API data format..
Local contrib check has passed, and now I'm testing it with running NiFi environments. It will take a while to go through a variety of S2S tests. But I think it's ready for code review.
Would you take a look and give me feedback?

@ijokarumawak thanks for the update. The new commit perfectly addresses the concern that I had with the Builder. I do think we should update the RemoteProcessGroup & DTO as well. The DTO could still have a getTargetUri() that would return the first value entered, just as the builder does now, and then also have a getTargetUris() that returns all of them. Along those lines, it may make sense to not event deprecated the getUri() method and simply document that it returns the first one entered by the user and the getUris() method returns all of them. The DTO may also require having a setTargetUri() and setTargetUris(). I'm just very hesitant to return a comma-separated list of URI's when we have a method named getUri(), etc. This would also have the added benefit of keeping the UI clean, showing a single URI if we want to. @mcgilman do you have any thoughts on how the RPG DTO should change (or not)?

I think what @markap14 outlined above would be fine. It would be consistent with the other changes in this PR. We can certainly address the UX in a subsequent JIRA however, we cannot go back if we start accepting a comma separate list in the target URI field of the RPG DTO.
Another option which I'm also fine with would be if we supported specifying either a single target URI or a listing. This would support backward compatibility while offering support for multiple going forward.

@markap14 @mcgilman Thank you for the comments. I will update the PR by adding getTargetUrist o make DTO and API consistent with other changes.

Hi @markap14 @mcgilman
I've updated the PR commit with the latest master.
Explicitly separated the URI and URIs.
I've also updated these RPG UI components to use the plural URLs.
These display Name and URIs. It looked too verbose if URLs are used for both Name and URIs, so I used URI for the Name if Name is not specified:

Tried to update all UI components that I was aware of, but I didn't want to make this PR huge and rush to change following components, so that we can discuss more about the details with separated JIRAs.
Known UI components those are not updated to use URLs
Search component
RPGs can be searched by target URI. I haven't updated search related components to use plural URIs. So only the first URI is searchable, and shown in the UI right now.

Component Status and Summary

Action History
When RPG is updated, audit event is stored. I think we should use URIs here. But in order to do so, we have to alter the REMOTE_PROCESS_GROUP_DETAILS table in database repository. I don't know what would be a proper migration process at this point, so didn't touch it within this PR.

Hope this PR would be sufficient to provide the core support of multiple URIs for better connectivity with NiFi cluster via Site-to-Site. Thanks for reviewing!

The last commit addresses Component Search to use URIs so that user can search RPG with any URI defined.

I took a closer look on the remaining UI components, Component Status, Summary, and Action History described in the previous comment. I think there won't be much benefits by using URIs with these component despite of the amount of lines of code required to be changed.
So, I left these as it is.
I have finished changing code and testing.
@markap14 Please review and give any feedback. Thank you!

@ijokarumawak Thanks for the updates! All looks good to me now. I've tested and everything seems to work well. Reviewed code and things look good. +1 merged to master!


If I use it before a public php function it won't work

e.g. public function instead of function.

You'll need to be more detailed. It works for me.

when you type /** before a function, it works for me only half the time.

Show the snippet of code that doesn't work. Also some information like OS, version, etc, which you can get from the sublime text log (Menu > View > Show Console).
More detailed information can be extracted from the log by enabling command and input logging. open the console: sublime.log_commands(True); sublime.log_input(True);.


request: language support for inverse relations

The ask
SpiceDB language could have native support to denote the inverse of a relationship
Example:
definition repository {
  relation owner: User
}

definition user {
  inverse relation repositories: repository#owner
}

The goal

At very large scale, writing 2 tuples instead of one would incur additional COGS for the underlying database system
It removes the burden from the client-side to maintain those two relationships. With large and complex schemas, it could be difficult to keep track of all the relationships that need to be written when a new entity is persisted in the application side. As complexity increases, there could be higher chances to have inconsistent data persisted in SpiceDB

Adding a +1 on this issue. We found ourselves needing to add inverse relationships for most of our use-cases.

+1


"Could not set timer: Invalid argument (22)" when trying to run laravel docker image with non-root

What happened?
I am trying to serve a laravel statamic application via the non-root version of the docker container
When accessing the admin panel I get the following error

Build Type
Docker (Debian Bookworm)
Worker Mode
No
Operating System
GNU/Linux
CPU Architecture
Apple Silicon
PHP configuration
phpinfo()
PHP Version => 8.3.4

System => Linux bd0174d41d00 6.5.0-15-generic #15-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan  9 22:39:36 UTC 2024 aarch64
Build Date => Mar 16 2024 00:20:58
Build System => Linux - Docker
Build Provider => https://github.com/docker-library/php
Configure Command =>  './configure'  '--build=aarch64-linux-gnu' '--with-config-file-path=/usr/local/etc/php' '--with-config-file-scan-dir=/usr/local/etc/php/conf.d' '--enable-option-checking=fatal' '--with-mhash' '--with-pic' '--enable-mbstring' '--enable-mysqlnd' '--with-password-argon2' '--with-sodium=shared' '--with-pdo-sqlite=/usr' '--with-sqlite3=/usr' '--with-curl' '--with-iconv' '--with-openssl' '--with-readline' '--with-zlib' '--enable-phpdbg' '--enable-phpdbg-readline' '--with-pear' '--with-libdir=lib/aarch64-linux-gnu' '--enable-embed' '--enable-zts' '--disable-zend-signals' 'build_alias=aarch64-linux-gnu'
Server API => Command Line Interface
Virtual Directory Support => enabled
Configuration File (php.ini) Path => /usr/local/etc/php
Loaded Configuration File => (none)
Scan this dir for additional .ini files => /usr/local/etc/php/conf.d
Additional .ini files parsed => /usr/local/etc/php/conf.d/docker-php-ext-gd.ini,
/usr/local/etc/php/conf.d/docker-php-ext-intl.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pdo_mysql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sodium.ini,
/usr/local/etc/php/conf.d/docker-php-ext-zip.ini,
/usr/local/etc/php/conf.d/php.ini

PHP API => 20230831
PHP Extension => 20230831
Zend Extension => 420230831
Zend Extension Build => API420230831,TS
PHP Extension Build => API20230831,TS
Debug Build => no
Thread Safety => enabled
Thread API => POSIX Threads
Zend Signal Handling => disabled
Zend Memory Manager => enabled
Zend Multibyte Support => provided by mbstring
Zend Max Execution Timers => enabled
IPv6 Support => enabled
DTrace Support => disabled

Registered PHP Streams => https, ftps, compress.zlib, php, file, glob, data, http, ftp, phar, zip
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, tls, tlsv1.0, tlsv1.1, tlsv1.2, tlsv1.3
Registered Stream Filters => zlib.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, convert.*, consumed, dechunk

This program makes use of the Zend Scripting Language Engine:
Zend Engine v4.3.4, Copyright (c) Zend Technologies

 _______________________________________________________________________

Configuration

Core

PHP Version => 8.3.4

Directive => Local Value => Master Value
allow_url_fopen => On => On
allow_url_include => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => UTF-8 => UTF-8
default_mimetype => text/html => text/html
disable_classes => no value => no value
disable_functions => no value => no value
display_errors => STDOUT => STDOUT
display_startup_errors => On => On
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => On => On
enable_post_data_reading => On => On
error_append_string => no value => no value
error_log => no value => no value
error_log_mode => 0644 => 0644
error_prepend_string => no value => no value
error_reporting => no value => no value
expose_php => On => On
extension_dir => /usr/local/lib/php/extensions/no-debug-zts-20230831 => /usr/local/lib/php/extensions/no-debug-zts-20230831
fiber.stack_size => no value => no value
file_uploads => On => On
hard_timeout => 2 => 2
highlight.comment => #FF8000 => #FF8000
highlight.default => #0000BB => #0000BB
highlight.html => #000000 => #000000
highlight.keyword => #007700 => #007700
highlight.string => #DD0000 => #DD0000
html_errors => Off => Off
ignore_repeated_errors => Off => Off
ignore_repeated_source => Off => Off
ignore_user_abort => Off => Off
implicit_flush => On => On
include_path => .:/usr/local/lib/php => .:/usr/local/lib/php
input_encoding => no value => no value
internal_encoding => no value => no value
log_errors => Off => Off
mail.add_x_header => Off => Off
mail.force_extra_parameters => no value => no value
mail.log => no value => no value
mail.mixed_lf_and_crlf => Off => Off
max_execution_time => 0 => 0
max_file_uploads => 20 => 20
max_input_nesting_level => 64 => 64
max_input_time => -1 => -1
max_input_vars => 1000 => 1000
max_multipart_body_parts => -1 => -1
memory_limit => 512M => 512M
open_basedir => no value => no value
output_buffering => 0 => 0
output_encoding => no value => no value
output_handler => no value => no value
post_max_size => 100M => 100M
precision => 14 => 14
realpath_cache_size => 4096K => 4096K
realpath_cache_ttl => 120 => 120
register_argc_argv => On => On
report_memleaks => On => On
report_zend_debug => Off => Off
request_order => no value => no value
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i => /usr/sbin/sendmail -t -i
serialize_precision => -1 => -1
short_open_tag => On => On
SMTP => localhost => localhost
smtp_port => 25 => 25
sys_temp_dir => no value => no value
syslog.facility => LOG_USER => LOG_USER
syslog.filter => no-ctrl => no-ctrl
syslog.ident => php => php
unserialize_callback_func => no value => no value
upload_max_filesize => 100M => 100M
upload_tmp_dir => no value => no value
user_dir => no value => no value
user_ini.cache_ttl => 300 => 300
user_ini.filename => .user.ini => .user.ini
variables_order => EGPCS => EGPCS
xmlrpc_error_number => 0 => 0
xmlrpc_errors => Off => Off
zend.assertions => 1 => 1
zend.detect_unicode => On => On
zend.enable_gc => On => On
zend.exception_ignore_args => Off => Off
zend.exception_string_param_max_len => 15 => 15
zend.max_allowed_stack_size => 0 => 0
zend.multibyte => Off => Off
zend.reserved_stack_size => 0 => 0
zend.script_encoding => no value => no value

ctype

ctype functions => enabled

curl

cURL support => enabled
cURL Information => 7.88.1
Age => 10
Features
AsynchDNS => Yes
CharConv => No
Debug => No
GSS-Negotiate => No
IDN => Yes
IPv6 => Yes
krb4 => No
Largefile => Yes
libz => Yes
NTLM => Yes
NTLMWB => Yes
SPNEGO => Yes
SSL => Yes
SSPI => No
TLS-SRP => Yes
HTTP2 => Yes
GSSAPI => Yes
KERBEROS5 => Yes
UNIX_SOCKETS => Yes
PSL => Yes
HTTPS_PROXY => Yes
MULTI_SSL => No
BROTLI => Yes
ALTSVC => Yes
HTTP3 => No
UNICODE => No
ZSTD => Yes
HSTS => Yes
GSASL => No
Protocols => dict, file, ftp, ftps, gopher, gophers, http, https, imap, imaps, ldap, ldaps, mqtt, pop3, pop3s, rtmp, rtmpe, rtmps, rtmpt, rtmpte, rtmpts, rtsp, scp, sftp, smb, smbs, smtp, smtps, telnet, tftp
Host => aarch64-unknown-linux-gnu
SSL Version => OpenSSL/3.0.11
ZLib Version => 1.2.13
libSSH Version => libssh2/1.10.0

Directive => Local Value => Master Value
curl.cainfo => no value => no value

date

date/time support => enabled
timelib version => 2022.10
"Olson" Timezone Database Version => 2024.1
Timezone Database => internal
Default timezone => UTC

Directive => Local Value => Master Value
date.default_latitude => 31.7667 => 31.7667
date.default_longitude => 35.2333 => 35.2333
date.sunrise_zenith => 90.833333 => 90.833333
date.sunset_zenith => 90.833333 => 90.833333
date.timezone => UTC => UTC

dom

DOM/XML => enabled
DOM/XML API Version => 20031129
libxml Version => 2.9.14
HTML Support => enabled
XPath Support => enabled
XPointer Support => enabled
Schema Support => enabled
RelaxNG Support => enabled

fileinfo

fileinfo support => enabled
libmagic => 543

filter

Input Validation and Filtering => enabled

Directive => Local Value => Master Value
filter.default => unsafe_raw => unsafe_raw
filter.default_flags => no value => no value

gd

GD Support => enabled
GD Version => bundled (2.1.0 compatible)
FreeType Support => enabled
FreeType Linkage => with freetype
FreeType Version => 2.12.1
GIF Read Support => enabled
GIF Create Support => enabled
JPEG Support => enabled
libJPEG Version => 6b
PNG Support => enabled
libPNG Version => 1.6.39
WBMP Support => enabled
XPM Support => enabled
libXpm Version => 30411
XBM Support => enabled
WebP Support => enabled
BMP Support => enabled
AVIF Support => enabled
TGA Read Support => enabled

Directive => Local Value => Master Value
gd.jpeg_ignore_warning => On => On

hash

hash support => enabled
Hashing Engines => md2 md4 md5 sha1 sha224 sha256 sha384 sha512/224 sha512/256 sha512 sha3-224 sha3-256 sha3-384 sha3-512 ripemd128 ripemd160 ripemd256 ripemd320 whirlpool tiger128,3 tiger160,3 tiger192,3 tiger128,4 tiger160,4 tiger192,4 snefru snefru256 gost gost-crypto adler32 crc32 crc32b crc32c fnv132 fnv1a32 fnv164 fnv1a64 joaat murmur3a murmur3c murmur3f xxh32 xxh64 xxh3 xxh128 haval128,3 haval160,3 haval192,3 haval224,3 haval256,3 haval128,4 haval160,4 haval192,4 haval224,4 haval256,4 haval128,5 haval160,5 haval192,5 haval224,5 haval256,5

MHASH support => Enabled
MHASH API Version => Emulated Support

iconv

iconv support => enabled
iconv implementation => glibc
iconv library version => 2.36

Directive => Local Value => Master Value
iconv.input_encoding => no value => no value
iconv.internal_encoding => no value => no value
iconv.output_encoding => no value => no value

intl

Internationalization support => enabled
ICU version => 72.1
ICU Data version => 72.1
ICU TZData version => 2022e
ICU Unicode version => 15.0

Directive => Local Value => Master Value
intl.default_locale => no value => no value
intl.error_level => 0 => 0
intl.use_exceptions => Off => Off

json

json support => enabled

libxml

libXML support => active
libXML Compiled Version => 2.9.14
libXML Loaded Version => 20914
libXML streams => enabled

mbstring

Multibyte Support => enabled
Multibyte string engine => libmbfl
HTTP input encoding translation => disabled
libmbfl version => 1.3.2

mbstring extension makes use of "streamable kanji code filter and converter", which is distributed under the GNU Lesser General Public License version 2.1.

Multibyte (japanese) regex support => enabled
Multibyte regex (oniguruma) version => 6.9.8

Directive => Local Value => Master Value
mbstring.detect_order => no value => no value
mbstring.encoding_translation => Off => Off
mbstring.http_input => no value => no value
mbstring.http_output => no value => no value
mbstring.http_output_conv_mimetypes => ^(text/|application/xhtml\+xml) => ^(text/|application/xhtml\+xml)
mbstring.internal_encoding => no value => no value
mbstring.language => neutral => neutral
mbstring.regex_retry_limit => 1000000 => 1000000
mbstring.regex_stack_limit => 100000 => 100000
mbstring.strict_detection => Off => Off
mbstring.substitute_character => no value => no value

mysqlnd

mysqlnd => enabled
Version => mysqlnd 8.3.4
Compression => supported
core SSL => supported
extended SSL => supported
Command buffer size => 4096
Read buffer size => 32768
Read timeout => 86400
Collecting statistics => Yes
Collecting memory statistics => No
Tracing => n/a
Loaded plugins => mysqlnd,debug_trace,auth_plugin_mysql_native_password,auth_plugin_mysql_clear_password,auth_plugin_caching_sha2_password,auth_plugin_sha256_password
API Extensions => pdo_mysql

openssl

OpenSSL support => enabled
OpenSSL Library Version => OpenSSL 3.0.11 19 Sep 2023
OpenSSL Header Version => OpenSSL 3.0.11 19 Sep 2023
Openssl default config => /usr/lib/ssl/openssl.cnf

Directive => Local Value => Master Value
openssl.cafile => no value => no value
openssl.capath => no value => no value

pcre

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.42 2022-12-12
PCRE Unicode Version => 14.0.0
PCRE JIT Support => enabled
PCRE JIT Target => ARM-64 64bit (little endian + unaligned)

Directive => Local Value => Master Value
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => On => On
pcre.recursion_limit => 100000 => 100000

PDO

PDO support => enabled
PDO drivers => sqlite, mysql

pdo_mysql

PDO Driver for MySQL => enabled
Client API version => mysqlnd 8.3.4

Directive => Local Value => Master Value
pdo_mysql.default_socket => no value => no value

pdo_sqlite

PDO Driver for SQLite 3.x => enabled
SQLite Library => 3.40.1

Phar

Phar: PHP Archive support => enabled
Phar API version => 1.1.1
Phar-based phar archives => enabled
Tar-based phar archives => enabled
ZIP-based phar archives => enabled
gzip compression => enabled
bzip2 compression => disabled (install ext/bz2)
Native OpenSSL support => enabled

Phar based on pear/PHP_Archive, original concept by Davey Shafik.
Phar fully realized by Gregory Beaver and Marcus Boerger.
Portions of tar implementation Copyright (c) 2003-2009 Tim Kientzle.
Directive => Local Value => Master Value
phar.cache_list => no value => no value
phar.readonly => On => On
phar.require_hash => On => On

posix

POSIX support => enabled

random

Version => 8.3.4

readline

Readline Support => enabled
Readline library => 8.2

Directive => Local Value => Master Value
cli.pager => no value => no value
cli.prompt => \b \>  => \b \>

Reflection

Reflection => enabled

session

Session Support => enabled
Registered save handlers => files user
Registered serializer handlers => php_serialize php php_binary

Directive => Local Value => Master Value
session.auto_start => Off => Off
session.cache_expire => 180 => 180
session.cache_limiter => nocache => nocache
session.cookie_domain => no value => no value
session.cookie_httponly => Off => Off
session.cookie_lifetime => 0 => 0
session.cookie_path => / => /
session.cookie_samesite => no value => no value
session.cookie_secure => Off => Off
session.gc_divisor => 100 => 100
session.gc_maxlifetime => 1440 => 1440
session.gc_probability => 1 => 1
session.lazy_write => On => On
session.name => PHPSESSID => PHPSESSID
session.referer_check => no value => no value
session.save_handler => files => files
session.save_path => no value => no value
session.serialize_handler => php => php
session.sid_bits_per_character => 4 => 4
session.sid_length => 32 => 32
session.upload_progress.cleanup => On => On
session.upload_progress.enabled => On => On
session.upload_progress.freq => 1% => 1%
session.upload_progress.min_freq => 1 => 1
session.upload_progress.name => PHP_SESSION_UPLOAD_PROGRESS => PHP_SESSION_UPLOAD_PROGRESS
session.upload_progress.prefix => upload_progress_ => upload_progress_
session.use_cookies => On => On
session.use_only_cookies => On => On
session.use_strict_mode => Off => Off
session.use_trans_sid => Off => Off

SimpleXML

SimpleXML support => enabled
Schema support => enabled

sodium

sodium support => enabled
libsodium headers version => 1.0.18
libsodium library version => 1.0.18

SPL

SPL support => enabled
Interfaces => OuterIterator, RecursiveIterator, SeekableIterator, SplObserver, SplSubject
Classes => AppendIterator, ArrayIterator, ArrayObject, BadFunctionCallException, BadMethodCallException, CachingIterator, CallbackFilterIterator, DirectoryIterator, DomainException, EmptyIterator, FilesystemIterator, FilterIterator, GlobIterator, InfiniteIterator, InvalidArgumentException, IteratorIterator, LengthException, LimitIterator, LogicException, MultipleIterator, NoRewindIterator, OutOfBoundsException, OutOfRangeException, OverflowException, ParentIterator, RangeException, RecursiveArrayIterator, RecursiveCachingIterator, RecursiveCallbackFilterIterator, RecursiveDirectoryIterator, RecursiveFilterIterator, RecursiveIteratorIterator, RecursiveRegexIterator, RecursiveTreeIterator, RegexIterator, RuntimeException, SplDoublyLinkedList, SplFileInfo, SplFileObject, SplFixedArray, SplHeap, SplMinHeap, SplMaxHeap, SplObjectStorage, SplPriorityQueue, SplQueue, SplStack, SplTempFileObject, UnderflowException, UnexpectedValueException

sqlite3

SQLite3 support => enabled
SQLite Library => 3.40.1

Directive => Local Value => Master Value
sqlite3.defensive => On => On
sqlite3.extension_dir => no value => no value

standard

Dynamic Library Support => enabled
Path to sendmail => /usr/sbin/sendmail -t -i

Directive => Local Value => Master Value
assert.active => On => On
assert.bail => Off => Off
assert.callback => no value => no value
assert.exception => On => On
assert.warning => On => On
auto_detect_line_endings => Off => Off
default_socket_timeout => 60 => 60
from => no value => no value
session.trans_sid_hosts => no value => no value
session.trans_sid_tags => a=href,area=href,frame=src,form= => a=href,area=href,frame=src,form=
unserialize_max_depth => 4096 => 4096
url_rewriter.hosts => no value => no value
url_rewriter.tags => form= => form=
user_agent => no value => no value

tokenizer

Tokenizer Support => enabled

xml

XML Support => active
XML Namespace Support => active
libxml2 Version => 2.9.14

xmlreader

XMLReader => enabled

xmlwriter

XMLWriter => enabled

zip

Zip => enabled
Zip version => 1.22.3
Libzip version => 1.7.3
BZIP2 compression => Yes
XZ compression => No
ZSTD compression => No
AES-128 encryption => Yes
AES-192 encryption => Yes
AES-256 encryption => Yes

zlib

ZLib Support => enabled
Stream Wrapper => compress.zlib://
Stream Filter => zlib.inflate, zlib.deflate
Compiled Version => 1.2.13
Linked Version => 1.2.13

Directive => Local Value => Master Value
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

Additional Modules

Module Name

Environment

Variable => Value
PUID => 1000
HOSTNAME => bd0174d41d00
PHP_VERSION => 8.3.4
PHP_INI_DIR => /usr/local/etc/php
GPG_KEYS => 1198C0117593497A5EC5C199286AF1F9897469DC C28D937575603EB4ABB725861C0779DC5C0A9DE4 AFD8691FDAEDF03BDF6E460563F15A9B715376CA
XDG_DATA_HOME => /data
GODEBUG => cgocheck=0
XDG_CONFIG_HOME => /config
PHP_LDFLAGS => -Wl,-O1 -pie
PWD => /app
PORT => 81
TZ => UTC
HOME => /var/www
PGID => 1000
PHP_SHA256 => 39a337036a546e5c28aea76cf424ac172db5156bd8a8fd85252e389409a5ba63
PHPIZE_DEPS => autoconf 		dpkg-dev 		file 		g++ 		gcc 	ibc-dev 		make 		pkg-config 		re2c
NVM_DIR => /usr/local/nvm
TERM => xterm
PHP_URL => https://www.php.net/distributions/php-8.3.4.tar.xz
SHLVL => 1
PHP_CFLAGS => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
NODE_PATH => /usr/local/nvm/v20.11.1/lib/node_modules
PATH => /usr/local/nvm/versions/node/v20.11.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NODE_VERSION => 20.11.1
DEBIAN_FRONTEND => noninteractive
PHP_ASC_URL => https://www.php.net/distributions/php-8.3.4.tar.xz.asc
PHP_CPPFLAGS => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
_ => /usr/local/bin/php

PHP Variables

Variable => Value
$_SERVER['PUID'] => 1000
$_SERVER['HOSTNAME'] => bd0174d41d00
$_SERVER['PHP_VERSION'] => 8.3.4
$_SERVER['PHP_INI_DIR'] => /usr/local/etc/php
$_SERVER['GPG_KEYS'] => 1198C0117593497A5EC5C199286AF1F9897469DC C28D937575603EB4ABB725861C0779DC5C0A9DE4 AFD8691FDAEDF03BDF6E460563F15A9B715376CA
$_SERVER['XDG_DATA_HOME'] => /data
$_SERVER['GODEBUG'] => cgocheck=0
$_SERVER['XDG_CONFIG_HOME'] => /config
$_SERVER['PHP_LDFLAGS'] => -Wl,-O1 -pie
$_SERVER['PWD'] => /app
$_SERVER['PORT'] => 81
$_SERVER['TZ'] => UTC
$_SERVER['HOME'] => /var/www
$_SERVER['PGID'] => 1000
$_SERVER['PHP_SHA256'] => 39a337036a546e5c28aea76cf424ac172db5156bd8a8fd85252e389409a5ba63
$_SERVER['PHPIZE_DEPS'] => autoconf 		dpkg-dev 		file 		g++ 		cc 		libc-dev 		make 		pkg-config 		re2c
$_SERVER['NVM_DIR'] => /usr/local/nvm
$_SERVER['TERM'] => xterm
$_SERVER['PHP_URL'] => https://www.php.net/distributions/php-8.3.4.tar.xz
$_SERVER['SHLVL'] => 1
$_SERVER['PHP_CFLAGS'] => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
$_SERVER['NODE_PATH'] => /usr/local/nvm/v20.11.1/lib/node_modules
$_SERVER['PATH'] => /usr/local/nvm/versions/node/v20.11.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
$_SERVER['NODE_VERSION'] => 20.11.1
$_SERVER['DEBIAN_FRONTEND'] => noninteractive
$_SERVER['PHP_ASC_URL'] => https://www.php.net/distributions/php-8.3.4.tar.xz.asc
$_SERVER['PHP_CPPFLAGS'] => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
$_SERVER['_'] => /usr/local/bin/php
$_SERVER['PHP_SELF'] => public/debug.php
$_SERVER['SCRIPT_NAME'] => public/debug.php
$_SERVER['SCRIPT_FILENAME'] => public/debug.php
$_SERVER['PATH_TRANSLATED'] => public/debug.php
$_SERVER['DOCUMENT_ROOT'] =>
$_SERVER['REQUEST_TIME_FLOAT'] =><PHONE_NUMBER>.0402
$_SERVER['REQUEST_TIME'] =><PHONE_NUMBER>
$_SERVER['argv'] => Array
(
    [0] => public/debug.php
)

$_SERVER['argc'] => 1
$_ENV['PUID'] => 1000
$_ENV['HOSTNAME'] => bd0174d41d00
$_ENV['PHP_VERSION'] => 8.3.4
$_ENV['PHP_INI_DIR'] => /usr/local/etc/php
$_ENV['GPG_KEYS'] => 1198C0117593497A5EC5C199286AF1F9897469DC C28D937575603EB4ABB725861C0779DC5C0A9DE4 AFD8691FDAEDF03BDF6E460563F15A9B715376CA
$_ENV['XDG_DATA_HOME'] => /data
$_ENV['GODEBUG'] => cgocheck=0
$_ENV['XDG_CONFIG_HOME'] => /config
$_ENV['PHP_LDFLAGS'] => -Wl,-O1 -pie
$_ENV['PWD'] => /app
$_ENV['PORT'] => 81
$_ENV['TZ'] => UTC
$_ENV['HOME'] => /var/www
$_ENV['PGID'] => 1000
$_ENV['PHP_SHA256'] => 39a337036a546e5c28aea76cf424ac172db5156bd8a8fd85252e389409a5ba63
$_ENV['PHPIZE_DEPS'] => autoconf 		dpkg-dev 		file 		g++ 		cc 		libc-dev 		make 		pkg-config 		re2c
$_ENV['NVM_DIR'] => /usr/local/nvm
$_ENV['TERM'] => xterm
$_ENV['PHP_URL'] => https://www.php.net/distributions/php-8.3.4.tar.xz
$_ENV['SHLVL'] => 1
$_ENV['PHP_CFLAGS'] => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
$_ENV['NODE_PATH'] => /usr/local/nvm/v20.11.1/lib/node_modules
$_ENV['PATH'] => /usr/local/nvm/versions/node/v20.11.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
$_ENV['NODE_VERSION'] => 20.11.1
$_ENV['DEBIAN_FRONTEND'] => noninteractive
$_ENV['PHP_ASC_URL'] => https://www.php.net/distributions/php-8.3.4.tar.xz.asc
$_ENV['PHP_CPPFLAGS'] => -fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
$_ENV['_'] => /usr/local/bin/php

 _______________________________________________________________________

PHP Credits

PHP Group
Thies C. Arntzen, Stig Bakken, Shane Caraveo, Andi Gutmans, Rasmus Lerdorf, Sam Ruby, Sascha Schumann, Zeev Suraski, Jim Winstead, Andrei Zmievski

Language Design & Concept
Andi Gutmans, Rasmus Lerdorf, Zeev Suraski, Marcus Boerger

                               PHP Authors
Contribution => Authors
Zend Scripting Language Engine => Andi Gutmans, Zeev Suraski, Stanislav Malyshev, Marcus Boerger, Dmitry Stogov, Xinchen Hui, Nikita Popov
Extension Module API => Andi Gutmans, Zeev Suraski, Andrei Zmievski
UNIX Build and Modularization => Stig Bakken, Sascha Schumann, Jani Taskinen, Peter Kokot
Windows Support => Shane Caraveo, Zeev Suraski, Wez Furlong, Pierre-Alain Joye, Anatol Belski, Kalle Sommer Nielsen
Server API (SAPI) Abstraction Layer => Andi Gutmans, Shane Caraveo, Zeev Suraski
Streams Abstraction Layer => Wez Furlong, Sara Golemon
PHP Data Objects Layer => Wez Furlong, Marcus Boerger, Sterling Hughes, George Schlossnagle, Ilia Alshanetsky
Output Handler => Zeev Suraski, Thies C. Arntzen, Marcus Boerger, Michael Wallner
Consistent 64 bit support => Anthony Ferrara, Anatol Belski

                               SAPI Modules
Contribution => Authors
Apache 2.0 Handler => Ian Holsman, Justin Erenkrantz (based on Apache 2.0 Filter code)
CGI / FastCGI => Rasmus Lerdorf, Stig Bakken, Shane Caraveo, Dmitry Stogov
CLI => Edin Kadribasic, Marcus Boerger, Johannes Schlueter, Moriyoshi Koizumi, Xinchen Hui
Embed => Edin Kadribasic
FastCGI Process Manager => Andrei Nigmatulin, dreamcat4, Antony Dovgal, Jerome Loyet
litespeed => George Wang
phpdbg => Felipe Pena, Joe Watkins, Bob Weinand

                              Module Authors
Module => Authors
BC Math => Andi Gutmans
Bzip2 => Sterling Hughes
Calendar => Shane Caraveo, Colin Viebrock, Hartmut Holzgraefe, Wez Furlong
COM and .Net => Wez Furlong
ctype => Hartmut Holzgraefe
cURL => Sterling Hughes
Date/Time Support => Derick Rethans
DB-LIB (MS SQL, Sybase) => Wez Furlong, Frank M. Kromann, Adam Baratz
DBA => Sascha Schumann, Marcus Boerger
DOM => Christian Stocker, Rob Richards, Marcus Boerger
enchant => Pierre-Alain Joye, Ilia Alshanetsky
EXIF => Rasmus Lerdorf, Marcus Boerger
FFI => Dmitry Stogov
fileinfo => Ilia Alshanetsky, Pierre Alain Joye, Scott MacVicar, Derick Rethans, Anatol Belski
Firebird driver for PDO => Ard Biesheuvel
FTP => Stefan Esser, Andrew Skalski
GD imaging => Rasmus Lerdorf, Stig Bakken, Jim Winstead, Jouni Ahto, Ilia Alshanetsky, Pierre-Alain Joye, Marcus Boerger, Mark Randall
GetText => Alex Plotnick
GNU GMP support => Stanislav Malyshev
Iconv => Rui Hirokawa, Stig Bakken, Moriyoshi Koizumi
IMAP => Rex Logan, Mark Musone, Brian Wang, Kaj-Michael Lang, Antoni Pamies Olive, Rasmus Lerdorf, Andrew Skalski, Chuck Hagenbuch, Daniel R Kalowsky
Input Filter => Rasmus Lerdorf, Derick Rethans, Pierre-Alain Joye, Ilia Alshanetsky
Internationalization => Ed Batutis, Vladimir Iordanov, Dmitry Lakhtyuk, Stanislav Malyshev, Vadim Savchuk, Kirti Velankar
JSON => Jakub Zelenka, Omar Kilani, Scott MacVicar
LDAP => Amitay Isaacs, Eric Warnke, Rasmus Lerdorf, Gerrit Thomson, Stig Venaas
LIBXML => Christian Stocker, Rob Richards, Marcus Boerger, Wez Furlong, Shane Caraveo
Multibyte String Functions => Tsukada Takuya, Rui Hirokawa
MySQL driver for PDO => George Schlossnagle, Wez Furlong, Ilia Alshanetsky, Johannes Schlueter
MySQLi => Zak Greant, Georg Richter, Andrey Hristov, Ulf Wendel
MySQLnd => Andrey Hristov, Ulf Wendel, Georg Richter, Johannes Schlüter
OCI8 => Stig Bakken, Thies C. Arntzen, Andy Sautins, David Benson, Maxim Maletsky, Harald Radi, Antony Dovgal, Andi Gutmans, Wez Furlong, Christopher Jones, Oracle Corporation
ODBC driver for PDO => Wez Furlong
ODBC => Stig Bakken, Andreas Karajannis, Frank M. Kromann, Daniel R. Kalowsky
Opcache => Andi Gutmans, Zeev Suraski, Stanislav Malyshev, Dmitry Stogov, Xinchen Hui
OpenSSL => Stig Venaas, Wez Furlong, Sascha Kettler, Scott MacVicar, Eliot Lear
Oracle (OCI) driver for PDO => Wez Furlong
pcntl => Jason Greene, Arnaud Le Blanc
Perl Compatible Regexps => Andrei Zmievski
PHP Archive => Gregory Beaver, Marcus Boerger
PHP Data Objects => Wez Furlong, Marcus Boerger, Sterling Hughes, George Schlossnagle, Ilia Alshanetsky
PHP hash => Sara Golemon, Rasmus Lerdorf, Stefan Esser, Michael Wallner, Scott MacVicar
Posix => Kristian Koehntopp
PostgreSQL driver for PDO => Edin Kadribasic, Ilia Alshanetsky
PostgreSQL => Jouni Ahto, Zeev Suraski, Yasuo Ohgaki, Chris Kings-Lynne
Pspell => Vlad Krupin
random => Go Kudo, Tim Düsterhus, Guilliam Xavier, Christoph M. Becker, Jakub Zelenka, Bob Weinand, Máté Kocsis, and Original RNG implementators
Readline => Thies C. Arntzen
Reflection => Marcus Boerger, Timm Friebe, George Schlossnagle, Andrei Zmievski, Johannes Schlueter
Sessions => Sascha Schumann, Andrei Zmievski
Shared Memory Operations => Slava Poliakov, Ilia Alshanetsky
SimpleXML => Sterling Hughes, Marcus Boerger, Rob Richards
SNMP => Rasmus Lerdorf, Harrie Hazewinkel, Mike Jackson, Steven Lawrance, Johann Hanne, Boris Lytochkin
SOAP => Brad Lafountain, Shane Caraveo, Dmitry Stogov
Sockets => Chris Vandomelen, Sterling Hughes, Daniel Beulshausen, Jason Greene
Sodium => Frank Denis
SPL => Marcus Boerger, Etienne Kneuss
SQLite 3.x driver for PDO => Wez Furlong
SQLite3 => Scott MacVicar, Ilia Alshanetsky, Brad Dewar
System V Message based IPC => Wez Furlong
System V Semaphores => Tom May
System V Shared Memory => Christian Cartus
tidy => John Coggeshall, Ilia Alshanetsky
tokenizer => Andrei Zmievski, Johannes Schlueter
XML => Stig Bakken, Thies C. Arntzen, Sterling Hughes
XMLReader => Rob Richards
XMLWriter => Rob Richards, Pierre-Alain Joye
XSL => Christian Stocker, Rob Richards
Zip => Pierre-Alain Joye, Remi Collet
Zlib => Rasmus Lerdorf, Stefan Roehrich, Zeev Suraski, Jade Nicoletti, Michael Wallner

                            PHP Documentation
Authors => Mehdi Achour, Friedhelm Betz, Antony Dovgal, Nuno Lopes, Hannes Magnusson, Philip Olson, Georg Richter, Damien Seguy, Jakub Vrana, Adam Harvey
Editor => Peter Cowburn
User Note Maintainers => Daniel P. Brown, Thiago Henrique Pojda
Other Contributors => Previously active authors, editors and other contributors are listed in the manual.

PHP Quality Assurance Team
Ilia Alshanetsky, Joerg Behrens, Antony Dovgal, Stefan Esser, Moriyoshi Koizumi, Magnus Maatta, Sebastian Nohn, Derick Rethans, Melvyn Sopacua, Pierre-Alain Joye, Dmitry Stogov, Felipe Pena, David Soria Parra, Stanislav Malyshev, Julien Pauli, Stephen Zarkos, Anatol Belski, Remi Collet, Ferenc Kovacs

                     Websites and Infrastructure team
PHP Websites Team => Rasmus Lerdorf, Hannes Magnusson, Philip Olson, Lukas Kahwe Smith, Pierre-Alain Joye, Kalle Sommer Nielsen, Peter Cowburn, Adam Harvey, Ferenc Kovacs, Levi Morrison
Event Maintainers => Damien Seguy, Daniel P. Brown
Network Infrastructure => Daniel P. Brown
Windows Infrastructure => Alex Schoenmaker

PHP License
This program is free software; you can redistribute it and/or modify
it under the terms of the PHP License as published by the PHP Group
and included in the distribution in the file:  LICENSE

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

If you did not receive a copy of the PHP license, or have any
questions about PHP licensing, please contact<EMAIL_ADDRESS>

### Relevant log output

```shell
no log output only the html output as shown in the image above.

This is the relevant section of my dockerfile:
# taken from https://frankenphp.dev/docs/docker/
FROM dunglas/frankenphp

LABEL maintainer="Rainer Burgstaller"

ARG USER=www-data
ARG WWWGROUP=www-data

RUN \
	# Use "adduser -D ${USER}" for alpine based distros
	useradd -D ${USER}; \
	# Add additional capability to bind to port 80 and 443
	setcap CAP_NET_BIND_SERVICE=+eip /usr/local/bin/frankenphp; \
	# Give write access to /data/caddy and /config/caddy
	chown -R ${USER}:${USER} /data/caddy && chown -R ${USER}:${USER} /config/caddy;

WORKDIR /app

ENV DEBIAN_FRONTEND noninteractive
ENV TZ=UTC

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN apt-get update && apt-get install -y  gnupg

RUN install-php-extensions gd pdo_mysql zip intl

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN chown -R $USER:$WWWGROUP /app

EXPOSE 80

RUN mkdir /.composer && \
    chmod -R ugo+rw /.composer && \
    chown -R $USER /.composer

# Switch to use a non-root user from here on
USER ${USER}

# Add application
WORKDIR /app

COPY --chown=${USER}:$WWWGROUP backend /app

RUN cd /app && \
    mkdir -p /app/storage/logs

RUN composer install --no-interaction --no-dev --prefer-dist --optimize-autoloader

This is the relevant section of the docker-compose.yml
services:
  fp:
    image: me/image
    ports:
      # laravel
      - '8002:443'
    environment:
      PORT: 81
      PUID: '${UID:-1000}'
      PGID: '${GID:-1000}'
    user: '33:33'
    networks:
      - app-network

Yes, I know running it with an external provided uid is uncommon, but I want to deploy on heroku and they run their images with arbitrary uids, so this is a first step to getting there.

My bet is that using timer_create() require some Linux capacities that are disabled in this rootless+Docker context.

Possibly CAP_WAKE_ALARM but we aren't using alarms, or an old kernel that doesn't have CLOCK_BOOTTIME available. It'd be worth trying to run as root, @rburgst, to see if that fixes it. If so, then it is a capability issue, if not, can you share the kernel version of your docker host: uname -r

$ uname -r
6.5.0-15-generic

when I run the container without the user: 33:33 directive, then the problem stays the same.

What are you passing to set_time_limit()? Is it a valid argument according to https://www.php.net/manual/en/function.set-time-limit.php?

I am not passing anything specific in there (as this is statatmic code), but when I debug, I see that -1 is being passed in.
When I run this locally on my mac this also is the same value and it doesnt cause a problem there (which doesnt say much). However, this code also works fine on a apache-fpm container as root.
@withinboredom I tried adding
	setcap CAP_WAKE_ALARM=+eip /usr/local/bin/frankenphp; \

to the dockerfile but that caused the container to stop immediately (note that I dont know how to properly use setcap, i just did copy paste from the existing setcap call).

-1 is in fact an invalid number to pass to timer_create and is also an invalid value to pass to set_time_limit(), which it appears to silently fail on in non-zts PHP (or is perhaps treated as 0).
Either way, -1 is undefined behavior, so I'm not sure what the proper fix is. I'll go open an issue on the statatmic repo for now, as a fix in PHP is likely to take much longer.

Looks like it is already merged: https://github.com/statamic/cms/pull/9297
Are your dependencies up to date?

Todo:

[ ] research how other SAPIs treat negative numbers
[ ] ideally, negative numbers should be an error (needs a PHP RFC to codify, earliest change 8.4 or 8.5)
[ ] implement changes

https://externals.io/message/123108 to discuss proper behavior

Closing in favor of externals.io/message/123108 and https://github.com/php/php-src/pull/13942 as it's a PHP bug, not related to FrankenPHP.

I am using
        {
            "name": "statamic/cms",
            "version": "v4.53.2",
            ```
which is approx 1 month old, the referenced fix was merged in January, so I would have thought that this should have been fixed already.
At any rate I did actually get it to work with a naked sample installation of statamic: https://github.com/rburgst/frankenphp-statamic

So now I only need to figure out what else is wrong in our setup.


SQLite3 adapter corrupts BLOB data on insert

It looks like the SQLite3 adapter does some encoding tricks when inserting BLOB values.
Here's a test script (change the requires to test with MRI and JRuby):
#require 'sqlite3' # uncomment to test with MRI
require 'activerecord-jdbcsqlite3-adapter' # uncomment to test with JRuby
require 'active_record'

ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: '/tmp/test.sqlite3')

connection = ActiveRecord::Base.connection

connection.execute("CREATE TABLE test (data BLOB)")

data_bytes = [207]
data_str = data_bytes.pack('c*')

connection.execute("INSERT INTO test (data) VALUES (#{connection.quote(data_str)})")
result_str = connection.select_value('SELECT data FROM test')
result_bytes = result_str.bytes

puts("Expected #{data_bytes.inspect}, got #{result_bytes.inspect}")

Output with JRuby <IP_ADDRESS>, activerecord 4.2.5 and activerecord-jdbc-adapter 1.3.19:
Expected [207], got [239, 191, 189]

Output with JRuby <IP_ADDRESS>, activerecord 4.2.5 and activerecord-jdbc-adapter 1.4.0.dev:
Expected [207], got [195, 143]

Output with MRI 2.2.3, activerecord 4.2.5 and sqlite3 1.3.11:
Expected [207], got [207]

After running the test script and then checking the data in the DB with
sqlite3 /tmp/test.sqlite3 'SELECT * FROM test' | od -t x1

shows that the data is corrupted on insert.

Well looks I went a bit astray in trying to minimize the reproduction. In the above case, the quote method does the right thing when given a binary column.
However, here's a case involving marshaling:
#require 'sqlite3' # uncomment to test with MRI
require 'activerecord-jdbcsqlite3-adapter' # uncomment to test with JRuby
require 'active_record'
require 'date'

ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: '/tmp/test.sqlite3')
ActiveRecord::Base.connection.execute("CREATE TABLE tests (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, data BLOB NOT NULL)")

module Marshalled
  def self.dump(value)
    Marshal.dump(value)
  end

  def self.load(data)
    Marshal.load(StringIO.new(data)) if data
  end
end

class Test < ActiveRecord::Base
  serialize :data, Marshalled
end

Test.create!(data: Date.parse('2001-02-03'))
p Test.first.data

This fails with “ArgumentError: dump format error” on JRuby with activerecord-jdbc-adapter 1.3.19 and 1.4.0.dev but works with MRI.

do you have a stack or back trace (preferably against <IP_ADDRESS>) ... seems like this might be a JRuby bug.

This is with <IP_ADDRESS> since I seem to be having some trouble with <IP_ADDRESS>.
ArgumentError: dump format error(ï)
                     load at org/jruby/RubyMarshal.java:145
                     load at /tmp/issue696.rb:15
  type_cast_from_database at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/type/serialized.rb:19
                type_cast at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute.rb:104
           original_value at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute.rb:42
                    value at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute.rb:37
              fetch_value at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute_set.rb:31
          _read_attribute at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute_methods/read.rb:93
                     data at $HOME/.rbenv/versions/jruby-<IP_ADDRESS>/lib/ruby/gems/shared/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute_methods.rb:50
                    <top> at /tmp/issue696.rb:24

The backtrace is identical on <IP_ADDRESS>.

recall there has been a marshalling issue somewhere along 9.0
have verified this works by now (also previous master - 1.4.0.dev is dead at this point - no resources there)


Confusing error message when Managed HSM name is wrong

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az keyvault role assignment list
Errors:
<urllib3.connection.VerifiedHTTPSConnection object at 0x0549A1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az keyvault role assignment list --hsm-name {}

Expected Behavior
When the HSM with the provided name does not exist, the above cryptic error message is shown, which is not very useful. 'getaddrinfo' failed means the DNS name <hsm_name>. could not be found. Therefore the error message should say "The DNS name '<hsm_name>.' was not found." instead of the above error message.
Environment Summary
Windows-10-10.0.19041-SP0
Python 3.6.8
Installer: MSI

azure-cli 2.15.1 *

Additional Context

hi @houk-ms could you pls  help to have a look? thanks

hi @houk-ms could you pls  help to have a look? thanks

let me investigate it.

let me investigate it.


Improve support for non-compliant DICOM and edge cases

I was playing around with the pydicom test files and noticed that parsing (with SkipProcessingPixelDataValue() option enabled) the following files result in errors:

ExplVR_BigEndNoMeta.dcm
ExplVR_LitEndNoMeta.dcm
MR_truncated.dcm
UN_sequence.dcm
meta_missing_tsyntax.dcm
nested_priv_SQ.dcm
no_meta.dcm
no_meta_group_length.dcm

These files were parseable by dcm4che.
For most of them, the issue is related to missing some or all meta tags.
Ideally, if the meta tags aren't included, we should ignore and try parsing the first data element. And we can guess the transfer syntax by peeking the first tag and VR of the first element. We try [implicit LE, explicit BE, explicit LE], whichever one decodes a known tag and VR is likely the correct transfer syntax and we should use that for the rest of the dataset.

Thanks @lnogueir! I agree that we can attempt some enhanced fallback behavior, particularly if that's standard practice in other parsers. Our current fallback behavior for no transfer syntax found in the Metadata is to proceed with little endian explicit but it shouldn't be too difficult to try a couple different transfer syntaxes speculatively.

For missing metadata group length we do hav a flag for that, but can consider automatic fallback behavior: https://github.com/suyashkumar/dicom/blob/65259e55c2dd96a811e752aff2864ef16b903351/read.go#L184
I think originally the idea was the safest option is to force callers to be as explicit as possible about what loosened restrictions they may want in case there are any safety issues with fallback behavior or in case the fact the dicom is not compliant is something the user would want raised to them (in case that led to other concerns). That being said, recently I've been thinking it's more reasonable to just have more automatic fallbacks, at least ones that are "standard" in the industry for dicom non-compliance.

Going through these in a little more detail, and adding some more notes:

Easy fix (have working locally)
Easy fix (have working locally)
This appears to parse fine, but iiuc the problem is the PixelData is intentionally truncated? So the vl says 8192 but there aren't that many bytes left in the dicom. We treat this as an error, which seems reasonable imo but we can discuss more.
Bug/implementation needed with UN unknown sequences
Needs more investigation, seems related to the UN sequences / tags possibly similar to 4.
Also seems possible related to UN sequences / tag handling
Same as 4/5/6 at first glance?
Should parse fine with allowMissingMetaElementGroupLength but need to check

The draft changes in #330 address 1 and 2.

For missing metadata group length we do hav a flag for that, but can consider automatic fallback behavior:
https://github.com/suyashkumar/dicom/blob/65259e55c2dd96a811e752aff2864ef16b903351/read.go#L184
I think originally the idea was the safest option is to force callers to be as explicit as possible about what loosened restrictions they may want in case there are any safety issues with fallback behavior or in case the fact the dicom is not compliant is something the user would want raised to them (in case that led to other concerns). That being said, recently I've been thinking it's more reasonable to just have more automatic fallbacks, at least ones that are "standard" in the industry for dicom non-compliance.

I agree. I think it's better to have these automatic fallbacks and just log warnings if non-compliant things are found. That's the behavior most libraries choose I believe.

Also, the changes in #331 address 6. They almost address 4 but some other changes are still needed for that.

Edited the original comment to update current status, link to PRs, etc.

I noticed that the following files (also from pydicom) do not error, but drop values:

rtdose_rle.dcm
rtdose_rle_1frame.dcm (seems to be the same issue as the file above)

The issue seems to be that they change the transfer syntax when writing the SQ dataset. The top level dataset is encoded with explicit little endian, but the dataset of the Referenced RT Plan Sequence is encoded with implicit little endian.
These were the pydicom issues that handled that: https://github.com/pydicom/pydicom/issues/1035, https://github.com/pydicom/pydicom/issues/1067


what is startup project for?

still failing to understand why we are creating a startup console project when we are not even going to use it? the steps described create migration folder in Database project rather than startup project, and we are not adding any code to startup project as well.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 439f9dd7-6fff-f3af-15a3-c4f407e2c041
Version Independent ID: 2779bb58-f0af-c9ee-cf40-50e24371a7fc
Content: Getting Started on UWP - New Database - EF Core
Content Source: entity-framework/core/get-started/uwp/getting-started.md
Product: entity-framework
GitHub Login: @rowanmiller
Microsoft Alias: divega

It's answer is  literally  at the beginning , in bold and marked !Important .
Don't skip while reading.

I know on a technical level that startup is supposed to run the commands, I was just confused as there was no code whatsoever in the project, but I guess it doesnt need any code to run migrations like commands. Thanks for the help anyways.


Add more test coverage

Connection with issue(s)

Close #???

Connected to #???
Solution description
Screenshots or Videos

To Do

[ ] Read contributing guide
[ ] Check the original issue to confirm it is fully satisfied
[ ] Add solution description to help guide reviewers
[ ] Add unit test to verify new or fixed behaviour
[ ] If apply, add documentation to code properties and package readme

Yes it still makes sense because they use the validate methods and other users can extend them.

Yes it still makes sense because they use the validate methods and other users can extend them.

Yeah, but maybe can create a CoreValidator without translatedErrorText and BaseValidator extends this? I don't if this could be work.
Ignore a thing that won't be used, I don't see like a good practice

@deandreamatias I've implemented this now. Can you check and merge?


Adds Squash Stacks RFC

Summary

Readable
Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.
[ ] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[ ] I have added an integration test, if necessary.
[x] I have reviewed the styleguide for guidance on my code quality.
[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

💯 This will be be fantastic!

I was curious how many layers this would save (i.e. how many layers are in each current stack) - it seems like each stack currently contains ~7 layers. So, while it won't permanently solve our problems regarding builders with too many layers, it'll create some wiggle room for people creating custom builders that add to the Paketo Full builders.

I was curious how many layers this would save (i.e. how many layers are in each current stack) - it seems like each stack currently contains ~7 layers. So, while it won't permanently solve our problems regarding builders with too many layers, it'll create some wiggle room for people creating custom builders that add to the Paketo Full builders.

Hmm, interesting. I think I misunderstood this RFC. Is this do just squash the run/build image layers? or is it to squash down the builder layers? or both?
I agree, squashing the run/build images isn't going to save a ton, but squashing the builder layers, that would solve the layer problem for us.

Perhaps a discussion of squashing builders should be added here. Right now, we create builders with pack builder create, which doesn't have a --squash option. A potentially more reusable implementation here is to add a jam squash-image command that flattens an arbitrary image. This could be used for both builders and stacks.

Something to bear in mind with potentially squashing builders: If builders squash all buildpacks into a single layer, whenever any buildpack in the builder changes, the entire cached layer will be invalidated. This will significantly hurt builder-pull performance on any platform that uses on OCI layer caching. Maybe it would be more prudent to squash composite buildpacks before distributing them in builders.

Something to bear in mind with potentially squashing builders: If builders squash all buildpacks into a single layer, whenever any buildpack in the builder changes, the entire cached layer will be invalidated. This will significantly hurt builder-pull performance on any platform that uses on OCI layer caching. Maybe it would be more prudent to squash composite buildpacks before distributing them in builders.

That's a fair point. I think it would be interesting to try both and see how it goes. I'd be curious to see how big the layer ends up being with all the buildpacks squashed into one layer. There are quite a few individual layers now with the builder image and that's not exactly speedy. Sometimes downloading one larger resource is faster than downloading many smaller resources.

All in all, if this RFC introduces a more general jam squash-image subcommand and sets the intent to (at least) squash stacks, we can experiment and decide on a best builder-squashing approach after implementation. I'm all for experimenting on this.

I don't think we'd likely want a builder squashed into a single layer. More likely, we'd want to keep the base stack layer, and then squash each of the buildpack groups into a layer. It feels like it'll be harder to do that from the outside.

WG 11-15-2022: [RM] think about a more intelligent way to squash layers
[FGJ] What about generalizing this, and implementing it as a CLI command?
[RM] API design would be needed

@paketo-buildpacks/stacks-maintainers I'm looking for any more feedback on this.

I'm fine with proceeding with this as-is. We can extend support further for builders, etc in the future if we want to.


Plugin label disabled

Question
Hello, thank you for your work.
I add the plugin as described by the readme. However, when I load the cloudcompare.exe, the plugin label is disabled(grey), and I can not load the dae and gltf file.
Screenshots

Additional context
Add any other context about the problem here.
Your Environment

CloudCompare version (2.11.3):
OS & Version: windows vs 2015
Qt Version: 5.7.0

Are there any messages  in the console at the bottom? It should say whether the plugin was found and loaded, and if the extensions were properly registered.
Does the plugin show up in the "About Plugins" window? (Not sure where that menu item is on Windows.)
If it does not, make sure that the plugin DLL is in one of the paths listed at the bottom of that window.

Thank you for your reply. I checked the console and found that the software failed to load the MeshIO.dll and says that
"MeshIO.dll does not seem to be a valid plugin".

And the files in the plugin folder are like this:

Do I need to add other related dll files into this folder to make the MESHIO.dll valid?

Oh! Yes you'll need those.
I didn't realize Windows was building assimp as a DLL. I'll take a look at the CMake to see why.

Yes, I try to copy the three dll file into the same folder with the cloudcompare.exe and it works

I just noticed that the dates on your DLLs are not consistent and the file size of MeshIO.dll is different between the build directory and what you have installed.
Did you first try to build a recent version of CloudCompare/MeshIO and then back off to the 2.11.x version? pugixml is in a recent version of assimp, but not the older one.
Could you please remove the whole build directory and rebuild from scratch to see what you get?

you are using the MeshIO commit tagged "cloudcompare-2.11.x", right?

Yes

I just noticed that the dates on your DLLs are not consistent and the file size of MeshIO.dll is different between the build directory and what you have installed.

Actually, the difference between the size of MeshIO.dll is that one is from release mode and another is from releasewithdebuginfo, sorry for the confusion.

Did you first try to build a recent version of CloudCompare/MeshIO and then back off to the 2.11.x version? pugixml is in a recent version of assimp, but not the older one.

no

I'm also not seeing how you end up with a DLL for zlib or where pugixml is coming from.

For convenience, the assimp, zlib and pugixml are all build from vcpkg.

set( ASSIMP_BUILD_ZLIB TRUE CACHE INTERNAL "override ASSIMP flags" FORCE )
I have tried this and rebuild the exe, still need to copy the three dll files

For convenience, the assimp, zlib and pugixml are all build from vcpkg.

I don't understand. So you've modified MeshIO's CMake files to work with externally-built libraries?

yes, I make a little adjustment to the meshio cmake files to work with vcpkg. Maybe it is due to the adjustment? Just curious, under your assumption, is only one dll file(meshio.dll) generated in the build folder？

在 2023年1月3日，下午10:44，Andy Maloney @.***> 写道：

For convenience, the assimp, zlib and pugixml are all build from vcpkg.
I don't understand. So you've modified MeshIO's CMake files to work with externally-built libraries?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

🤦  That's kind of critical information that would have saved me a couple of hours of digging through CMake files.
If you have modified it to use other libraries for some reason, then I have no idea what it's doing. It's designed to use only the parts it needs from a specific version of assimp which is compiled into the MeshIO plugin.
And yes it's supposed to be one DLL - I mentioned my surprise about assimp as a DLL back here.


Prophet Model Deployment ERROR

Description:
I am trying to deploy Prophet Model.
Model is logged as mlflow.prophet flavour.
When I am creating image from Model Artifact this error came :
WebserviceException: WebserviceException:
"error": {
"message": "Image creation polling reached non-successful terminal state, current state: Failed\nError response from server:\nStatusCode: 400\nMessage: Docker image build failed."
}
}
Code to reproduce issue
import azureml
import mlflow.azureml
from azureml.core import Workspace
from azureml.core.authentication import ServicePrincipalAuthentication
from azureml.core.webservice import AciWebservice, Webservice

principal_auth = ServicePrincipalAuthentication(tenant_id, principal_id, app_secret)
workspace = Workspace.get(name=workspace_name, subscription_id=subscription_id, auth=principal_auth, resource_group=workspace_rg)
model_image, azureml_model = mlflow.azureml.build_image(model_uri='runs:/016fc3f2d5c34c8eadbeb20ad16d3/082358timeseriesforecasting', workspace=workspace)
model_image.wait_for_creation(show_output=True)

Other info / logs
In Container logs error is shown below:
[0mThe command '/bin/sh -c CONDA_ROOT_DIR=$(conda info --root) && if [ -n "$AZUREML_CONDA_ENVIRONMENT_PATH" ]; then conda env update -p "$AZUREML_CONDA_ENVIRONMENT_PATH" -f '/var/azureml-app/conda.yaml'; else conda env update -n base -f '/var/azureml-app/conda.yaml'; fi && conda clean -aqy && rm -rf /root/.cache/pip && rm -rf "$CONDA_ROOT_DIR/pkgs" && find "$CONDA_ROOT_DIR" -type d -name pycache -exec rm -rf {} +' returned a non-zero code: 137
2021/11/23 12:21:39 Container failed during run: acb_step_0. No retries remaining.
failed to run step ID: acb_step_0: exit status 137
Run ID: cj58 failed after 6m19s. Error: failed during run, err: exit status 1
System information
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Azure Distributed cluster
MLflow installed from (source or binary): source
MLflow version (run mlflow --version): 1.12.0
Python version: 3.7.3

@NumtraAdmin This repository's issues forum deals with azure/aci-deploy github action only. If you are using this action inside github workflow and facing this issue please do share your workflow here.

Hi @NumtraAdmin - As we see no response, we are closing this issue.
Please feel free to reopen the issue in case you have anything related with.
Thanks.


[Service Bus] ATOM API: Do not override status code of the get response for a non-existing entity

For a non-existing entity, service returns a response with 200 status code for the get request, we override the status code to 404 and throw a new RestError with MessagingEntityNotFoundErr as the error code.
TODO: Retain the logic to parse the response, not override the status code but keep it as is.

So, we would still be throwing the MessagingEntityNotFoundError, but keep the status code same as what the service returns?
I believe we had changed this to 404 based on prior discussions with @bterlson when we last touched this piece of code.
Perhaps git blame will find the right PR and the right discussion?

So, we would still be throwing the MessagingEntityNotFoundError, but keep the status code same as what the service returns?

Yes, that also makes us align with .NET.

git blame

https://github.com/Azure/azure-sdk-for-js/pull/6054#discussion_r346647767
https://github.com/Azure/azure-sdk-for-js/pull/6586#discussion_r360254693

Done with #10231


[SILOpt] Added flag to disable destroy hoisting.

It is on by default.  The default changes with the value of the CopyPropagation setting.  But it can still be overridden.

@swift-ci please test and merge

It even caught a bug!  The = was missing at the end of "enable-destroy-hoisting" in
def enable_destroy_hoisting :
    Joined<["-"], "enable-destroy-hoisting=">,
    HelpText<"Whether to enable destroy hoisting">,
    MetaVarName<"true|false">;


Pick a few grammars in autoscaling.md

A few changes about grammar for improvement.

@windsonsea  Thank you for putting it in notice. The Changes looks good to me
/lgtm


Feature Request, Expire members of a set

Hi,
One of the much requested features, to expire individual members of a set, is unlikely to get implemented in Redis, hoping KeyDb can take to improve it
https://github.com/antirez/redis/issues/135

Its good timing since I’m already looking at the expire code heavily.
What do you think of triggering a lua script on expire?  That way custom behavior can be added for any type as needed.
The one hesitation I have is if you have this ability for sets - then what about the other set-like data types?

I would foresee other types like hashes too. I'm not a great fan of calling another script. My use case is quite simple, without expire on individual keys in a set, I'm forced to put them in global space, and there will be collisions.
Perhaps I would think it's an extenion method, like EXPIREMEMBER

The trouble is that internally the values are not strongly typed. So often KeyDB/Redis doesn’t know if its a set/ziplist/etc. You can even craft values that are valid as multiple types.
It would have to be EXPIREMEMBER [type] [key] [subkey].
It is definitely doable though.

A prototype of EXPIREMEMBER is now in the subexpire branch: https://github.com/JohnSully/KeyDB/tree/subexpire
The command is: EXPIREMEMBER [key] [subkey] [seconds]
This is still a prototype so please don't try this in production - but I think this feature will work very well.  I'd love to hear any suggestions you have for improving this command.   Right now only sets are supported but this is a temporary limitation.
I will continue to update this issue as the feature makes it way to our stable release.

Looks good, thanks!

This has been merged into the unstable branch in prep for its v5.1 release.  The unstable docker updates at 4am EST each night if you would like to try it via docker.

Hello @JohnSully ,
I am very happy to have found this fork which support member expiration. I've tested it and it works very well.
However there is a feature I'd like to see: have the possibility to know/get the deleted value on the "notify-keyspace-events" notification system. Currently, we can only get the key of the set by parsing the channel.
To be honest, I've just started digging into Redis/KeyDB and I don't know much about it, so I don't even know if it's possible to add this feature without rewriting the whole project.
So, do you think that's possible? And if yes, would you agree to implement it? (Or maybe it's already possible?)
Thanks in advance.

Thanks @vfrz this is an oversite.  I'll add a new subexpire keyspace event so you can track them.

@vfrz I moved your feature request to a new issue #85.  Because I'm hoping to release v5.1 on Monday there won't be time to do it as part of that release.


Mouse roller and OS 3.2, level selector.

On UAE with OS 3.2, mouse roller moves the scrollbar of the level selector window, when the window is active. However I do not get any IDCMP for this, so the list is not scrolled, which looks weird.

It turned out that OS 3.2 converts mouse roller events to series of IDCMP_GADGETDOWN events. Then fixing issue #24 also fixed this one (in commit a3a38a0).


Is there any train shell file?

Could you show me how to execute train or test, is there any shell to start training?

By the way, a requirement.txt is required.


CornerstoneImageLoadProgress not firing anymore

This is related to cornerstone #135, the quick fix is replacing
$(cornerstone).trigger('CornerstoneImageLoadProgress', {

with this:
$(cornerstone.events).trigger('CornerstoneImageLoadProgress', {

Can anyone apply the fix?.

Where is this happening? It looks fine on master to me?
https://github.com/chafey/cornerstoneWADOImageLoader/search?utf8=✓&q=CornerstoneImageLoadProgress&type=

Mmm, I was using 0.14.3, I'm upgrading.


Feature request: Cloudflare Worker support

Hi,
Are there any plans to make Vento compatible with Cloudflare Workers? I tried bundling the npm package (using esbuild), but ran into the dreaded Dynamic require of "node:fs" is not supported errors (et al.).
I know Vento is specifically a Deno project, but it would be awesome for my projects.

Vento has an NPM distribution. Did you try it?

Yes, but that results in errors like Dynamic require of "node:fs" is not supported. Cloudflare Workers is a bit like node, but not enough :|

Do you know in which package/file where this error occurs?
I don't see any node:fs in Vento code (npm version). Maybe it's caused by a dependency.

Yup, the dependency is @deno/shim-deno. The Cloudflare worker environment has no access to the filesystem, so node:fs is not available.

If Cloudflare has no access to file system, you have to replace the default loader with other with support for Cloudflare API.
The default loader is this. You can do something like this:
class CloudflareLoader {
  async load(file) {
    return {
      source: "File content here"
    }
  }

  resolve(from, file) {
    return "Resolved file";
  }
}

const env = vento({
  includes: new CloudflareLoader()
});

Creating a Cloudflare workerd loader is the way to do this, so closing this issue ✌


Use "short ID" for the course slug

As a course author, I'd like to be able to reliably control the URL my course will be published under, and include information that might not be part of the course title.
Currently, the course slug is based off the course title.
It would be better and more reliable & predictable to make the course slug exactly the same as the value put into short ID. This would require that we make some changes to the help text for short ID.

This is not what we want to do.


Undefined behaviour in ssl_session_calculate_timeout() in 3.1/3.0

The function ssl_session_calculate_timeout adds a time to a timeout value but does not check for possible overflow in 3.1/3.0. Therefore setting a very large timeout value using SSL_SESSION_set_timeout can cause undefined behaviour.
This is not the case in 3.2/3.3/master where we do the addition safely.

Given that openssl 3.1 and 3.0 will be supported until sometime in 2026, we should consider fixing this, but its perhaps not a high priority unless someone is encountering this bug


README Improvements

Various README Improvements

That's very nice of you to improve the readme file if you need info or anything just ask for it

@ccgauche matter of fact I wanted to ask if it was cool if I added my AUR package to the install section as explicitly a 3rd party package.

@ccgauche https://github.com/Rustmilian/ytermusic/tree/master#YTerMusic

@ccgauche It's in a state I'm happy with now. If there's any changes you want feel free to say so.

@ccgauche matter of fact I wanted to ask if it was cool if I added my AUR package to the install section as explicitly a 3rd party package.
Also are there any more colors supported other then Blue, White, Green, Black, Red, & Yellow?

I'm ok with 3rd party distributions if they are cited as 3rd parties.
Yes we can technically use all RGB colors. I'm working on theming the app to make it more customizable.
Apart from that I'm pretty happy with the changes.

Yes we can technically use all RGB colors

Good to know. I figured how too, I'll make some changes.
Also, I just want so that the download speed is way faster now. ❣

@ccgauche I think we're near the final product, give it one last look through and tell me whatcha think! 🥰
one last change needed is adding paths for windows, then it can be merged.

If you wanna add paths you can or you can link to the directories doc with some info about which file is in which directories folder

If you wanna add paths you can or you can link to the directories doc with some info about which file is in which directories folder

This one? https://docs.rs/directories/latest/directories/struct.ProjectDirs.html#method.config_dir

@ccgauche If that was the right place to link, it's ready for merge now.


Remove confluent schema registry dependency

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Any movement on this PR? I am really looking forward to it.

Hi @tmnd1991 !
Thanks for this contribution. We are going to close this PR until further notice. This is a great start obviously but there's much more work to do to fully remove the registry.
We're still very much interested in removing the registry, so feel free to reopen if you've got time to work on it again!
John


LightOccluder2D hides normal map

https://godotengine.org/qa/14215/is-it-possible-to-have-a-normal-mapped-sprite-cast-shadows
Create a 2D light and a 2D sprite, attach a normal map. Add a light occluder 2D to the sprite to create shadows. The normal map is hidden.
Godot 2.1 - Ubuntu 16.04

Have you tried different cull modes on the occluder?

Wow you're amazing! Never heard of cull mode, sorry to bother. Clockwise worked.


updated the badges

updated the badges now that Alex has transferred the GitHub repo.

Coverage remained the same at 100.0% when pulling 687dde620e4fe683ce7f25c83c8dd9307b5f7259 on update-badges into 1663eb778f9854135b42779d898b5f5bed5be51a on master.


OCM-10477 | test: fix id:75210

ocm-10477
$ ginkgo --focus 75210 tests/e2e/
Ginkgo detected a version mismatch between the Ginkgo CLI and the version of Ginkgo imported by your packages:
Ginkgo CLI Version:
2.11.0
Mismatched package versions found:
2.17.1 used by e2e
Ginkgo will continue to attempt to run but you may see errors (including flag
parsing errors) and should either update your go.mod or your version of the
Ginkgo CLI to match.
To install the matching version of the CLI run
go install github.com/onsi/ginkgo/v2/ginkgo
from a path that contains a go.mod file.  Alternatively you can use
go run github.com/onsi/ginkgo/v2/ginkgo
from a path that contains a go.mod file to invoke the matching version of the
Ginkgo CLI.
If you are attempting to test multiple packages that each have a different
version of the Ginkgo library with a single Ginkgo CLI that is currently
unsupported.
Running Suite: ROSA CLI e2e tests suite - /home/akanni/OCP-Repository/rosa/tests/e2e
Random Seed:<PHONE_NUMBER>
Will run 1 of 179 specs
SSSSSSSSS•SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
Ran 1 of 179 Specs in 538.786 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 178 Skipped
PASS
Ginkgo ran 1 suite in 10m46.024667257s
Test Suite Passed

/retest

/retest

/retest

/lgtm

/retest

/override e2e-presubmits-pr-rosa-hcp-advanced  e2e-presubmits-pr-rosa-sts-advanced

/override ci/prow/e2e-presubmits-pr-rosa-hcp-advanced ci/prow/e2e-presubmits-pr-rosa-sts-advanced


Preparation or improvement of the guide for usage of list generators of module classes (issue #888 related)

The goal is to prepare or improve a guide or report that is generated by the following code:
from dynpy.utilities.documents.document import BasicsOfReportComponentImplementationGuide
BasicsOfReportComponentImplementationGuide()

Help was added to the ModuleStructure method was added and expanded


fix priority queue and set tree.lb

Implementation of correct priority queue. Storing the whole node in the priority queue doesn't work.
Therefore there is now a pq and nodes the former is the priority queue which stores the lower bound of a node and its node id and nodes itself is a dict with key node_id and value node.
The priority gets correctly updated and the tree.lb is set as the lowest lower bound of open nodes.
Also it breaks early if the incumbent is nearly the lower bound as specified by atol and rtol.
Closes: #21 #27

I should add a test case for callback back in but I think the last one doesn't make sense anymore.

I would rename pq to queue or node_queue


Update Chinese translations

Ref: https://github.com/newsboat/newsboat/pull/2766#issuecomment-2156020794
I update several unresolved items by the way. query feed is translated into 集合种子 which means aggregated feed and with original English by its side as a navigation to docs.

coverage: 59.814%. remained the same
when pulling 0f49e7d411d789e95076a2dd62860cb717c13744 on CookiePieWw:zh-trans
into bd55d67f453ce249bbc933fe5ae2b34ee5ea8f4a on newsboat:master.

coverage: 59.814%. remained the same
when pulling 0f49e7d411d789e95076a2dd62860cb717c13744 on CookiePieWw:zh-trans
into bd55d67f453ce249bbc933fe5ae2b34ee5ea8f4a on newsboat:master.

Excellent, than you very much!


How to check all *.md files under one folder_1 and also under folders under folder_1

Hi,
I use --verbose folder/*.md to let lychee check links in all md files, but I also need it to check other md files under other folders. Is this supported?
Thanks

Yeah, just use multiple arguments:
lychee --verbose 1/*.md 2/*.md

There is no limit on the maximum number of inputs. You can also mix inputs from different sources:
lychee --verbose 1/*.md 2/*.md https://endler.dev

This checks the markdown files in the folders 1 and 2 as well as all links on endler.dev. 😃

Yeah, just use multiple arguments:
lychee --verbose 1/*.md 2/*.md

There is no limit on the maximum number of inputs. You can also mix inputs from different sources:
lychee --verbose 1/*.md 2/*.md https://endler.dev

This checks the markdown files in the folders 1 and 2 as well as all links on endler.dev. 😃

Or use extended globs to check all Markdown files in all directories below the current one recursively:
lychee -v '**/*.md'

Note the required single quotes since lychee itself does the file matching here as extended globs (** to match directories recursively) are not supported by default in most shells.

Or use extended globs to check all Markdown files in all directories below the current one recursively:
lychee -v '**/*.md'

Note the required single quotes since lychee itself does the file matching here as extended globs (** to match directories recursively) are not supported by default in most shells.

@Wuliyuanulb can we close this or do you have any follow-up questions?

@Wuliyuanulb can we close this or do you have any follow-up questions?

I have a question on this one. How do I specify a root directory under which I want all the markdown files scanned for broken link?

lychee -v '/path/to/**/*.md'

or
cd /path/to
lychee -v '**/*.md'

with /path/to being the path to that root directory.

We use glob for that. According to the docs about patterns,

** matches the current directory and arbitrary subdirectories. This sequence must form a single path component, so both a and b are invalid and will result in an error. A sequence of more than two consecutive * characters is also invalid.

This also explains why **.md doesn't work: ** can only stand on its own.
Just checked if lychee was picking up the current directory when using a glob.
Created the following file tree:
❯❯❯ tree
.
├── 1
│   └── bar.md
├── 2
│   ├── 3
│   │   └── bla.md
│   └── baz.md
└── foo.md

3 directories, 4 files

Here's the glob test for ls:
ls **/*.md
1/bar.md   2/3/bla.md 2/baz.md   foo.md

And here's the lychee command:
❯❯❯ lychee --verbose --progress '**/*.md'
✅ http://foo.com/ [200 OK]
✅ http://bla.com/ [200 OK]
✅ http://baz.com/ [200 OK]
✅ http://bar.com/ [200 OK]

📝 Summary
-------------------
🔍 Total: 4
✅ Successful: 4
⏳ Timeouts: 0
🔀 Redirected: 0
👻 Excluded: 0
🚫 Errors: 0

So it did pick up http://foo.com. 😃 Also works without the quotes around **/*.md on zsh.

That should answer all questions and we can close this I guess. 😄

Also works without the quotes around **/*.md on zsh.

True, but if the shell did not support it or has ** support disabled (default in bash), the shell could theoretically expand * and lychee would then get a glob with * possibly pre-expanded, hence won't find all matches anymore. It's unlikely that there is a file path with literal **, but I like to be explicit when spamming shell commands into the world 😄.

Oh yeah, absolutely. I recommend to always quote it so that there are no differences with varying environments. Thanks for pointing that out. 👍


Add all Ollama options

Hi There!
Thanks for the great extension!
I was running into some issues with Chat models have too short of a context length when running them through Ollama. With some of the messages I was sending with big code snippets the AI wasn't remembering most of it.
I dived in to change the num_ctx setting and noticed that alot of the options you can tweak for Ollama was missing, so I thought it would be handy to add them all in.
The options are all defined here https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values, I just used their defaults and descriptions.
Adjusting the context I managed to greatly improve the output of llama3 as seen here: https://gist.github.com/JPyke3/3d9a87f138613cb00bd82a828f418fba vs the shorter context window: https://gist.github.com/JPyke3/24f995a900b968381ab6f21f3727e0dd
This is my config for the record with this change:
require("codecompanion").setup({
	adapters = {
		llama3 = require("codecompanion.adapters").use("ollama", {
			schema = {
				model = {
					default = "llama3",
				},
				num_ctx = {
					default = 16384
				},
				num_predict = {
					default = -1
				},
			},
		}),
		codegemma = require("codecompanion.adapters").use("ollama", {
			schema = {
				model = {
					default = "codegemma"
				},
			},
		}),
	},
	strategies = {
		chat = "llama3",
		inline = "codegemma",
		agent = "llama3",
	},
})

Hopefully the formatting/syntax is okay, I am still very much a lua rookie so I am still trying to catch my bearings.
Thanks so much for the extension! It has greatly improved my productivity 😄

Thank you so much! This is awesome.
I'm hoping to add cmp completion for model settings at some point this week to make it more intuitive to know what to change when in the chat buffer.
Btw, do you find all of the plugin functionality works with Ollama?

No probs at all!
That's a great idea, if you need me to flesh out descriptions for cmp or whatever you need feel free to give me a ping.
I have found all the functionality has worked really well! The default prompts have especially lent themselves well to llama3:latest (I wouldn't be able to write them better myself), I found gemma2:latest to waffle a bit too much and doesn't keep it relevant to the code. For inline code I am using codegemma:latest - Haven't fully been satisfied with that one yet but so far the best of the 2 i tested haha. Still playing around alot though.
I have found it effectively essential to set num_predict to -1 in order to ensure the AI can generate until it stops, if you're using your own PC so it shouldn't matter if the token count is high, it'll just churn until it's done.
If you have a wiki page or adapter best practice page i'd be happy to flesh out my testing and configs so that people can have a starting point for ollama. Happy to help out as I am heavily using the extension.

Good shout on llama3:latest. I've just tried it out. The cold starts are brutal but then she picks up speed. I've not been able to look past claude-3-5-sonnet. The speed and quality of response is amazing. In a chat buffer if you type @ you'll now get a completion menu for some additional helpers. Being able to send the current buffer to the LLM as context is a huge time saver.
I've just pushed a commit which outputs virtual text next to each of the settings in the chat buffer. Nice way of displaying all of your hard work in this PR.
I wondered if you'd be willing in the next week or two to PR some sensible Ollama defaults? I'd like to have a dedicated section in the README which informs users which models to download.

Shouldn't the default be to let ollama pick those values?

Shouldn't the defaults be nil so that the ollama modelfile parameters are used?
Also, the ollama modelfile readme is inaccurate! The default value for num_predict is actually -1:
https://github.com/ollama/ollama/blob/d0634b15961558fd5a06119ea0a08d3bd8511e43/api/types.go#L583

That's a really good point and you're absolutely right. We should nil them out for the defaults. Do you know if there is any way to pull the values from the modelfile via the CLI? would be neat to grab them to transparently how the user what the model is using.
Thanks for the heads up on num_predict I will fix that in a PR

Ah, seems like ollama show llama3 --parameters will show this:
num_keep                       24
stop                           "<|start_header_id|>"
stop                           "<|end_header_id|>"
stop                           "<|eot_id|>"

That's probably what I am after. Alright I will snap together a PR that reads from this first, and if there is no value present it will read from the defaults. That way the user knows exactly what the model is using/doing.


Make transformer for detrending support multivariate series

Is your feature request related to a problem? Please describe.
For the EEG applications we need a transformer that detrends each channel. Currently the detrending transformer only supports univariate time series.
Describe the solution you'd like
Detrend each univariate series separately

@SveaMeyer13, what do you think about the solution in #1038? That way, you can apply everything, not just the detrender, to multivariate EEG data.

@fkiraly, yes I like the idea in #1038 - for now, I just took inspiration by how it was done in the imputer.

well, we can always do both.
It's a bit of a sktime bad habit to ask for more general solutions that take more time, than go for the "fit for purpose" fix - sorry for doing that here.
I think the solution you proposed is robust, and definitely more suitable to get you quicker to your goal and less of a distraction than mine.


Can dmmf be passed from context?

I don't want to create PrismaClient while building schema. Luckily the prisma client can be passed to pothos from context.
but not the dmmf.
Without dmmf,
TypeError: Cannot read properties of undefined (reading 'modelMap')
    at getModel (file:///D:/project/node_modules/@pothos/plugin-prisma/esm/util/datamodel.js:30:28)
    at getDefaultIDSerializer (file:///D:/project/node_modules/@pothos/plugin-prisma/esm/util/cursors.js:78:19)
    at SchemaBuilder.prismaNode (file:///D:/project/node_modules/@pothos/plugin-prisma/esm/schema-builder.js:41:81)

What can I do in this case?
I want something like:
prisma: {
    client: context => context.prisma,
    // Because the prisma client is loaded dynamically, we need to explicitly provide the some information about the prisma schema
    dmmf: context => (context.prisma as unknown as { _baseDmmf: Prisma.DMMF.Document })._baseDmmf,
  },

No, this isn't possible.  The ddmf is what describes the Prisma schema.  This is what Pothos uses to understand when you do something like t.relation('author') that author is a User type.  The separation of dmmf from the client used was to allow client to be instantiated at run time, but you still need to understand the Prisma schema to build the graphql schema

I think there is a getDmmf method that Prisma added recently to a utils package that might help you get the dmmf without creating a cliebt instance

Made a small change to the prisma plugin so you can now pass the dmmf like this:
import { Prisma, PrismaClient } from '@prisma/client';

prisma: {
    client: context => context.prisma,
    dmmf: Prisma.dmmf,
  },

This avoids needing to instantiate the prisma client outside of context and is a lot cleaner

Made a small change to the prisma plugin so you can now pass the dmmf like this:
import { Prisma, PrismaClient } from '@prisma/client';

prisma: {
    client: context => context.prisma,
    dmmf: Prisma.dmmf,
  },

This avoids needing to instantiate the prisma client outside of context and is a lot cleaner

Cool, let me try.


Long link times with --incompatible_sandbox_hermetic_tmp

When using Bazel 7, --incompatible_sandbox_hermetic_tmp is flipped, leading to incredibly long link times.
I ran into this when bumping one of my projects to Bazel 7 where it caused the aarch64 cross-compilation job that normally takes 10-15 minutes to hit the 30-minute limit I've got set for it.
This can short-term be worked around with --noincompatible_sandbox_hermetic_tmp, but it feels like something that needs to be looked into somewhere.
robin@x:~/code/hastur$ bazel clean
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

robin@x:~/code/hastur$ time bazel test --config=linux-aarch64-musl util/unicode_test
INFO: Analyzed target //util:unicode_test (48 packages loaded, 5896 targets configured).
INFO: From Linking util/unicode_test:
In file included from external/zig_sdk/lib/libc/musl/crt/rcrt1.c:3:
external/zig_sdk/lib/libc/musl/crt/../ldso/dlstart.c:146:20: warning: a function declaration without a prototype is deprecated in all versions of C and is treated as a zero-parameter prototype in C2x, conflicting with a subsequent definition [-Wdeprecated-non-prototype]
        GETFUNCSYM(&dls2, __dls2, base+dyn[DT_PLTGOT]);
                          ^
external/zig_sdk/lib/libc/musl/crt/rcrt1.c:11:13: note: conflicting prototype is here
hidden void __dls2(unsigned char *base, size_t *sp)
            ^
1 warning generated.
INFO: Found 1 test target...
Target //util:unicode_test up-to-date:
  bazel-bin/util/unicode_test
INFO: Elapsed time: 25.911s, Critical Path: 25.37s
INFO: 10 processes: 3 internal, 7 linux-sandbox.
INFO: Build completed successfully, 10 total actions

Executed 1 out of 1 test: 1 test passes.

real    0m25.921s
user    0m0.018s
sys     0m0.000s

robin@x:~/code/hastur$ bazel clean
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

robin@x:~/code/hastur$ time bazel test --config=linux-aarch64-musl --noincompatible_sandbox_hermetic_tmp util/unicode_test
INFO: Analyzed target //util:unicode_test (48 packages loaded, 5896 targets configured).
INFO: Found 1 test target...
Target //util:unicode_test up-to-date:
  bazel-bin/util/unicode_test
INFO: Elapsed time: 2.562s, Critical Path: 2.04s
INFO: 10 processes: 3 internal, 7 linux-sandbox.
INFO: Build completed successfully, 10 total actions

Executed 1 out of 1 test: 1 test passes.

real    0m2.571s
user    0m0.010s
sys     0m0.000s

Thanks! Covered in #135, please review that and see if it works for you.
We recommend --sandbox_add_mount_pair=/tmp or, if you can, --sandbox_add_mount_pair=/tmp/zig-cache instead of incompatible_sandbox_hermetic_tmp.


Disallow calls to async functions from async functions without using 'await'

Fixes #13376

@RyanCavanaugh I think it would very useful also to check calls without await in control flow condition statements
Because to catch errors like this I need tslint with slow "strict-boolean-expressions" rule
async function bar(){return false}

async function foo() {
  if (bar()) {}
  while (bar()) {}
  do {} while (bar())
  var x = bar() ? 1 : 2;
  for (var x = 1; i++; bar()){}
  switch (bar()) {case bar(): break;}
}

Hi,
is there an ETA when this PR will be merged to master (e.g. after a 2.4 branch was created)?
Currently I'm merging this branch for a custom TS build and each time I have to fix the merge conflict in diagnosticMessages.json.  :wink:
Thanks!

Would love to have this! Currently relying on tslint rule no-floating-promises but having this guard in TypeScript itself seems preferable.

This would be especially useful for VSCode users, where no-floating-promises doesn't work out-the-box with the TSLint plugin. Having code look fine but then throw when hitting tslint via the CLI or VSCode task is annoying.

in light of the discussion about errors vs warnings we should instead move this to tslint.

It already is in tslint. This is problematic enough to be in TS proper. The tslint plugin in VS code doesn’t catch it and those without a separate linting step in their build pipeline could be shipping very broken code. I hit a MAJOR security issue with not awaiting a Promise— no way should this be downgraded to a warning.

The TSLint rule also requires type info to be on, which we do not do because it is so much slower (and causes tslint to run out of memory with our repo last I tried it).

Merging this would prevent things like this:
try {
  await someImportantSecurityCheck(); // <-- ditching `await` is a MAJOR security flaw
  runSomePrivilegedFunction();
} catch (e) {
  // deal with unauthorised...
}

// ...

Which incidentally, is exactly the issue I ran into.
Not awaiting promises can cause major headaches. Not to mention unhandled Promise rejections are deprecated, and will eventually terminated with a non-zero error code.
Again - this is not linting. It's a significant flaw in an otherwise decent type system.

Will pick this up later


Allow override of some paths

Clear and concise description of the problem
I have some vitesse projects where there is common code I keep in a separate git repository. So I have that as a subfolder of the projects and git clone there.
So that folder is in the .gitignore file as I wasn't going to add it as a submodule.
Because it's in the .gitignore file it's not running eslint on that common code.
Suggested solution
Would be great if we could somehow configure an override to allow some paths so they don't get ignored even though they are in the gitignore file
Alternative
I've currently disabled gitignore being used in the eslint.config.js but that means having to explicitly enter the paths to ignore and missing out on the goodness :)
Also considered using the common files as a git submodule but it comes with other complexities and git noise on the parent as it's the common repository that has the most dev activity.
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guide.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

How would you like to see the API? A custom filter function allowing you to decided programmatically>

In reality for me it would just be hard-coded configuration. Although a function may work well for other developers who have more complex requirements than I do.
To be honest I wasn't wanting to dictate how it should be done - you've got a better idea on how the code all fits together and I'm more than happy to follow your guidance :-)


Need option to minimize Exception (including shortcut)

After Exception we have something like this:

But when we trying repair code, modify, then this layer doing problems if some part of code is upper, and other in down.
My idea is to made "indention" for exception and offer some icon to control layer to minimize/maximize (eq. yellowe arror)

This same by some shortcut (on/off layer) and the result what I'm expecting

Other ideas

This is a fair feature request, however we do not have it on our immediate plan. Thus assigning to backlog and we might add this in the future. Thanks

@isdorn I hope that is possible to made here more improvements
https://github.com/microsoft/debugpy/issues/321
Debuging is still hell here

The Pylance have something like this for Problems

This aditional btn as minimize next to close can be ok with shortcuts as up,down, minimize, close

@isidorn  Take look nowe as upper
@alexr00 want corupt MS to realise this in this way : )
Need more Kansei.

This must be re-open and resolved.


Add Dev Environment

This will allow us to don't mess with current info

Codecov Report

Merging #108 into development will decrease coverage by 0.37%.
The diff coverage is 0%.

@@               Coverage Diff               @@
##           development     #108      +/-   ##
===============================================
- Coverage        71.21%   70.83%   -0.38%
===============================================
  Files               33       33
  Lines             1504     1512       +8
===============================================
  Hits              1071     1071
- Misses             433      441       +8

Impacted Files
Coverage Δ

MyWeight/Log.swift
54.83% <0%> (-19.08%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9bcca3a...8d8d93f. Read the comment docs.