Pass context and info to CRUD service, which will handle selected fields, aggregations etc.

See Release notes below for what has changed in this PR..
This PR improves the user API by decreasing reliance on user modification of the context in resolvers. Now you can create a custom resolver without the context if you wish:
const customResolvers = {
  Query: {
    getDraftNotes: async (_: any, __: any, context: GraphbackContext, info: GraphQLResolveInfo) => {
      const noteService = context.graphback.services.Note;

      const results = await noteService.findBy({
        filter: {
          title: {
            startsWith: '[WIP]'
          }
        }
      })

      return results.items;
    }
  }
}

It is a matter of passing the optional context and info to the CRUD layer, where Graphback will handle field selection, aggregrations etc.
This PR also adds other fixes in the same area (like usng generics instead of any)
Release notes
Changed GraphbackCRUDService findBy method signature. This also applies to all services that implement this interface.
- findBy(filter: QueryFilter<Type>, context: GraphbackContext, page?: GraphbackPage, orderBy?: any): Promise<ResultList<Type>>;
+ findBy(args?: FindByArgs, context?: GraphbackContext, info?: GraphQLResolveInfo): Promise<ResultList<Type>>;

args
findBy now accepts a new interface, FindByArgs, which wraps the filter, page and orderBy optional arguments.
await noteService.findBy({
  filter: {
    id: {
      gt: 100
    }
  },
  page: {
    offset: 0,
    limit: 10
  },
  orderBy: {
    field: 'id'
  }
})

context
The context parameter is now optional.
info
You can now optionally pass the GraphQLResolveInfo info object to the CRUD service, to perform extra optimizations like retrieving only the selected fields from the query.
await noteService.findBy(filter, context, info);

context parameter removed from subscribeToCreate, subscribeToDelete, subscribeToUpdate methods in GraphbackCRUDService.
This method was unused.
Removed context parameter from all GraphbackDataProvider methods. This also applies to all providers that implement this interface.
All CRUD methods in GraphbackDataProvider had a context parameter used to get the selected fields.
These have been replaced with (optional) selectedFields - you can pass a string array of the fields you want to select from the database.
await noteProvider.findBy(filter, ['id', 'name']);

Changed GraphbackDataProvider findBy method signature. This also applies to all providers that implement this interface.
args
findBy now accepts a new interface, FindByArgs, which wraps the filter, page and orderBy optional arguments.
await noteService.findBy({
  filter: {
    id: {
      gt: 100
    }
  },
  page: {
    offset: 0,
    limit: 10
  },
  orderBy: {
    field: 'id'
  }
})

Remove resolver options from GraphbackContext
Resolver options was removed from the context because the count aggregation and selectedFields extraction logic was moved to the CRUDService method.
CRUDService, DataSyncCRUDService now accepts a ModelDefinition as the first constructor parameter.
To instantiate a CRUDService you must pass the full ModelDefinition instead of the model name.
const myService = new CRUDService(modelDefinition, ...);

KeycloakCrudService now accepts a ModelDefinition as the first constructor parameter.
To instantiate a CRUDService you must pass the full ModelDefinition instead of the model name.
const myService = new KeycloakCrudService(modelDefinition, ...);

DataSyncProvider sync method signature has been changed
- sync(lastSync: Date, context: GraphbackContext, filter?: any, limit?: number): Promise<Type[]>;
+ sync(lastSync: Date, selectedFields?: string[], filter?: QueryFilter, limit?: number): Promise<Type[]>;

The context argument has been replaced with (optional) selectedFields.
context parameter is removed from the create method in MongoDBDataProvider and DatasyncMongoDBDataProvider providers.
This parameter did not do anything.
Follow up

This enables https://github.com/aerogear/graphback/issues/1627 to be completed.

This ranks as the most awesome PR of this year! Adding to my queue to check even further.
Amazing work!

I'd have loved to actually not pass the GraphQLResolverInfo down to service layer but if we consider that layer to be like a Controller or Router layer then I can live with that.

Done review. This is really what we needed to make things right (apart from typesafety), how would you see this done?

Done review. This is really what we needed to make things right (apart from typesafety), how would you see this done?

Apart from typesafety of what? The services?

Done review. This is really what we needed to make things right (apart from typesafety), how would you see this done?

Apart from typesafety of what? The services?

I guess he's talking about https://github.com/aerogear/graphback/issues/1637

Thought so but was not entirely sure. I am in the middle of investigating that (feels like follow up PR).


Required field with autoValue fails on update

Firstly, thanks for your meteor packages, they're the cornerstone of all of our meteor projects. A hugely valuable contribution.
Apologies in advance if I've misunderstood some part of the docs and this is the expected behaviour.
I'm trying to copy the createdAt pattern to add unique IDs to sub documents. I've created a simple reproduction of my issue here.
Steps to reproduce

Insert a document, add a name "Bob" and age "10"
Run AutoForm.debug(true) in the browser console
Click submit on the Update form

Expected result

The submission should be accepted

Actual result

Validation error: Error in EditNameForm update Error: ID is required

I think the relevant part of the schema is here:
@ChildSchema = new SimpleSchema
  _id:
    type: String
    regEx: SimpleSchema.RegEx.Id
    autoValue: ->
      if @isInsert
        Random.id()
      else if @isUpsert
        $setOnInsert: Random.id()
      else
        @unset()
    autoform:
      #omit: true
      type: 'hidden'
  age:
    type: Number

@Schema = new SimpleSchema
  name:
    type: String
  children:
    type: [@ChildSchema]

The same approach appears to work in a top level document, but not in a sub document inside an array. That is, if I've understood everything correctly.

After some more digging it looks like the problem is that formToModifier shows a single $set which writes all the properties for every field. The result is that nested fields get completely wiped as a whole unit. For example, the modifier for the schema above looks something like:
$set:
  name: 'Bobby'
  children: [
    age: 10
  ]

The result is that the nested _id value does not get set.

Of course, the answer was in the docs! Apologies for the spurious ticket...
https://github.com/aldeed/meteor-collection2#subobjects-and-arrays-of-objects


State and prove the universal property of the free magma quotiented by a theory.

null
Closes #412

Thanks Pietro!

Thank you @codyroux for your contribution!


Support basic auth for forge

Add support for http basic auth for custom forge repo's while still ensuring that the uri's userinfo is not printed to console.
Result of changes

puppet module install will be passed the basic auth information when run
Now adds basic auth information to Puppetfile.lock to prevent the error Puppetfile and Puppetfile.lock are out of sync!

This works for me, would be good to see it merged/released


go install fails due to crypto library error

go install github.com/smallstep/step-kms-plugin@latest

will not proceed with the following error.
# go.step.sm/crypto/tpm/attestation
.go/pkg/mod/go.step.sm/crypto@v0.32.3/tpm/attestation/client.go:216:26: ac.baseURL.JoinPath undefined (type *url.URL has no field or method JoinPath)
.go/pkg/mod/go.step.sm/crypto@v0.32.3/tpm/attestation/client.go:260:26: ac.baseURL.JoinPath undefined (type *url.URL has no field or method JoinPath)

My env.
2023-10-27T12:11:25 [✖ USAGE  2] ⬢ [Systemd] ❯ uname -a
Linux SSD0086 <IP_ADDRESS>-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

~
2023-10-27T12:12:40 ⬢ [Systemd] ❯ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ncaq/.cache/go-build"
GOENV="/home/ncaq/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ncaq/.go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ncaq/.go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.18"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.18/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2420606553=/tmp/go-build -gno-record-gcc-switches"

Hi @ncaq, thank you for reporting this issue. You can try installing a specific version by specifying @v... or use a more recent Go version to install the binary. I think the JoinPath method was added in Go 1.19.
I've opened a PR to bump the Go version in go.mod to 1.20, so that it's in line with our crypto library: https://github.com/smallstep/step-kms-plugin/pull/106.


Facebook SDK 4.27.0 Issue - WebDialog

Environment

Android device: Emulator
Android OS version: 8.0.0
Google Play Services version: 11.5.09 (on Emulator)
Firebase/Play Services SDK version: 11.4.2
FirebaseUI version: 3.0.0
Facebook SDK Version: 4.27.0

I upgraded my firebase-ui dependency to 3.0.0 and Facebook-SDK to 4.27.0
    // Firebase UI
    implementation "com.firebaseui:firebase-ui-database:3.0.0"
    implementation "com.firebaseui:firebase-ui-auth:3.0.0"

    // Facebook Login
    implementation "com.facebook.android:facebook-android-sdk:4.27.0"

I looked into the FacebookProvider.java class and it does have updated code from https://github.com/firebase/FirebaseUI-Android/pull/929
WebDialog.setWebDialogTheme(theme);

Not sure what I need to do. Please help.
Observed Results:
java.lang.NoSuchMethodError: No static method setWebDialogTheme(I)V in class Lcom/facebook/internal/WebDialog; or its super classes

Expected Results:

App compiles and runs without this error

I placed a breakpoint in FacebookProvider.java and seems it's consuming new code now. No other action was necessary.

Hi! I am having this exact problem. Could you give more details about your solution?
Thank you.

@lalo2302 make sure to be using FirebaseUI 3.1 and com.facebook.android:facebook-login:4.27.0. Also try cleaning your cache.

I was using an outdated version of firebase ui. Thank you very much.


BaseObject design reconciliation points

There are some open points that we need to clarify with the @rnkuhns design. It also needs to work for sktime.
Some key points, from an architecture integration perspective:

@rnkuhns merged the BaseObject and the _HeterogeneousMetaEstimator. I think this is not a good idea, it merges concerns, and overloads the BaseObject with pipeline functionality. I don't think sktime can live with that... too many concerns in one place, and refactor of all the pipelines would become a nightmare. I'd really prefer to move away from this god object direction...
@rnkuhns removed the testing functions, but sktime needs those. These should also be public, not private, since other classes interact with them. So from a formal class design perspective, these need to be public unified interface points, even if they are not user facing.
I like the addition of checking whether the tag containers exist, although instead of raising an error I simply would consider it empty
is_composite is something I added out of a hurry. Perhaps we should sit down and design the components interface more cleanly?

@franz, I've got a couple of thoughts related to these:

BaseObject (and scikit-learn's BaseEstimator ) have a get_params and set_params interface that already works with certain composites (when parameter values have the parameter getting/setting interface themselves).  _HeterogenousMetaEstimator adds the parameter getting/setting for another form of composite objects (an iterable of name, object tuples). The composition supported by BaseObject and _HeterogenousMetaEstimator both allow for composition of heterogenous objects that use the BaseObject style interface, just in different forms. By combining those into the same implementation of get_params and set_params in BaseObject we are providing functionality for the range of supported (out-of-the-box) composition in the same place.
I still want to add the testing interface -- it just needs a slight tweak to work as an external package (it can't rely on a hard-coded location to look for test instance parameters; there are a couple options for generalizing that we need to consider)
I'm fine with assuming the tag container is empty if it doesn't exist. Maybe we should raise a warning instead of an error (e.g., we noticed you tried to use the tag interface, but don't have _tags or _tags_dynamic attributes. Did you forget to add them to your class?)
Agreed on is_composite. I also think it depends on what we determine for the parameter getting/setting. For example, if BaseObject only supports getting/setting params for "direct" composites (parameter value has the interface), then is_composite should determine it is a composite object if that case is found. If BaseObject more broadly supports composite parameter gettting/setting then is_composite should indicate the object is a composite more broadly.

Re merging BaseObject and _HeterogeneousMetaEstimator

By combining those into the same implementation of get_params and set_params in BaseObject we are providing functionality for the range of supported (out-of-the-box) composition in the same place.

I see this. I think there is a stronger counterargument, in that we are merging too much the generic and the specific. I would separate BaseObject having the functionality which every object needs, and having functionality which only some objects need in another class. BaseObject is already heavy enough imo.
Re testing interface: ok, agreed.
Re tag container: ok, agreed with assuming it is empty if does not exist. I would also not raise a warning, I think it should be consistent with attribute inheritance which also does not raise a warning if one of the inheriting classes doesn't have it.
Re is_composite: well, then, according to your opinion, it's not too bad in its current state?


[camera_android_camerax]: error certain methods don't override or implement methods from a supertype.

Describe the bug
I can't run my flutter app without it crashing and throwing the error
C:\Users\Nnam0\AppData\Local\Pub\Cache\hosted\pub.dev\camera_android_camerax-0.6.8+2\android\src\main\java\io\flutter\plugins\camerax\PreviewHostApiImpl.java:84: error: cannot find symbol
            new TextureRegistry.SurfaceProducer.Callback() {
                                               ^
  symbol:   class Callback
  location: interface SurfaceProducer
C:\Users\Nnam0\AppData\Local\Pub\Cache\hosted\pub.dev\camera_android_camerax-0.6.8+2\android\src\main\java\io\flutter\plugins\camerax\PreviewHostApiImpl.java:85: error: method does not override or implement a method from a supertype
              @Override
              ^
C:\Users\Nnam0\AppData\Local\Pub\Cache\hosted\pub.dev\camera_android_camerax-0.6.8+2\android\src\main\java\io\flutter\plugins\camerax\PreviewHostApiImpl.java:91: error: method does not override or implement a method from a supertype
              @Override
              ^
3 errors

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':camera_android_camerax:compileDebugJavaWithJavac'.
> Compilation failed; see the compiler error output for details.

* Try:

To Reproduce
Steps to reproduce the behavior:

Run the command flutter run after adding livelyness_detection: ^0.0.1+4 to pubspec.yaml
See error

Expected behavior
App is expected to build and run the app in debug mode
Desktop (please complete the following information):
$ flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 3.22.2, on Microsoft Windows [Version 10.0.22631.4037], locale en-US)
[√] Windows Version (Installed version of Windows is version 10 or higher)
[√] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[√] Chrome - develop for the web
[!] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.10.6)
    X The current Visual Studio installation is incomplete.
      Please use Visual Studio Installer to complete the installation or reinstall Visual Studio.
[√] Android Studio (version 2022.3)
[√] VS Code (version 1.92.2)
[√] Connected device (5 available)
[√] Network resources

Additional context
minSdkVersion 23

Just downgrade your camera plugin to 0.10.6, camera: 0.10.6


Downloading a late portion of a very long twitch video

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I'm running yt-dlp version 2022.10.04 (update instructions) or later (specify commit)
[X] I've searched the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
I am attempting to download a 2 minute section of a 46 hour long (twitch subathon) video. The time stamp I am interested in is 33:54:00 to 33:56:00. Is there a known issue with this use case?
The program seems to start at the correct chunked/*.ts (verified by copying the url from the yt-dlp output and downloading the file through my browser); however, the job does not download any sections and continues to scan all chunks until the end of the video.  Upon reaching the end, it creates a 1 byte junk file. I have interrupted my verbose log as there are about 3000 additional chunks scanned before reaching the end of the video which I presume would not be relevant to the report. (Note also that the absence of downloading may not be reflected in the log, as I have noticed the "download" text, e.g. frame= 1799 fps=150 q=-1.0 size=   23040kB time=00:00:30.01 bitrate=6288.1kbits/s speed=2.51x does not remain in the log of a successful job.)
The program works as expected if I use an earlier time range, e.g. 3:54:00 to 3:56:00, downloading the chunks appropriate for that range, stopping the scan, and combining what it downloaded into a single, functional, output file.
I am uncertain if this is a known issue with very long twitch videos. I was not able to find any open or closed issues referring to a similar problem, although it could be that I did not use the correct terms. The only reference to a similar issue that I have found was a reddit post (https://old.reddit.com/r/youtubedl/comments/x8g6lk/downloading_parts_of_a_long_video_fails_past_a/), with no solution indicated. In that case, the user said ytdlp fails after hour ~26 and that he did not have a similar issue with long youtube videos. I have not been rigorous in checking at what time range the issue begins to occur for the video in my case.
My assumption is that there is either an issue on twitch's end with how it timestamps chunks on very long videos, or yt-dlp is unable to read the time range for late portions of very long twitch videos.
Thank you for your time.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
:\MISC\Video\dlp>yt-dlp -vU "https://www.twitch.tv/videos/1595417284" --download-sections "* 33:54:00  -33:56:00" -o "20220919 nina - %(title)s [%(id)s] - misery state.%(ext)s" -P "temp:tmp" -P "D:\Misc\Video"
[debug] Command-line config: ['-vU', 'https://www.twitch.tv/videos/1595417284', '--download-sections', '* 33:54:00  -33:56:00', '-o', '20220919 nina - %(title)s [%(id)s] - misery state.%(ext)s', '-P', 'temp:tmp', '-P', 'D:\\Misc\\Video']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.10.04 [4e0511f] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-7-6.1.7601-SP1
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.0.1-full_build-www.gyan.dev (setts), ffprobe 5.0.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1690 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.10.04, Current version: 2022.10.04
yt-dlp is up to date (2022.10.04)
[debug] [twitch:vod] Extracting URL: https://www.twitch.tv/videos/1595417284
[twitch:vod]<PHONE_NUMBER>: Downloading stream metadata GraphQL
[twitch:vod]<PHONE_NUMBER>: Downloading video access token GraphQL
[twitch:vod]<PHONE_NUMBER>: Downloading m3u8 information
[twitch:vod]<PHONE_NUMBER>: Downloading storyboard metadata JSON
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] v1595417284: Downloading 1 format(s): 1080p60
[info] v1595417284: Downloading 1 time ranges: 122040.0-122160.0
[debug] Invoking ffmpeg downloader on "https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/index-dvr.m3u8"
[download] Destination: D:\Misc\Video\tmp\20220919 nina - Day 3 SUBATHON | Games, Fun and Wowee !schedule !subathon [v1595417284] - misery state.mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -headers "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Sec-Fetch-Mode: navigate
" -ss 122040.0 -t 120.0 -i "https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/index-dvr.m3u8" -c copy -f mp4 "file:D:\Misc\Video\tmp\20220919 nina - Day 3 SUBATHON | Games, Fun and Wowee !schedule !subathon [v1595417284] - misery state.mp4.part"
ffmpeg version 5.0.1-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (Rev7, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplaceb
o --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
[tcp @<PHONE_NUMBER>4535c0] Starting connection attempt to <IP_ADDRESS> port 443
[tcp @<PHONE_NUMBER>4535c0] Successfully connected to <IP_ADDRESS> port 443
[hls @<PHONE_NUMBER>44f940] Skip ('#EXT-X-VERSION:3')
[hls @<PHONE_NUMBER>44f940] Skip ('#ID3-EQUIV-TDTG:2022-09-21T13:22:42')
[hls @<PHONE_NUMBER>44f940] Skip ('#EXT-X-TWITCH-ELAPSED-SECS:0.000')
[hls @<PHONE_NUMBER>44f940] Skip ('#EXT-X-TWITCH-TOTAL-SECS:166887.219')
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/0.ts', offset 0, playlist 0
[hls @<PHONE_NUMBER>44f940] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/0.ts' for reading
[tcp @<PHONE_NUMBER>766540] Starting connection attempt to <IP_ADDRESS> port 443
[tcp @<PHONE_NUMBER>766540] Successfully connected to <IP_ADDRESS> port 443
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/1.ts', offset 0, playlist 0
[hls @<PHONE_NUMBER>44f940] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/1.ts' for reading
[tcp @<PHONE_NUMBER>7c1980] Starting connection attempt to <IP_ADDRESS> port 443
[tcp @<PHONE_NUMBER>7c1980] Successfully connected to <IP_ADDRESS> port 443
[mpegts @<PHONE_NUMBER>763800] parser not found for codec timed_id3, packets or times may be invalid.
[h264 @<PHONE_NUMBER>29f640] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @<PHONE_NUMBER>29f640] Increasing reorder buffer to 1
[AVIOContext @<PHONE_NUMBER>958380] Statistics: 330843 bytes read, 0 seeks
[AVIOContext @<PHONE_NUMBER>1874c0] Statistics: 0 bytes read, 0 seeks
Input #0, hls, from 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/index-dvr.m3u8':
  Duration: 46:21:27.22, start: 62.017000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
  Stream #0:0: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
  Stream #0:1: Video: h264 (High), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, left), 1920x1080 (1920x1088) [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn
    Metadata:
      variant_bitrate : 0
  Stream #0:2: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:D:\Misc\Video\tmp\20220919 nina - Day 3 SUBATHON | Games, Fun and Wowee !schedule !subathon [v1595417284] - misery state.mp4.part':
  Metadata:
    encoder         : Lavf59.16.100
  Stream #0:0: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, bt709, left), 1920x1080 (0x0) [SAR 1:1 DAR 16:9], q=2-31, 60 fps, 60 tbr, 90k tbn
    Metadata:
      variant_bitrate : 0
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
  Stream #0:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12203.ts', offset 0, playlist 0
[hls @<PHONE_NUMBER>44f940] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12203.ts' for reading
[tcp @<PHONE_NUMBER>3ccfc0] Starting connection attempt to <IP_ADDRESS> port 443
[tcp @<PHONE_NUMBER>3ccfc0] Successfully connected to <IP_ADDRESS> port 443
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12204.ts', offset 0, playlist 0
[hls @<PHONE_NUMBER>44f940] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12204.ts' for reading
[tcp @<PHONE_NUMBER>3cd180] Starting connection attempt to <IP_ADDRESS> port 443
[tcp @<PHONE_NUMBER>3cd180] Successfully connected to <IP_ADDRESS> port 443
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12205.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12205.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12206.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12206.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12207.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12207.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12208.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12208.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12209.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12209.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12210.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12210.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12211.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12211.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12212.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12212.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12213.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12213.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12214.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12214.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12215.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12215.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12216.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12216.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12217.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12217.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12218.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12218.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12219.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12219.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12220.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12220.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12221.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12221.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12222.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12222.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12223.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12223.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12224.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12224.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12225.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>953f40] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12225.ts' for reading
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12226.ts', offset 0, playlist 0
[https @<PHONE_NUMBER>288c00] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12226.ts' for reading
[AVIOContext @<PHONE_NUMBER>17c740] Statistics: 87074785 bytes read, 0 seeks
[hls @<PHONE_NUMBER>44f940] HLS request for url 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12226.ts', offset 0, playlist 0
[hls @<PHONE_NUMBER>44f940] Opening 'https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/12226.ts' for reading
[tcp @<PHONE_NUMBER>3cd180] Starting connection attempt to <IP_ADDRESS> port 443
Error writing trailer of file:D:\Misc\Video\tmp\20220919 nina - Day 3 SUBATHON | Games, Fun and Wowee !schedule !subathon [v1595417284] - misery state.mp4.part: Immediate exit requested
frame=    0 fps=0.0 q=-1.0 Lsize=       0kB time=00:00:00.00 bitrate=N/A speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (https://dgeft87wbj63p.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/index-dvr.m3u8):
  Input stream #0:0 (audio): 0 packets read (0 bytes);
  Input stream #0:1 (video): 0 packets read (0 bytes);
  Input stream #0:2 (data): 0 packets read (0 bytes);
  Total: 0 packets (0 bytes) demuxed
Output file #0 (file:D:\Misc\Video\tmp\20220919 nina - Day 3 SUBATHON | Games, Fun and Wowee !schedule !subathon [v1595417284] - misery state.mp4.part):
  Output stream #0:0 (video): 0 packets muxed (0 bytes);
  Output stream #0:1 (audio): 0 packets muxed (0 bytes);
  Total: 0 packets (0 bytes) muxed
[AVIOContext @<PHONE_NUMBER>fa1e80] Statistics: 48 bytes written, 2 seeks, 3 writeouts
Error closing file file:D:\Misc\Video\tmp\20220919 nina - Day 3 SUBATHON | Games, Fun and Wowee !schedule !subathon [v1595417284] - misery state.mp4.part: Immediate exit requested
[AVIOContext @<PHONE_NUMBER>437f80] Statistics: 90155776 bytes read, 0 seeks
[AVIOContext @<PHONE_NUMBER>6a6980] Statistics: 406335 bytes read, 0 seeks
Exiting normally, received signal 2.

ERROR: Interrupted by user
Terminate batch job (Y/N)? n

D:\MISC\Video\dlp>pause
Press any key to continue . . .

This is due to an ffmpeg bug with very long HLS playlists:
https://trac.ffmpeg.org/ticket/7673
You can reproduce the issue using ffmpeg alone:
ffmpeg -y -loglevel verbose -headers "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.19 Safari/537.36"$'\r\n' -ss 122040.0 -t 120.0 -i "https://d1ymi26ma8va5x.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/index-dvr.m3u8" -c copy -f mp4 file:output.mp4

Whereas if you try to download a section from before the 26.5 hour mark it will work (e.g. -ss 93600 -t 120).
As a workaround for this specific case, you could use curl to download the chunks and then remux to mp4:
curl -qf --fail-early "https://d1ymi26ma8va5x.cloudfront.net/2ca48245fc2ff5d9a05c_ninaninin_39848342232_1663599666/chunked/[12204-12214].ts" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.19 Safari/537.36" >> "output.ts"

ffmpeg -i "output.ts" -c copy -f mp4 "output.mp4"

This is due to an ffmpeg bug with very long HLS playlists:
https://trac.ffmpeg.org/ticket/7673

Thank you for looking into this further. As you note, it's an issue with ffmpeg and a long-standing one at that.
My own workaround had to been to run the command in order to get the url and the first chunk it starts at, guess how many chunks cover the range I'm looking for, download them each manually, then ffmpeg concat into a single file; curl will save some effort there. (For any future users still on windows 7: it will need to be installed, but windows 10+ includes it by default.)
Thanks again for your time.


[FLINK-34467] add lineage integration for jdbc connector

Add native lineage support for JDBC connector. It is an ongoing effort. The PR is not ready for review.

@HuangZhenQiu I notice your comment "The PR is not ready for review.", I wonder if you could turn this PR into a draft until it is ready for review - thanks.


reset password via link error

Reset password via link should bypass the confirm password requirement, which obviously the user can't provide.

fixed by storing resetCode in session by 9e6afe88b9734647c1c20c43cd19b12a3e175fb7


Pass through HTTP status 102 processing

Title: Envoy should pass through all 102 processing status from downstream services.
Description:
Downstream services might send a 102 processing as a form of keep-alive. Envoy should simply pass them on without interfering.
(Optional: also pass through other unknown 1xx status)
Relevant Links:

WebDAV: https://datatracker.ietf.org/doc/html/rfc2518#section-10.1
HTTP 1.1: https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

A client MUST be prepared to accept one or more 1xx status responses prior to a regular response [...]
Proxies MUST forward 1xx responses, unless the connection between the proxy and its client has been closed, or unless the proxy itself requested the generation of the 1xx response.

cc @alyssawilk for thoughts on this.

Btw I edited the description. I just realized I mixed up "downstream" and "upstream" :-/

yeah I think arguably we should rename the encode100ContinueHeaders functions to encode1xxHeaders and allow any no-op 100s
Matt, if you agree that's reasonable I can add it to my queue

Matt, if you agree that's reasonable I can add it to my queue

Sure makes sense to me.

@alyssawilk First of all: thank you so much for implementing this (so fast!).
I tried with the recent master, but was not quite able to figure out if I am doing it wrong, or if it doesn't work (yet?) as it should.
If I open a connection directly to my (test) service, it sends a 102 every 2 seconds. So an example request looks like this:
> curl -vvv 'http://<IP_ADDRESS>:3000/test?wait=10s'
*   Trying <IP_ADDRESS>:3000...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 3000 (#0)
> GET /test?wait=10s HTTP/1.1
> Host: <IP_ADDRESS>:3000
> User-Agent: curl/7.80.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 102 Processing
* Mark bundle as not supporting multiuse
< HTTP/1.1 102 Processing
* Mark bundle as not supporting multiuse
< HTTP/1.1 102 Processing
* Mark bundle as not supporting multiuse
< HTTP/1.1 102 Processing
* Mark bundle as not supporting multiuse
< HTTP/1.1 102 Processing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 18 Nov 2021 08:07:18 GMT
< Content-Length: 18
<
* Connection #0 to host <IP_ADDRESS> left intact
Returned after 10s⏎

When I do the same with envoy (commit 1eb1841f) in between, it now looks like this:
> curl -vvv 'http://<IP_ADDRESS>:8080/test?wait=20s'
*   Trying <IP_ADDRESS>:8080...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080 (#0)
> GET /test?wait=20s HTTP/1.1
> Host: <IP_ADDRESS>:8080
> User-Agent: curl/7.80.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 102 Unknown
* Mark bundle as not supporting multiuse
< HTTP/1.1 504 Gateway Timeout
< content-length: 24
< content-type: text/plain
< date: Thu, 18 Nov 2021 08:08:45 GMT
< server: envoy
<
* Connection #0 to host <IP_ADDRESS> left intact
upstream request timeout⏎

It seems to only forward the first 102, and apparently it turns "Processing" into "Unknown" (or strips it right away; might be curl that turns it into "Unknown"). The remaining 102s seem to be swallowed, so the request now also runs into a timeout.
Am I missing something or is this so far expected behavior?
Maybe I am using the feature flag wrong?
I have this in my config:
layered_runtime:
  layers:
    - name: static_layer
      static_layer:
        envoy.reloadable_features.proxy_102: true

ah yeah, if you check the release notes, as with 100s and 103s, Envoy only handles the first, sorry that wasn't clear to you
Handling multiple 1xxs is significantly more complicated.  Simply passing them through in the happy path isn't so bad, but when things are backed up, we only have one "slot" in the HTTP processing machine to buffer 1xxs,and handling more than one gets hella complicated and I think would take more time than I can afford to tackle this quarter.
Filed https://github.com/envoyproxy/envoy/issues/19044 for posterity

Thanks for clarification. I probably have a pretty naive view on this and hoped that upstream responses can simply be passed  through. But given the many protocol conversions envoy supports, this likely is not possible without some state machine around it.
Again many thanks for tackling this (so soon) and also thanks for even considering looking into it some time later for an even broader solution! :smiley:


Use serverId (server url) in kernel connection metadata when connecting to remote jupyter server

Part of #9167

Code cleanup
Basically uses the serverId (url) in the kernel connection metadata to connect to the remote jupyter server.
Also paves the way for connecting to multiple remotes (without assuming theres only ever one remote)

Codecov Report

Merging #9853 (44f145e) into main (9f58327) will increase coverage by 0%.
The diff coverage is n/a.

:exclamation: Current head 44f145e differs from pull request most recent head 4d85d19. Consider uploading reports for the commit 4d85d19 to get more accurate results

@@         Coverage Diff          @@
##           main   #9853   +/-   ##
====================================
  Coverage    63%     63%
====================================
  Files       203     203
  Lines      9810    9810
  Branches   1556    1556
====================================
+ Hits       6224    6232    +8
+ Misses     3087    3078    -9
- Partials    499     500    +1

Impacted Files
Coverage Δ

src/platform/errors/errorHandler.ts
68% <0%> (+1%)
:arrow_up:

...rc/platform/common/dataScienceSurveyBanner.node.ts
70% <0%> (+5%)
:arrow_up:


Disabling Log correlation when agent is inactive

[x] Implement code
[x] Add tests
[ ] Update documentation
[ ] Update CHANGELOG.md

What if active is set to false after the activation and before the deactivation? We should have a transaction-scoped flag (can we use the noop flag?) which is consistent for a whole transaction. If that is false, call no scope listeners.

What if active is set to false after the activation and before the deactivation?

I only check in adding to MDC but always removing from MDC.


Not reexport RecordBatch twice

Which issue does this PR close?
There are two duplicated reexports with different path. But actually they point to the same source.

After this pr,

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

This is for backwards compatibility, what is the issue with this?

Just a little confused.

Prior to splitting apart the crate RecordBatch lived under arrow::record_batch, it was then moved to live under arrow_array and therefore re-exported under arrow::array. To avoid breaking downstreams an export at the old path was preserved

Understand, we might add deprecation info, but feel free to close this pr if you don't think we need this.

I think this isn't worth the downstream churn, but thank you for raising this


Options with Resolver

If https://github.com/chartjs/Chart.js/pull/8374 goes through, this would make the plugin compatible and most things scriptable.

Filename
Size
Change

dist/chartjs-plugin-annotation.esm.js
6.64 kB
+2 B (0%)

dist/chartjs-plugin-annotation.js
6.76 kB
+6 B (0%)

dist/chartjs-plugin-annotation.min.js
4.57 kB
-14 B (0%)


Conditional exclude items

Actual Situation
I would like to exclude files / folders conditionally in the copier.yaml file. Currently, if you add a file/folder to the exclude list, it is always excluded, even if there is a suffixed file that converts. Therefore, the exclusion list applies to the result (this has been discussed in past issues).
My real use case is that I have a few configuration files that apply to my template, and may or not apply to the destination depending on a certain variable. If they don't apply to the destination I would like to exclude them and not copy irrelevant files. If they apply, I would not exclude and normal overwrite rules apply.
Desired Situation
Being able to exclude files/folders conditionally would allow having config files in my template that are copied over conditionally on variables. This would be a new powerful feature.
Proposed solution
I would like to have something in the copier.yaml that would allow me to exclude files/folders:
_exclude:
  - "{% if variable is defined %}myconfig.yml{% endif %}"

I think this solution would not impact current behavior of exlusions applying to final names. It just modifies applying the exclusion conditionally.

Are you aware that Copier allows to do that thanks to file names?
A file named {% if some_condition %}filename.ext{% ending %}.jinja will only be created if the some_condition evaluates to true.

Not exactly, because the file in question is configuring the template, it cannot have any other name. That file has to exist as is, and excluded on copy conditionally. At the moment I have to overwrite it and make it clean not to copy my template's config over, but it is not used in the destination. Does it make sense?

Oh, IIUC you have files that are used both for maintaining the template, and as files to copy to destination projects. Interesting. I'd recommend keeping two separate copies indeed, and using the subdirectory feature to separate them. Just my opinion but if you need to change config for your template you might not want this change to apply to projects too? Better explicit than implicit?

Other than that, no strong opinion :) If other maintainers think this feature is worth it, I won't mind.

Yeah, let me give you an example. I have a acl/access.yml file that specifies who has right permissions in the template repo. When I create a copy of the template to apply to that same server, I ask the pertinent question of who should have access, and format a brand new file from acl/access.yml.jinja file, which will replace the template's original. If I am not creating a project for that server, I don't want to have this file at all in the new repo, so I would exclude the folder.
subdirectory feature is not ideal. I like to keep files in the root because I can test everything. I can run CI and builds exactly as they are going to run in the copies (or pretty close).

Actually could you post snippets of your file layout? Because it's quite hard to understand how exactly you're using Copier from text alone.

subdirectory feature is not ideal. I like to keep files in the root because I can test everything. I can run CI and builds exactly as they are going to run in the copies (or pretty close).

Could you elaborate on that? I'm not sure to understand either.

Here an example. The access.yml file is used for compliance of the template repo. Copies of the template will inherit from the processed access.yml.jinja, satisfying their own compliance. Copies of the template where I could conditionally exclude folders from, would not get the folder.

Regarding the CI that I mentioned. I have a .github/workflows/ci.yaml file running pytest and mkdocs builds. Those tests are run on the template repo, so I know that tests are being found and run, and documentation is being built. Because the same modules and CI is being copied to new repos, I can check that tests running for the template repo, will more likely pass in new copies. If I had a subdirectory with all my template files, I would have to have one ci workflow for the template, and another ci workflow for the subdirectory, completely different between themselves, I would not be able to test the subdirectory workflow at all (because it wouldn't be run, only the template's workflow would be run).
Hope this makes it all clearer.

OK I see, thanks a lot :slightly_smiling_face:

If I had a subdirectory with all my template files, I would have to have one ci workflow for the template, and another ci workflow for the subdirectory, completely different between themselves, I would not be able to test the subdirectory workflow at all (because it wouldn't be run, only the template's workflow would be run).

Yeah that's one of the challenges I'm facing indeed. Tracking my progress here: https://github.com/pawamoy/copier-uv/issues/24.

I am running into the same issue. I can't edit the file name because it is needed for the template itself to run some checks. I tried to create a duplicate filename and then add that to the exclude but it ends up skipping both directories.

I'd like an option that doesn't involve messing with the filename too because the conditionals can become pretty unwieldy and it also seems my editor (vim) doesn't really like it either, for some reason. I can open the file, but vim fails to save it because it seems to interpret some things incorrectly.

I'm not sure I'm understanding the use case properly, but if I do, it seems to me like you can solve it with a combination of subdirectory and symlinks.
One live example here:

The project itself uses CI definitions you can find in .gitlab/ci/*.yml
The template lives under template, and there's a symlink from template/{% if gitlab_ci != "none" %}.gitlab{% endif %}/ci to that parent folder.
A question lets the user choose if they want to enable gitlab ci

As a result, the template CI itself uses the same CI primitives as the produced projects. But produced projects can configure that CI at will.
Would this strategy solve your problems?

Note: To make it even more fun, the template is applied to itself!

I mean it would but at some point the conditional expressions get really large, and trying to embed all that logic into a file name becomes untenable.

You can use a macro then, or a computed field

Thank you @yajo for your input. Yes, using subdirectory would work, and I could have specific pipelines to test the template folder. As I mentioned above, it is not ideal, I think it becomes pretty cumbersome and repetitive.
This is btw the solution I currently have. I have many symlinked files to try to not repeat myself a lot. But it is not clear and it is not easy to mantain. I think the solution I am proposing would be cleaner and easier, without any breaking changes.

There's a subtle difference between using _exclude and Jinja conditions in directory/file names: _exclude refers to generated directories/files, so it may be difficult/unintuitive/verbose to come up with a correct pattern when using dynamic directory/file names.
Think of this example template for a Python project with configurable project layout and optional CLI:
layout:
  type: str
  help: Which Python project layout would you like?
  choices:
    flat: ""
    src: null

package_slug:
  type: str
  help: Your Python package name/slug

cli:
  type: bool
  help: Do you need a CLI?

.
└── {{ layout }}
    └── {{ package_slug }}
        ├── {% if cli %}__main__.py{% endif %}.jinja
        └── __init__.py

Here, the Jinja condition for the CLI-related file is part of its filename, so it works for either project layout. But if we were to use the _exclude setting, we'd need to include path-related template variables to construct the generated file path (without the .jinja suffix):
+_exclude:
+  - "{% if not cli %}{{ layout }}/{{ package_slug }}/__main__.py{% endif %}"

 layout:
   type: str
   help: Which Python project layout would you like?
   choices:
     flat: ""
     src: null

 package_slug:
   type: str
   help: Your Python package name/slug

 cli:
   type: bool
   help: Do you need a CLI?

 .
 └── {{ layout }}
     └── {{ package_slug }}
-        ├── {% if cli %}__main__.py{% endif %}.jinja
+        ├── __main__.py.jinja
         └── __init__.py

TBH, I don't find the latter more intuitive nor more robust. If the conditions become more complex, a computed value may be used as @yajo already suggested. Note that macros may not work at the moment because of https://github.com/copier-org/copier/issues/1164, and my upstream PR https://github.com/pallets/jinja/pull/1852 does not seem to get merged for some reason.

thanks @sisp! I agree my proposal being more intuitive / robust is subjective. But it gives the user the option to choose how they want to encode the logic. I personally prefer to read simpler file names and a use the exlude pattern that is still pretty explicit. Using dynamic names & computed values in the exclude section would be pretty powerful. Maybe it does not become simpler, but it is cleaner to me, your exclude logic is confined clearly.
Furthermore, it allows not requiring the subdirectory feature, which we have to use otherwise. Using subdirectory makes everything messier in my opinion, as you may have to symlink or duplicate files.

I agree with @sisp but it's also true that I don't see that as a strong reason to oppose including templating in _exclude. However it's a strong reason to me to not want to develop such a feature. 😆
But if some contributor wants to open the PR, I'd be glad to review it. I'll label the issue as such.


Proposal: Bulk Upload and Download APIs

Currently the only way to write data to SpiceDB is with the WriteRelationships API, and the only way to read relationships is the ReadRelationships API. These APIs are designed around OLTP use-cases, and are not suitable for bulk reading and writing of data. Use-cases like backup and restore, and backfill require a large amount of relationships to be accessed at once.
We should explore adding more performant APIs for these bulk operations to facilitate these use cases.

These APIs are now experimentally exposed via the experimental service: https://buf.build/authzed/api/docs/main:authzed.api.v1#authzed.api.v1.ExperimentalService
See https://github.com/authzed/spicedb/pull/1326 and https://github.com/authzed/spicedb/pull/1281

Hi. Can you please share how to consume method using Java client?
I  am not finding 'bulkImportRelationships' method on ExperimentalServiceGrpc stub.
Ex:  experimentalService.bulkImportRelationships(request);

You can find an example here: https://github.com/authzed/authzed-java/blob/a8ae90a3f8f008ab6e3ea71299064a7b83938981/src/intTest/java/V1ClientTest.java#L212-L239


Update dependency version of analyzer

Hi,
I was just about to add a new library to a project and version solving seems to fail since floor requires analyzer ^0.40.6. Apperently, analyzer is now available with support for nullsafety (https://pub.dev/packages/analyzer/versions/0.42.0-nullsafety.0). Could you update the dependency?
Thank you in advance!

Found out that I could simply override the dependencies.


fix: Message actions popover not showing after open app download links page and go back

Details

Escalate PopoverReportActionContextMenu to [Expensify.js (https://github.com/Expensify/App/blob/main/src/Expensify.js) to become an app-wide component, thus avoiding race conditions when using the app-wide ReportActionContextMenu.showContextMenu method
Fixed Issues

$ https://github.com/Expensify/App/issues/14131
https://github.com/Expensify/App/issues/14131#issuecomment-1381298507
Tests

Open any chat room.
Right click/Long press on any message to open actions popover. (Popover will be display)
Go to Settings => About => App download links.
Dismiss/close modal on outside click.
Try to right click on any message to open actions popover/modal.
Verify that the actions popover/modal shows correctly

[x] Verify that no errors appear in the JS console

Offline tests

Turn off the internet
Open any chat room.
Right click/Long press on any message to open actions popover. (Popover will be display)
Go to Settings => About => App download links.
Dismiss/close modal on outside click.
Try to right click on any message to open actions popover/modal.
Verify that the actions popover/modal shows correctly

QA Steps

Open any chat room.
Right click/Long press on any message to open actions popover. (Popover will be display)
Go to Settings => About => App download links.
Dismiss/close modal on outside click.
Try to right click on any message to open actions popover/modal.
Verify that the actions popover/modal shows correctly

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

https://user-images.githubusercontent.com/113963320/213670370-6155d3d1-f294-4509-b8a3-140b624ea132.mov

Mobile Web - Chrome

https://user-images.githubusercontent.com/113963320/213669161-1f2b6b64-1047-42c5-9037-52fcc0add485.mov

Mobile Web - Safari

https://user-images.githubusercontent.com/113963320/213670333-cdbfee10-5e0e-402f-9f6f-c26ab07d0853.mov

Desktop

https://user-images.githubusercontent.com/113963320/213678369-d9aaa0ab-0eb3-4e5a-8b21-38cdf1e73e04.mov

iOS

https://user-images.githubusercontent.com/113963320/213678274-7cc674c0-4af5-4864-afec-87d0ced04162.mov

Android

https://user-images.githubusercontent.com/113963320/213703377-e8c6c8ca-9418-4274-baa9-6265a2a63f57.mp4

Other than that the code looks good, so I'm going to start testing and going through the checklist

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/luacmartins in version: 1.2.58-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/chiragsalian in version: 1.2.58-4 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

Leaving a note that this has caused a regression in https://github.com/Expensify/App/issues/15870.
There was a fix that was added before to resolve https://github.com/Expensify/App/issues/5197 and https://github.com/Expensify/App/issues/5250, but this PR eliminated the need for it, in turn introducing a new regession.


python312Packages.dirigera: 1.2.1 -> 1.2.2

Diff: https://github.com/Leggin/dirigera/compare/refs/tags/v1.2.1...v1.2.2
Changelog: https://github.com/Leggin/dirigera/releases/tag/v1.2.2

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 363583

x86_64-linux

:fast_forward: 1 package marked as broken and skipped:

home-assistant-custom-components.dirigera_platform

:white_check_mark: 4 packages built:

python311Packages.dirigera
python311Packages.dirigera.dist
python312Packages.dirigera
python312Packages.dirigera.dist


Consider using icon webcomponents

Hi, I see this package is wrapping the svgs in vue components using generate.js. While it works, this script will need to be re-run every time a new version of @itwin/itwinui-icons is published.
There is a webcomponents icons package called @itwin/itwinui-icons-elements, which you might consider using instead of creating vue wrappers. The only reason we use React wrappers is the poor support for webcomponents in React. But Vue has good support for webcomponents so you should be good.
Cheers!

Thanks, I'll try it.


feat: added new package @instana/opentelemetry-sampler

refs 99429
Samplers are not supported in the contrib repo of Otel.
The docs say that samplers need to be outside of their main repo.

[ ] replace NPM reference for propagator (currently a local tgz, build is red)

The propagator has been released now: https://www.npmjs.com/package/@opentelemetry/propagator-instana

I think this needs to be rebased? For some reason, parts of the work on the GH action appear in the diff. 😕 🤷

Rebased and updated NPM package reference 👍


feat: add refresh address button for qr code

Description
This PR addresses issue #431

Add Refresh Address button to receive on-chain dialog
Add loading and error handling during refresh (using previously exisiting SetStateAction functions)
Add related unit tests
Add translations (to the best of my ability, happy to remove if this is not needed)

Screen Recording
This is a short demo of the resulting behaviour (with an overridden response, given that the mock backend seems to return the same address from the new-address endpoint).

Awesome, thank you very much!


License

Hi Daniel,
I was just wondering under which license the code of this repository is published?
Best,
René

I have added a LICENSE-MIT file.
Note that this code is only for research purposes. It is an unfinished project and there might be very stupid things in there.


HTML Text Formatting

Description 📜
I am creating issue for types of text formatting in HTML

Domain of Contribution 📊

[x] Frontend Dev HTML/CSS/JS

/assign

/assign


General: Ignore decode error of stdout/stderr in run_subprocess

Brief description
Ignore decode errors and replace invalid character (byte) with escaped byte character.
Description
Calling of run_subprocess may cause crashes if output contains some unicode character which (for example Polish name of encoder handler).
Testing notes:
Issue was discovered by a client so testing notes are related to the case.

Open TrayPublisher
Select family (e.g. Render)
Drop video from client to both (source and reveiw) input fields
Publish
Publishing should went smoothly without issues in ExtractReview plugin

Now it crashes on Extract burnins.

Now it crashes on Extract burnins.

Should be fixed

Maybe it should do de decoding on the args and kwargs of the subprocess?

It's not issue with args but with output from process.

still crashing on burnins
publish-report.txt

still crashing on burnins

Forgot tu push one commit


Scrolling after update weirdly slow (and high cpu consumption)

Description
I created a video showing the lagginess of the scroll and the high cpu usage
Video: https://imgur.com/o8EJqxO
DBeaver Version
<IP_ADDRESS>309041200
Operating System
macOS Sonoma 14.0 (23A339)
Database and driver
No response
Steps to reproduce
No response
Additional context
No response

Your video is unavailable. Could you attach it to the github comment directly?

https://github.com/dbeaver/dbeaver/assets/686843/19829e4a-5bc4-4192-8d25-566aa2bd8cdc
Sorry didn't knew that was possible to upload videos here

Thanks

Haven't you updated you mac recently up to 14.0 version?

Yes, I updated DBeaver at the same time, maybe that is the problem.

We don't have macOS Sonoma 14.0 to reproduce this issue, but we couldn't reproduce it on the older mac versions.

Hi, the issue persists, macOS Sonoma 14.0 is now live

Thank you for update

I can confirm the issue after updating to Sonoma.  When scrolling a large number of tables there is noticeable lag and high CPU usage.  My symptoms are much worse than what is shown in video.  M1/16Gb machine.

Same Here , when scrolling a large number of package in macOS Sonoma.

Same here. I've tried to downgrade to 3 months version back. it's still occur. It seems the sonoma issues.
I can confirm this (Sonoma 14.0 with Intel last macbook generation, 2020)

I'm having this issue too after updating to macOS Sonoma. It happens in the database navigator, but not script editor/result panes. Autocomplete is also extremely slow. 2019 Intel MacBook Pro.

I'm seeing the same thing just after update to Sonoma. It seems to be related to the size (number of entries) in the navigator -- I have a DB connection with thousands of schemas and it's painful to navigate. And while most operations in the SQL editor are fine, I do see a delay the first time that I engage auto-complete (i.e. when I type SELECT * FROM A and it's trying to find the schemas starting with A).

I'm seeing the same thing just after update to Sonoma. It seems to be related to the size (number of entries) in the navigator -- I have a DB connection with thousands of schemas and it's painful to navigate. And while most operations in the SQL editor are fine, I do see a delay the first time that I engage auto-complete (i.e. when I type SELECT * FROM A and it's trying to find the schemas starting with A).
This is also on a 2019 Intel MacBook Pro with Sonoma 14.0.

I noticed the same thing with the number of entries. Just to make sure there wasn't an issue with my configuration, I renamed the "DBeaverData" folder in my Library to "DBeaverData-old" then I added my main database back. At first I thought things were better, but as soon as I started drilling down in the database navigator (database, schema, tables, procedures...) it just gets slower and slower.

Same issue after updating to macOS Sonoma. Navigation in database is very slow and difficult.
Tested on MacBook Pro 2019 - Intel.

Hello, if it helps, I'm experiencing this but not on macOS Sonoma. I wonder if this is a general issue that occurred with a DBeaver update?
Here's a short video showing how at idle DBeaver consumes negligible CPU, but resizing the window the CPU quickly jumps to near 100%, and scrolling a table the CPU also stays level at near 100%.
https://github.com/dbeaver/dbeaver/assets/392214/a30e70de-6106-4f02-91b4-84f0f81ba8e2
I'm using DBeaver  <IP_ADDRESS>310081633 and running macOS Ventura (13.6)
Stab in the dark, perhaps DBeaver is doing UI rendering on the CPU when it should be using the GPU..

Also having this issue. Apple M1 Pro, 16 GB, macOS 14.0. It just started this week, immediately after updating to 14.0. Gonna have to switch clients if it keeps up :(

Tried switching to Oracle JDK 17, still the same problem, Oracle JDK 21, also the same problem.
This issue makes DBeaver on Macos a pain to use.

I've switched to VS Code. The Snowflake extension works perfectly well, and there's zero lag. The cutover took me less than 10 minutes, would recommend (even if only as a temporary fix) until someone can track down and resolve this issue.

Another fact: Tried DBeaver Ultimate Edition 23.2 (trial) and it works perfectly, the CPU consumption gets a little high when scrolling but there is no lag visible. Is 23.2 Ultimate maybe based on an even older Community Branch?

I am experiencing this as well on macOS Sonoma, 2019 Mac Pro with Intel chip.

Also having this issue. macOS Sonoma M1 Max.

As a workaround, unchecking User Interface -> Navigator -> Show statistics info makes it much better for me

@giovanni-landolina thanks for pointing that out. It can greatly assist in our research.

I started experiencing this same issue here after upgrading DBeaver CE and macOS around the same time, about a month ago.
Sonoma 14.1.1 - 2.3Ghz Intel Quad-Core i7
Only seems to affect my "Database Navigator" menu: extreme lag while scrolling up and down, gets worse as more database sub-groups are expanded, before that recent update, scrolling this same Navigator list was smooth and lag-free.

So,

With <IP_ADDRESS>312122044 the issue still persists on my side.

So, very strange, today it seems to be working fine. Different from my last comment, I'm now on DBeaver <IP_ADDRESS>312031432 and macOS Sonoma 14.2. Whatever it is with that combination, I have happy scrolling again!

Can confirm, it looks that it was a weird Sonoma problem after all.

With <IP_ADDRESS>312122044 the issue still persists on my side. MacOS: 14.1.1 (23B81). Updating to 14.2 and will report.

Confirming that with MacOS 14.2 and the latest version of DBeaver things are fixed.

It's a good news! Thank you!

With <IP_ADDRESS>312122044 the issue still persists on my side. MacOS: 14.1.1 (23B81). Updating to 14.2 and will report.

Confirming that with MacOS 14.2 and the latest version of DBeaver things are fixed.

Confirm MacOS 14.2 are fixed.