Request to change behavior of Soundcloud permalink extractor

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm requesting a site-specific feature
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
United States
Example URLs
https://w.soundcloud.com/player/?url=https://api.soundcloud.com/users/164828530
Provide a description that is worded well enough to be understood
Verbose log uses a "user:permalink" which is based on the artist ID:
https://w.soundcloud.com/player/?url=https://api.soundcloud.com/users/164828530
When you open this link in a browser, it shows just the artist's tracks, as if you added /tracks to the end of the normal user URL like so:
https://soundcloud.com/rabidzen/tracks
However, in the verbose log, downloading this permalink begins downloading the artist's entire page, including reposts of songs by other artists, as well as playlists, as if you had not added /tracks to the end of the URL (i.e. equivalent behavior to downloading just https://soundcloud.com/rabidzen). My opinion is that yt-dlp should treat this permalink as equivalent to URL with /tracks, not the base artist URL. Or provide some way to choose between the two behaviors.
I've been changing this myself in a forked version: https://github.com/DunnesH/yt-dlp/commit/aaac7f42dc226201375ffb8b6218e2d48a075c9e. I just wanted to create an issue here so potentially this could be changed in the main project. I'm not experienced in github or programming so I apologize for overcomplicating this.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', 'https://w.soundcloud.com/player/?url=https://api.soundcloud.com/users/164828530']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [12b248ce6] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.2.3
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.2, sqlite3-3.35.5, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1820 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[SoundcloudEmbed] Extracting URL: https://w.soundcloud.com/player/?url=https://api.soundcloud.com/users/164828530
[debug] Loading soundcloud.client_id from cache
[soundcloud:user:permalink] Extracting URL: https://api.soundcloud.com/users/164828530
[soundcloud:user:permalink] 164828530: Downloading user info
[download] Downloading playlist: rabidZen
[soundcloud:user:permalink] 164828530: Downloading track page 1
[soundcloud:user:permalink] 164828530: Downloading track page 2
[soundcloud:user:permalink] Playlist rabidZen: Downloading 123 items of 123
[download] Downloading item 1 of 123
[debug] Loading soundcloud.client_id from cache
[soundcloud] Extracting URL: https://soundcloud.com/upscale-recordings/entity
[soundcloud] upscale-recordings/entity: Downloading info JSON
[soundcloud] 809680825: Downloading hls_mp3 format info JSON
[soundcloud] 809680825: Downloading http_mp3 format info JSON
[soundcloud] 809680825: Downloading hls_opus format info JSON
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 809680825: Downloading 1 format(s): hls_opus_64
[debug] Invoking hlsnative downloader on "https://cf-hls-opus-media.sndcdn.com/playlist/5cc6db95-2840-45e9-9ea2-3bdda412773b.64.opus/playlist.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1vcHVzLW1lZGlhLnNuZGNkbi5jb20vcGxheWxpc3QvNWNjNmRiOTUtMjg0MC00NWU5LTllYTItM2JkZGE0MTI3NzNiLjY0Lm9wdXMvcGxheWxpc3QubTN1OCoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3MTkxMDQ1MTd9fX1dfQ__&Signature=H~dKjkDvAnsvHW8004O9qHHOBm9AG2CQbo-8SFp9tnXj3LuE6soIcioZfwrxGysxbsr6JWixeZ00StA8G2KKJ-i9DpWSimooojRl5BElEIRJP7k53AJ1kgIh94tBXNDcH0KszW-u1BuZ4cOpEkcR0pksd0tbDJx2XWvYPkDqaOoTuo7-H4w~Zo0N~ebmnczzvxRYh0t54MtGOvVnpFHaNz53mFzl9xY421m2jsLh8UQnmDseF~-hF0RKKkimCefKV4~nRk3AF4tV4Mf56e437UxZ2eR0kwT2uTVBWH8pAMa5Ui2XQd139oIJIw9cY74J4KTstZ712-H8Oir5tpAWpw__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 31
[download] Destination: Echo Map - Entity [809680825].opus
[debug] File locking is not supported. Proceeding without locking
[download] 100% of    2.33MiB in 00:00:02 at 1.03MiB/s
[download] Downloading item 2 of 123
[soundcloud] Extracting URL: https://soundcloud.com/upscale-recordings/echo-map-awoken
[soundcloud] upscale-recordings/echo-map-awoken: Downloading info JSON
[soundcloud] 804743230: Downloading hls_mp3 format info JSON
[soundcloud] 804743230: Downloading http_mp3 format info JSON
[soundcloud] 804743230: Downloading hls_opus format info JSON
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 804743230: Downloading 1 format(s): hls_opus_64
[debug] Invoking hlsnative downloader on "https://cf-hls-opus-media.sndcdn.com/playlist/6d967bdd-2582-4a64-bc59-079e17b95f38.64.opus/playlist.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1vcHVzLW1lZGlhLnNuZGNkbi5jb20vcGxheWxpc3QvNmQ5NjdiZGQtMjU4Mi00YTY0LWJjNTktMDc5ZTE3Yjk1ZjM4LjY0Lm9wdXMvcGxheWxpc3QubTN1OCoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3MTkxMDQ1MjB9fX1dfQ__&Signature=Amt5cJ5s5uB~zgNDLgMsOdGFUwbCWc7TZRHTDM3gqS8h02ff0JxO-XX1pTU8lJ9yrXgdFop0MrOOCxFG2GzYXuTw8JjR3h6Xxxod8IqucsvkKfLIhooHajFRdtTjF78BfN3cWjmV-BCPap0qvERJPxxbTgv0kAEADkjsg5ZVhJJuCRMRzlah417KtVp-3iE1IxcpvXNlPSH33NV9MxmF4Eeuklhs8lts9IZqJj-3DS3xfYJ3UmCUBQf7hlzKvGCZkl1wN891ROjvL3eQ7AAj1t9-QxZKDpL7OglsgWS9wDQfwhGGif9AnsE2Fi6Tr-JtNSrwrzVGdcnf-t2iOlzDYw__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 31
[download] Destination: Echo Map - Awoken [804743230].opus
[download] 100% of    2.27MiB in 00:00:02 at 1.11MiB/s
[download] Downloading item 3 of 123
[soundcloud] Extracting URL: https://soundcloud.com/nightowlcollective/lapis-lazuli
[soundcloud] nightowlcollective/lapis-lazuli: Downloading info JSON
[soundcloud] 804072046: Downloading hls_mp3 format info JSON
[soundcloud] 804072046: Downloading http_mp3 format info JSON
[soundcloud] 804072046: Downloading hls_opus format info JSON
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 804072046: Downloading 1 format(s): hls_opus_64
[debug] Invoking hlsnative downloader on "https://cf-hls-opus-media.sndcdn.com/playlist/0a34b574-c79e-471c-8b0b-3551108c9a91.64.opus/playlist.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1vcHVzLW1lZGlhLnNuZGNkbi5jb20vcGxheWxpc3QvMGEzNGI1NzQtYzc5ZS00NzFjLThiMGItMzU1MTEwOGM5YTkxLjY0Lm9wdXMvcGxheWxpc3QubTN1OCoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3MTkxMDQ1Mzh9fX1dfQ__&Signature=bsQ-VK-apIMf4h4DEWVac88Am6IFSsOTMcht0mpBp3hmalXjWifcZzTIlq0cskUhphW0UdaBOEtSWTqnf21a-4VivYeFTby3hAvyYM2l3goZ6rp94c2HDs7GlMXKALFqfultsXPiq-axwX33pGrYAnQsTeP1NwWsRW29yCARcvbmQ-h~F50R0bsOEMxNIwhCybWk2BB78qIjSMec2D8Mu1SamU-HXzVj7nqYRp0wgKi4BBQ63FMOQMSqbsM6l6QVGmklSsyur1B66VslxkELnvih4ogyFuRTtpuF5bfV7VSGPya9miCsNB0v3aXfbL17TqqxODLF1TVJPp4ko6-O7w__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 32
[download] Destination: rabidZen & White Zoo - Lapis Lazuli [804072046].opus
[download] 100% of    2.33MiB in 00:00:02 at 1.12MiB/s
[download] Downloading item 4 of 123
[soundcloud] Extracting URL: https://soundcloud.com/nightowlcollective/complexive-alone
[soundcloud] nightowlcollective/complexive-alone: Downloading info JSON
[soundcloud] 796901368: Downloading hls_mp3 format info JSON
[soundcloud] 796901368: Downloading http_mp3 format info JSON
[soundcloud] 796901368: Downloading hls_opus format info JSON
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 796901368: Downloading 1 format(s): hls_opus_64
[debug] Invoking hlsnative downloader on "https://cf-hls-opus-media.sndcdn.com/playlist/abe75bb5-134f-474a-a5c7-1a594876efce.64.opus/playlist.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiKjovL2NmLWhscy1vcHVzLW1lZGlhLnNuZGNkbi5jb20vcGxheWxpc3QvYWJlNzViYjUtMTM0Zi00NzRhLWE1YzctMWE1OTQ4NzZlZmNlLjY0Lm9wdXMvcGxheWxpc3QubTN1OCoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3MTkxMDQ0NDh9fX1dfQ__&Signature=Dcz3mEV4MjC4FB5skuK9o-oBerJ2eQXt9hRXqXkX86lK66gyjIbv7Rr1pqyVhDdT8dQUuBdFcwGtf40MZikq-v9QsVVPI-FdSis-019rwz8WCFJURnu~u6cVdkuH~NKiK1HtAbykfYmuw-9J~ZxrNuTItqpPuZImSwl5TqNxNMIulOkeYXOTPhVkSrdjfWKZ21onjkci9t9N3MLaFhIkVJNveTUtPadCYdg21xa5awqceB5jUlLWuj5O-otFnIllufhcAkyVR~Lc8sePz6oIrRYcWvW1gahL3Oloq-O2tAoH12-~R~MtjKAecuSS8C53sAo19PGt-RG0s7hM~Uxs-w__&Key-Pair-Id=APKAI6TU7MMXM5DG6EPQ"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 23
[download] Destination: Complexive - Alone [796901368].opus
[download]   5.5% of ~ 796.79KiB at   36.87KiB/s ETA Unknown (frag 2/23)
ERROR: Interrupted by user

I think this is a good change that will result in more-expected behavior

As the original contributor of the feature (#5842), bringing this change in as-is would be fine with me.
If I recall correctly the PR was designed to match the existing behavior of the soundcloud:user extractor, where a bare URL resolves to all of the user's activity.  I think originally I wanted to cover all the endpoints SoundcloudUserIE does, but I ended up not doing so because of the larger refactoring work involved and test coverage requirements.


Unsupported operation: Infinity or NaN toInt

The following error was reported by an Android app (currently in internal testing) using Flutter 2.0.1 and Dart 2.0.12, release build with null-safety:
UnsupportedError: Unsupported operation: Infinity or NaN toInt
  File "double.dart", line 219, in _Double.toInt
  File "semantics.dart", line 2328, in _BoxEdge.compareTo
  File "comparable.dart", line 88, in Comparable.compare
  File "list.dart", line 359, in ListMixin._compareAny
  File "sort.dart", line 69, in Sort._insertionSort
  File "sort.dart", line 58, in Sort._doSort
  File "sort.dart", line 33, in Sort.sort
  File "list.dart", line 355, in ListMixin.sort
  File "semantics.dart", line 2523, in _childrenInDefaultOrder
  File "semantics.dart", line 2117, in SemanticsNode._childrenInTraversalOrder
  File "semantics.dart", line 2055, in SemanticsNode._addToUpdate
  File "semantics.dart", line 2653, in SemanticsOwner.sendSemanticsUpdate
  File "object.dart", line 1083, in PipelineOwner.flushSemantics
  File "binding.dart", line 458, in RendererBinding.drawFrame
  File "binding.dart", line 874, in WidgetsBinding.drawFrame
  File "binding.dart", line 319, in RendererBinding._handlePersistentFrameCallback
  File "binding.dart", line 1144, in SchedulerBinding._invokeFrameCallback
  File "binding.dart", line 1082, in SchedulerBinding.handleDrawFrame
  File "binding.dart", line 998, in SchedulerBinding._handleDrawFrame
  File "zone.dart", line 1354, in _rootRun
  File "zone.dart", line 1258, in _CustomZone.run
  File "zone.dart", line 1162, in _CustomZone.runGuarded
  File "hooks.dart", line 161, in _invoke
  File "platform_dispatcher.dart", line 253, in PlatformDispatcher._drawFrame
  File "hooks.dart", line 120, in _drawFrame

After a short look into the Flutter source code it looks like a _BoxEdge was constructed with an invalid offset -- Unfortunately the stack trace does not tell us, where the invalid _BoxEdge instance has been created.

Hi @MichaelTamm
is there a way to reproduce this locally?
Can you please provide your flutter doctor -v and flutter run -release --verbose logs, and a minimal complete reproducible code sample?
Thank you

Output of flutter doctor -v:
[✓] Flutter (Channel stable, 2.0.1, on Linux, locale de_DE.UTF-8)
    • Flutter version 2.0.1 at /home/Michael/flutter.stable
    • Framework revision c5a4b4029c (vor 7 Tagen), 2021-03-04 09:47:48 -0800
    • Engine revision 40441def69
    • Dart version 2.12.0

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /home/Michael/Android/Sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /usr/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_282-b08)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).

[✓] VS Code (version 1.54.1)
    • VS Code at /usr/share/code
    • Flutter extension version 3.20.0

[✓] Connected device (2 available)
    • SM G930F (mobile) • ce041824782daa3c02 • android-arm64  • Android 8.0.0 (API 26)
    • Chrome (web)      • chrome             • web-javascript • Google Chrome 89.0.4389.82

Note: I have Android Studio installed via Jetbrains Toolbox, but that's another issue (https://github.com/flutter/flutter/issues/75590)

I would really like to provide more help for the triage. Right now I cannot run the Flutter App in release mode:
$ flutter run --release --flavor local
Launching lib/main.dart on SM G930F in release mode...

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:lintVitalLocalRelease'.
> Could not resolve all artifacts for configuration ':app:prodReleaseRuntimeClasspath'.
   > Failed to transform libs.jar to match attributes {artifactType=processed-jar, org.gradle.libraryelements=jar, org.gradle.usage=java-runtime}.
      > Execution failed for JetifyTransform: /home/Michael/src/autovio-dev/build/app/intermediates/flutter/prodRelease/libs.jar.
         > Transform's input file does not exist: /home/Michael/src/autovio-dev/build/app/intermediates/flutter/prodRelease/libs.jar. (See https://issuetracker.google.com/issues/158753935)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 55s
Running Gradle task 'assembleLocalRelease'...
Running Gradle task 'assembleLocalRelease'... Done                 56,4s
Exception: Gradle task assembleLocalRelease failed with exit code 1

In debug mode everything works fine and there aren't any assertion errors.

The device, from which the Exception orginated was a Nexus 5X (Model: MTC20F) running Android 6.0.1

/cc @HansMuller

I faced this problem too. In the Crashlytics log, I see such a trace, but it is not clear from it where the problem begins:
Non-fatal Exception: io.flutter.plugins.firebase.crashlytics.FlutterError: Unsupported operation: Infinity or NaN toInt. Error thrown Instance of 'ErrorDescription'.
       at _Double.round(_Double.java)
       at RenderSliverFixedExtentBoxAdaptor.getMinChildIndexForScrollOffset(RenderSliverFixedExtentBoxAdaptor.java:70)
       at RenderSliverFixedExtentBoxAdaptor.performLayout(RenderSliverFixedExtentBoxAdaptor.java:192)
       at RenderObject.layout(RenderObject.java:1858)
       at RenderSliverEdgeInsetsPadding.performLayout(RenderSliverEdgeInsetsPadding.java:137)
       at _RenderSliverFractionalPadding.performLayout(_RenderSliverFractionalPadding.java:167)
       at RenderObject.layout(RenderObject.java:1858)
       at RenderViewportBase.layoutChildSequence(RenderViewportBase.java:510)
       at RenderViewport._attemptLayout(RenderViewport.java:1580)
       at RenderViewport.performLayout(RenderViewport.java:1489)
       at RenderObject._layoutWithoutResize(RenderObject.java:1713)
       at PipelineOwner.flushLayout(PipelineOwner.java:885)
       at RendererBinding.drawFrame(RendererBinding.java:453)
       at WidgetsBinding.drawFrame(WidgetsBinding.java:883)
       at RendererBinding._handlePersistentFrameCallback(RendererBinding.java:319)
       at SchedulerBinding._invokeFrameCallback(SchedulerBinding.java:1143)
       at SchedulerBinding.handleDrawFrame(SchedulerBinding.java:1080)
       at SchedulerBinding._handleDrawFrame(SchedulerBinding.java:996)

Without information of how to reproduce this crash we can't really debug this.


indication that user is logged in is missing

We miss any indication that I am logged in under my account (username, icon, name of the depositor, orcid number) as a personal site bottom.

Yes, will be implemented.

This was fixed in #223


properly set transaction outcome

What does this pull request do?
properly set transaction outcome for flask instrumentation when an exception occurs

Related issues
Closes #1636

And thank you for taking a look at this so quickly!

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-19T18:07:05.165+0000

Duration: 48 min 1 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
4986

Skipped
3338

Total
8324

:green_heart: Flaky test report
Tests succeeded.
:robot: GitHub comments
Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

/test linters : Run the Python linters only.

/test full : Run the full matrix of tests.

/test benchmark : Run the APM Agent Python benchmarks tests.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

:globe_with_meridians: Coverage report

Name
Metrics % (covered/total)
Diff

Packages
100.0% (66/66)
:green_heart:

Files
100.0% (225/225)
:green_heart:

Classes
100.0% (225/225)
:green_heart:

Lines
89.808% (17473/19456)
:-1: -0.004

Conditionals
77.144% (3193/4139)
:+1:

/test linters


fix(mux): Cannot move across mux panes when wrap_at_edge and both panes are running nvim

Fixes #74

@rahulaggarwal965 can you please give this a thorough test?


Problems with kbdinfo

I am currently trying to test whether caps lock is pressed (because vim does weird things in caps lock which I'm always confused of). Since I don't have any x server running, I found that kbdinfo can check whether caps lock is enabled:

My Windows build number: Microsoft Windows [Version 10.0.19041.264]

Install the kbd package under wsl (I am running Linux DESKTOP-0IB64NU 4.19.84-microsoft-standard #1 SMP Wed Nov 13 11:44:37 UTC 2019 x86_64 GNU/Linux).
Run sudo kbdinfo gkbled. Should return scrolllock:off numlock:off capslock:off
Now retype the command and activate at least one of scroll lock, numlock or caps lock and hit enter. It still returns scrolllock:off numlock:off capslock:off.

What's wrong / what should be happening instead:
kbdinfo should return the enabled locks correctly.

Strace of the failing command, if applicable:  https://pastebin.com/0iduYDCm

Does anyone know of a different way to check whether caps is enabled in wsl?
Thanks for any help in advance :)

Since I don't have any x server running

WSL (both 1 and 2) does not have a keyboard device, pretty much by-design. /dupe #1806 ref #2730. You are going to be better off drilling-down "my vim does weird things" in a forum like serverfault. TLDR If you really (really) want to query the caps lock state you need something like X11 or Wayland.


finish an example file for netlist to graph using graph_tool

TODO

[ ] requires  kf.KCell.netlist()

fixes #154
@Hsuan-Tung

@joamatab I guess this is dead?

yes, feel free to close


ONNX: Cannot load models with complex data initializers

Netron app and version: 5.7.4 Desktop app, 5.7.4 Website
OS and browser version: Windows 10.0.19042 Build 19042 + Edge 100.0.1185.50 (Official build) (64-bit)

Steps to Reproduce:

Load a model containing complex data initializer(s)
An error is thrown: Error loading ONNX model. Unsupported tensor data type '14' in 'model.onnx'.

I believe this error is thrown here: https://github.com/lutzroeder/netron/blob/9a49e86a9a3989e3a92dbca117a8e1728373e5ee/source/onnx.js#L853
There seems to be no switch cases for complex64 and complex128 datatypes. Unfortunately I cannot share the model I'm using while getting this error, but before the line of code linked above was added I was able load the model successfully (though if I recall correctly I couldn't view complex data initalizers)

complex_init.zip
Here's a small onnx model which should reproduce the error

complex_init.onnx.zip


Named placeholders

If it's not too difficult, please consider to add named placeholders.
example:
SELECT  id FROM location WHERE
facility_name LIKE :search
OR address LIKE :search
OR address2 LIKE :search
OR city LIKE :search
OR state LIKE :search
OR zip LIKE :search
OR rep_fname LIKE :search

I'm not sure that it's simple, but it's definitely a desired feature.
I'll try to find time for this.

Implemented in v5.0.0

wow, cool! thank you!


Migrate database and file-based tutorials into module template form

Purpose

Migrates the following tutorials into module template form, so that they can be pushed to Ballerina Central.

Database Integrations

data_backed_service
managing_database_transactions
querying_mysql_database
sending_a_message_to_datasource
insert_csv_into_mongodb

File-Based Integrations

file_integration_using_ftp
file_integration_using_smb

Related issue: https://github.com/wso2/ballerina-integrator/issues/411

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Added location controller and tests

In this PR, we add an controller that uses the /api/locations endpoint to connect to location query service
also tested using dev server
Closes #9

@Tommygithubaccount123 Fixed doc description and summary according to swagger UI


Issue: Silent Failure When Creating Organization with Invalid Characters

Issue: Silent Failure When Creating Organization with Invalid Characters
Steps to Reproduce:

Sign up with GitHub and navigate to the "Let's get started" page (https://polar.sh/dashboard/create).
Enter an organization name with apostrophes ("Emmanuel's Org").
Select all options in the section below.
Click "Create".

Expected Behavior:

User should receive immediate feedback about invalid characters in the organization name.
Error message should be displayed on the page.

Actual Behavior:

The "Create" button becomes disabled without any visible feedback.
No error message is shown on the page.
Error is only visible in browser developer tools (Network and Console tabs).

Error Details:
Console Error:
Failed to load resource: the server responded with a status of 422 ()
POST https://api.polar.sh/v1/organizations/ 422 (Unprocessable Content)

Network Tab Response:
{
    "error": "RequestValidationError",
    "detail": [
        {
            "type": "value_error",
            "loc": [
                "body",
                "slug"
            ],
            "msg": "Value error, The slug can only contain ASCII letters, numbers and hyphens.",
            "input": "emmanuel's-org",
            "ctx": {
                "error": {}
            }
        }
    ]
}

Suggested Improvement:
Add a validation message or error status display below the organization name input field to provide immediate feedback to users about invalid characters.
Browser/OS Information:
Microsoft Edge 127.0.2651.98 (Official build) (64-bit)
Windows 10 Version 22H2 (Build 19045.4651)
Screenshots:

Thank you for the super-detailed bug report, @Jemeni11. I'll take care of this rapidly 🙂

The fix is live 👍 I improved the slug generation/validation on client's side so it matches server's expectations + it'll display the server error message in case something is wrong.


Remove Garrison reference from code

Reference https://gerrit.openbmc-project.xyz/#/c/10964/

https://gerrit.openbmc-project.xyz/#/c/10965/

This one too as per discussion with @gtmills


2,823 tokens/s seems extremely high!

Hey,
I think the tokens/s calculation might be incorrect. I can see that you are computing time by timing the CPU clock here: https://github.com/likejazz/llama3.cuda/blob/master/llama3.cu#L789
This might result in an incorrect number because the actual code is running on the GPU and the CPU is just dispatching the kernel. Hence, I suspect that the time that you are getting is CPU execution time of dispatching the kernel.
The correct way will be using cuda events.
Reference: https://developer.nvidia.com/blog/how-implement-performance-metrics-cuda-cc/

@romitjain oh, thank you for the clarification. could you please send me a patch?


Add shutdownhook to make MoP shutdown gracefully

Motivation
As https://github.com/streamnative/mop/issues/484 mentioned, if user kill MoP server, we need to close mop gracefully.

here


PROJECTLIST

This is a list of projects to be written/edited on the website.  It is a live list.
Topic - Contact - To be done

OSS-SCM - Vineet - Project description, how it's going, what's to come from it
IronHacks - Elizabeth - CHECK Project description, note new competition, what's to come from it
RAPID - Bryan - Project description, how it's going, what's to come from it

Split the issue out


macOS build error (Collecting native-compiler info with '/usr/bin/cc -v ' failed)

Things to check before reporting a bug

[x] I have searched GraalVM GitHub repository and could not find a similar issue
[x] I have searched Spring experimental GraalVM Native project and could not find a similar issue
[x] I have attached a sample project so that the issue can be reproduced on other computers

Describe the bug
build failed with:
Error: Collecting native-compiler info with '/usr/bin/cc -v ' failed
Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
com.oracle.svm.core.util.UserError$UserException: Collecting native-compiler info with '/usr/bin/cc -v ' failed
To prevent native-toolchain checking provide command-line option -H:-CheckToolchain

To Reproduce
./gradlew buildNativeImage

Expected behavior
image should be built
Screenshots

Desktop (please complete the following information):

OS: macOS
Version 11.0.1

I believe this is a duplicate of #33
I will work on it (when I have time)

@vitalyster could you do this with --stacktrace --info and print log here? It depends on the exact error

disableToolchainChecking   = true configuration option give me the same result as in #33

1
On Wed, 9 Dec 2020, 10:27 vitalyster<EMAIL_ADDRESS>wrote:

disableToolchainChecking = true configuration option give me the same
result as in #33
https://github.com/ayltai/spring-graalvm-native-plugin/issues/33
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ayltai/spring-graalvm-native-plugin/issues/44#issuecomment-741648168,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFMVSYS2A3NSYR7ECYK4TH3ST47I7ANCNFSM4UTEZYBA
.


google search beta

Not perfect but I want to get this out so people can begin working on it and providing feedback.

how does this work?

@mckaywrigley @rossman22590 It looks like this is enable via const [showPluginSelect, setShowPluginSelect] = useState(false); and is hardcoded to false, so the Plugin dropdown doesn't show.
I have a fix for this here: https://github.com/mckaywrigley/chatbot-ui/pull/477


Emit chat events for non-translated messages to improve compatibility

The 'chat' event is emitted only when the chat.type.text translation is present (on modern server versions, in parseJsonMessage7) but this key is not used on all server software. Instead (tested on Glowstone 1c8634, Spigot/CraftBukkit 1.7.9-R0.1), the player name and text are embedded in the same string:
chat packet { id: 2,
  message: '{"extra":["\\u003cplayer\\u003e foobar"],"text":""}' }
parseJsonMessage { extra: [ '<player> foobar' ], text: '' }
parseJsonMessage7 { extra: [ '<player> foobar' ], text: '' }

mineflayer parseOldMessage can already parse this string format, but it isn't called (despite the "used by minecraft <= 1.6.1 and craftbukkit >= 1.6.2" comment) since the JSON message doesn't have translate. This PR handles this case, calling parseOldMessage from parseJsonMessage7 when needed.
Fixes the chat examples on Glowstone/Spigot/CraftBukkit in my testing (used examples/echo.js).
ref https://github.com/andrewrk/mineflayer/issues/175

Added this as part of https://github.com/andrewrk/mineflayer/pull/195


How can I find the pre-trained model "cats_and_dogs_small_2.h5" ?

Sir, how can I find the pre-trained model "cats_and_dogs_small_2.h5" mentioned by 5.4-visualizing-what-convnets-learn.ipynb ? Thank you!

The chapter assumes you're running it from beginning to end. You should execute all the cells in https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/5.2-using-convnets-with-small-datasets.ipynb

@morenoh149 Thank you for informing. I've already tried as 5.2-using-convnets-with-small-datasets.ipynb and gotten the right *.h5 file needed for next action.

please close the issue if it is solved @wizephen


new where clause functions

hi @izniburak
how about having clean whereNull and whereNotNull

  public function whereNull($where)
  {
    $where = $where . ' is null';
    if (is_null($this->where))
      $this->where = $where;
    else
      $this->where = $this->where . ' ' . 'and ' . $where;
    return $this;
  }

// will produce ``` where tbl.colmn is null```

  public function whereNotNull($where)
  {
    $where = $where . ' is not null';
    if (is_null($this->where))
      $this->where = $where;
    else
      $this->where = $this->where . ' ' . 'and ' . $where;
    return $this;
  }

// will produce ``` where tbl.colmn is not null```

Actually, I really missed these functions in my last project, so i had to extend your class.
Thanks !

Why is it closed if it's not implemented? Current where doesn't offer any IS NULL support. I'd argue that such functionality should be implemented in current where.


Fix feedback display

Short description

This PR fixes HIX feedback display in the page form.
Proposed changes

For bug A: check whether the HIX feedback is empty so feedbackCount does not go up
For bug B: accept the old format and recast feedbackResult in the case.

Side effects

I hope none 🙈

Resolved issues

Fixes: #2877

Pull Request Review Guidelines

Thanks, this fixes the issue.
Approve since this is prio:high, but I don't really like that our database store two different versions of hixFeedback currently. I think the more robust way is to create migrations when we change the data that is stored in the db. Then there would be no need to handle different versions of hix feedback in the code. Do you think this is worth opening an issue for?

Opened an issue before forgetting :) #2881 Feel free to modify the issue


Multiple files in a stream

I find one of the behaviours that has crept in to some of the modules (notably normalization) suboptimal.
This is that some streams - notably the structural streams, and things to do with BET - have started passing many files with different purposes. So, for example, the structural stream contains both the non-normalized (mms_) and normalized (w_) image. And the segmentation images (in some versions of the code) are output in a single "anatomical" stream.
The problem with this is that it means a module that receives this either has to do file filtering - so assuming something about the filename structure of the inputs - or rely upon the order of the files, which is opaque and could easily break.
I would suggest we revert to having all of the files within a stream being in some sense exchangeable, and use different streams to represent different kinds of data. It comes with a small extra programming overhead, but I think will save pain in the long run.
What do you all think?

Discussion May 9, 2018 (!) we agreed that streams should not contain multiple kinds of information. Specifically:

"structural" should not change to something different after normalization (just like epi can be not normalized, or normalized)

norm_noss modules may combine different segmentations in the "segmentation" stream which should be changed

@jpeelle, I thought we decided to drop norm_noss
Following #159, here is the list of modules and thir current behaviours:

checking (size(Simg,1) > 1) and setting according to settings.structural
./aamod_ANTS_build_template.m
./aamod_ANTS_struc2epi.m
./aamod_ANTS_struc2template.m
./aamod_LoAd.m
./aamod_bet.m
./aamod_bet_freesurfer.m
./aamod_bet_premask.m
./aamod_coreg_extended.m
./aamod_coreg_extended_1.m
./aamod_freesurfer_initialise.m
./aamod_fsl_FAST.m
./aamod_fsl_FIRST.m
./aamod_fsl_robustFOV.m

checking (size(Simg,1) > 1) and setting according to prefix (w, b)
./aamod_mask_fromstruct.m - prefix match (w, b)
./aamod_structural_overlay.m - prefix match (w)

checking (size(Simg,1) > 1) and using first (hard-coded)
./aamod_coreg_extended_2.m
./aamod_norm_noss.m
./aamod_pewarp_estimate.m

processing all according to settings.structural
./aamod_biascorrect.m
./aamod_biascorrect_ANTS.m
./aamod_denoiseANLM.m

I am happy to edit all to omit any checking and settings and make them expect only one file.

I believe we did decide to drop norm_noss.


Allow nexus configuration from files

To make life a bit easier, in particular during testing, we want to ability to load or bootstrap mayastor with a given, specific config file.

I left two comments, one resolved.
I don't know where/how dump_info_json() and config_json() get called as I don't understand the architecture well enough, but otherwise it looks reasonable AFAIUI.


[FLINK-14904][table] Rename ANY type to RAW type

What is the purpose of the change
Renames the ANY type to RAW type.
Brief change log
Updates docs, parser, logical types, and API.
Verifying this change
This change is already covered by existing tests.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? docs  and JavaDocs

Thank you @dawidwys. Merging...


[[FIX]] Correct parsing of class method names

@rwaldron I found an unrelated bug while fixing gh-3381, and I've included a
patch for that here, too. gh-3381 only effected class methods, but I added a
regression tests for method defined in object initializers for good measure.

Coverage increased (+0.0003%) to 98.545% when pulling b619b1c66ede0ed6888ba0d5183b13dc37c07ed7 on jugglinmike:class-method-names into 07ff25d3f4379415f0b438d9fb7cc6f6cabb7f7a on jshint:master.

Coverage increased (+0.0003%) to 98.545% when pulling b619b1c66ede0ed6888ba0d5183b13dc37c07ed7 on jugglinmike:class-method-names into 07ff25d3f4379415f0b438d9fb7cc6f6cabb7f7a on jshint:master.

https://github.com/jshint/jshint/pull/3379 has landed

Rebased and passing as expected, @rwaldron


history mixin - separate path and query concerns.

Unless I'm mistaken, it is not possible to update the search query through React Router without setting the path at the same time. This is rather inconvenient for non-routed components which nonetheless need to manipulate query parameters.
It would be very nice to have something similar to the $location.search() functionality in angular which allows you to do exactly this.
For example
history.search({x: 'b', y: 'c'});
How angular implements this:
https://docs.angularjs.org/guide/$location

This would come via https://github.com/rackt/react-router/issues/2186. You'd do essentially:
history.push({...location, query: {foo: 'bar'}})

Note that there will have to be a way to get at the path & query params outside routed components in order for this to be useful. I  believe this used to be possible with the State mixin, can't see a clean way  to achieve this in 1.0.rc4
I am currently using a window.location hack to achieve this, would be nice to avoid that in the future :-)

You just add your own listener to the history.

Can't find any documentation for that... should that be here?
https://github.com/rackt/react-router/blob/master/docs/API.md#history-mixin

Uhh, no? Why would it be? Are you talking about outside components or inside your component hierarchy? Because for the latter it's just on context.

Outside, if I'm following your terminology correctly. I'm getting at the history object through the History mixin.
var FoundationAccordion = React.createClass({
    mixins:[History],

    componentDidMount: function () {
        this.history.listen(function(err, updateToPath){
            this.setState(updateToPath.location.query);
        })
    },

This works, so thanks for getting me on the right path.

No, that's not the right way to do it - you're still in component hierarchy, just read location from context.

There is no location on this.context object here...only history, which comes from the History mixin. I would guess because this component is not instantiated/mounted by a Route. Which is what I thought you meant by "component hierarchy" ;-/

@stefaneg Check this docs.
We made the issue tracker for bugs and feature requests only, let's follow up on Reactiflux or StackOverflow.


Edit contributing.md to clarify step 8

Add and commit changes relevant to the issue only (do not push changes to dependencies unless the issue calls for it)

PR #88


feat: "Group" column for rollup/tree tables

Have a virtual column that just shows the rolled up value
Displays the value from the "source" cell (ie. the value from the grouping column at that depth)
The context menu shows the filter options for the source cell, so you can filter by options
Virtual column you cannot use quick filters or sort on
Closes #1555
Tested with rollup from docs:

from deephaven import read_csv, agg

insurance = read_csv("https://media.githubusercontent.com/media/deephaven/examples/main/Insurance/csv/insurance.csv")

agg_list = [agg.avg(cols=["bmi", "expenses"])]
by_list = ["region", "age"]

test_rollup = insurance.rollup(aggs=[], by=by_list, include_constituents=True)
insurance_rollup = insurance.rollup(aggs=agg_list, by=by_list, include_constituents=True)

Also shows up in the find column menu

Still an issue that you can link into/out of the virtual column. At least for the link into, it creates an invalid filter on the column which is unclearable unless you use the clear all filters keyboard shortcut (ctrl+e).
Linking out works, but feels odd since it's the key column and can have multiple types. I'd say just don't make it a linkable column at all


Adapt to new migration for umbrel 0.5.0 including core lightning data

fixes #3159

@openoms I works that I can migrate the core lightning data - but their hsm_secret is not encrypted as far as i can see, should we leave this as default after migration?

Yes, we are not encrypting it by default either.


java.io.IOException: Cannot download 'https://github.com/HotswapProjects/HotswapAgent/releases/download/1.0/hotswap-agent-1.0.jar': connect timed out

Install the "hotswap-agent-1.0.jar" manually from the folder, but after save the configuration, seems the plugin will still try to download the "hotswap-agent-1.0.jar" from the website.
BTW, I'm using the latest IDEA version.

Hi @cloudhuang,
The hotswap agent configuration in IDEA is for current project only. If you change the "Agent installation" path in one IDEA's project and open another instance of IDEA with different project then plugin will try to download it and put the jar file to the default folder unless you specify another location for it.
@cloudhuang Is it something what you did?

Error occurred during initialization of VM
agent library failed to init: instrument
Error opening zip file or JAR manifest missing : C:\Users\HZhan.IntelliJIdea2017.2\config\plugins\hotswap-agent-intellij-plugin\lib\hotswap-agent-intellij-plugin.jar
Disconnected from server

Hi. I'm not sure I understand the above statement (sorry). I'm just trying a vanilla install and I get:
java.io.IOException: Cannot download 'https://github.com/HotswapProjects/HotswapAgent/releases/download/1.2.1-SNAPSHOT/hotswap-agent-1.2.0.jar': Status: 404
Uninstalled and reinstalled and the issue persists. curling that location also fails.


Disponibilty for Power BI Desktop Report Server

Could you please add a GA/release date for Power BI Desktop report server ?
It is not possible to use field parameters for now. Is-it due to preview release or technical limitation of Report Server ?
I don't understand why we still can't use a DAX field parameter, so it would be nice to have clues on that in this documentation.
Parameter = {
("Customer", NAMEOF('Customer'[Customer]), 0),
("Category", NAMEOF('Product'[Category]), 1),
("Color", NAMEOF('Product'[Color]), 2),
("Product", NAMEOF('Product'[Product]), 3)
}
Thank you

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5221841f-f47d-838f-7dae-a4b5a94cefa2
Version Independent ID: 5221841f-f47d-838f-7dae-a4b5a94cefa2
Content: Let report readers use field parameters to change visuals (preview) - Power BI
Content Source: powerbi-docs/create-reports/power-bi-field-parameters.md
Service: powerbi
Sub-service: pbi-reports-dashboards
GitHub Login: @Sujata994
Microsoft Alias: sunaraya

Thanks for your comment, @pedrion. This issue has been assigned to the doc author for review.

Hi @pedrion,
We don't publish release dates in our documentation. There's more information (not exact dates) in Microsoft Fabric release plan documentation.
For product feedback or a product question, you can submit product feedback or feature ideas by going to Power BI Ideas.
You can also engage with the engineering team by being active in the Community.
Thanks,
Kesem


Feature Request: Pause animation for couple of seconds

Hello,
I would like to pause short animations as the viewer cannot see what is going on in a short loop.
Command
Sleep/Pause in ms:
svg-term --cast xxx --out xxx.svg --pause 500

Example

Thanks for the suggestion. I don't want to introduce features like this in svg-term-cli, editing/extended recording tools will not be part of its scope.
I am rolling around the idea to provide something like a separate asciicast editor, which would include functionality like this.
You might achieve the result you are after with some scripting (pseudo code):
#!/bin/sh
# screencast.sh
printf '\e[01;32mredaxmedia@redaxmedia\e[00m:\e[01;34m/mnt/WIP/ncss-linter\e[00m$ '
sleep 0.1
printf 'bin/ncss-linter --url=https://redaxmedia.com --namespace=rs --log-level=info'
bin/ncss-linter --url=https://redaxmedia.com --namespace=rs --log-level=info

sleep 500
echo ""

I wonder why --command is not documented!?
Here comes a even more elegant way as I want to extend an the existing cast
svg-term --command="asciinema play https://asciinema.org/a/123683 && sleep 5 && echo" --out=123683.svg

I wonder why --command is not documented!?

Forgot to sync the docs with the cli, done via e22ca85.

@marionebl Is there instead some way to just stop the animation at the end instead of looping?


ESM support

Would be great to have ESM support on next-session so it supports named imports:
import { MemoryStore, session } from 'next-session'

the ESM support guide is here: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c
probably because it's a more Next.js-centered project the better way would be to produce both CJS and ESM outputs.

Just a note here: next-session v4 is ESM compatible.


modify the navbar

Related Issue #686
Proposed Changes

Changed the css property of background of header

Additional Info

Any additional information or context

Checklist

[x ] ✅ My code follows the code style of this project.
[ x] 📝 My change requires a change to the documentation.
[ ] 🎀 I have updated the documentation accordingly.
[x ] 🌟 ed the repo

[ x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

@agamjotsingh18 Please review it
Light mode
https://github.com/agamjotsingh18/codesetgo/assets/101205226/5e6848d1-6123-4930-90ab-1ac89233d4c1
Dark mode
https://github.com/agamjotsingh18/codesetgo/assets/101205226/0da40578-4002-45c9-b933-f75f47cbd7b0

Mobile view has issues


carla make launch erro

CARLA version:0.9.13
Platform/OS:ubuntu 20.04
Problem you have experienced:
i'm aready finish the other step ,but when i run make launch ,i found the error:
BuildCarlaUE4.sh: Build CarlaUE4 project.
make[1]: 进入目录“/home/omen/carla/Unreal/CarlaUE4”
bash "/home/omen/UnrealEngine_4.26/Engine/Build/BatchFiles/Linux/RunMono.sh" "/home/omen/UnrealEngine_4.26/Engine/Binaries/DotNET/UnrealBuildTool.exe" CarlaUE4Editor Linux Development  -project="/home/omen/carla/Unreal/CarlaUE4/CarlaUE4.uproject"
Running Mono...
Fixing inconsistent case in filenames.
Setting up Mono
~/UnrealEngine_4.26/Engine ~/carla/Unreal/CarlaUE4
Using 'git status' to determine working set for adaptive non-unity build (/home/omen/UnrealEngine_4.26).
Invalidating makefile for CarlaUE4Editor (Module.OnlineSubsystemUtils.cpp.o.rsp modified)
ERROR: Couldn't touch header directories: Access to the path "/home/omen/carla/Unreal/CarlaUE4/Plugins/Carla/Intermediate/Build/Linux/B4D820EA/UE4Editor/Inc/Carla/Timestamp" is denied.
if you had met this error before,please told me how to fix this error,thanks !
-->


Merge up

Superceded by my merge :)
Issue was caused by a change in $_FILE_TO_URL_MAPPING that needed it to be defined as global in travis-support module.


add nodejs 4.3.2 support. Closes #356

Worth to mention that updating nodejs version on already deployed function will have no effect. apex delete is needed before.

@tj I'm not sure about inference. Should we switch to 4.3? Hmm, I will add it.

Hmm good question haha, I suppose if it's unspecified at both levels assuming the recent is fine

If we need later we could have a RuntimeLatest = RuntimeWhatever


api index

Fixes #1752.
Enumerates every method with a brief description pulled from the documentation:

Also enumerates every option, showing which interface the option is defined:

It would perhaps be better to provide more specific links (e.g., to the #bend anchor fragment instead of the general arrow mark documentation), but I think that’s non-blocking and we could add that in the future as desired.

Trying this for now. Let me know if you want changes. I think I’d like to use shorter, explicit anchor fragments across the site, for example, such as #pointerX instead of #pointerx-options.

Love it!


Virtual Concrete Pangolin

Hotspot Name
Virtual Concrete Pangolin
Hotspot b58 Address
111Aw9N7yyhGZDMrp7t8n1hLANi93DG3bxGRBSjZFVqa5E7EXqu
Discord Handle
No response
Hotspot Manufacturer
panther
Removal Reason
Now it has to be redeployed, even so. Then I will share this hot spot with my friends. You can check and verify this through your technical test methods. Please remove it from the deny list for me, thank you.
Modifications
No
Extra forwarders
No
Extra antennas
No
Additional Information
No response

We have taken into account this removal request in the next iteration of the denylist. It may take some time before it's processed. This is an automated comment due to the volume of addition and removal requests. A future tool is in the works to provide more insight into the analysis approach made by the Helium team and community members that maintain this list.


Investigate Ceras and CompilerServices.Unsafe dependency conflict

I'm honestly not sure. I may fork Ceras to resolve the CompilerServices.Unsafe conflict (it wasn't a complicated issue if memory serves me well). Not sure about the Reflection.Metadata conflict as I don't actually know what's using that.
Originally posted by @NarryG in https://github.com/redscientistlabs/RTCV/pull/239#discussion_r491750415

I poked at this a bit and it seems like whatever issue I was previously having may be gone. I still need to test more but I brought everything up to the latest version and didn't appear to get any errors with my quick test

Closed with https://github.com/redscientistlabs/RTCV/commit/e46acba8e36f1227bbf1dd494531d3c541f0eb0e


cudaPackages: .pc files should not contain /usr paths

Describe the bug
We need to patch cuda's pkg-config files
Steps To Reproduce
Steps to reproduce the behavior:

❯ NIXPKGS_ALLOW_UNFREE=1 nix build --impure nixpkgs#cudaPackages.cuda_cudart
❯ cat result/pkg-config/cudart-11.7.pc
observe cudaroot=/usr/local/cuda-11.7

Expected behavior
Paths relative to outputs
Notify maintainers
@NixOS/cuda-maintainers

Hellol! I was looking for good first bug issues and tried this.
Describe the bug:
The pkg-config files for the cuda_cudart package in the Nix package repository nixpkgs contain incorrect paths. The paths should be relative to the package outputs, but they currently point to fixed paths like /usr/local/cuda-11.7.
Steps to reproduce the bug:
Build and install the cuda_cudart package using the following command:
NIXPKGS_ALLOW_UNFREE=1 nix build --impure nixpkgs#cudaPackages.cuda_cudart

Check the contents of the pkg-config file for the package:
cat result/pkg-config/cudart-11.7.pc
cudaroot=/usr/local/cuda-11.7
libdir=${cudaroot}/targets/x86_64-linux/lib
includedir=${cudaroot}/targets/x86_64-linux/include

Name: cudart
Description: CUDA Runtime Library
Version: 11.7
Libs: -L${libdir} -lcudart
Cflags: -I${includedir}

Observe that the cudaroot path is set to /usr/local/cuda-11.7, which is incorrect.
Expected behavior
The paths in the pkg-config files should be relative to the package outputs, rather than fixed paths like /usr/local/cuda-11.7 . For instance, the cudaroot path should be replaced with the correct output path in the Nix store.
Let's begin:
Fork the nixpkgs repository:
git clone https://github.com/dooygoy/nixpkgs.git
Cloning into 'nixpkgs'...
remote: Enumerating objects: 3783643, done.
remote: Counting objects: 100% (458/458), done.
remote: Compressing objects: 100% (172/172), done.
remote: Total 3783643 (delta 332), reused 390 (delta 286), pack-reused 3783185
Receiving objects: 100% (3783643/3783643), 3.11 GiB | 6.33 MiB/s, done.
Resolving deltas: 100% (2576941/2576941), done.
Updating files: 100% (34603/34603), done.

cd nixpkgs/

Search for the package:
grep -r --include='*.nix' -l 'cuda_cudart' .
./pkgs/applications/science/math/mathematica/generic.nix
./pkgs/development/libraries/science/math/tiny-cuda-nn/default.nix
./pkgs/development/libraries/science/math/nccl/default.nix
./pkgs/development/libraries/science/math/faiss/default.nix
./pkgs/development/libraries/science/math/magma/generic.nix
./pkgs/development/libraries/opencv/4.x.nix
./pkgs/development/libraries/nvidia-thrust/default.nix
./pkgs/development/python-modules/torchvision/default.nix
./pkgs/development/python-modules/openai-triton/default.nix
./pkgs/development/python-modules/bitsandbytes/default.nix

I see there is the cudatoolkit but I am not sure if this is the package I have to fix:
https://github.com/NixOS/nixpkgs/blob/nixos-22.11/pkgs/development/compilers/cudatoolkit/

There is also the build-cuda-redist-package.nix with the:
# TODO: choose whether to install static/dynamic libs
  installPhase = ''
    runHook preInstall
    rm LICENSE
    mkdir -p $out
    mv * $out
    runHook postInstall
  '';

How should I go from here? I am not sure how to continue for now or whether I am looking at the correct file:

@dooygoy hi, and thank you for taking interest in this issue! In the hindsight, I see I didn't include all of the inputs in the description, sorry about that. I also should warn you in advance that although I added the good-first-bug label, merging a PR to close this issue might take a little time and a bit of experimenting. The individual steps should be very simple though!
I won't assume familiar you are with nixpkgs, so I'll include details that may possibly be redundant.
A note on impact: hypothetically, we could just delete the pkg-config files, because most of our downstream packages seem to be using cmake, meson, or bazel for builds, and they either rely on proper mechanisms like FindCUDAToolkit.cmake to discover CUDA, or they hard-code paths like /opt/something/cuda. However, the upstream keeps publishing the .pc files, so some people we don't see may be relying on them, so it's reasonable for us to keep them, as long as making them correct isn't too much of a burden. I think we should.
Also, do not worry if you decide to spend your time elsewhere: I think it's good to have these notes written up anyway
You guessed right, most of our CUDA-related expressions live in pkgs/development/compilers/cudatoolkit, and the actual cudaPackages.cuda_cudart package (and all other cudatoolkit components from recent releases) is built from build-cuda-redist-package.nix, which you also have already found[^1]. The expression in build-cuda-redist-package.nix is re-used for every CUDA package, and the actual hashes (package names, URLs, hashes) come from JSON files published by nvidia along with cudatoolkit. It should be sufficient that we edit build-cuda-redist-package.nix appropriately

[ ] It might be sufficient that we simply patch the existing .pc files to replace /usr/local/cuda-XX.Y with $out. This can be done in the installPhase before runHook postInstall. I imagine we could start by just trying sed, e.g. sed -i 's|^cudaroot=.*$|cudaroot=${placeholder "out"}|' $out/pkg-config/*.pc.
[ ] While we're at it, we probably should start using the validatePkgConfig and testMetaPkgConfig hooks: in build-cuda-redist-package.nix we'd need to append validatePkgConfig to the nativeBuildInputs list. We'll take validatePkgConfig as an argument: you add it after autoAddOpenGLRunpathHook at the top of the file, and it should get auto-wired from there. The hook itself is defined in all-packages.nix.

[ ] The testMetaPkgConfig is more useful: we can expose "test derivations" that verify availability of specific pkg-config targets e.g. as cudaPackages.cuda_cudart.tests.pkg-config, and we can configure our CI to run these tests. However, testMetaPkgConfig is also a little more complicated. We'd need to populate the pkgConfigModules list in meta, cf. the example. E.g. cuda_cudart includes a pkg-config/cuda-11.7.pc file that defines a Name: cuda target, so it would go in the list as meta.pkgConfigModules = [ "cuda" ];. However, different cuda components define different pkg-config targets and under different file names: e.g. cuda_nvrtc includes a pkg-config/nvrtc-11.7.pc with Name: nvrtc, and cudaPackages.libcublas contains pkg-config/cublas-11.7.pc with Name: cublas. I think we'd have write these out by hand, and it would also cumbersome to do that in build-cuda-redist-package.nix (we want to keep that generic expression simple). For per-package modifications we have the overrides.nix file. But we can chat about this one in details later, and there's enough stuff to do before we touch testMetaPkgConfig

[ ] Some packages do not ship any .pc files, e.g. cuda_nvcc doesn't seem to. We don't want to break them with our hooks
[ ] You can also test the .pc files manually by entering a nix-shell such as mkShell { packages = [ pkg-config ]; buildInputs = with cudaPackages; [ cuda_cudart cuda_nvrtc cuda_nvtx libcublas libcufft ]; } and running pkg-config --list-all, pkg-config --cflags --libs cuda cudart nvrtc ...
[ ] We'd want to verify that the configured flags actually are correct in the sense that we can use them to build a cuda program. Ideally we'd find a package that uses pkg-config to discover cuda, although, as I mentioned, people mostly use CMake (amen!)
[ ] Eventually we'd run a nixpkgs-review on the PR to verify we haven't broken any builds

This should be enough to get started
[^1]: By the way, there's a quick way to locate the expression for a package: nix edit nixpkgs#cudaPackages.cuda_cudart opens build-cuda-redist-package.nix in your editor.

Thank you very much for such a long description and thank you for including the details that may possibly be redundant (which are not!).
Since this is my very first package issue and I wish to blog about it too (and I am pretty excited about it too, yay!) I will document it in detail including my analysis which may also go into much detail. So let's begin:

A note on impact: hypothetically, we could just delete the pkg-config files, because most of our downstream packages seem to be using cmake, meson, or bazel for builds, and they either rely on proper mechanisms like FindCUDAToolkit.cmake to discover CUDA, or they hard-code paths like /opt/something/cuda. However, the upstream keeps publishing the .pc files, so some people we don't see may be relying on them, so it's reasonable for us to keep them, as long as making them correct isn't too much of a burden. I think we should.

Analysis:
I see downstream packages and upstream is mentioned and I wasn't sure about this terminology. Upstream refers to the original software projects, while downstream refers to the packages in the Nix package repository (nixpkgs) that depend on those upstream projects. When a bug is fixed or a new version is released upstream, it's usually the responsibility of the downstream package maintainers to update their packages to incorporate those changes.
I see this quote "hard-code paths like /opt/something/cuda," is referring to the practice of hardcoding absolute paths in the build process or configuration files which looks to me as impure with the regard to how Nix packages are made. Why is that? Hardcoding paths can cause issues because it makes assumptions about the file system layout and the location of dependencies, which may not hold true for every user or system. Nix reproducible builds actually solve this problem, in this case by making these paths relative instead of absolute.

You guessed right, most of our CUDA-related expressions live in pkgs/development/compilers/cudatoolkit, and the actual cudaPackages.cuda_cudart package (and all other cudatoolkit components from recent releases) is built from build-cuda-redist-package.nix, which you also have already found1. The expression in build-cuda-redist-package.nix is re-used for every CUDA package, and the actual hashes (package names, URLs, hashes) come from JSON files published by nvidia along with cudatoolkit. It should be sufficient that we edit build-cuda-redist-package.nix appropriately

I open the linked JSON file published by nvidia and I see the relative paths which seem to be important to make the package functional. In the Nix expression, the relative_path is used along with the fetchurl function to download the appropriate source file for building the package. This part caught my attention that shows the relative paths for each platform. For example for the Linux it shows:
"linux-x86_64": {
            "relative_path": "cuda_cudart/linux-x86_64/cuda_cudart-linux-x86_64-11.7.60-archive.tar.xz",
            "sha256": "1c079add60a107f6dd9e72a0cc9cde03eb9d833506f355c22b9177c47a977552",
            "md5": "1ef515eb31691f2c43fb0de1443893a3",
            "size": "854744"
        },

But let's read further instructions.

It might be sufficient that we simply patch the existing .pc files to replace /usr/local/cuda-XX.Y with $out. This can be done in the installPhase before runHook postInstall. I imagine we could start by just trying sed, e.g. sed -i 's|^cudaroot=.*$|cudaroot=${placeholder "out"}|' $out/pkg-config/*.pc.

I am familiar with sed but have never actually used it and I am wondering at the same time if there is a more Nix-way to solve this too. It seems the suggested solution is to patch the existing .pc files by replacing /usr/local/cuda-XX.Y with $out in the installPhase before runHook postInstall.
installPhase = ''
  runHook preInstall
  rm LICENSE
  mkdir -p $out
  mv * $out

  # Patch the .pc files to replace /usr/local/cuda-XX.Y with $out
  sed -i 's|^cudaroot=.*$|cudaroot=${placeholder "out"}|' $out/pkg-config/*.pc

  runHook postInstall
'';

What is sed actually doing here? The sed command is used to replace the cudaroot path in all .pc files under $out/pkg-config. The ${placeholder "out"} expression is used to reference the $out variable within the single quotes of the sed command.
Now is there a more Nix way to solve this? Is there a Nix function that would do this operation instead of sed? There is a Nix function called substituteInPlace  that we could use in the postInstall phase of the installPhase. This function is designed to replace strings in files within Nix derivations. This function is like the substitute function. For example I am looking at the nixpkgs manual and searching for examples of this function and we see this:
postPatch = ''
        substituteInPlace lib/zoneinfo.rs \
          --replace "/usr/share/zoneinfo" "${tzdata}/share/zoneinfo"
      '';

How could we apply this function in our case instead of sed and would that be a better alternative? Something like this:
postInstall = ''
  substituteInPlace $out/pkg-config/cudart-11.7.pc \
    --replace "cudaroot=/usr/local/cuda-11.7" "cudaroot=${placeholder "out"}"
'';

But another issue is that unlike sed, substituteInPlace does not support regular expressions it seems, so I wonder what would be the best path to take here. Let's move on.

While we're at it, we probably should start using the validatePkgConfig and testMetaPkgConfig hooks: in build-cuda-redist-package.nix we'd need to append validatePkgConfig to the nativeBuildInputs list. We'll take validatePkgConfig as an argument: you add it after autoAddOpenGLRunpathHook at the top of the file, and it should get auto-wired from there. The hook itself is defined in all-packages.nix.

OK, let's look at the build-cuda-redist-package.nix:

Add validatePkgConfig as an argument at the top of the file, after autoAddOpenGLRunpathHook.

{ lib
, stdenv
, backendStdenv
, fetchurl
, autoPatchelfHook
, autoAddOpenGLRunpathHook
, validatePkgConfig
}:

Append validatePkgConfig to the nativeBuildInputs list:

nativeBuildInputs = [
  autoPatchelfHook
    # This hook will make sure libcuda can be found
    # in typically /lib/opengl-driver by adding that
    # directory to the rpath of all ELF binaries.
    # Check e.g. with `patchelf --print-rpath path/to/my/binary
  autoAddOpenGLRunpathHook
  validatePkgConfig
];

How is the hook defined in all-packages.nix? Let's see that too:
validatePkgConfig = makeSetupHook
    { name = "validate-pkg-config"; propagatedBuildInputs = [ findutils pkg-config ]; }
    ../build-support/setup-hooks/validate-pkg-config.sh;

Now I read test derivations and CI are mentioned too which sounds cool and definetly as something useful and functional to have when building packages. Tests are super important, but I am not familiar with how these things work

The testMetaPkgConfig is more useful: we can expose "test derivations" that verify availability of specific pkg-config targets e.g. as cudaPackages.cuda_cudart.tests.pkg-config, and we can configure our CI to run these tests.

We must somehow create a kind of test-derivation.nix that would use the testMetaPkgConfig hook within which would then check the availability of pkg-config targets.
That sounds super interesting!

For per-package modifications we have the overrides.nix file. But we can chat about this one in details later, and there's enough stuff to do before we touch testMetaPkgConfig.

OK so we move on and leave this for later. I am not familiar with overrides.nix and how they work but I do remember seeing a nice 2019? Nixcon talk by beloved Graham. Maybe I should check that too? Excellent!

Some packages do not ship any .pc files, e.g. cuda_nvcc doesn't seem to. We don't want to break them with our hooks

In order to ensure that the hooks don't break packages that don't ship any .pc files, how can we conditionally apply the hooks based on the presence of .pc files in the package? Upon further investigation I see that one way to achieve this is by using the optional function provided by Nix, which conditionally adds an element to a list based on a condition. Maybe we could use this function in nativeBuildInputs to conditionally add the hooks based on the presence of .pc files in the package.

You can also test the .pc files manually by entering a nix-shell such as mkShell { packages = [ pkg-config ]; buildInputs = with cudaPackages; [ cuda_cudart cuda_nvrtc cuda_nvtx libcublas libcufft ]; } and running pkg-config --list-all, pkg-config --cflags --libs cuda cudart nvrtc ...

Oh, let's try this right now!
mkShell { packages = [ pkg-config ]; buildInputs = with cudaPackages; [ cuda_cudart cuda_nvrtc cuda_nvtx libcublas libcufft ]; }
bash: syntax error near unexpected token `}'

Oh, we got an error! Was there a typo? Is something missing? I am instructed to enter nix shell and it seems to me I didn't actually start the nix shell. Maybe I need to pull this command after that:
nix-shell -E 'with import <nixpkgs> {}; mkShell { buildInputs = with cudaPackages; [ pkg-config cuda_cudart cuda_nvrtc cuda_nvtx libcublas libcufft ]; }'
error: Package ‘cuda_cudart-11.7.60’ in /nix/store/79fr5k4kky1p59y865ivnaxxq23y0nxg-source/pkgs/development/compilers/cudatoolkit/redist/build-cuda-redist-package.nix:55 has an unfree license (‘unfree’), refusing to evaluate.

       a) To temporarily allow unfree packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_UNFREE=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowUnfree = true; }
       in configuration.nix to override this.

       Alternatively you can configure a predicate to allow specific packages:
         { nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
             "cuda_cudart"
           ];
         }

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowUnfree = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)

Another error! This tells us we need to export NIXPKGS_ALLOW_UNFREE=1 into our shell. So this tells me that by default the nix shell provides an environment where unfree packages aren't allowed? Let's export it and then try again:
export NIXPKGS_ALLOW_UNFREE=1

~ via 🐍 v3.10.10
❯ nix-shell -E 'with import <nixpkgs> {}; mkShell { buildInputs = with cudaPackages; [ pkg-config cuda_cudart cuda_nvrtc cuda_nvtx libcublas libcufft ]; }'
these 8 derivations will be built:
  /nix/store/53qvvhwj1mqdssx6lnpgymdpvv15jbxh-cuda_nvrtc-linux-x86_64-11.7.50-archive.tar.xz.drv
  /nix/store/4zk729lbyn8381ijj1ff1vh8wg03mv41-cuda_nvrtc-11.7.50.drv
  /nix/store/6ciynn4hdamj6ihym3icyh37ndybancc-libcufft-linux-x86_64-<IP_ADDRESS>-archive.tar.xz.drv
  /nix/store/maz1aq725zxk8wdp38b7nlfcx5lq45qv-libcublas-linux-x86_64-<IP_ADDRESS>-archive.tar.xz.drv
  /nix/store/7lvyc15c2f3qqjhwpxfgq0hc5w2wfic8-libcublas-<IP_ADDRESS>.drv
  /nix/store/hfr7kk26wqfbqs89qdi61n1f0wlza0c6-libcufft-<IP_ADDRESS>.drv
  /nix/store/sfccgfg4f0ry6d44grwa5zshxzryqy6j-cuda_nvtx-linux-x86_64-11.7.50-archive.tar.xz.drv
  /nix/store/xwqbkra4rhg02nadk8v792f93w51pi73-cuda_nvtx-11.7.50.drv
building '/nix/store/53qvvhwj1mqdssx6lnpgymdpvv15jbxh-cuda_nvrtc-linux-x86_64-11.7.50-archive.tar.xz.drv'...
building '/nix/store/sfccgfg4f0ry6d44grwa5zshxzryqy6j-cuda_nvtx-linux-x86_64-11.7.50-archive.tar.xz.drv'...
building '/nix/store/maz1aq725zxk8wdp38b7nlfcx5lq45qv-libcublas-linux-x86_64-<IP_ADDRESS>-archive.tar.xz.drv'...
building '/nix/store/6ciynn4hdamj6ihym3icyh37ndybancc-libcufft-linux-x86_64-<IP_ADDRESS>-archive.tar.xz.drv'...
...

And now we are back in business! There is a huge output running so let's see.. I wonder if it's necessary to copy the whole output here? Now it is unpacking a source archive into the /nix/store. Let's wait because it is taking a little while.. The output ends with fixupPhase completed in 31 seconds. Now let's try some of the commands I was instructed to do:
pkg-config --list-all

~ via 🐍 v3.10.1 via impure (nix-shell)

We get nothing which means it is somehow not finding it in nix store? I thought while in nix shell everything is magically available. Well, let's try to find it?
echo $cuda_cudart

Hmm, still nothing. It seems that the environment variables are not set when entering a nix-shell. Instead, let's find the location of the .pc files by using find command to search the Nix store for the .pc files associated with the CUDA packages:
find /nix/store -name "*.pc" | grep cuda

/nix/store/vqqdcl22a5cynh9m7jxnmmhi4ylyy47r-cuda_nvtx-11.7.50/pkg-config/nvToolsExt-11.7.pc
/nix/store/4qz7zqrxd1anyhhi0fyix8dn4hqbifps-cuda_cudart-11.7.60/pkg-config/cudart-11.7.pc
/nix/store/4qz7zqrxd1anyhhi0fyix8dn4hqbifps-cuda_cudart-11.7.60/pkg-config/cuda-11.7.pc
/nix/store/vz5jd0y7wlj94cj4k3abcb94njz3f52d-cuda_nvrtc-11.7.50/pkg-config/nvrtc-11.7.pc

Oh and what do we do now? We must find a way to extract these directory paths and add them to our environment.
Great! Now we can set the PKG_CONFIG_PATH environment variable by concatenating these paths with colons.
export PKG_CONFIG_PATH="/nix/store/vqqdcl22a5cynh9m7jxnmmhi4ylyy47r-cuda_nvtx-11.7.50/pkg-config:/nix/store/4qz7zqrxd1anyhhi0fyix8dn4hqbifps-cuda_cudart-11.7.60/pkg-config:/nix/store/vz5jd0y7wlj94cj4k3abcb94njz3f52d-cuda_nvrtc-11.7.50/pkg-config"

Now when we try to run the pkg-config --list-all it still shows nothing! What do we do now? Let's make more checks:
Let's verify that the PKG_CONFIG_PATH environment variable is correctly set:
echo $PKG_CONFIG_PATH
/nix/store/vqqdcl22a5cynh9m7jxnmmhi4ylyy47r-cuda_nvtx-11.7.50/pkg-config:/nix/store/4qz7zqrxd1anyhhi0fyix8dn4hqbifps-cuda_cudart-11.7.60/pkg-config:/nix/store/vz5jd0y7wlj94cj4k3abcb94njz3f52d-cuda_nvrtc-11.7.50/pkg-config

Let's make sure the .pc files exist in the specified directories:
ls -l /nix/store/vqqdcl22a5cynh9m7jxnmmhi4ylyy47r-cuda_nvtx-11.7.50/pkg-config
ls -l /nix/store/4qz7zqrxd1anyhhi0fyix8dn4hqbifps-cuda_cudart-11.7.60/pkg-config
ls -l /nix/store/vz5jd0y7wlj94cj4k3abcb94njz3f52d-cuda_nvrtc-11.7.50/pkg-config
total 4
-r--r--r-- 2 root root 249 sij   1  1970 nvToolsExt-11.7.pc
total 8
-r--r--r-- 2 root root 234 sij   1  1970 cuda-11.7.pc
-r--r--r-- 2 root root 239 sij   1  1970 cudart-11.7.pc
total 4
-r--r--r-- 2 root root 259 sij   1  1970 nvrtc-11.7.pc

But still, pkg-config --list-all is not showing anything. I seem to be running in circles now. I am not sure what to do next. Maybe I did something wrong with the nix shell and missed some environment setting? I will need to look into this too. Let's finish and see what else we will have to do.

We'd want to verify that the configured flags actually are correct in the sense that we can use them to build a cuda program. Ideally we'd find a package that uses pkg-config to discover cuda, although, as I mentioned, people mostly use CMake (amen!)

Oh, then can we make a small hello-world cuda program to test if this works?
Conclusion
Overall I think this was a wonderful experience and thank you very much for insightful advices and directions. I would very much like to continue wrapping this up nicely if possible and along the way learning about Nix. I will study again the steps made and try to learn more about it. Let me know how to continue from here too. <3

pkg-config --list-all ... empty output

Ah, I guess we need to move $out/pkg-config to $out/lib/pkg-config for pkg-config setup hooks to pick these up automatically. E.g. once you enter mkShell { packages = [ pkg-config ]; buildInputs = [ zlib ]; }, PKG_CONFIG_PATH should already be extended with a path to zlib.pc's directory
You can do this with mv and in the same installPhase

But another issue is that unlike sed, substituteInPlace does not support regular expressions it seems

Yes, and these .pc files include the cuda version (e.g. 11.7) which is different e.g. for cudaPackages_11_7 and cudaPackages_11_8. It would be better to do without sed, but we also don't want to overcomplicate things without good reason

Is the sed imperative solution?

Yes, and in fact a lot of nixpkgs is (managed) "imperative". A way to think about this is that Bash is the effect system for the purely-functional Nix. Of course, we do prefer declarative solutions

with import <nixpkgs> {};

Note that <nixpkgs> is impure and it resolves into the channel your system is configured to follow (you can just enter nix repl and see what <nixpkgs> evaluates to). In particular, this way you won't see the effect of the changes you made to cuda packages. You'll have to import the particular nixpkgs revision you're working on, e.g. you can enter its directory and import ./. { ... }

$ export NIXPKGS_ALLOW_UNFREE=1

Yes, and note that this also only works because nix-shell (unlike the new nix shell, which may be confusing) is impure and reads environment variables. As the message suggests, you can also just import <nixpkgs> { config.allowUnfree = true; } if you like, instead of relying on environment variables

OK so we move on and leave this for later. I am not familiar with overrides.nix

The file itself is specific to our cuda packages, but the mechanisms are common in nixpkgs. One is overrideAttrs which lets you modify which arguments end up going into mkDerivation. For example, in build-cuda-redist-package.nix we already pass the attribute "meta" to mkDerivation, but that meta doesn't include pkgConfigModules. For testMetaPkgConfig to work we need to piggyback that attribute, and we need to piggyback different values for different cuda packages. This is what we can do in overrides.nix:
  cuda_cudart = prev.cuda_cudart.overrideAttrs (oldAttrs: {
    meta = oldAttrs.meta // {
        pkgConfigModules = [ "cuda" "cudart" ];
    };
  });

Here, oldAttrs will be the exact same dictionary we've defined in build-cuda-redist-packages.nix, but the dictionary that final mkDerivation will see will have meta replaced with one that additionally has pkgConfigModules.
Extra info that you can totally live without. There are, in fact, two mechanisms at play here, and the second has to do with where prev and final variables come from in that file. As you can guess prev.cuda_cudart refers to an intermediate version of the cudart derivation that hasn't been embellished with meta.pkgConfigModules yet and that will never in fact get fully evaluated or built. The prev itself is an intermediate version of the cudaPackages dictionary

Oh, then can we make a small hello-world cuda program to test if this works?

Yes, that would be the reasonable next step once we get pkg-config working "automatically"

I open the linked JSON file published by nvidia and I see the relative paths which seem to be important to make the package functional

Aha, maybe I should have pointed out that this is somewhat specific to how we handle cudatoolkit, which is unlike "normal" Nix packages: we don't have access to sources, so we download pre-built binaries from nvidia and ensure that they can be used within nixpkgs. We do so in a semi-automated manner by re-using these jsons in the nix expression

Thinking of it, maybe we could wrap this one up quickly and switch to some issue where you could see the practical benefit of your work 🙃 E.g. https://github.com/NixOS/nixpkgs/issues/224533 shouldn't be hard to do, but you would be able to measure the impact in gigabytes of saved storage when including something like pytorch in Nix-built docker images
I think you can already open a draft PR with the .pc patches and @ reference this issue. There we can link and discuss concrete pieces of code

I hope it is OK I am writing all the steps which are obvious for you and thank you for the patience and guidance. As you can tell I am a beginner so I know if you see some weird steps that I am making you will let me know
I begin like this:
cd nixpkgs

Create and switch to a new branch for changes:
git checkout -b fix-cuda-pkg-config

Quick way to locate the expression for a package, in our case this opens opens build-cuda-redist-package.nix in your editor:

Great tip!
nix edit nixpkgs#cudaPackages.cuda_cudart

Now the fix for the missing pkg-config --list-all output is:

Ah, I guess we need to move $out/pkg-config to $out/lib/pkg-config for pkg-config setup hooks to pick these up automatically. E.g. once you enter mkShell { packages = [ pkg-config ]; buildInputs = [ zlib ]; }, PKG_CONFIG_PATH should already be extended with a path to zlib.pc's directory

You can do this with mv and in the same installPhase

Here's the breakdown as I understand:

Create a new directory for pkg-config: mkdir -p $out/lib/pkg-config. I guess this doesn't exist so we have to create it. Aha, the lib directory should already exist so with -p flag in mkdir -p we make sure it creates the directory path if any part of it is missing.

Move the contents of pkg-config to the new directory: mv $out/pkg-config/* $out/lib/pkg-config.

Update the path in the sed command to patch the .pc files: $out/lib/pkg-config/*.pc.

# Patch the .pc files to replace /usr/local/cuda-XX.Y with $out
    sed -i 's|^cudaroot=.*$|cudaroot=${placeholder "out"}|' $out/lib/pkg-config/*.pc

A way to think about this is that Bash is the effect system for the purely-functional Nix. Of course, we do prefer declarative solutions

Mhmm this makes sense. Does this also mean, and I'm just thinking out loud, that since bash is the actual effect system (I'm not aware how huge is this) that there are numerous opportunities for declarative programs to be built on top on Nix that would functionally encapsulate the effects? In a way Nix is a very fertile ground. OK

You'll have to import the particular nixpkgs revision you're working on, e.g. you can enter its directory and import ./. { ... }

What does this mean actually? Are you refering to a specific nixpkgs version and leaving me with a placeholder { ... } or can one put {} too which will by default load the available one? Am I supposed to make a shell.nix environment where this specific <nixpkgs> will be declared and then work from that? I understand that it is probably importing the available nixpkgs in my cloned directory. Something like:
with import ./. { ... };

mkShell {
  packages = [ pkg-config ];
  buildInputs = [ zlib ];
}

I do have some reading to do to better understand all this and the difference between pure and impure nix-shell, so thank you also for being patient with me. I wonder if nix shell is out of the question with these packages since it is a newer experimental command?
OK, this is all super interesting, and i am doing it in spare time so I will keep digging at this. Regarding the second issue you referenced I investigated it a bit and will work on it tonight after which I will write about it too.

mkdir -p $out/lib/pkg-config ... mv $out/pkg-config/* $out/lib/pkg-config

Or just
mkdir $out/lib
mv $out/pkg-config $out/lib/

What does this mean actually? Are you refering to a specific nixpkgs version

So, by "nixpkgs" I refer to any check out of the nixpkgs git repo. When you write import /path/to/nixpkgs/ what Nix does is it reads and evaluates /path/to/nixpkgs/default.nix. An example of that is import <nixpkgs>, where <nixpkgs> is being expanded depending on how your system is configured. E.g. I get:
❯ nix eval --impure --expr "<nixpkgs>"
/etc/nixpkgs/channels/nixpkgs
❯ ls /etc/nixpkgs/channels/nixpkgs -l
lrwxrwxrwx 1 root root 50 Mar 17 02:29 /etc/nixpkgs/channels/nixpkgs -> /nix/store/rmw0kq593g38kj9dwpg5ka81bbskh9dm-source
❯ ls /etc/nixpkgs/channels/nixpkgs
CONTRIBUTING.md  COPYING  default.nix  doc  flake.nix  lib  maintainers  nixos  pkgs  README.md

Your path will likely be different, but the structure will be the same and it'll match that of https://github.com/NixOS/nixpkgs/

and leaving me with a placeholder { ... } or can one put {} too which will by default load the available one

Now, after import <nixpkgs> comes the import <nixpkgs> { }.
If you look at <nixpkgs>/default.nix, you'll find that it is actually a function.
It accepts arguments such as config, overlays, system.
For example, you can call nixpkgs as import <nixpkgs> { config = { allowUnfree = true; } } and then you'll get an instance of nixpkgs where unfree errors are disabled.

with import ./. { ... };

A required argument of import <nixpkgs> is system, which is a string that looks like "x86_64-linux".
However, when you use nix-shell this argument is allowed to default builtins.currentSystem which evaluates "impurely" into the current system's platform. We call this "impure" because it depends on environment: on one machine you'll get x86_64-linux, and on another you'll get aarch64-darwin.
As a result, you can just use with import ./. { } to get an instance of the nixpkgs package set.
The result of import ./. { } (or import <nixpkgs> { }) is a dictionary that contains attributes like lib, and cudaPackages, and bash, etc.

I wonder if nix shell is out of the question with these packages since it is a newer experimental command?

We use both in practice

Am I supposed to make a shell.nix

Yes, what I meant is that you can test pkg-config like so:
$ cat << EOF > pkg-config-shell.nix
with import ./. { };

mkShell {
  packages = [ pkg-config ];
  buildInputs = [ zlib ];
}
EOF
$ nix-shell pkg-config-shell.nix
$ pkg-config --list-all
$ pkg-config --cflags --libs zlib

Update the path in the sed command to patch the .pc files: $out/lib/pkg-config/*.pc.

Github PR web interface is rather convenient to discuss such details!

Update the path in the sed command to patch the .pc files: $out/lib/pkg-config/*.pc.

Github PR web interface is rather convenient to discuss such details!

Oh I think I understand now what you mean!


add scroolbars to wide graphs.

Ref #599
This will use all available space to display the graph.

less file?

What do you mean? I just made som changes in the existing template, css and js for the graph.

Maybe you have to modify less file since you have modified css file.

There is no less file. Its just a standard css file for this feature.

for further improvements, it might be possible to make the graph more compact, but then We would have to change the lib graph.js, this should probably be done upstream for the lib.
It accepts one parameter for "unitSize", We now use default value, which is 20. But If we could set unitSize-x to 10 and unitSize-y to 20, the graph could be more compact and still have the same height. (the y value have to match the font-size used for the text.)
but, I am not sure how many parallel branches there is useful to display. Maybe there should be a limit?

@kjellkvinge Maybe you could add a horizontal scrollbar. Otherwise we'll keep running into the same problem.

LGTM

@kjellkvinge Nitpicking: could reset the width to 80%? I personally think the width of 100% looks kinda ugly :sweat_smile:

@Bwko yes, this is difficult :)
you mean somethink like this:
-#git-graph-container {overflow-x:auto; width:100%}
+#git-graph-container {overflow-x:auto; width:80%; margin-left:10%}

This will bring more air on the sides, but it looks miss placed when you have a repo with only one, or a few branches in parallell.
it also depends on the screen size.
maybe we should keep the "ui container" unless there is more than 5 or 10 paralell branches??  any thoughts?

@kjellkvinge Thanks :+1: I agree it's hard to get it right. I've tried this solution but it still looks kinda weird. Maybe we should keep the width to 100% width and remove the .full class. Perhaps not the best solution but it prevents jumping of the page width.

ok, now trying a new approach. This was originally a PR for using full page width. Now it just adds scrollbars if the graph is to wide. This might solve both maintaining the same layout as the rest of the pages, and still usable even if there is a lot of paralell branches in the graph.

@kjellkvinge I find this to be the best solution thus far.
Does someone else have an opinion on this?

Not to much opinions out there :) Maybe I can provide some screenshots, or a live demo somewhere, so people can test without without rebuilding?

@kjellkvinge Screenshots are good enough (I guess)

LGTM


update to libsass 3.2.0-beta.4

Hey thanks for the great work.
I run into 'Support parent selector & in values / and/or interpolations' (https://github.com/sass/libsass/issues/548). That seems to be fixed in libsass 3.2.0-beta.4.
Anything I can do to help with a new release that includes libsass 3.2.0-beta.4?

@fsmanuel, thanks for the offer. I have a very busy week going on. The only stopper is the recent changes in importer. I will try to spend sometime over the weekend.
@mgreter, here are the changes I was referring to: https://github.com/am11/node-sass/commit/55aadd29ac006899be7972bc970ad3332f8e4b28. It throws at line 11 because Sass_Compiler (comp) is null with the following test:
// in node-sass dir
require('./').render({
  data: '@import \'doesNotExist.scss\'',
  importer: function(url, prev, done) {
    done({file: url, contents: 's { a: ss; }'});
  }
}, function(err, result){
  if(err) {
    console.log(err); return;
  }

  console.log(result, result.css.toString());
});

(hopefully missing something fundamental and not the bug in libsass..)
Note this is the first step to make the previous implementation just work. Once this passes the existing test suite, I will modify it for multi importers support.

+1
Can't wait to try libsass 3.2.0-beta4

Libsass beta4 had a bug which is fixed in their master. We are waiting on the next nightly.

v3.0.0-beta.5 is released pointing to libsass v3.2.0-beta.5: https://github.com/sass/node-sass/releases/tag/v3.0.0-beta.5.

@am11 :+1:

Yay! Great work everyone

Thanks! Much better now but still not quite working. Specifically, singularitygs doesn't compile yet due to using negative numbers in maps keys: https://github.com/sass/libsass/issues/1086


Allow consecutive dashes in hostname

fixes: #529
This allows consecutive dashed in hostname. Thus, http://foo--bar.dk/ is validated as valid URL.

The CI is failing due to a misconfiguration which I have just fixed in #541. If you can base this PR off of the latest master then you should get the updated CI config and your tests should pass.

Thanks for submitting the PR :+1:


modify the project UI

add UI to modify the title and description
only if the logged in user is a project owner, the modify buttons are available

2h


FAQ update: force USB2 mode

update on how to force USB2 mode using code (rather than CLI)

So I'm thinking we should leave in the CLI option as well with depthai_demo.py.  I think that still works in Gen2 depthai_demo.py right @VanDavv ?

Surprisingly there is no fusb2 flag in the demo script, I'll add it quickly in a minute and circle back with PR (and check which other flags might be missing)
Regarding the PR itself, the changes from @cafemoloko just add additional info about USB mode, they do not remove the mention about CLI flags

Ah cool thanks.  I mis-read the diff file I think.


Kafka <IP_ADDRESS>

This is created to track our upgrade to support Kafka <IP_ADDRESS> client.

May I know the status of the upgrade? Is there a timeline as in when the support for Kafka <IP_ADDRESS> will be out? Thanks!

This feature looks very interesting because 0.10.2 client supports all 0.10 servers starting with 0.10.0:
https://issues.apache.org/jira/browse/KAFKA-4462
To me it would make maintaining Logstash Kafka plugin branches 5.x  (0.10.0) and 6.x (0.10.1) unnecessary.

@suyograo in principle this should just work out of the box when upping the version to 0.10.2, any reason not to put in that one-liner PR + bumped version and changelog here?

I like your proposal, Armin. Let's verify the claim that it really works
with all 3 versions (10.2, 10.1, 10.0) before we merge though.
On Wed, May 10, 2017 at 1:12 PM Armin Braun<EMAIL_ADDRESS>wrote:

@suyograo https://github.com/suyograo in principle this should just
work out of the box when upping the version to 0.10.2, any reason not to
put in that one-liner PR + bumped version and changelog here?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/logstash-plugins/logstash-input-kafka/issues/178#issuecomment-300598821,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIC6kzN4Q_LtN4ibGr09ZOp4wWpdh5eks5r4hongaJpZM4MLqbm
.

@jordansissel

. Let's verify

As in manually try it out real quick or as in add an IT to run through those 3 versions (given we seem to have 0 tests involving an actual Kafka broker, this may not be the worst idea)?

Manually is ok for now. Long term I imagine we will add more complex
integration tests, but manual (one time) is good enough to satisfy my
distrust :)
On Wed, May 10, 2017 at 1:48 PM Armin Braun<EMAIL_ADDRESS>wrote:

@jordansissel https://github.com/jordansissel
. Let's verify
As in manually try it out real quick or as in add an IT to run through
those 3 versions (given we seem to have 0 tests involving an actual Kafka
broker, this may not be the worst idea)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/logstash-plugins/logstash-input-kafka/issues/178#issuecomment-300607889,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIC6o5KAbatn3WZLyHTMe7qmFX--wvLks5r4iKggaJpZM4MLqbm
.

@jordansissel alright :) I actually tried this already by accident when looking into another issue. At least 10.1 works (I'll try the other combinations real quick too tomorrow). Two other problems come to mind:

Ok not a problem, but <IP_ADDRESS> is out, I guess we'd rather go for that ? :)
Actual problem: https://github.com/logstash-plugins/logstash-output-kafka/issues/127 => don't we have to upgrade both plugins at the same time and clearly document that you need to keep versions aligned when upgrading? Otherwise some user configurations may end up with a mix of Kafka versions in the class path?

@jordansissel one way to fix the classpath issue (If we actually care) would be to build a separate jar with all the jar dependencies and shade the Kafka library there.
Already did that for https://github.com/logstash-plugins/logstash-filter-useragent

@original-brownbear Lets bump to <IP_ADDRESS> directly.

As in manually try it out real quick or as in add an IT to run through those 3 versions (given we seem to have 0 tests involving an actual Kafka broker, this may not be the worst idea)?

We do have integration tests with real Kafka brokers which runs on travis. See here: https://github.com/logstash-plugins/logstash-input-kafka/blob/master/kafka_test_setup.sh#L8
That needs to be changed as well. Anyhow this should work and integration tests should validate if so. +1 on moving forward with this on both input and output.

Otherwise some user configurations may end up with a mix of Kafka versions in the class path?

This is typically users understanding. They would upgrade both (input and output) or none.

@suyograo here we go: https://github.com/logstash-plugins/logstash-input-kafka/issues/178 sec adding one for output as well

Fixed in #189


Error with numeric subreddits in GDPR mode

shreddit reliably chokes on parsing comments.csv when the subreddit field is all-numeric, e.g. for /r/404 or /r/2012. For example, if I change ...,technology,... to ...,2012,... in the first record of my comments.csv in the GDPR export, I get the following error:
  2023-07-08T21:34:11.153874Z  INFO  Shredding Comments...
    at src/main.rs:52

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error(Deserialize { pos: Some(Position { byte: 63, line: 1, record: 1 }), err: DeserializeError { field: None, kind: Message("data did not match any variant of untagged enum Source") } })', src/sources/gdpr.rs:17:41
stack backtrace:
   0: rust_begin_unwind
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:578:5
   1: core::panicking::panic_fmt
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1687:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1089:23
   4: shreddit::sources::gdpr::list::{{closure}}
             at ./src/sources/gdpr.rs:17:39
   5: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:310:13
...

Maybe the csv Reader is producing a numeric type when there's an all-numeric sequence, and then deserialize fails because the types don't match?
My current workaround is to remove the subreddit: String field from the Gdpr enum variant in comment.rs, as it's not currently used for anything. Removing the offending lines from the .csv should also work.

Note that this problem doesn't happen with the id field, which can also be all-numeric and is also supposed to be a String. This might have something to do with serde and enums.


Landing update

Technology setup
Generally the technical setup is done by a local Platform Service Provider like Grassroots Economics.
The setup is GNU/Linux-first. No guarantee or support will be provided for proprietary OS platforms.
Open Source and Copyleft
Ensure that you trust the software being used. Even if it is open source now, that improvements and upgrades will remain open source.
Even if it is open source, ensure that it is clear and transparent enough to reasonably understand and maintain.
Distributed Ledger
We highly recommend using abstractions that enable portability to other ledger systems. Even if a ledger chain is benevolent and reasonable to use now, does not mean it remains that way forever.
A long term goal should be for communities to collaborate in creating their own ledger systems, where members of the communities hold nodes that decentralize and secure the ledger with low harfdware requirements.
Interfaces / Wallets
Provide and preserve abstractions that enable use of any wallet and network provider.
Offer wallet custody system, aswell as recovery systems for lost keys and passwords.
Data Sharing
Given the consent of the communities, anonymized transaction data can be recorded and displayed.
Provide full understanding and transparency of the scope of data that is shared, and the possible risks and effects caused by data sharing.
Ensure that the intelligence that is yielded from shared data is given back in full to the communities, to own and to do with as they see fit.
Guiding principles
The guiding principles for system design and contribution are described below with emphasis on software development and infrastructure:

Avoid lock-ins: No part of the system should rely on and/or integrate with libraries or services that cannot be reasonably understood and adapted. Nor should the work required to replace a component be unreasonable large as a result of external dependencies. Adaptable dependencies should expose an implementable interface.

Lean dependency graph: The base should make use of primitives and standard libraries as much as possible.

No soft lock-in to virtual environments: Too many modern software projects only provides reasonable ease of use through mechanisms like Docker or similar frameworks. Care should be taken to incur minimum penalty for using the application and tooling in a local host environment. Documentation must exist and be maintained to facilitate this.

Coherent configurations regardless of entry-point: Merged ini/toml-format configuration files constitute configuration schemas and overrides, translating to a configuration directive vocabulary overridable by environment variables of the same name.

CLI tooling framework: Layered definitions of command line arguments, so that concepts common for each layer is only defined in one place. Full and intuitive control of overriding configuration directives with command line arguments and flags. Easy generation of base resource objects (such as rpc, wallet) directly from rendered configurations. Facilitate and encourage use of chainable commands (e.g. UNIX pipelines).

closed in 4259029