Fix Picarto.tv video downloads (not stream)

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2022.02.04. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
USA
Description
When trying to download this stream recording that is going to be auto-deleted soon I found that yt-dlp and JDownloader2 both can't fetch the stream. I found issue #1373 but it's a little old and the error is different, plus I've figured out roughly how to fix this, so I'm opening this new issue to put all the info in one place.
The actual player data is hiding behind a blob.

It loads the video info through a Websocket.

Swap wss:// to https:// and you get playlist info, BUT the playlists only contain a reference to ANOTHER playlist.

This is what's inside the HLS index.m3u8 file:

If you use the relative URL in that file, you'll get the ACTUAL m3u8 playlist for the MPEG-TS segments.

From there, it's a straightforward "download all segments and concat with ffmpeg" strategy. I'm fetching it with wget as I type this, and I've verified the segments are playable.
Verbose log
FYI: I rename to youtube-dl.exe so I don't have to change my scripts; it's actually yt-dlp latest master.

X:\>youtube-dl --throttled-rate 20k --match-filter "!is_live & !live" --cookies "Y:\cookies.txt" --add-metadata --write-description --write-info-json --write-thumbnail --write-subs --write-auto-subs --sub-lang en,en_US --convert-subs srt -ciw -o "%(title)s.%(ext)s" -v https://picarto.tv/DarkCookie/videos/616575
[debug] Command-line config: ['--throttled-rate', '20k', '--match-filter', '!is_live & !live', '--cookies', 'Y:\\cookies.txt', '--add-metadata', '--write-description', '--write-info-json', '--write-thumbnail', '--write-subs', '--write-auto-subs', '--sub-lang', 'en,en_US', '--convert-subs', 'srt', '-ciw', '-o', '%(title)s.%(ext)s', '-v', 'https://picarto.tv/DarkCookie/videos/616575']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2022.02.04 [c1653e9ef] (win_exe)
[debug] Lazy loading extractors is disabled
[debug] Python version 3.9.0 (CPython 64bit) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg 2021-01-12-git-ca21cb1e36-full_build-www.gyan.dev, ffprobe 4.3.1-2020-11-19-full_build-www.gyan.dev
[debug] Optional libraries: Crypto, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [Picarto] Extracting URL: https://picarto.tv/DarkCookie/videos/616575
[Picarto] DarkCookie: Downloading JSON metadata
ERROR: [Picarto] DarkCookie: Stream is offline
  File "yt_dlp\extractor\common.py", line 616, in extract
  File "yt_dlp\extractor\picarto.py", line 50, in _real_extract

As a workaround, once u have the m3u8 url, you can just pass it to yt-dlp. No need to manually download and concat
❯ yt-dlp "https://recording-eu-1.picarto.tv/stream/hls/govod%2bgolive+DarkCookie_2<IP_ADDRESS>.05.50_nsfw.mkv/index.m3u8?video=maxbps"
[generic] index: Requesting header
[generic] index: Downloading m3u8 information
[info] index: Downloading 1 format(s): 2128
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 3684
[download] Destination: index [index].mp4
[download]   0.1% of ~978.54MiB at 50.17KiB/s ETA 06:22:47 (frag 1/3684)
ERROR: Interrupted by user

As a workaround, once u have the m3u8 url, you can just pass it to yt-dlp. No need to manually download and concat

Sometimes I like to see things through the hard way...just to make sure ;-)

Something to add: I've discovered that when a live stream is broken, the video (recording) segments end prematurely, with the m3u8 handing out segments that don't exist on the server. If this gets added, it'll need to handle a consistently failed segment (usually in the 90th percentile completion) as the end of the video rather than an error.


Update libraries (libde265 1.0.12, libaom 3.6.1, libheif 1.16.2)

Depends on https://github.com/carsales/pyheif/pull/86.
Partially based on #85.

Hey @homm are you able to merge the changes from #86 into this?

Rebase on master

Any chance this could get a version bump?


Proof overview rst

Edited overview.rst for typos, clarity, etc.

Thank you very much, @OlyDLG , I like that, but I'll not merge this because the two extra files: .lock & .cache. Would you mind to submit another PR, only modifying the overview?
Probably a good way to do it is to move back to your master branch, create another branch from that, apply the modifications on the overview.rst, and than commit only the overview file on this new branch.
How did you apply that commit? I usually do
git add -p
So I can choose which files, and even which modifications, I want on this commit.
cp overview.rst overview.rst.bk
git checkout master
git checkout -b newoverview
mv overview.rst.bk overview.rst
git add -p
Double check the modifications, and including each one by 'y'
git commit
And we'll be ready to PR and modify the main release.
Thank you!

I edited overview.rst myself, and credited it to @OlyDLG.
Commits:
de0d5f6afc8
abadee277


ci: add dependency-review job

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
555

Relevant Lines:
555

💛  - Coveralls


javascript for-loo.js

sorry guys, someone know the answer for this questionnarie?
For loops allow you to repeatedly run a block of code a certain number of
times. This for loop logs to the console ten times:
 for (let i = 0; i < 10; i++) {
   // log the numbers 0 through 9
   console.log(i)
 }

The first part, let i = 0, is run once at the beginning of the loop. The
variable i is used to track how many times the loop has run.
The second part, i < 10, is checked at the beginning of every loop
iteration before running the code inside the loop. If the statement is
true, the code inside the loop is executed. If it is false, then the loop
is complete. The statement i < 10; indicates that the loop will continue
as long as i is less than 10.
The final part, i++, is executed at the end of every loop. This increases
the variable i by 1 after each loop. Once i reaches 10, the loop will
exit.
The challenge:
Create a file named for-loop.js.
In that file define a variable named total and make it equal the number 0.
Define a second variable named limit and make it equal the number 10.
Create a for loop with a variable i starting at 0 and increasing by 1 each
time through the loop. The loop should run as long as i is less than
limit.
On each iteration of the loop, add the number i to the total variable. To
do this, you can use this statement:
 total += i

When this statement is used in a for loop, it can also be known as an
accumulator.  Think of it like a cash register's running total while each
item is scanned and added up.  For this challenge, you have 10 items and
they just happen to be increasing in price by 1 each item (with the first
item free!).
After the for loop, use console.log() to print the total variable to the
terminal.
Check to see if your program is correct by running this command:
 javascripting verify for-loop.js

Not really an issue related to this repo. Therefore closing this.


Issue 164: improve hashmail proof dist

I've rebased onto the event notification endpoint commits. I'm working on an itest to check that the backoff procedure works as expected: https://github.com/lightninglabs/taro/pull/222/commits/78fb94b2f7386a43cda734838842953f0a8aa2e9
I'm running into a problem where the test doesn't end, it hangs at some point.

Doesn't save state in SQL for the data to be reused again in the state machine.  Merely means it might be just redundant actions.

^ rebase on main

I think we need approval from @Roasbeef because he requested changes in his previous review.


The definition of the use cases.

The definition the user actions on the projects.

I created issues and I wrote the comments.


Escape external links macro

The inclusion_tag registration method doesn't trigger auto-escaping of our fields. This adds escaping to most of the fields (in addition to converting a test file).

Thanks @tadhg-ohiggins


Add Cross Platform Support to Helm

what

Install helm based on OS
Remote local repo by default
Install to /usr/local/bin

why

Make it compatible with OSX
/usr/bin/ is read-only on OS X
local repo is not used by cloudposse

who
@cloudposse/engineering

ok


Python3 support for CDH 5.x

Hi!
Is there any plan to make a CDH 5.x release to allow Python 3.5+ given https://www.python.org/doc/sunset-python-2/ ? Hue is also currently relying on very old dependencies that would need to get upgraded if possible :)

As of today, there is no plan for CDH5/6, only CDP (the new distribution).

@romainr thanks a lot for the answer. Last question - is it possible to use the Hue package on CDH6 with packages in CDH5 as workaround, or due to the Hadoop 2->3 switch nothing will ever work? I am trying to understand if I can workaround this issue, we are not ready to migrate to CDH6 yet but we'd like to avoid running python2 until we do it.

@romainr thanks a lot for the answer. Can I ask why the update will not even be present in CDH6 given the promise of cloudera to support it up to 2022? Running python2 after the next January is not really an option (https://www.python.org/doc/sunset-python-2/).

Please follow https://issues.cloudera.org/browse/HUE-8737 and contact Cloudera support(here it is upstream Hue which is a bit different from Cloudera's distribution)


Support for Qdrant Vector DB

Support for Qdrant Vectors is requested.
More Info: https://qdrant.tech/

@amirkhan-ak-sf what about the connector size we need to keep under control?

Yes, the size need to be under control. We shouldn't exceed the overall size in the mule app of 350 MB. Currently we are at 326.1 MB. @tbolis-at-mulesoft maybe we can think about similar pattern to seperate into 2 connectors (maybe on-premise vs SaaS)?

@amirkhan-ak-sf Let's discuss about this after the new version release.

Thank you @Anush008 we sull’ look into your PR asap. Is there a trial we can use to test it out? What’s the best option to test it? Thx.

Hey @tbolis-at-mulesoft . Qdrant can run locally using Docker.
I've mentioned the steps in the PR description.

Thx awesome.

Quick question… does quadrant offer a way to efficiently query all segments (only metadata, no text and vector) using some sort of pagination? When possible we are offering a list sources method. Thx

Yes.
https://qdrant.tech/documentation/concepts/points/#scroll-points
https://api.qdrant.tech/v-1-12-x/api-reference/points/scroll-points

Awesome we will need then to implement as well the list source method as done already for others (eg. milvus, ai search, pgvector, …)
Thx

I am running into this super weird issue with my VSCode Java extension completely failing to work on the project.
So that's blocking the list sources update.

@Anush008 no problem I'll look into this on Friday.

I switched over to IntelliJ to continue.

@Anush008 I had the same issue. IntellJ is perfect for Java.

listSources() is supported now.


fix: misprints in the MTLS env. variables definition

Changelog: Title
Ticket: None

The current docs don't give result in a working example. We're rewriting this whole section: #2165 It is almost done to give a copy-paste way to set up everything.
I'd reject this and hope to have your review on the other PR soon.

I checked the other one, and you fixed it there, so I think when you merge #2165 you can close this one.


cart has more tabs than fit on screen at once

while this is only a minor problem, it would be nice if all tabs fit on screen at once. possible options:

consolidate some of the commentary tabs
move some of the commentary to another file

resolved by 3b158a653a1b5a0d74c62485041bb36916c4de0f


Error building with pydata-sphinx-theme-0.9.0

Hi,
I've found my pages do not build with the recent release of pytdata-sphinx-theme-0.9.0 and fails with the following error...
[app] emitting event: 'build-finished'(ExtensionError("Handler <function update_templates at 0x7f689e710ee0> for event 'html-page-context'

Traceback (most recent call last):
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/events.py", line 94, in emit
    results.append(listener.handler(self.app, *args))
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/pydata_sphinx_theme/__init__.py", line 155, in update_templates
    sizes=favicon["sizes"],
KeyError: 'sizes'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/application.py", line 330, in build
    self.builder.build_update()
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 286, in build_update
    self.build(to_build,
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 350, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 524, in write
    self._write_serial(sorted(docnames))
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 534, in _write_serial
    self.write_doc(docname, doctree)
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/builders/html/__init__.py", line 625, in write_doc
    self.handle_page(docname, ctx, event_arg=doctree)
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/builders/html/__init__.py", line 1026, in handle_page
    newtmpl = self.app.emit_firstresult('html-page-context', pagename,
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/application.py", line 457, in emit_firstresult
    return self.events.emit_firstresult(event, *args,
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/events.py", line 112, in emit_firstresult
    for result in self.emit(name, *args, allowed_exceptions=allowed_exceptions):
  File "/home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages/sphinx/events.py", line 102, in emit
    raise ExtensionError(__("Handler %r for event %r threw an exception") %
sphinx.errors.ExtensionError: Handler <function update_templates at 0x7f689e710ee0> for event 'html-page-context' threw an exception (exception: 'sizes')

Extension error (pydata_sphinx_theme):
Handler <function update_templates at 0x7f689e710ee0> for event 'html-page-context' threw an exception (exception: 'sizes')

❱ pip show pydata-sphinx-theme
Name: pydata-sphinx-theme
Version: 0.9.0
Summary: Bootstrap-based Sphinx theme from the PyData community
Home-page:
Author:
Author-email:
License:
Location: /home/user/miniconda3/envs/pgfinder/lib/python3.9/site-packages
Requires: sphinx, packaging, docutils, beautifulsoup4
Required-by:

Respected sir,
I have also faced the same error in the new 0.9.0 version. try falling back to 0.8.1 this will resolve the issues.
@ns-rse this can be currently be resolved by using( pydata-sphinx-theme==0.8.1 ) in requirements.txt until a patch has not been submitted.
Thanks and Regards,
Henil Panchal

@henilp105 Thank you already pegged it to 0.8.1 in the GitHub action that builds my documentation GitHub pipeline.

The favicon documentation suggests that you need to provide a sizes keyword...are you not providing this? If not that explains the error:
https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/configuring.html?highlight=favicon#adding-favicons

I have favicons defined but no size, currently with...
html_theme_options = {
    "show_nav_level": 2,
    "favicons": [
        {"rel": "icon", "href": "https://github.com/Mesnage-Org/pgfinder/actions/workflows/ci-tests.yml/badge.svg"},
        {"rel": "icon", "href": "https://codecov.io/gh/Mesnage-Org/pgfinder"},
    ],
}

...and building with pydata_sphinx_theme==0.9.0 fails as per above.
If however I add a size as you suggest...
html_theme_options = {
    "show_nav_level": 2,
    "favicons": [
        {
            "rel": "icon",
            "sizes": "156x20",
            "href": "https://github.com/Mesnage-Org/pgfinder/actions/workflows/ci-tests.yml/badge.svg",
        },
        {
            "rel": "icon",
            "sizes": "115x20",
            "href": "https://codecov.io/gh/Mesnage-Org/pgfinder",
        },
    ],
}

Then the documentation builds fine.

Next step
So I think this is not a bug per-se, but we could improve the UX by defining a default "size" in case the user doesn't present one.

I was building favicon for my documentation and I remembered this issue. I just found this small favicon extention: https://pypi.org/project/sphinx-favicon/. Maybe instead of duplicate the work we could also have a look.

If that seems relatively well maintained, the functionality is there, and it's not a heavy dependency, I'd be +1 on just adding it as a dependency and using it for the favicon functionality


Audio/mpeg

It's not support on mime types audio/mpeg??

@blackmamb41 You filled in the mime type to valid_file_mimetypes, you should go to http://localhost:8000/laravel-filemanager?type=Files and upload your file.

mp3 mimetypes is 'application/octet-stream'. Then ????

@Ariunbold13 fill it into valid_file_mimetypes.

Please follow the upgrade document and upgrade to v1.7.1.


[3.2] Provide support for buttons and D-pads mapped to half axes.

3.2 version of #42799.

Thank you @madmiraal for this work. I did try the build with my 2in1 Controller and it didn't actually change anything of the current 3.2.3 build.
I tried the joypad demo from godot and the results were the same. :(

In this picture I was keep pressing the left arrow on the Dpad. Only that button

@salvob41 Thank you for testing. The picture provides really useful feedback. If you don't mind, please provide similar pictures for pressing each of the other directions on the D-pad.

THis is at the start of the scene:

right button Dpad only:

Up button DPAD only:

Down button DPAD only:

Analogic:
If I move the analogic up and/or right it seems that it keeps stucked in that position
Left or right analogic:

Up or down analogic:

Things to note:

Right joypad, right buttons, Back triggering buttons (R1, L1, L2, R2), and Start button and "select button" work good!
when, as I mentioned earlier, move analogic buttons it seems as I still keep pressing them (example: in this pictures I moved the analogic, but at the time of the picture I am not pressing anything, the joypad is resting on the table with nothing being pressed)

If I move the window of the joypad scene, it seems that the buttons pressed are resetted (weird behaviour)
2in1 controller have a button in the middle that change a behaviour of Dpad and Analogic: that is if I pressed the HOME button the dpad buttons seems to become the same input as the analogic and viceversa for the analogic

Hope this shed more lights on this issue

the pencil in the picture is to give a nice little "arty" touch ...

I've updated this PR to include two new fixes:

Fixes the mapping to a negative half-axis
Fixes the axis mappings to D-Pad buttons not resetting.

@salvob41 Please test it again.

things are happening:
Analogic left now works well. Dpad has still same problems.

Hope you can see here:

Pressing RIGHT triggers the RIGHT arrow on the DPAD in joypads demo - and stay pressed for some reasons
Pressing LEFT does the same
Pressing DOWN triggers the DOWN arrow on the DPAD in joypads demo - and stay pressed for some reasons
Pressing UP does the same

@salvob41 Thanks for all the testing and images. You're my eyes with this one. I think I've fixed the negative axis (i.e up and left) to button mapping; so hopefully it will work now. Please try again.

glad to help, @madmiraal. Thanks for your patience and work.
Dpad still gives problems. I put some testing video here:
https://imgur.com/a/JSwlZEX

Marked as draft, because I've added a debug warning message in the axis to button mapping process. Hopefully this will provide the information needed to work out what is going wrong.
@salvob41 Please test again, moving left, right, up and then down, and post the output console messages.

Good idea putting some debug message.
At the start:
WARNING: joy_axis: Axis0 = 0.501961 -> Button15 = 0.501961
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis1 = 0.501961 -> Button13 = 0.501961
   At: main/input_default.cpp:828.

Nothing happens when I press RIGHT and DOWN buttons.
when I press and release the UP button (this debug print happens at the act of releasing the button)
WARNING: joy_axis: Axis1 = 0.4 -> Button13 = 0.4
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis1 = 0.501961 -> Button13 = 0.501961
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis1 = 0.4 -> Button13 = 0.4
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis1 = 0.501961 -> Button13 = 0.501961
   At: main/input_default.cpp:828.

similar with LEFT button:
WARNING: joy_axis: Axis0 = 0.4 -> Button15 = 0.4
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis0 = 0.501961 -> Button15 = 0.501961
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis0 = 0.4 -> Button15 = 0.4
   At: main/input_default.cpp:828.
WARNING: joy_axis: Axis0 = 0.501961 -> Button15 = 0.501961
   At: main/input_default.cpp:828.

it seems that the last buld works good!! I will test it better in a few minutes

I've added a fix for the mapping not taking into account that an axis range may be only positive even if it represents negative values, and removed the debug message.

Yeahh!  I tested it with my controllers and I confirm that this work!
Congratulation @madmiraal on your work, perseverance and patience.
Great job!

But there is a problem @madmiraal .
If I use the get_action_strength on the left analogic it gives me 0 when not pressed or 1 when pressed. not middle values.
In 3.2.2 it gives me values between 0 and 1

If I use the get_action_strength on the left analogic it gives me 0 when not pressed or 1 when pressed. not middle values.

The controller's mapping takes D-Pad hat events and maps them to axis events. Hat events only contain a direction, not a magnitude, so they can only be mapped to 0 or 1.

Yes.... I am diving into the mapping and there is something weird.
I tried it on a Linux and it recognised the joypad as PS3 controller.
Will try to see what is wrong with the mapping

So I put this mapping:
030000008f0e00000300000009010000,2In1 USB Joystick,a:b2,b:b1,x:b3,y:b0,back:b8,start:b9,leftstick:b10,rightstick:b11,leftshoulder:b4,rightshoulder:b5,dpup:h0.1,dpdown:h0.4,dpleft:h0.8,dpright:h0.2,leftx:a0,lefty:a1,rightx:a2,righty:a3,lefttrigger:b6,righttrigger:b7,platform:Mac OS X
and the analogic works. I only see that when I press the Dpad (it works well) but the buttons seems  to be held pressed.

when I press the Dpad (it works well) but the buttons seems to be held pressed.

Are you saying that when you release the D-pad, the buttons (12, 13, 14 and 15) are remaining pressed?

when I press the Dpad (it works well) but the buttons seems to be held pressed.

Are you saying that when you release the D-pad, the buttons (12, 13, 14 and 15) are remaining pressed?

Yes

@salvob41 I'm confused. Please provide one of your videos with what you're seeing now.

@salvob41 I'm confused. Please provide one of your videos with what you're seeing now.
@madmiraal Sorry for the confusion. Here's a video:
https://imgur.com/a/WyvgzXC

Dpad button works great, but seems the are kept pressed (weird).
Analogic works well (you can see it even shows the strength)

I think I've fixed the bug that was causing the D-Pad buttons to stick.
@salvob41 If you don't mind, please test it again.

@madmiraal Something added recently on 3.2 doesn't make me build from source. I receive this error message:
$ scons platform=osx arch=x86_64 --jobs=$(sysctl -n hw.logicalcpu)
scons: Reading SConscript files ...
xcrun: error: SDK "iphoneos" cannot be located
xcrun: error: SDK "iphoneos" cannot be located
xcrun: error: unable to lookup item 'SDKVersion' in SDK 'iphoneos'
Failed to find SDK version while running xcrun --sdk iphoneos --show-sdk-version.
CalledProcessError: Command '['xcrun', '--sdk', 'iphoneos', '--show-sdk-version']' returned non-zero exit status 1.:
  File "/Users/panoramix/Desktop/coding/notapixelstudio/godot/SConstruct", line 43:
    if detect.can_build():
  File "./platform/iphone/detect.py", line 17:
    if get_darwin_sdk_version("iphone") < 13.0:
  File "/Users/panoramix/Desktop/coding/notapixelstudio/godot/methods.py", line 717:
    return float(decode_utf8(subprocess.check_output(["xcrun", "--sdk", sdk_name, "--show-sdk-version"]).strip()))
  File<EMAIL_ADDRESS>line 411:
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File<EMAIL_ADDRESS>line 512:
    raise CalledProcessError(retcode, process.args,

I should open a issue, but I cannot test it otherwise

Rebased following merge of #43091.

🥳  @madmiraal !! It works! Everything is in place.
Analogic stick: perfect.
Dpad correctly pressed.
Dpad correctly released.
For the 2in1 USB Joystick in particular: This needs to be merged
as the correct mapping https://github.com/gabomdq/SDL_GameControllerDB/pull/391 .
Then we will sync the game_controllerdb.txt in Godot.
Thank you, @madmiraal

Thanks a ton to both of you, amazing work!


Solarized dark theme

Would you be open to implement the Solarized dark theme (or accept a PR)?

Hello Mr. @hlissner, I just wanted to let you know I'm working on it. It currently looks like this:

I used doom-city-lights as a template because it's a similar color pallete and I've been using VS Code as reference as well as the official theme page (https://ethanschoonover.com/solarized/), tweaking the colors until I'm able to reach a point in which it looks as similar as the original as possible. I'll be glad to hear your feedback! This is one of the most used themes overall and I consider doom-themes to be the best theme packages for Emacs so I think this will be a great addition. I need to finish tweaking the terminal colors but I think it's looking good, what do you think?

Hi there! Sorry for the late response. I think this looks great! However, I must refuse merging this: I am currently rewriting doom-themes and its API, and intend to split the package into multiple packages:

One for the theme API (not sure yet if it will be doom-themes-core or if I'll come up with a new name for it)
This repo will be reworked to only contain this package's flagship themes: doom-one, doom-one-light and doom-one-vibrant
And two plugins, one for our neotree theme and the other for treemacs

All our other themes will either be moved into their own individual packages (and submitted to melpa separately) or to https://github.com/Emiller88/emacs-doom-community-themes. I would encourage you to do one of these things. In fact, if you are willing to maintain the solarized theme, I'd be happy to give it to you if you'd like to createa doom-solarized pack or similar.
In any case, the rewrite will be done within the next two weeks.

Hi Henrik! I am willing to maintain my implementation of Solarized, however, may I ask why are you going to separate the package? I think the greatest thing about this plugin is that instead of having a lot of theme packages hanging around in MELPA the user has access to a lot of nice themes in a single package. In fact that's why it has become my favourite theme package, so I'm really intrigued about why you've made this decision.
As a matter of fact I'm implementing my own multi theme package using the same core porting multiple popular themes into one package using the same file structure.
By the way, I've seen that @Emiller88 has taken my Solarized implementation so how should I proceed?

After some thought and discussion, I've come around to keeping the themes together -- well, only for the themes that I am certain I can maintain on my own or with help.
I'll still split the api and extensions out into their own thing, but I want to be more selective about the themes I allow into this megapack.
I had three concerns going into this decision:

The project's scope is too beg to handle on my own: many of its current themes I have never used and are no longer maintained by their original authors, but I still get bug reports for. That's before the requests and PRs.

I'm a bit bothered by the duplication in the emacs ecosystem. Counting this one, there are 3 solarized theme packages, two nord themes, two nova themes, and so on. It doesn't need to be this way. I'd rather encourage people to use the new doom api to improve those ones, rather than bring a copy into this package.

The API is poorly written, terribly inefficient and undocumented, first written when I knew little about Emacs and elisp. The compile-time magic involved with every theme makes M-x package-install RET doom-themes dreadfully slow, and makes it a pain point for any doom theme author aspirant who pulls it in as a base. I can't imagine how it'd be for folks using their own local, un-byte-compiled doom themes.

In any case, the rewrite addresses many of these issues, so I think I can tolerate the workload of maintaining a megapack.

By the way, I've seen that @Emiller88 has taken my Solarized implementation so how should I proceed?

Ah, I forgot to respond to this. I'll merge #303 -- solarized is one theme I'd like in this pack. I realize this confuses things, but I'll remove it from the community pack shortly.

Nice nice, just one thing, are you ok with the org appearance for that theme? I changed it in my local  but I'm not sure if you'd like me to upload those changes


Chad's comments & questions

inline questions & comments added via comments in the code, to facilitate an asynchronous discussion about various aspects of the AssemblyScript example

Partially addresses #3


Propagate error when network interface name couldn't be detected

Cherry pick of upstream PR #2851.

@steven-sheehy thanks!

Coverage decreased (-0.1%) to 16.769% when pulling bd121cb8d4f289387f8b09f135946276eb90622f on steven-sheehy:iface-errors into 5e67002de62ae466bb9d15ba73090baf5e6526ca on aledbf:master.


Story/csi 753 create vol from snapshot

No description provided.

please mention here the actual content change

@igorgonibm don't forget to delete the branch once done


[Feature Request]: Find elements in shadow root in the same way selenium 4 handling it

description of what the problem is :
We need to handle shadow root elements in the same way it is handled with selenium 4
WebElement shadowHost = driver.findElement(By.cssSelector("#shadow_host"));
SearchContext shadowRoot = shadowHost.getShadowRoot();
WebElement shadowContent = shadowRoot.findElement(By.cssSelector("#shadow_content"));
Resources :
https://github.com/titusfortner/website-examples/blob/shadow_dom/java/selenium4/src/test/java/com/titusfortner/selenium4/ShadowDomTest.java#L93-L95
https://titusfortner.com/2021/11/22/shadow-dom-selenium.html?fbclid=IwAR1lYByXhUC8o7C81X89mIQVcSF-OIg8KWWs_eLIRvXzxkXqWKulmcsf810

this was one of the harder requests... thanks for the challenge :)
https://github.com/ShaftHQ/SHAFT_ENGINE/pull/918


Downgrade issue - bun/yarn

Verify canary release

[X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue
see below
What package manager are you using / does the bug impact?
Yarn v1
What operating system are you using?
Mac
Which canary version will you have in your reproduction?
2.0.11
Describe the Bug
I am using bun as a package manager, package.json contains:
  "packageManager"<EMAIL_ADDRESS>The build process is reporting an error (see below) - the regex is fine so there is another issue somewhere between bun and turbo.
invalid_package_manager_field

  × could not parse the packageManager field in package.json, expected to match regular expression (?P<manager>bun|npm|pnpm|yarn)@(?
  │ P<version>\d+\.\d+\.\d+(-.+)?)
    ╭─[/Users/shuther/Documents/devP/web-main1/package.json:38:1]
 38 │   ],
 39 │   "packageManager"<EMAIL_ADDRESS>    ·                     ──────┬──────
    ·                           ╰── invalid `packageManager` field

So I tried to downgrade the migration using:
px @turbo/codemod migrate --to 2.0.9
but I am getting:
? Where is the root of the repo to migrate? .
No codemods required to migrate from 2.0.11 to 2.0.9

Upgrading turbo from 2.0.11 to 2.0.9 (no codemods required)

Upgrading turbo with yarn add<EMAIL_ADDRESS>--dev -W

Migration failed
Unable to upgrade turbo: Error: Command failed: yarn add<EMAIL_ADDRESS>--dev -W
error This project's package.json defines "packageManager": "yarn<EMAIL_ADDRESS>However the current global version of Yarn is 1.22.22.

Presence of the "packageManager" field indicates that the project is meant to be used with Corepack, a tool included by default with all official Node.js distributions starting from 16.9 and 14.19.
Corepack must currently be enabled by running corepack enable in your terminal. For more information, check out https://yarnpkg.com/corepack.

The yarn downgrade is a known issue.
The package manager is not picked up properly.
Migration fails.
Expected Behavior
downgrade should work.
To Reproduce
Source of the project was: https://github.com/saasfly/saasfly
then:
npx @turbo/codemod migrate
Additional context
No response

This is an unfortunate yarn bug, from deep in the archives: https://github.com/yarnpkg/yarn/issues/9015


fix: bitswap query timeout

Summary of changes

Changes introduced in this pull request:

Call cancel if bitswap requests are not responded to after 5s
Update changelog, adding fixed section

Feature is not user-facing
CI log looks correct:
libp2p_messsage_total{libp2p_message_kind="bitswap_block_request_out"} 21
libp2p_messsage_total{libp2p_message_kind="bitswap_block_response_in"} 19
...
network_container_capacities{kind="bitswap_outgoing_query_ids"} 14
network_container_capacities{kind="cx_request_table"} 7
network_container_capacities{kind="hello_request_table"} 14

Reference issue to close (if applicable)

Closes https://github.com/ChainSafe/forest/issues/2398
Other information and links

Change checklist

[x] I have performed a self-review of my own code,
[x] I have made corresponding changes to the documentation,
[x] I have added tests that prove my fix is effective or that my feature works (if possible),
[x] I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Not sure if already ready but looks like I have some stall when syncing (ctrl-c forest after 5 minutes):
./target/release/forest --encrypt-keystore false --target-peer-count 50
2023-01-09T19:05:57.323019Z  INFO forest: Using default mainnet config
2023-01-09T19:05:57.323806Z  INFO forest::daemon: Starting Forest daemon, version 0.4.1+git.0b76e4c5
2023-01-09T19:05:57.324126Z  INFO forest_libp2p::service: Recovered libp2p keypair from "/Users/guillaume/Library/Application Support/com.ChainSafe.Forest/libp2p/keypair"
2023-01-09T19:05:57.324176Z  INFO forest::daemon: PeerId: 12D3KooWQGY99GfNXhHRCoeUnU4cRK9T9TsZA9vpxbqh6gGXmjSp
2023-01-09T19:05:57.324182Z  WARN forest::daemon: Warning: Keystore encryption disabled!
2023-01-09T19:05:57.324612Z  INFO forest::daemon: Admin token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJBbGxvdyI6WyJyZWFkIiwid3JpdGUiLCJzaWduIiwiYWRtaW4iXSwiZXhwIjoxNjc4NDc1MTU3fQ.OdpOVsRgr9cfnc0PgMnm56PXNchzNYQAfaw1nbBJjp0
2023-01-09T19:05:57.363770Z  INFO forest::daemon: Prometheus server started at <IP_ADDRESS>:6116
2023-01-09T19:05:57.370888Z  INFO forest_genesis: Initialized genesis: BlockHeader: Cid(bafy2bzacecnamqgqmifpluoeldx7zzglxcljo6oja4vrmtj7432rphldpdmm2)
2023-01-09T19:05:57.371036Z  INFO forest::daemon: Using network :: mainnet
2023-01-09T19:05:57.373733Z  INFO forest_chain_sync::chain_muxer: Evaluating network head...
2023-01-09T19:05:57.373837Z  INFO forest::daemon: JSON-RPC endpoint started at <IP_ADDRESS>:1234
2023-01-09T19:05:57.374028Z  INFO forest_rpc: Ready for RPC connections
2023-01-09T19:05:57.710737Z  INFO forest_libp2p::service: Running libp2p service
2023-01-09T19:05:58.828298Z  INFO forest_chain_sync::chain_muxer: Local node is behind the network, starting BOOTSTRAP from LOCAL_HEAD = 2499415 -> NETWORK_HEAD = 2499491
Downloading headers 76 / 76 [====================================================================================================================================] 100.00 % 1030.58/s 2023-01-09T19:05:59.005508Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499415, N blocks = 2
2023-01-09T19:06:08.933515Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499416, N blocks = 4
2023-01-09T19:06:12.091095Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499417, N blocks = 5
2023-01-09T19:06:18.812607Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499418, N blocks = 5
2023-01-09T19:06:23.405985Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499419, N blocks = 6
2023-01-09T19:06:25.391653Z  WARN forest_libp2p::service: RPCResponse receive failed
2023-01-09T19:06:37.040348Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499420, N blocks = 11
2023-01-09T19:06:42.816320Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499421, N blocks = 4
2023-01-09T19:06:47.358377Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499422, N blocks = 3
2023-01-09T19:06:50.972237Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499423, N blocks = 2
2023-01-09T19:06:54.735976Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499424, N blocks = 5
2023-01-09T19:06:58.808168Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499425, N blocks = 5
2023-01-09T19:07:04.600038Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499426, N blocks = 1
2023-01-09T19:07:08.119494Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499427, N blocks = 3
2023-01-09T19:07:11.648083Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499428, N blocks = 6
2023-01-09T19:07:15.443440Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499430, N blocks = 2
2023-01-09T19:07:22.537771Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2499431, N blocks = 5
2023-01-09T19:07:52.387728Z ERROR forest_chain_sync::tipset_syncer: Sync messages check state failed for tipset range
2023-01-09T19:07:52.387822Z ERROR forest_chain_sync::chain_muxer: Bootstrapping failed, re-evaluating the network head to retry the bootstrap. Error = TipsetRangeSyncer(NetworkMessageQueryFailed("ChainExchange request failed for all top peers. 26 network failures, 0 lookup failures, request:\nChainExchangeRequest { start: [Cid(bafy2bzacecwbb5qfvfeczfgehoaemnf3zgvbfa4veiljxxj3ug25dhblyhqyq), Cid(bafy2bzacedtmxupghl6m7prauc7m4gcc6a5vkntg6nasoomcpzvpdsuvsp5ps), Cid(bafy2bzaceanspck2ialpeqgud4ffqh7pthtyiyysxw4mflpfdx6i6ktffveiq), Cid(bafy2bzaceda5rpns545fq2buycixrsgw5dsdtko6nkxo4wfalsettti25y5he), Cid(bafy2bzaceb3us2frwohd6rc6ljhfs6yqj7uc6vzxsiaxxfqyvlcecyjaohvp4)], request_len: 4, options: 2 }"))
2023-01-09T19:07:52.387913Z  INFO forest_chain_sync::chain_muxer: Evaluating network head...

^C2023-01-09T19:12:12.788345Z  WARN forest::cli: Got interrupt, shutting down...
2023-01-09T19:12:12.788941Z  WARN forest_libp2p::service: bitswap query cancellation err: sending on a closed channel
2023-01-09T19:12:12.788960Z  WARN forest_libp2p::service: bitswap query cancellation err: sending on a closed channel
2023-01-09T19:12:12.788969Z  WARN forest_libp2p::service: bitswap query cancellation err: sending on a closed channe
```l

@elmattic Thanks for testing! It's caused by a deadlock, trying to read the lock while holding the write lock, now fixed
if let Some((cid, _)) = outgoing_bitswap_query_ids.write().await.remove(&query_id) {
                    metrics::NETWORK_CONTAINER_CAPACITIES
                        .with_label_values(&[metrics::values::BITSWAP_OUTGOING_QUERY_IDS])
                        .set(outgoing_bitswap_query_ids.read().await.capacity() as u64);
...
}

I would demote those info logs, else it will a bit noisy:
2023-01-10T12:09:39.897969Z  WARN forest_libp2p::service: RPCResponse receive failed
2023-01-10T12:09:40.719572Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2501533, N blocks = 7
2023-01-10T12:09:41.547254Z  WARN forest_libp2p::service: RPCResponse receive failed
2023-01-10T12:09:44.273600Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2501534, N blocks = 6
2023-01-10T12:09:48.305293Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2501535, N blocks = 4
2023-01-10T12:09:52.011478Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2501536, N blocks = 4
2023-01-10T12:10:02.388090Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2501537, N blocks = 5
2023-01-10T12:10:07.261786Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 2501538, N blocks = 4
2023-01-10T12:10:10.836646Z  INFO forest_chain::store::chain_store: New heaviest tipset: TipsetKeys { cids: [Cid(bafy2bzacec7nhxp4bhvcur62tgflqfzyruixasspbtvqf5z5k4rvoxfdlbqwa), Cid(bafy2bzacecm3kv4sqrkmyd5nb56jcxzvcpf4pbjt76bfqbyuhja52xworykxa), Cid(bafy2bzaceanvxvll5uik7jajefsidlsipxh5k5xl3w6tnkfbkoc3hdadiv472), Cid(bafy2bzacebu5ppmacnkqpvi3ll6ezvsas5mtm2qzmrjgq4kh4dy5qrmbkum64)] }
2023-01-10T12:10:10.836678Z  INFO forest_chain_sync::chain_muxer: Bootstrap successfully completed, now evaluating the network head to ensure the node is in sync
2023-01-10T12:10:10.836680Z  INFO forest_chain_sync::chain_muxer: Evaluating network head...
2023-01-10T12:10:15.711986Z  INFO forest_libp2p::service: Cancelling bitswap query 1134
2023-01-10T12:10:15.712082Z  INFO forest_libp2p::service: Cancelling bitswap query 19708
2023-01-10T12:10:15.712113Z  INFO forest_libp2p::service: Cancelling bitswap query 21457
2023-01-10T12:10:15.712138Z  INFO forest_libp2p::service: Cancelling bitswap query 8046
2023-01-10T12:10:15.712163Z  INFO forest_libp2p::service: Cancelling bitswap query 27340
2023-01-10T12:10:15.712181Z  INFO forest_libp2p::service: Cancelling bitswap query 17323
2023-01-10T12:10:15.712208Z  INFO forest_libp2p::service: Cancelling bitswap query 27658
2023-01-10T12:10:15.712229Z  INFO forest_libp2p::service: Cancelling bitswap query 28506
2023-01-10T12:10:15.712247Z  INFO forest_libp2p::service: Cancelling bitswap query 10910
2023-01-10T12:10:15.712265Z  INFO forest_libp2p::service: Cancelling bitswap query 28612
2023-01-10T12:10:15.712281Z  INFO forest_libp2p::service: Cancelling bitswap query 29301
2023-01-10T12:10:15.712304Z  INFO forest_libp2p::service: Cancelling bitswap query 29937
2023-01-10T12:10:15.712331Z  INFO forest_libp2p::service: Cancelling bitswap query 2052
2023-01-10T12:10:15.712352Z  INFO forest_libp2p::service: Cancelling bitswap query 22358
2023-01-10T12:10:15.712370Z  INFO forest_libp2p::service: Cancelling bitswap query 22411

I would demote those info logs, else it will a bit noisy:

@elmattic fixed

@hanabi1224 What about internal containers size in bitswap crate?

By reading the code I think it should be fixed by this PR, maybe I can add some metrics to my fork if that makes ur testing easier

@hanabi1224 What about internal containers size in bitswap crate?

By reading the code I think it should be fixed by this PR, maybe I can add some metrics to my fork if that makes ur testing easier

That's ok, I have some logs in my local fork as well, so I can check.

@hanabi1224 What about internal containers size in bitswap crate?

By reading the code I think it should be fixed by this PR, maybe I can add some metrics to my fork if that makes ur testing easier

Yes it would help to add those metrics, just to be sure.
I did some testing and I can see capacity of queries: FnvHashMap<QueryId, Query> increasing like this over a period of 90 minutes:

457,443
917,240
1,834,975

Yes it would help to add those metrics, just to be sure. I did some testing and I can see capacity of queries: FnvHashMap<QueryId, Query> increasing like this over a period of 90 minutes:

457,443
917,240
1,834,975

@elmattic  Sure I can work on that, will update u once it's done

@elmattic metrics updated, looks like it's still leaking HashMap space with my fix, will investigate more, maybe we can merge this PR first as it has completed its goal
https://github.com/ChainSafe/forest/actions/runs/3931143058/jobs/6722023668#step:13:8454
bitswap_container_capacities{type="BITSWAP_DB_QUERIES"} 448
bitswap_container_capacities{type="BITSWAP_REQUESTS"} 224

@elmattic metrics updated, looks like it's still leaking HashMap space with my fix, will investigate more, maybe we can merge this PR first as it has completed its goal

Sure.


[DROOLS-1174] rehash instead of remove deleted RightTuples from the RightTupleMemory before processing right updates

@ge0ffrey Please test this fix with optaplanner and tell me if it is ok for you.

This doesn't seem to fix it, I get:
java.lang.IllegalStateException: Score corruption: the workingScore (-3290) is not the uncorruptedScore (-3190) after completedAction (Lily {8.9} <-> Allison {0.3}):
The corrupted scoreDirector has 1 ConstraintMatch(s) which are in excess (and should not be there):
org.optaplanner.examples.dinnerparty.solver/twoSameJobTypePerTable/level0/[8, SOCIALITE]=-100
The corrupted scoreDirector has no ConstraintMatch(s) which are missing.
Check your score constraints.
at org.optaplanner.core.impl.score.director.AbstractScoreDirector.assertWorkingScoreFromScratch(AbstractScoreDirector.java:418)

@ge0ffrey I updated my PR, please give it a second try

ok to test

I confirm that the first commit only fixed the isolated reproducer but not the original dinner party test. The second commit also fixes the original reproducer.
I'm now running complete OptaPlanner test suite locally to verify no regression was introduced.

Merged by https://github.com/droolsjbpm/drools/commit/abdf21f2d


net: tls: Add DLTS support to secure sockets

This PR adds support to DTLS with sockets API.
A new socket option was added to select DTLS role. A user should specify on a socket whether it should behave as a DTLS server or client (by default it's a client). Of course all options implemented for TLS can be used with DTLS as well.
For starters, DTLS server supports only one connection at a time. If needed, it could be extened in the future to support multiple client connections, yet it would require more sophisticated resource and datagram management (each connection would need it's own mbedtls_ssl_context).
The handshake procedure is almost transparent to the user. It will take place during initial call to sendto/recvfrom. DTLS client will report an error on handhshake failure. DTLS server will silently ignore the error and prepare TLS context for another handshake.
Because DTLS requires extra resources per TLS context and some extra code, it can be disabled by Kconfig if not used.
As DTLS proven itself to be trickier than TLS, there are quite a lot of changes. I've tried to keep them split in a logically separated commits, to keep reviewing simple. Still, let me know if it needs any refactoring.
As for the samples, DTLS support was added for socket echo_client/echo_server. By default, the samples will run both TLS and DTLS simultaneously, 4 connections in total. They can be tested with dtls-client/dtls-server from net-tools as well.

Codecov Report

Merging #9338 into master will not change coverage.
The diff coverage is 28.57%.

@@          Coverage Diff           @@
##           master   #9338   +/-   ##
======================================
  Coverage    52.5%   52.5%
======================================
  Files         201     202    +1
  Lines       25339   25339
  Branches     5293    5293
======================================
  Hits        13304   13304
  Misses       9945    9945
  Partials     2090    2090

Impacted Files
Coverage Δ

include/net/net_ip.h
69.07% <ø> (ø)
:arrow_up:

include/net/socket.h
100% <ø> (ø)
:arrow_up:

subsys/net/lib/sockets/sockets.c
42.44% <ø> (+0.38%)
:arrow_up:

subsys/net/lib/sockets/sockets_internal.h
28.57% <28.57%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a564e64...a190445. Read the comment docs.

@pfalcon do you have any comments to this PR?

This was meant for 1.13

@ceolin Thank you for the feedback, I've pushed the updated branch.

For starters, DTLS server supports only one connection at a time. If needed, it could be extened in the future to support multiple client connections, yet it would require more sophisticated resource and datagram management (each connection would need it's own mbedtls_ssl_context).

So, layering violation catches and bites us again, eh? ;-)
Note that I can't claim to be a UDP usage expert, let alone DTLS one, so I learn as I go, plus some previous thinking (which isn't backed much by practical experience, so is "original research" in Wikipedia terms).
So, a feature of UDP is that it is both datagram and connectionless transport. Indeed, most well-known usages of UDP are "single request - single response" protocols, like DNS or NTP, which fit perfectly "connectionless". But sooner or later need for a conversation-like protocol over UDP may emerge, which means storing state, which effectively means going connection-oriented. And here UDP leaves that to application-level, there may be different choices, and again, I'm not sure about the "best practices" or at least well-known choices here. I can imagine at least 2 options:

Emulate TCP with its accept(). This means keeping "listening" UDP socket, but on first received pkt there create a new pkt and send reply from it. Expecting client to follow this switch, i.e. send initial request to a well-known port, but be ready to receive reply from another port, and then use that port for follow-up communication. This approach pushes client multiplexing down/back to the IP stack, which may be more performant, but may also use more resources. This choice would be seem to be good match for thread-based sever model: threads are already bloat, so having one socket for each thread would be also expected bloat, while would allow to avoid any resource sharing problems.
Multiplex by client address, fully on application-level. This duplicates multiplexing work done by the IP stack (which already multiplexes by port number), but might allow to save resources on creating socket objects just for purpose of multiplexing.

Now getting back to DTLS, how it is different from UDP is that it is inherently connection-oriented, as there's cipher state to keep for each connection (RFC6347 uses term "association"). Still, the RFC allows to keep as much UDP freedom as possible with the statement "It is designed to run in application space, without requiring any kernel modifications." For example, when implementing (or more exactly, integrating) DTLS on application level, it would be still possible to implement "accept-like" model (p.1) above. But pushing it under sockets API, only choice 2 is available.
But then yes, DTLS server implementation must track all clients, an implementation with just 1 connection is nothing but a toy (but a typical starting choice for Zephyr implementations, yeah). This means applying some (user-defined) constraints on the number of clients. YMMV, but (ab)using listen() call for this may be only better than defining bunch of sockopt's. Another problem is garbage collection of clients. All these problems would be easier if done on application level, but instead now need to find workarounds/define interfaces for all this, likely rather rigid (and at the same time probably already hard to match in various "offloading engine" - if viable DTLS offloading exists at all of course).

Due to @jukkar's hastiness, here comes postreview, still raising some points worth attention IMHO.

I waited several days for your comments that never seemed to come, you also did not indicate that you needed more time for this. Also other people commented, Robert send new version which updated the code accordingly. After all this there is normally no need to wait more if people do not indicate they need more time.

I waited several days for your comments that never seemed to come, you also did not indicate that you needed more time for this.

I didn't? https://github.com/zephyrproject-rtos/zephyr/pull/9338#issuecomment-412054498 reads "My suggestion would be not to haste with this, and let other people (not just me) to review it, e.g. until middle of next week.". Anyway, if @rlubos would want to act on some suggestions (nothing critical - beyond multi-client server support which can for sure wait), he surely can do it in other PRs.

@pfalcon Thank you for the feedback, I'll apply the fixes and propose them in another PR.

So, layering violation catches and bites us again, eh? ;-)

I wouldn't say that it's layering that bites us, but resource management. Eventlually, something like point 2 from your proposals should be implemented for DTLS. Yet, as I've found even the simple single-client DTLS implementation quite complex, I've gave up on this for now, as it would surely delay the PR and make it harder to review. Especially, that every DTLS server sample in Zephyr so far, follow this convention (net_app, bare mbedTLS, all use single SSL context, therefore allow to connect only one client).
But I do agree, that it's a limitation and there's a room for improvement.

I wouldn't say that it's layering that bites us, but resource management.

Yeah, but that's part of the layering issue - in classical model, it's flexible and all is left to application to do it in completely in different/orthogonal ways they like (at the expense of having to do it). We're trying to hide that in kernel, which saves apps from dealing with it, but now we need to find good schedule for resource management and implement it, still risking that it may be not flexible enough for some apps.

Yet, as I've found even the simple single-client DTLS implementation quite complex, I've gave up on this for now, as it would surely delay the PR and make it harder to review.

Sure, ack on that, as I told, I barely was familiar with DTLS at all myself.

Especially, that every DTLS server sample in Zephyr so far, follow this convention (net_app, bare mbedTLS, all use single SSL context, therefore allow to connect only one client).

I wanted to check how net_app had it, thanks for confirming that it supports just one connection either. (I remember times when net_app's TCP server supported just one connection ;-) ).

But I do agree, that it's a limitation and there's a room for improvement.

Yeah, we just need to document it as such, and fixing that would definitely be based on the actual need to support multiple DTLS clients per server in implemented in Zephyr, which unlikely to arise soon. Actually, we may need that sooner to define DTLS offloading interface, but even that will be queued-behind TLS offloading for quite some time I guess.
Thanks for responses and the new PR, will proceed to it now!


Investigate prebuilt poisoning in new ArPow tarball build

Previously we separated poisoning into two steps: after the production build, poison assemblies and packages.  Then, after using those poisoned packages in an offline build, check if any of them made it through to the final output.  We no longer really have this distinction of production/offline builds (see https://github.com/dotnet/installer/pull/11091), so we'll have to do something different here.

[Triage]  Closing as this is being addressed by https://github.com/dotnet/source-build/issues/2652.


docker compose doesn't pass cli arguments

sorry to re-open this issue. But now seems that the version on pip updated as well. So I got my version updated today and it produces the same errors (regardless of pip or docker image directly)
registration            | 2021-05-26T14:20:49.003879113Z Usage: matrix-registration [OPTIONS] COMMAND [ARGS]...
registration            | 2021-05-26T14:20:49.003935786Z
registration            | 2021-05-26T14:20:49.003952889Z   a token based matrix registration app
registration            | 2021-05-26T14:20:49.003966510Z
registration            | 2021-05-26T14:20:49.003979523Z Options:
registration            | 2021-05-26T14:20:49.003992439Z   --config-path TEXT  specifies the config file to be used
registration            | 2021-05-26T14:20:49.004005534Z   --version           Show the flask version
registration            | 2021-05-26T14:20:49.004017721Z   -h, --help          Show this message and exit.

my command in docker-compose looks like this
[...]
volumes:
            - "./configs/synapse-registration/config.yaml:/etc/config.yaml:ro"
            - "./configs/synapse-registration/register.html:/usr/local/lib/python3.9/site-packages/matrix_registration/templates/register.html:ro"
            - "./data/matrix-registration/:/data/"
command: /bin/sh -c 'matrix-registration --config-path /etc/config.yaml serve'
[...]

so atm matrix-registration is completly broken for me
The only way that works for me is to change my Dockerfile for the build to install matrix-registration==0.8.0 from pip
Originally posted by @jahlives in https://github.com/ZerataX/matrix-registration/issues/64#issuecomment-848817518

hmm so afaik this help page only appears if no arguments are passed or the -h/--help argument.
You're command looks correct though

@ZerataX
Could hardly imagine that the arguments are not passed, as the very same command does properly work with matrix-registration v 0.8.0

weird is the following message from logs
registration            | 2021-05-28T07:25:05.547871948Z Usage: matrix-registration [OPTIONS] COMMAND [ARGS]...
registration            | 2021-05-28T07:25:05.547940420Z Try 'matrix-registration -h' for help.
registration            | 2021-05-28T07:25:05.547953560Z
registration            | 2021-05-28T07:25:05.547962189Z Error: No such command 'matrix-registration'.

first claims wrong (missing) params then claiming command does not exist at all???

okay it works now with latest version from pip. Had to change config.yaml
As for reasons above I cannot use the image. But also with version from pip I have to open "issues"

client_redirect dos NOT redirect to given url. Instead after register it redirects to register page again
not sure that -m when creating a token does what it should. Registered 3 test users with that token but api/token still shows me its used once only

it will use the last provided --config-path. so you should be able to just overwrite it

if I try it with your image and overwrite with the following command
"Cmd": [
                "matrix-registration",
                "--config-path",
                "/etc/config.yaml",
                "serve"
            ],

I still get
registration            | 2021-05-28T11:32:30.542062146Z Usage: matrix-registration [OPTIONS] COMMAND [ARGS]...
registration            | 2021-05-28T11:32:30.542109005Z Try 'matrix-registration -h' for help.
registration            | 2021-05-28T11:32:30.542118073Z
registration            | 2021-05-28T11:32:30.542125135Z Error: No such command 'matrix-registration'.
registration            | 2021-05-28T11:32:41.920007513Z Usage: matrix-registration [OPTIONS] COMMAND [ARGS]...
registration            | 2021-05-28T11:32:41.920069286Z Try 'matrix-registration -h' for help.
registration            | 2021-05-28T11:32:41.920086826Z
registration            | 2021-05-28T11:32:41.920100459Z Error: No such command 'matrix-registration'.

but anyway that's okay with me as I just can use the latest version from pip and with that it works as expected

no I mean can't you just do:
"Cmd": [
                "--config-path",
                "/etc/config.yaml",
                "serve"
            ],

that should result in:
$ /nix/store/y611r8b86zv0l2r2xnlf408j718f62xl-python3.8-matrix-registration/bin/matrix-registration \
  --config-path /data/config.yaml \
  --config-path /etc/config.yaml \
  serve

where only the last config-path should be used

@ZerataX
tried that which results in the following process being fired up
/nix/store/qy5z9gcld7dljm4i5hj3z8a9l6p37y81-python3-3.8.8/bin/python3.8 /nix/store/y611r8b86zv0l2r2xnlf408j718f62xl-python3.8-matrix-registration/bin/.matrix-registration-wrapped --config-path=/data/config.yaml --config-path /etc/config.yaml serve

but then it throws exceptions that the db cannot be opened
registration            | 2021-05-29T12:45:17.137905524Z Traceback (most recent call last):
registration            | 2021-05-29T12:45:17.137970719Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2338, in _wrap_pool_connect
registration            | 2021-05-29T12:45:17.137987552Z     return fn()
registration            | 2021-05-29T12:45:17.137995117Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 364, in connect
registration            | 2021-05-29T12:45:17.138002701Z     return _ConnectionFairy._checkout(self)
registration            | 2021-05-29T12:45:17.138009755Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
registration            | 2021-05-29T12:45:17.138017387Z     fairy = _ConnectionRecord.checkout(pool)
registration            | 2021-05-29T12:45:17.138024434Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
registration            | 2021-05-29T12:45:17.138031856Z     rec = pool._do_get()
registration            | 2021-05-29T12:45:17.138038726Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 239, in _do_get
registration            | 2021-05-29T12:45:17.138046129Z     return self._create_connection()
registration            | 2021-05-29T12:45:17.138053047Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
registration            | 2021-05-29T12:45:17.138060454Z     return _ConnectionRecord(self)
registration            | 2021-05-29T12:45:17.138067311Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
registration            | 2021-05-29T12:45:17.138074679Z     self.__connect(first_connect_check=True)
registration            | 2021-05-29T12:45:17.138081592Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
registration            | 2021-05-29T12:45:17.138088986Z     pool.logger.debug("Error on connect(): %s", e)
registration            | 2021-05-29T12:45:17.138095986Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
registration            | 2021-05-29T12:45:17.138103421Z     compat.raise_(
registration            | 2021-05-29T12:45:17.138110248Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
registration            | 2021-05-29T12:45:17.138117597Z     raise exception
registration            | 2021-05-29T12:45:17.138126212Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
registration            | 2021-05-29T12:45:17.138134089Z     connection = pool._invoke_creator(self)
registration            | 2021-05-29T12:45:17.138141083Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
registration            | 2021-05-29T12:45:17.138148523Z     return dialect.connect(*cargs, **cparams)
registration            | 2021-05-29T12:45:17.138161494Z   File "/nix/store/pw7vpy2x39sc50ydmqf2klqaymi51cqf-python3.8-SQLAlchemy-1.3.19/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 493, in connect
registration            | 2021-05-29T12:45:17.138169981Z     return self.dbapi.connect(*cargs, **cparams)
registration            | 2021-05-29T12:45:17.138177044Z sqlite3.OperationalError: unable to open database file

guess it tries the /data path to find the db (from first --config-path)

it definitely shouldn't. the db is handled like everything else:
$ matrix-registration --config-path=configA.yaml --config-path=configB.yaml generate
UserColonyTaboo
$ ls db*
dbB.sqlite3
$ grep db configA.yaml
db: 'sqlite:///{cwd}dbA.sqlite3'
$ grep db configB.yaml
db: 'sqlite:///{cwd}dbB.sqlite3'

Are you sure there isn't just a mistake with your config file? sqlalchemy expects an absolute path, so if you want to do a relative path it should look like this
db: 'sqlite:///{cwd}db.sqlite3'

where cwd will be replaced by the current working directory

@ZerataX
I have the following three mounts in my docker-compose
        volumes:
            - "./configs/synapse-registration/register.html:/usr/local/lib/python3.9/site-packages/matrix_registration/templates/register.html:ro"
            - "./data/matrix-registration/:/data/"
            - "./configs/synapse-registration/config.yaml:/etc/config.yaml:ro"

and config.yaml defines db as db: 'sqlite:///{cwd}/data/db.sqlite3'
If I use this command command: ["--config-path","/etc/config.yaml","serve"] together with the image it throws the shown exceptions.

@ZerataX
tried that but still same exceptions. Tried with 3 and 4 / although I think 3 / should be correct. Anyway the pip version works perfectly, so I keep using this one. Thank you for your patience and your work for the registration. Really a great tool that I love to use :+1:

since i can't reproduce this at all and you seem to have found a solution I'll consider this fixed until someone says otherwise ^^


How to build Rocketchat on ubuntu 14.04 ?

Hello, I had install  Rocketchat via  this url<EMAIL_ADDRESS>When I launch " meteor build " nothing happen.
It downloads but nothing.
.
Someone can help me ?

If you just want to run Rocket.Chat, please follow https://rocket.chat/docs/installation/manual-installation/ubuntu/
If you are trying to develop and build a new version, please be aware that the initial download of Meteor, it can take a long time indeed.


Improved toolbox look

Fixed scrollbars for toolbox. Changed to preload instead of lazy load every time on expand for the panels.

LGTM!+1


wsl.exe nor any wslapi.dll Add-type request can't be called from Powershell on Windows for ARM

Is your feature request related to a problem? Please describe.
Lubos is working on WSL enablement for Windows for ARM on his private Surface Pro X in his spare time.
He's trying to avoid using WSL-DistroLaucnher as building it in build.opensuse.org requires cross-compilation by using mingw64 cross compiler test suite, that currently works only on x86 and not aarch64/arm64 (see link bellow).
He wrote a simple PowerShell wrapper that possibly could in the end probably do the trick, however he figured out that Microsoft is providing only Power Shell x86 on Windows for ARM. Therefore this would be dead end for any user-convenient installation.
Uploading application to Windows Store which we can't rebuild in OpenBuildService is a no go for SUSE nor openSUSE as confirmed on our WSL weekly call.
How does openSUSE/SUSE build x86_64 images?
https://build.opensuse.org/project/show/Virtualization:WSL (our arm builds currently simply generate linux aarch64 binaries, instead of windows ones, unlike on x86)
Why is this a problem?
Powershell x86 will not see wsl.exe nor will access the System32 directory with arm libraries. https://en.wikipedia.org/wiki/WoW64  ( https://github.com/lkocman/WSL-PowerLauncher/issues/1 ) this generally limits many use cases for WSL on ARM (I was surprised how many people were using powershell/WSL on x86).
Describe the solution you'd like
I'd like to be able to call wsl.exe and also be able to use [DllImport("wslapi.dll", CharSet = CharSet.Unicode)] in the PowerShell shipped on Windows for ARM, no extra fetching from github and extracting/installing content manually. Therefore I can write a platform agnostic WSL launcher.
Describe alternatives you've considered
Installing Latest PowerShell from github.com release archive. Solves it but this wouldn't work for user who just wants to download arm64 WSL Image from AppStore and have it working.
Porting ming64 cross-compiler suite to ARM and re-build WSL-DistroLaucher (Way too much effort).
Import pre-built ARM binaries of WSLDistroLauncher from VisualStudio into build.opensuse.org (rejected by engineering)
Upload WSL Image builds manually build by VisualStudio (rejected by engineering)
Additional context
PowerShell indeed seems like a very convenient way to solve our WSL/arm64 issue, but the lack of PowerShell for ARM on Windows for ARM default installation completely blocks the effort.

Powershell x86 will not see wsl.exe

Can you access it as C:\Windows\sysnative\wsl.exe?

nor will access the System32 directory with arm libraries.

Well, that seems harder.


Encrypted Slatepack, sender address

For encrypted slatepack wallet shows incorrect sender address. Was testes self case.
@vinayaga07  - I create that ticket base on what you show to me.

Closed because it is correct behavior for self case.

Closed because it is correct behavior for self case.


date picker dd.MM.yyyy

hi there, i'm testing the datepicker within angular 1.2.2 and bs 3 using our common swiss (and german) date format dd.MM.yyyy
just recognised that entering the date value manually in the textbox does not work as expected. it still uses the first number as the month.
http://plnkr.co/edit/1pdDca5KuWOWh3vPVRvz?p=preview

Why is this issue and all its referenced ones are closed? DatePicker still don't work with dd/MM/yyyy format when date is entered manually.
Try the plunker on https://angular-ui.github.io/bootstrap/#/datepicker and enter 24/05/2015
current version is:
angularjs: 1.3.13
bootstrap-ui: 0.13.0

@svarjavand it seems like either this is broken again or @bekos closed it too fast. Reopening.

On a second thought @svarjavand please create a new issue to prevent this one from being polluted. This is closed because the actual issue is fixed; dates with dd.MM.yyyy format work.

@rvanbaalen I can create a new issue but even dd.MM.yyyy format doesn't work.
Try the plunker on the demo page and enter 24.05.2015 in the popup date picker. Selected date will be empty.

Fair enough. Reason enough to keep this open.

That's likely the issue described in #3644 (potential fix in #3682)

With version 0.13 the console shows the following error when typing the date "23.12.2007" (pattern: dd.MM.yyyy) directly after typing 23:
Datepicker directive: "ng-model" value must be a Date object, a number of milliseconds since 01.01.1970 or a string representing an RFC2822 or ISO 8601 date.(anonymous function) @ angular.js:11655render @ ui-bootstrap-tpls.js:1071init.ngModelCtrl.$render @ ui-bootstrap-tpls.js:1059ngModelWatch @ angular.js:23423
Tis is working with version 0.12

I would like to know why the datepicker popup date not updating as expected using dd.MM.yyyy format.
http://plnkr.co/edit/7Fmcsz2rRDsi9U5eaeGz?p=preview


feat: Add RHEL9 support

Add RHEL9 kickstart for bastion host (#160).
Support for openvpn must be removed in an additional PR, because the robertdebock.openvpn packages can not be installed on RHEL9.

Will this also work if user has rhel 8 bastion?.

Will this also work if user has rhel 8 bastion?.

I did not test it with rhel 8.

ok just checking. It might be good to validate this. We dont know how quickly people will shift to rhel 9 bastion.

ok just checking. It might be good to validate this. We dont know how quickly people will shift to rhel 9 bastion.

But, we know that there will be issues. e.g. openvpn can not be installed on rhel9, some package name changed, the default python3 version will change from 3.6 to 3.9, ...

Might be good to keep both ks files then and pick based on rhel type?
wdyt @AmadeusPodvratnik

If agreed, then we should close the PR and use this PR instead: https://github.com/IBM/Ansible-OpenShift-Provisioning/pull/274

We will use the PR #274


Fixed graphics context allocation issue

Title: Resolved Graphics Context Allocation Issue
Description:
Proposed Changes
This PR resolves an issue pertaining to graphics context allocation in the FFFastImageViewManager. The previous implementation utilized older graphics context handling methods, which could potentially lead to errors. This PR updates the code to leverage the newer UIGraphicsImageRenderer API, mitigating any context allocation problems.
Changes Made

Updated FFFastImageViewManager.m to utilize UIGraphicsImageRenderer for managing image context.
Eliminated manual handling of context creation and cleanup, reducing the likelihood of errors.

Graphics Context Handling:
It now employs UIGraphicsImageRenderer, a more modern API introduced in iOS 10. This API handles the creation, rendering, and cleanup of the image context automatically.
Renderer Usage:
It now creates a UIGraphicsImageRenderer object (renderer) and employs it to execute the drawing operations.
Error Handling:
It leverages a more modern API that is less susceptible to context allocation errors.
This PR addresses: #1006

Works well for me in iOS 17

Working fine on iOS 17. Thanks

@DylanVann Please merge

someone please tell me how to install this pull by npm

You can apply a patch, see https://github.com/DylanVann/react-native-fast-image/issues/1002#issuecomment-1742663000

@AdamGerthel I already have a patch applied for fast image in podsec file. Can u tell how can we apply multiple patches to same lib ?

@AdamGerthel I already have a patch applied for fast image in podsec file. Can u tell how can we apply multiple patches to same lib ?

I don't understand exactly what you mean by "I already have a patch applied for fast image in podsec file" - could you be a bit more specific? If you want to understand how to apply a patch for a node module, the best way is to read the usage instructions for patch-package.

Reference in new

react-native-fast-image+8.5.11.patch
If u see this file, i have patch for fastimage which modifies RNFastImage.podspec. Just wanted to understand how can i apply multiple patches to same library

Thank you for this!

@DylanVann please, man, merge this.

@DylanVann  please, merge this.

having the same issue in flutter, how can we solve that?