Downloads from tv.nrk.no no longer works

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.05.10. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2016.05.10

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue
Downloads from NRK Web-TV no longer works.
According to my logs, this used to work up until May 3rd. Since then youtube-dl has failed to download videos from this site, with the error found in the report below.
I've verified in a web-browser that these videoes are available to mobile clients and Flash-enabled browsers like Google Chrome.

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
% youtube-dl --verbose 'https://tv.nrk.no/serie/knoettene/MSUI23006714/12-05-2016'
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'https://tv.nrk.no/serie/knoettene/MSUI23006714/12-05-2016']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.6 - Linux-3.13.0-83-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: avconv 9.18-6, avprobe 9.18-6, rtmpdump 2.4
[debug] Proxy map: {}
[NRKTV] MSUI23006714: Downloading webpage
WARNING: unable to extract thumbnail; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
WARNING: unable to extract duration; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 341, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/nrk.py", line 316, in _real_extract
    self._sort_formats(formats)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 862, in _sort_formats
    raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue required an account credentials please provide them or explain how one can obtain them.

I have the exact same issue. It worked yesterday, but not today. Might be that NRK has changed something in their web/streaming solution. I had an old version of youtube-dl, so I updated to 2016.05.10, but the error was still the same. This is my output:
$ youtube-dl --verbose http://tv.nrksuper.no/serie/den-ultimate-spider-man/MSUI33005613/sesong-3/episode-26
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'http://tv.nrksuper.no/serie/den-ultimate-spider-man/MSUI33005613/sesong-3/episode-26']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.6 - Linux-3.13.0-85-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: avconv 9.18-6, avprobe 9.18-6, ffmpeg N-79139-gde1a0d4, ffprobe N-79139-gde1a0d4, rtmpdump 2.4
[debug] Proxy map: {}
[NRKTV] MSUI33005613: Downloading webpage
WARNING: unable to extract thumbnail; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
WARNING: unable to extract duration; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 341, in extract
return self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/nrk.py", line 316, in _real_extract
self._sort_formats(formats)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 862, in _sort_formats
raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
I noticed I ran youtube-dl from the same Ubuntu-version as josteink, so I tried running it from OS X also, and it was the same result, but --verbose gave a little different output:
$ youtube-dl --verbose http://tv.nrksuper.no/serie/den-ultimate-spider-man/MSUI33005613/sesong-3/episode-26
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'http://tv.nrksuper.no/serie/den-ultimate-spider-man/MSUI33005613/sesong-3/episode-26']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.10 - Darwin-15.4.0-x86_64-i386-64bit
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/bin/youtube-dl/main.py", line 19, in
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 421, in main
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 388, in _real_main
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 352, in init
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1986, in print_debug_header
File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 64, in get_versions
File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 48, in init
File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 105, in _determine_executables
File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 105, in
File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1690, in get_exe_version
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1334, in _execute_child
child_exception = pickle.loads(data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
dispatchkey
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 966, in load_string
raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

Experiencing the same issue. Not all videos on the site are geo-restricted, though; this one shouldn't be.

Running from git master, I can confirm this now again works.
Thanks for the awesomely fast fix :)


[SMALLFIX] Removed explicit type argument in PermissionCheckTest

Removed an explicit type parameter in the PermissionCheckTest class.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9930/
Test PASSed.

LGTM. Thanks.


Major problems with SDK code samples for Swift 2.0 / Xcode 7

I am not going to list all the problems I've had and continue to have with the Swift code samples, but basically I would have thought that given that Swift 2 is now Swift and that XCode 7 is now XCode that code samples in Swift should compile under Xcode 7 and run successfully.  Right now - they don't, even when converted to Swift 2.0.
I have been watching the Issues people have been having and your responses and there are all sorts of crazy 'fixes' like compiling with bitcode off etc, hack plist etc, but this shouldn't be necessary. Call me old fashioned but surely code samples for a major OS in a language the vendor is clearly committed to and that is growing like crazy should just work.  Right now is just a painful time suck.
Maybe I have missed something, maybe there are working code samples out there and I've not found them, and if so I apologize, but if not can you please let me know when we can expect updated SDK and associated code samples that work reliably for todays iOS environment/Swift?  Thanks.  :)

The code sample is currently not compatible with Xcode 7/Swift 2. When we update the samples, we'll update the thread. Thanks.

Thanks for getting back to me.  So the question remains - when is that
likely to be?
On Wed, Sep 23, 2015 at 10:56 AM, Yosuke Matsuda<EMAIL_ADDRESS>wrote:

The code sample is currently not compatible with Xcode 7/Swift 2. When we
update the samples, we'll update the thread. Thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-ios-samples/issues/26#issuecomment-142679810
.

+1

+1

We've updated the samples. Thanks.

Thats great. Thanks. Will check them out right away. :)
On Fri, Oct 9, 2015 at 6:14 PM, Yosuke Matsuda<EMAIL_ADDRESS>wrote:

We've updated the samples. Thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-ios-samples/issues/26#issuecomment-147019897
.

Good job guys. Much appreciated.


Fix logic for setting cid-* classes

Fix an incorrect check for null-or-empty-string.
Only set a cid- class on the body element if the value won't actually be cid-.
This is a cleanup; it doesn't affect site rendering.
/area web-development

hey @sftim  Can i take this issue

/approve
Thanks!


Is there a way to select all the items in a multiple select?

I am trying to find a way to select all the options inside a select, but I can not. Is there an easy way to do that?
Sorry for the trouble, I am learning.

You could have a separate button which copies from your items array to your selected items array in your controller. (Outside of ui-select)
For example from availableColors tomultipleDemo.colors for this ui-select:
<ui-select multiple ng-model="multipleDemo.colors">
    <ui-select-match placeholder="Select colors...">{{$item}}</ui-select-match>
    <ui-select-choices repeat="color in availableColors | filter:$select.search">
        {{color}}
    </ui-select-choices>
</ui-select>

@user378230 could you be more clear about your idea? Thanks

Ok resolved:
i've added the whole array to my ng-model. Thanks!

@ibbatta How did you achieve this ? I want the user to select All / Unselect All option too.
Can you pls help? Thx


nrlvmd.py - ERROR - Daemon process id is unreachable

Hello,
I am running Ubuntu 16.04 LTS and I have a New Relic license key for the New Relic Infrastructure product.
I installed your plugin via apt, following the instructions in your README.  I created the file /etc/newrelic/nrsysmond.cfg and populated it with my license key and hostname.
When I try to start the daemon using sudo systemctl start newrelic-lvm-plugin and then run sudo systemctl status newrelic-lvm-plugin I see this error in the logs nrlvmd.py[31390]: 2017-06-01 11:28:21,967 - ERROR - Daemon process id is unreachable.
I tried to run the script, without demonizing it, which produced the following output:
$> sudo /usr/share/python/newrelic-lvm-plugin/bin/nrlvmd.py -f --log DEBUG start
2017-06-01 11:34:46,469 - WARNING - Starting daemon.
File descriptor 3 (/run/nr_lvm_thinpool.pid) leaked on lvs invocation. Parent PID 2317: /usr/share/python/newrelic-lvm-
2017-06-01 11:34:46,496 - ERROR - could not convert string to float:
2017-06-01 11:34:46,497 - ERROR - Traceback (most recent call last):
2017-06-01 11:34:46,497 - ERROR -   File "/usr/share/python/newrelic-lvm-plugin/local/lib/python2.7/site-packages/daemonize.py", line 245, in start
2017-06-01 11:34:46,497 - ERROR -     self.action(*privileged_action_result)
2017-06-01 11:34:46,497 - ERROR -   File "/usr/share/python/newrelic-lvm-plugin/bin/nrlvmd.py", line 140, in launch_daemon
2017-06-01 11:34:46,497 - ERROR -     datas = set_datas()
2017-06-01 11:34:46,497 - ERROR -   File "/usr/share/python/newrelic-lvm-plugin/bin/nrlvmd.py", line 93, in set_datas
2017-06-01 11:34:46,497 - ERROR -     raise e
2017-06-01 11:34:46,497 - ERROR - ValueError: could not convert string to float:
2017-06-01 11:34:46,497 - ERROR -
2017-06-01 11:34:46,498 - WARNING - Stopping daemon.

Hello,
Could you give me the result of the following command please ?:
sudo lvs --noheadings -o lv_name,lv_size,data_percent,metadata_percent --separator ,

On 2 June 2017 at 03:07, Sebastien Pro<EMAIL_ADDRESS>wrote:

Could you give me the result of the following command please ?:
sudo lvs --noheadings -o lv_name,lv_size,data_percent,metadata_percent
--separator ,

LV-home,952.00m,,
LV-root,9.31g,,
LV-swap,9.31g,,
LV-tmp,476.00m,,
LV-var,2.79g,,
LV-var-log,952.00m,,
LV-var-log-audit,476.00m,,
bastion,1.00g,92.82,
carrot,1.00g,72.71,
carrot-eventchain-workers,10.00g,2.91,
lxc-data,500.00g,0.12,0.25
lxc-os,100.00g,3.39,0.27
monitor,1.00g,99.95,
rabbit,1.00g,73.05,
rabbit-eventchain-mq,10.00g,2.91,

Hello, I've try to make a fix.
You can test it by downloading the file nrlvmd.py from the opened pull request on your server:
curl -O https://raw.githubusercontent.com/WebGeoServices/newrelic_lvm_plugin/fix/monitor_several_volumes/nrlvmd.py
and launch it with the following command line:
sudo python3 ./nrlvmd.py -f --log DEBUG start
If it fix your problem, I'll build a new release.

Hello Sebastien,

You can test it by downloading the file nrlvmd.py from the opened pull
request on your server:
curl -O https://raw.githubusercontent.com/WebGeoServices/newrelic_
lvm_plugin/fix/monitor_several_volumes/nrlvmd.py
and launch it with the following command line:
sudo python3 ./nrlvmd.py -f --log DEBUG start

I followed your instructions above.  At first I got an error saying that
module daemonize wasn't available so I installed that using APT.  Then when
I ran it again, I got this error:
Traceback (most recent call last):
File "./nrlvmd.py", line 14, in
from assets.lvs_commands_datas import lvs_asset
ImportError: No module named 'assets'
I couldn't find a python module called assets, so I can't progress any
further.

My Bad, I make a mistake in my commit.
I have fix it so, you can re-download the file to try again. And you will need an other dependency, the package python-requests that you can install with APT.

@sebastienpro, I installed the python-requests package and re-downloaded your plugin from the URL - https://raw.githubusercontent.com/WebGeoServices/newrelic_lvm_plugin/fix/monitor_several_volumes/nrlvmd.py
Now when I run python3 ./nrlvmd.py -f --log DEBUG start I see this error message.
Traceback (most recent call last):
  File "./nrlvmd.py", line 204, in <module>
    exit(main())
  File "./nrlvmd.py", line 190, in main
    foreground=args.foreground
TypeError: __init__() got an unexpected keyword argument 'foreground'

If I run the command without the -f I get exactly the same error.

Hi,
Sorry to answer back so late.
Apparently the version of daemonize dependency is too old in the ubuntu's repository.
So i've build a .deb with the right dependency, it will be easier.
Here the link to download it:
https://github.com/WebGeoServices/newrelic_lvm_plugin/raw/fix/monitor_several_volumes/packages/python3-newrelic-lvm-plugin_1.1_beta.deb
Install it with the dpkg command:
sudo dpkg -i python3-newrelic-lvm-plugin_1.1_beta.deb
After that you should run the daemon with your first command:
sudo /usr/share/python/newrelic-lvm-plugin/bin/nrlvmd.py -f --log DEBUG start

Hi Sebastien,
It looks like New Relic has implemented monitoring of LVM thin volumes.
I'm now seeing graphs for my LVM thin volumes and am receiving alerts for
disk usage for these volumes.
So, it looks like I don't need this plugin anymore.
Thanks for your efforts!
Mason
On 19 June 2017 at 06:47, Sebastien Pro<EMAIL_ADDRESS>wrote:

Hi,
Sorry to answer back so late.
Apparently the version of daemonize dependency is too old in the ubuntu's
repository.
So i've build a .deb with the right dependency, it will be easier.
Here the link to download it:
https://github.com/WebGeoServices/newrelic_lvm_plugin/raw/fix/monitor_
several_volumes/packages/python3-newrelic-lvm-plugin_1.1_beta.deb
Install it with the dpkg command:
sudo dpkg -i python3-newrelic-lvm-plugin_1.1_beta.deb
After that you should run the daemon with your first command:
sudo /usr/share/python/newrelic-lvm-plugin/bin/nrlvmd.py -f --log DEBUG
start
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/WebGeoServices/newrelic_lvm_plugin/issues/3#issuecomment-309444673,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AV8fDpWtVPujt-0-dF-lfTfa7gqlhaC9ks5sFnvsgaJpZM4NtYFa
.

Hi,
Unfortunately i don't see them on my own newrelic's ... it is on the server's monitoring or in infrastructure?
It's not a big deal, i always have my plugin which work with my config. :-)
Thanks for the feedback.
Sébastien

On 23 June 2017 at 05:25, Sebastien Pro<EMAIL_ADDRESS>wrote:

Hi,
Unfortunately i don't see them on my own newrelic's ... it is on the
server's monitoring or in infrastructure?

I have an infrastructure essentials account.
I configure LVM thin pools on the host machine (where the New Relic
infrastructure agent is running), then create thin volumes on the host
machine and attach them to LXC containers.  The agent is sending the LVM
info to New Relic, displaying it in the storage graphs (%free, %used, etc)
and I'm able to successfully configure alerts for when these volumes begin
to fill up.

Nice, I will try that !
Thanks.


Memory leaks inside FirebasePerformance InstrumentSessionWithConfiguration

[REQUIRED] Step 1: Describe your environment

Xcode version: 12.5
Firebase SDK version: 8.1.1
Installation method: CocoaPods
Firebase Component: Analytics, Crashlytics, RemoteConfig, Core, Performance

[REQUIRED] Step 2: Describe the problem
Steps to reproduce:
What happened? How can we make the problem occur?
We're seeing memory leaks on startup seemingly stemming from FirebasePerformance. On startup, we're calling FirebaseApp.configure(withOptions:) inside func application(_:didFinishLaunchingWithOptions:), followed by a call to initialize another third-party SDK, Optimizely. (That's where the DefaultDataFileHandler.swift in the screenshot below comes from.)

Upon editing our Podfile to remove the Performance subspec, another run via Instruments shows no leaks occur.

Just digging a bit further here, this is the getSession method from the stack trace in my screenshot:
    open func getSession(resourceTimeoutInterval: Double?) -> URLSession {
        let config = URLSessionConfiguration.ephemeral
        if let resourceTimeoutInterval = resourceTimeoutInterval,
            resourceTimeoutInterval > 0 {
            config.timeoutIntervalForResource = TimeInterval(resourceTimeoutInterval)
        }
        return URLSession(configuration: config)
    }

If I just grab the URLSession initialization code and manually trigger it in my application (e.x. by tapping a button), I'll again see the same FirebasePerformance method triggering a memory leak in instruments:
_ = URLSession(configuration: URLSessionConfiguration.ephemeral)

@khaptonstall Thank you for reporting the issue.
I could not reproduce the memory leak with a quickstart app. It looks like some code in your app or the project setup may be a precondition for the memory leak.
We need more information to make progress on the issue. Could you please provide us a sample app where the leak is reproducible to debug? Alternatively I would recommend trying to get more information on the root cause of the leak with the profiler or by inspecting the Memory Graph after the request.

@maksymmalyhin Thanks for the reply. I've created a simple demo application replicating the issue upon tapping the button on screen when the app loads. (You'll just need to add your own GoogleService-Info.plist file. Let me know if you need any specific settings from our application's plist to debug.)
FirebaseLeakDemo.zip
Here's Instruments after tapping the button:

Here's the Memory Graph after tapping the button:

Thank you for the provided details! I can see the memory leak. We will keep investigation possible reasons and fixes. In the meanwhile you may temporary disable instrumentation with Performance.sharedInstance().isInstrumentationEnabled = false which disables problematic Firebase Performance logic.

The reason of the issue I can see so far:

Firebase Performance swizzles URLSession init to set a delegate proxy to be able to get metrics about the requests
According to the docs if the delegate is configured for a URLSession instance then either invalidateAndCancel or finishTasksAndInvalidate method must be called to prevent memory leaks
currently Firebase Performance doesn't call either of these methods

@khaptonstall You can workaround the issue on your end by calling finishTasksAndInvalidate method on your disposable URLSession instance after scheduling last request intended to run on this session. In this case you will be able to benefit from metrics collected by Firebase Performance without memory leaks.

@visumickey Could you please investigate possible Firebase Performance fixes or consider documentation/logs updates to warn about the required session invalidation.

Thanks @maksymmalyhin, that helps me better understand what's going on under-the-hood. I'd still like to see some way of fixing this on Firebase's end. As I reported in my original issue description the source of the URLSession is located in a third-party framework and out of our control. While a PR could be submitted on that third-party repo to call one of the invalidate methods, I don't think it's a reasonable expectation that all third-party frameworks utilizing URLSession should be calling the invalidate methods when they themselves aren't setting up delegates on the session.

@khaptonstall Possibility of the fix will be investigated but so far I cannot see how Firebase Performance can reliably know when a URLSession instance can be safely invalidated. Just for context, we considered alternative way to get requests data without using delegate by using KVO on URLSessionTask
.sate property but encountered threading issue so had to rollback the change.
We will keep you posted on the results.
cc @visumickey.

Marked P1 since the leak could grow large with some apps.

Update: The best way to go forward on this would be to move away from Swizzling on Firebase performance. It is hard for Firebase Performance to understand if a session is not used anymore for making network requests, so Fireperf calling the API finishTasksAndInvalidate would lead to unexpected outcomes. So, a temporary workaround for this issue is to call the API finishTasksAndInvalidate as a part of the session used to make network requests.
In the meanwhile, Firebase Performance would consider moving away from Swizzler to look for alternates.

@visumickey One possible intermediate step might be for Firebase to add some more granular controls as to what gets tracked via Performance.
To go back a step, I raised this issue due to a third-party library which spun up a disposable URLSession. There may be other third-party libraries I'm using now or in the future that will do the same, and I won't be able to access the URLSession in order to call finishTasksAndInvalidate.
Because of this, my only other option is to either set isInstrumentationEnabled to false (shutting down all types of monitoring) or remove Performance entirely. What would be more desirable is if I could instead just shut down the type of monitoring that is problematic in my scenario (network monitoring) and leave other performance metrics enabled (like app start time, screen rendering, etc.).

Hi! It's happening to me. Actually, I'm calling finishTasksAndInvalidate after the request is done but it has no effect on Firebase Performance's memory leak.

In my scenario, URLSession exists only for one request. After it is done, we call the finishTasksAndInvalidate method.


PSR-11 dependency injection containers compatibility with Workerman workers

Hi,
I am interested to know how Workerman workers behave when we have PSR-11 dependency injection containers in the php application?
PSR-11 specification: https://www.php-fig.org/psr/psr-11/
popular containers include:

Laravel container
PHP-DI (https://php-di.org/)
etc...

please check and share your inputs
Thank you

Workerman works fine with them.

This means, my php class to extend Worker class, that has to be called similar to http server or web socket server example right?

This means, my php class to extend Worker class, that has to be called similar to http server or web socket server example right?

Sorry, I can't understand what you said

what I meant is,
my class has psr-11 compatible dependency injection container utilized in it, so when integrating workerman, we will define number of worker instances know, and when we do that, that many instances happen with the same dependency injection container in each?
I hope, my queries will be cleared as I integrate workerman with my code over and above trying existing code samples, will try soon

I don't know much about Laravel container/PHP-DI, so I can't give you a clear answer. If you want to know about something with them, the best way is to actively test them.

sure will do, thank you


Publish to npm

http://plugins.jquery.com/
jQuery now recommends publishing to npm. I can do this if you want, but maybe you want your name on it etc.

Yeah, probably makes sense if I do it. I'm currently looking to resolving an issue with kinetic scrolling so will publish once that's resolved.

:+1:

:+1:

This is done now
https://www.npmjs.com/package/jquery-scrollify

thanks @lukehaas


Build worker fails due to "Cannot find type definition file for xxx"

🐞 bug report
Affected Rule
The issue is caused by the rule:
ts_library
Is this a regression?
Not sure.
Description
When building a ts_library in worker mode, the build will fail if the rule has transitive @npm dependencies and their @types targets are not included as a direct dependency.
🔬 Minimal Reproduction
tsconfig.json
{
  "compilerOptions": {
    "allowSyntheticDefaultImports": true,
    "allowUnreachableCode": false,
    "allowUnusedLabels": false,
    "experimentalDecorators": true,
    "forceConsistentCasingInFileNames": true,
    "lib": [
      "esnext",
      "dom",
    ],
    "moduleResolution": "node",
    "noFallthroughCasesInSwitch": true,
    "noImplicitReturns": true,
    "preserveSymlinks": true,
    "pretty": true,
    "strict": true,
  },
  "types": [],
}

index.tsx
import * as React from 'react';

BUILD.bazel
ts_library(
    name = "ts_default_library",
    srcs = ["index.tsx"],
    deps = [
        "@npm//@types/react",
        "@npm//react",
    ],
)

🔥 Exception or Error

$ bazel build --strategy=TypeScriptCompile=worker //test:ts_default_library

INFO: Analysed target //test:ts_default_library (0 packages loaded, 2 targets configured).
INFO: Found 1 target...
ERROR: /Users/xiaoyi/Projects/depot/test/BUILD.bazel:3:1: Compiling TypeScript (devmode) //test:ts_default_library failed (Exit 1)
error TS2688: Cannot find type definition file for 'classnames'.
error TS2688: Cannot find type definition file for 'connect'.
error TS2688: Cannot find type definition file for 'dom4'.
error TS2688: Cannot find type definition file for 'events'.
error TS2688: Cannot find type definition file for 'express'.
error TS2688: Cannot find type definition file for 'express-serve-static-core'.
error TS2688: Cannot find type definition file for 'faker'.
error TS2688: Cannot find type definition file for 'google-libphonenumber'.
error TS2688: Cannot find type definition file for 'google-protobuf'.
error TS2688: Cannot find type definition file for 'history'.
error TS2688: Cannot find type definition file for 'hoist-non-react-statics'.
error TS2688: Cannot find type definition file for 'http-proxy'.
error TS2688: Cannot find type definition file for 'http-proxy-middleware'.
error TS2688: Cannot find type definition file for 'jest'.
error TS2688: Cannot find type definition file for 'jwt-decode'.
error TS2688: Cannot find type definition file for 'loglevel'.
error TS2688: Cannot find type definition file for 'long'.
error TS2688: Cannot find type definition file for 'memory-fs'.
error TS2688: Cannot find type definition file for 'mime'.
error TS2688: Cannot find type definition file for 'node'.
error TS2688: Cannot find type definition file for 'object-path'.
error TS2688: Cannot find type definition file for 'pluralize'.
error TS2688: Cannot find type definition file for 'prop-types'.
error TS2688: Cannot find type definition file for 'qiniu-js'.
error TS2688: Cannot find type definition file for 'range-parser'.
error TS2688: Cannot find type definition file for 'react'.
error TS2688: Cannot find type definition file for 'react-dom'.
error TS2688: Cannot find type definition file for 'react-router'.
error TS2688: Cannot find type definition file for 'react-router-dom'.
error TS2688: Cannot find type definition file for 'recompose'.
error TS2688: Cannot find type definition file for 'serve-static'.
error TS2688: Cannot find type definition file for 'tapable'.
error TS2688: Cannot find type definition file for 'uglify-js'.
error TS2688: Cannot find type definition file for 'underscore'.
error TS2688: Cannot find type definition file for 'webpack'.
error TS2688: Cannot find type definition file for 'webpack-dev-middleware'.
error TS2688: Cannot find type definition file for 'webpack-dev-server'.
error TS2688: Cannot find type definition file for 'webpack-env'.
error TS2688: Cannot find type definition file for 'yup'.

🌍  Your Environment
Operating System:

OS X 10.14.4

Output of bazel version:

Build label: 0.24.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Apr 2 16:32:47 2019<PHONE_NUMBER>)
Build timestamp:<PHONE_NUMBER>
Build timestamp as int:<PHONE_NUMBER>

Rules version (SHA):

urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.29.0/rules_nodejs-0.29.0.tar.gz"],
sha256 = "1db950bbd27fb2581866e307c0130983471d4c3cd49c46063a2503ca7b6770a4",

"<EMAIL_ADDRESS>  version "0.29.0"
  resolved<EMAIL_ADDRESS>  integrity sha512-Dp5ucrE1vXTORGiwEi6Ur4dlICpLsmZ1dscsEQT4ywF7xTT0/NmIG0ecBghiCFPFQTxt1D05TR3SH06rPtTAew==

Anything else relevant?

Thanks this is a great bug report.
The problem is that TypeScript reads the @types directory and discovers all these typings files, because it has an aggressive and non-hermetic automatic discovery of ambient (non-imported) typings.
Under Bazel we only allow reads of files that are declared as inputs to an action, ensuring that the typescript compilation could be run remotely for example.
Our typical solution for this is to change tsconfig.json to disable that ambient typings discovery using "compilerOptions": { "types": [] }. Then in a compilation unit which actually requires some ambient typings we use the ///<reference types="foo"/> syntax to explicitly "import" that typing.
Some ideas:

we should improve the error message to indicate the possibility that typescript discovered a typing that isn't in the deps, and suggest what to do
figure out why TypeScript automatic discovery picks up these typings - I know we implemented a readDirectory in the compilerHost, maybe we can fix there
we could have Bazel always add the empty types option to your tsconfig (ideally we should do this only if the user didn't explicitly declare it, which means it should happen in tsconfig.ts where we read the options and set defaults)


Custom Admin Page not working

When I try to create a custom admin page for my application using the code snippet provided in the documentation, it does not seem to work.
# admin.py file in my user_organization Django app

from django.views.generic import TemplateView
from unfold.admin import ModelAdmin
from unfold.views import UnfoldModelAdminViewMixin

class AddUserOrganizationView(UnfoldModelAdminViewMixin, TemplateView):
  title = "Custom Title"  # required: custom page header title
  permissions_required = () # required: tuple of permissions
  template_name = "core/user_organization/templates/add_user_organization.html"

class CustomAdmin(ModelAdmin):
  def get_urls(self):
    return super().get_urls() + [
      path(
          "add-user-organization",
          AddUserOrganizationView.as_view(model_admin=self),  # IMPORTANT: model_admin is required
          name="add_user_organization",
      ),
    ]

The error I get when trying to visit http://localhost:8000/admin/add-user-organization  is-
Page not found (404)
  Request Method:	GET
  Request URL:	http://localhost:8000/admin/add-user-organization
  Raised by:	django.contrib.admin.sites.catch_all_view

And I think this might be the issue because CustomAdmin is not getting registered.
How can I work around this issue?

You must register CustomAdmin to make it work.

But for doing that I would require a corresponding / proxy model.
Is there any way to it without a model?


Update Developer Guide with Implementation Details

Describe the implementation of current/ future enhancement(s)
Progress:

[x] @prokarius
[ ] @wn96
[ ] @xantho09
[x] @Kelly9373
[x] @FongYuan

"Each of you were supposed to add about 1 page of content (including some UML diagrams) to the Developer Guide by v1.2, so that we can give you feedback on your DG-writing and UML-modelling skills. If you did not do that, please do that soon or else you lose this opportunity to recieve feedback. Once all team members have done that, please post a comment in your team PR (here) requesting the tutor to review new additions to the DG."


Handling more than 2 levels

This is the only feature keeping me from jumping ship from MkDocs to Hugo with this theme. :) Glad to see it's already being investigated!

Love the theme, and likely to use it for our team's documentation once the third level of pages has been implemented... How far along is this solution?

At what stage is the development of this enhancement?

@materkey not really started. Code base changed a lot with latest commits, so first PR on this subject might not suit with current code base.
Do you want to work on it ?

Hello @matcornic, on my forked version, i've coded 5 levels of menus.
https://github.com/vjeantet/hugo-theme-docdock
Feel free to cherry pick the code as you want.

everything happens in menu.html partial
caveat : menu structure depends on "parent-child" info set on each content page.


BAH-2758 | Update error messages on registration page [Lite Env]

BAH-2758 requirement was to update the error messages for registration page for lite-env
sample changes are as follows

@binduak @kk-tw

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Abhishek seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Plugin is broken on macOS Sierra

I've received several reports that the plugin is broken on macOS Sierra.
One of the bug reporters mentioned he had tweaked the Xcode project file to make it work; once he's sent it over I'd really appreciate any help verifying that the issue is fixed as I don't currently have access to a machine with Sierra.

I am running Sierra, if you send the fixed .bundle to me, I'll test if it works.
Though I can only respond with "works" or "does not work" due to lack of coding experience.

Thanks! I'm waiting to hear back; I'll let you know when I have a test build available.

I'll email you what I have later tonight when I'm back on my laptop.  I have the main "functionality" working, but if I remember right trying to go to the preferences pane crashes the client.
The twitch.tv API also now requires a client-id to be passed, so a setting option would probably need to be added to allow people to add their own appid to make API calls.

@Hiramas I have an updated bundle available here, please let me know if you run into any issues. I wasn't able to reproduce the preferences pane crash on Yosemite.
@stapler2025 thank you for notifying me about the Client-ID requirement 👍

Hey, I tested it. Moving this version into the "Extensions" folder crashes the client on startup.
Are there logfiles I can provide?

Thanks, I was able to get an old machine updated to Sierra and reproduced the crash issue. I replaced the linked bundle with a patched version, would you mind giving it another shot? If it still crashes, please email me at the address listed on my profile and we can dig through some logs.

No crash with this version.
The emotes work when coming from twitch.
I don't know if that is a natural limitation but when you write the emote in textual it shows as text in the client. Comes through as an emote in twitch chat itself, though.

Yeah, supporting emotes in sent text is on the to-do list. Unfortunately it has to be handled completely differently than received emotes--the Twitch server tells you exactly where emotes are in received messages, but there is no such information available in sent messages.
Thanks for verifying this test build! It's now available as v0.0.2.


Bump production release number for 1.26

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm

/approve
/unhold


dep ensure -update

This is a:
go get -u in the master branch
and then in this branch add dep ensure -update; which the changes committed first and then the deletes

Codecov Report

Merging #697 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #697   +/-   ##
======================================
  Coverage    58.2%   58.2%
======================================
  Files          37      37
  Lines       10121   10121
======================================
  Hits         5891    5891
  Misses       3169    3169
  Partials     1061    1061

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9cd01b...f5c4e2b. Read the comment docs.

looks good to me.

thanks
merging this (because smaller) instead of #696


Add math-related intrinsics/functions for JsValues

Adds handling for BigInt, among other things.

Thanks! Overall though I'd prefer to keep the set of intrinsics and such to a minimum unless there's a good reason to expand things. Could you elaborate a bit more on the motivation for this? It seems, for example, that some of the conversions (e.g. From<JsValue> for f64 aren't necessarily idiomatic in Rust since they can be fallible. If it's applying to a specific JS operation then ideally that'd either be done with the Reflect structure from js-sys but if necessary it seems fine to have an intrinsic.

@alexcrichton JS operators generally don't have corresponding methods, so Reflect isn't an option in those cases. I could remove the Object.is intrinsic, though. From definitely shouldn't be fallible, I'll fix that. But do you think it's a problem if traits like Add are fallible?

I've removed the Object.is intrinsic and made the From<JsValue> f64 impl into a TryFrom

Can you elaborate more on the motivation for this? This is a lot of library support with a lot of traits and a lot of intrinsics. It's a good deal to maintain in what should ideally be a pretty lean core library.

The motivation for this is mostly to allow performing mathematical operations on JS bigints. There currently isn't any other method in wasm-bindgen for handling these values. And the only way to add that functionality is to allow accessing the JS math operators, because there aren't any equivalent methods. Note that plan to make a follow-up PR to add a BigInt type to js-sys to expose this functionality in a more strongly-typed manner, so I'm not particularly attached to the current API in this PR.
The only two intrinsics this PR adds that aren't useful for bigints are the unary +,  >>>, and in. I added these for completeness (if every other JS operator is going to be made available, why leave out these three?), but I'm not attached to them. I suppose + and >>> could be marginally useful when dealing with objects with a custom Symbol.toPrimitive, but I don't think that's a very common use case.

follow-up PR to add a BigInt type to js-sys to expose this functionality in a more strongly-typed manner

I've been working on this, but I've run into a roadblock: the BigInt() constructor doesn't use new, and as far as I can tell wasm-bindgen doesn't have any facilities to account for this.

Ah ok, if possible then I think it would be best to bind BigInt in js-sys without adding new features to wasm-bindgen itself. Perhaps we could try to work through the issues that you're seeing there?
For example for the BigInt constructor you could do something like:
impl BigInt {
    pub fn new(arg: &JsValue) -> BigInt {
        // manually call the `BigInt` name bound as a function
    }
}

where it's not necessarily pure extern "C" { ... } blocks but that's ok.

Ready for review

This BigInt support is working perfectly for me! 👍🏼

@alexcrichton The num-traits dep is just there to allow easier integration with other Rust crates that also use num-traits, you don't miss out on features by not enabling it. There's a standalone pow method you can use instead of its Pow trait, and all of the other trait impls are trivial.

Makes sense, yeah, but I'd prefer to leave it out and keep wasm-bindgen at least somewhat slim in deps.


removing .idea directory

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: babaknaderi sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


[Site Request] tagesschau.de

Checklist

[ x] I'm reporting a new site support request
[ x] I've verified that I'm running yt-dlp version 2021.09.25
[ x] I've checked that all provided URLs are alive and playable in a browser
[ x] I've checked that none of provided URLs violate any copyrights
[ x] The provided URLs do not contain any DRM to the best of my knowledge
[ x] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

Single video: https://www.tagesschau.de/multimedia/video/video-924833.html
Single video: https://www.tagesschau.de/multimedia/sendung/ts-45041.html
Playlist: https://www.tagesschau.de/100sekunden/index.html

Description

German news from ARD (public service broadcaster)

There is already an extractor. Unfortunately it's broken at the moment.


Why not support train images without ground truth?

The Class CustomDataset's method prepare_train_img ignored images without ground truth.
So why do this? Training with images without ground truth can reduce the false positive and this is important.

Duplicate with #425 .


ScheduleLock doesn't support SPEL with Spring references from v5.1.0

With this PR #1298, a SPEL that works before, now is not working:
The next configuration:
@SchedulerLock(name = "#{shedlockCouchbaseDataSource.getAvailabilityShedlockName()}", lockAtMostFor = "120m", lockAtLeastFor = "1m")

where shedlockCouchbaseDataSource is a Spring bean with a public method: getAvailabilityShedlockName()
was working with version 4.43.0 (until 5.0.1),
but from 5.1.0 doesn't work any more.
Until v5.0.1, SpringLockConfigurationExtractor.getName() was resolved by org.springframework.util.StringValueResolver implemented by org.springframework.beans.factory.config.EmbeddedValueResolver that uses org.springframework.context.expression.StandardBeanExpressionResolver. It is work fine.
But, from v5.1.0, the SPEL implementation delegates into org.springframework.expression.spel.support.SimpleEvaluationContext, that doesn't support beans references.
In SimpleEvaluationContext javadoc:

A basic implementation of EvaluationContext that focuses on a subset of essential SpEL features and customization options, targeting simple condition evaluation and in particular data binding scenarios.
In many cases, the full extent of the SpEL language is not required and should be meaningfully restricted. Examples include but are not limited to data binding expressions, property-based filters, and others. To that effect, SimpleEvaluationContext is tailored to support only a subset of the SpEL language syntax, e.g. excluding references to Java types, constructors, and bean references.

Expected behavior
A schedulerlock must be created with name returned by shedlockCouchbaseDataSource.getAvailabilityShedlockName()
Actual behavior
An exception is launched:
2024-02-05 17:01:45,983 [availability-scheduler1] ERROR com.xxx.yyy.config.ShedlockConfiguration - Scheduled task threw an exception: EL1007E: Property or field 'shedlockCouchbaseDataSource' cannot be found on null
org.springframework.expression.spel.SpelEvaluationException: EL1007E: Property or field 'shedlockCouchbaseDataSource' cannot be found on null

Hi, I have added a test for it here and it works as expected. What am I missing?

Btw. the embeddedValueResolver is still used here.

Hi, I have added a test for it here and it works as expected. What am I missing?

I don't understand. I am checking this test.

Hi, I have added a test for it here and it works as expected. What am I missing?

In our case, we don't use @Schedule (the task is scheduled by code) and we have parameters in the method.
@SchedulerLock(name = "#{shedlockCouchbaseDataSource.getAvailabilityShedlockName()}", lockAtMostFor = "120m", lockAtLeastFor = "1m")
public void execute(final String param1) {
...
}

Then embeddedValueResolver is not executed because it fails before in parseSpel (getEvaluationContext doesn't return an optional empty)

Cool, I will look into that. Thanks

Fixed in 5.11.0

Thank you @lukas-krecan

@lukas-krecan I'm using Spring Boot 2 thus it seems I need to use 4.x.x but I need this feature of using parameter to lock name. Will it be supported in 4.x.x?

If I am not mistaken, it works in older versions of ShedLock. This ticket is about the fact that it was broken in 5.0.1


Add views for the reference sets of a project

Pull Request Test Coverage Report for Build 714272560

9 of 29 (31.03%)  changed or added relevant lines in 2 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-1.3%) to 72.062%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

dtf/templatetags/dtf/custom_filters.py
5
7
71.43%

dtf/views.py
4
22
18.18%

Files with Coverage Reduction
New Missed Lines
%

dtf/templatetags/dtf/custom_filters.py
1
60.68%

Totals

Change from base Build 711863042:
-1.3%

Covered Lines:
650

Relevant Lines:
902

💛  - Coveralls


Is this support horizontal bar?

Hi kingwang09,
Thanks for your plugin. Is this support horizontal bar chart? I couldn't found it in your example

@magicvn
Hi
Thank you for using react native pure chart.
Sadly, we have a next plan to implement horizontal bar chart
but it is not supported now.
We'll try our best to support mixed chart soon.
Thank you.


docs: rename source-app

What does this PR do?
we will have three docs entities so this is partial revert of #16676

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

the true docs build is here: https://github.com/Lightning-AI/lightning/actions/runs/4290369163/jobs/7474316858


Default speed limits via westnordost/osm-legal-default-speeds

For those cases where no maxspeed is given, osm2lanes could test out https://github.com/westnordost/osm-legal-default-speeds to get a set of default speed limits per country.
Unfortunately that would only work if no maxspeed is given; otherwise osm2lanes would need to compare what is received (or interpreted) from OSM with what the library responds with and try to merge both, which I image would be hard given the complexity of the table that the library relies on.

Thanks for  forwarding!
If maxspeed is explicitly given, we could probably trust it. Maybe in the eventual web editor interface, we could warn if the tagged value disagrees significantly with the legal default.
We may have to figure out how to actually call this library, but it runs natively and on web, so it should be possible. Or maybe we just consume https://github.com/westnordost/osm-legal-default-speeds/blob/master/demo/distribution/legal_default_speeds.json (or rather, use the Python script to generate up-to-date values)

Unfortunately that would only work if no maxspeed is given

Reverse search is implemented too:
https://westnordost.github.io/osm-legal-default-speeds/#tags=maxspeed%3D100&cc=DE

Or maybe we just consume https://github.com/westnordost/osm-legal-default-speeds/blob/master/demo/distribution/legal_default_speeds.json

So in other words, port the code that interprets that JSON (to Rust) (?)
Possible, it is about ~1000 lines of code (without test code).
About 700 lines of code is the parsing of the filter expression syntax as used in the second table on the wiki page, the rest is the matching logic.
However, the library should be callable from native code as Kotlin can be compiled to native. Though, I have never tried it and don't know what it would take to do that.

However, the library should be callable from native code as Kotlin can be compiled to native. Though, I have never tried it and don't know what it would take to do that.

We'd also need to compile to WASM to run in web browsers, but this should work too. It's more a question of how much it complicates the build process to have non-Rust dependencies. I can investigate this when I find more time to focus on osm2lanes


Added more accurate GPU Timing

Problem:
From doing GPU Captures & Traces, I've noticed that adding the GPU Markers together doesn't paint the full picture of how long a frame is being drawn. GPU Idle time,
Solution:
Added new Timer: RENDER_FRAME, to capture the time between PostDrawScene() & the Submit in Draw().
Caveat:
This does mean it'll capture the GPU Idle time between the compute & submit, but overall I've found this to be more accurate when testing changes.

Noticed that both checkGlError() & rendering the FrameTimerOverlay were adding almost a whole millisecond to the CPU timer, both of these are only ever opened in developer builds, so we should try to remove them whilst the UI is opened


Fix broken link and mark up problem.

This fixes a broken link (erroneous syntax) and also fixes up
markup that was causing quote mode when it should not have been.
Fixes: #3448

Codecov Report

Merging #3452 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3452   +/-   ##
=======================================
  Coverage   80.62%   80.62%
=======================================
  Files         390      390
  Lines       78643    78643
  Branches     8885     8885
=======================================
  Hits        63402    63402
  Misses      13844    13844
  Partials     1397     1397


[now-client] Fix file permissions for source files

There was a regression since 16.3.0 (now-client refactor) causing file permissions for source files to be not be preserved.
The solution was to add the FS mode to each file.
Fixes #3172

Codecov Report

Merging #3180 into canary will increase coverage by 1.36%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           canary    #3180      +/-   ##
==========================================
+ Coverage   12.47%   13.83%   +1.36%
==========================================
  Files         267      267
  Lines       10110    10249     +139
  Branches     1302     1238      -64
==========================================
+ Hits         1261     1418     +157
- Misses       8756     8769      +13
+ Partials       93       62      -31

Impacted Files
Coverage Δ

src/util/prefer-v2-deployment.ts
56.25% <0%> (-5.66%)
:arrow_down:

src/util/projects/get-project-by-id-or-name.ts
25% <0%> (-3.58%)
:arrow_down:

src/util/metrics.ts
70.58% <0%> (+2.4%)
:arrow_up:

src/util/dev/builder-cache.ts
84.76% <0%> (+2.51%)
:arrow_up:

src/util/dev/yarn-installer.ts
90.74% <0%> (+5.02%)
:arrow_up:

src/util/dev/builder.ts
75.32% <0%> (+5.85%)
:arrow_up:

src/util/dev/server.ts
66.83% <0%> (+7.97%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7f8f7c...19e3af5. Read the comment docs.


High-level version of lax.scatter variants for more intuitive use of scatter

lax.gather has its high-level counterpart as jax.numpy.take_along_axis, which is similar to torch.gather. However, for scatter operations there is no such counterpart, and one has to use lax.scatter variants, which require understanding of semantics of XLA's scatter: https://www.tensorflow.org/xla/operation_semantics#scatter.
To this day, I still don't understand the semantics, and I think so are many users. When they want to translate PyTorch to JAX, this may pose a problem given the ubiquity of scatter operations. I wrote a high-level scatter similar to PyTorch using nested vmaps for example (not sure if this implementation is the most straightforward one ... if not, plz lmk). I think it'd be great to add something like this to jax.numpy.
import jax
import jax.numpy as jnp
from functools import partial
from jax import vmap

def scatter(input, dim, index, src, reduce=None):
   # Works like PyTorch's scatter. See https://pytorch.org/docs/stable/generated/torch.Tensor.scatter_.html

   dnums = jax.lax.ScatterDimensionNumbers(update_window_dims=(), inserted_window_dims=(0,), scatter_dims_to_operand_dims=(0,))

   if reduce is None:
       _scatter = jax.lax.scatter
   elif reduce == "add":
       _scatter = jax.lax.scatter_add
   elif reduce == "multiply":
       _scatter = jax.lax.scatter_mul

   _scatter = partial(_scatter, dimension_numbers=dnums)
   vmap_inner = partial(vmap, in_axes=(0, 0, 0), out_axes=0)
   vmap_outer = partial(vmap, in_axes=(1, 1, 1), out_axes=1)

   for idx in range(len(input.shape)):
       if idx == dim:
           pass
       elif idx < dim:
           _scatter = vmap_inner(_scatter)
       else:
           _scatter = vmap_outer(_scatter)

   return _scatter(input, jnp.expand_dims(index, axis=-1), src)

index = jnp.asarray([[0, 1], [1, 0]])
src = jnp.asarray([[1, 2], [3, 4]])
input = jnp.zeros((2, 2)).astype(jnp.int32)

output = scatter(input, 0, index, src)
print(output)

Thanks for the comment. For what it's worth, JAX does offer a number of high-level wrappers around XLA's scatter, which are discussed in https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.ndarray.at.html#jax.numpy.ndarray.at
I think the root issue here is that what is meant by "scatter" in pytorch is quite different than what is meant by "scatter" in XLA.

I was looking at it; If I'm understanding the function, I think a simpler implementation of pytorch's scatter_ using these higher-level wrappers might look something like this:
def scatter2(input, dim, index, src, reduce=None):
  assert index.shape == input.shape == src.shape
  idx = jnp.meshgrid(*(jnp.arange(n) for n in index.shape))
  idx = tuple([*idx[:dim], index, *idx[dim + 1:]])
  return getattr(input.at[idx], reduce or "set")(src)

Thank you very much for your response! Admittedly, I didn't know that there's ndarray.at option, and your solution is very elegant! However, I tried your scatter on the above example and got a result slightly different from PyTorch's output. With a different example, I got "size incompatible" error. Maybe there was an error somewhere.

Yeah - I'm not sure I totally understand what pytorch's scatter is meant to do (particularly when input shapes don't match), but I suspect it can be implemented in terms of XLA's scatter using an approach similar to the one in my comment.

@AranKomat There is still no implementation of PyTorch scatter in JAX. I have tested def scatter(input, dim, index, src, reduce=None):   idx = jnp.meshgrid(*(jnp.arange(n) for n in input.shape), sparse=True)   idx[dim] = index   return getattr(input.at[tuple(idx)], reduce or "set")(src), it outputs different results with PyTorch. Can you provide information about how you solve this finally?

@guozhiyu It's impossible to say what you should do without a minimum reproducible example. Can you share concrete inputs and outputs in both JAX and pytorch that differ?

Thank you for your reply! I tested the example Aran provided:
index = jnp.asarray([[0, 1], [1, 0]])
src = jnp.asarray([[1, 2], [3, 4]])
input = jnp.zeros((2, 2)).astype(jnp.int32)

output = scatter(input, 0, index, src, reduce='add')
print(output)

output is
[[1 4]
 [2 3]]

, but in PyTorch,
index = torch.tensor([[0, 1], [1, 0]])
src = torch.tensor([[1, 2], [3, 4]],dtype=torch.int32)
input = torch.zeros((2, 2),dtype=torch.int32)
output = input.scatter_(0, index, src, reduce="add")

the output is
[[1, 4],
[3, 2]]

For what it's worth, I never intended to imply that the four-line scatter function in my comment above was a full-featured semantic equivalent of pytorch's scatter, so it's not surprising that it produces different results in some cases. I still don't fully understand the indended semantics of pytorch's scatter... For what it's worth my suggestion would be to re-express your intended computation in terms of JAX's native primitives, rather than trying to replicate pytorch's native primitives in JAX.
From your example inputs, perhaps this is what you're looking for?
import jax
import jax.numpy as jnp

index = jnp.asarray([[0, 1], [1, 0]])
src = jnp.asarray([[1, 2], [3, 4]])

out = jax.vmap(lambda x, i: x[i], in_axes=1, out_axes=1)(src, index)
print(out)
# [[1 4]
#  [3 2]]

@guozhiyu Hi. The post was made such a long time ago, so I can't guarantee if my implementation of PyTorch scatter works or not, but this is what I've found from my old repo:
def scatter(input, dim, index, src, reduce=None):
    # JAX-port of PyTorch's scatter. See https://pytorch.org/docs/stable/generated/torch.Tensor.scatter_.html
    # One can simplify this implementation with ndarray.at (ref: https://github.com/google/jax/issues/8487)

    dnums = jax.lax.ScatterDimensionNumbers(update_window_dims=(), inserted_window_dims=(0,), scatter_dims_to_operand_dims=(0,))

    if reduce is None:
        _scatter = jax.lax.scatter
    elif reduce == "add":
        _scatter = jax.lax.scatter_add
    elif reduce == "multiply":
        _scatter = jax.lax.scatter_mul

    _scatter = partial(_scatter, dimension_numbers=dnums)
    vmap_inner = partial(vmap, in_axes=(0, 0, 0), out_axes=0)

    for _ in range(len(input.shape)-1):
        _scatter = vmap_inner(_scatter)
    swap = lambda x: jnp.swapaxes(x, dim, -1)
    input, index, src = list(map(swap, (input, index, src)))
    return swap(_scatter(input, jnp.expand_dims(index, axis=-1), src))

@AranKomat Your code works well. Thank you very much!


create user from api

create user from api (remove from system menu and add only in page users)
users/insert_user.php
params:

email,
password,
role

fixed


Add ability to Clear registries

I couldn't find a way to clear a whole registry.
In fact with proc macro way of measuring I couldn't find a way to clear even a single metric;
Maybe just the documentation is lacking?

Hey, seems like I missed to derive Clear on registries -- even though individual metrics must implement it.
It's a pretty straightforward fix in metered-macro, would you like to try build the PR?


Using in React

Hi, do you know if I should use the same installation steps for a React app? This has been my experience:

Installed with 'yarn add algotrader'
Added 'const algotrader = require('algotrader');'
Compile fails.

I really like your work, thank you for making it available.
ERROR in ./node_modules/algotrader/objects/broker/robinhood/User.js
Module not found: Error: Can't resolve 'fs' in 'C:\AndroidAppDev\xcharts-react-webview\node_modules\algotrader\objects\broker\robinhood'
 @ ./node_modules/algotrader/objects/broker/robinhood/User.js 19:9-22

I realized this is an issue with webpack, so sorry about posting it here. If someone has a clean solution, please post it. I found I could replace 'fs' with 'fs-web', but for others like 'net', I haven't found a replacement.


Fix wrong log for fuse chown method

Can one of the admins verify this patch?

Codecov Report

Merging #12826 (0d44738) into master (1835fa4) will decrease coverage by 25.50%.
The diff coverage is 0.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1835fa4...0d44738. Read the comment docs.

alluxio-bot, merge this please


Data Loss Issue with Hudi Table After 3 Days of Continuous Writes[SUPPORT]

Description:
I have created a Hudi table named ods_table_v1 using the following SQL command:
sql
CREATE TABLE if not exists test_simulated_data.ods_table_v1(
id int,
count_field double,
write_time timestamp(0),
_part string,
proc_time timestamp(3),
WATERMARK FOR write_time AS write_time
)
PARTITIONED BY (_part)
WITH(
'connector'='hudi',
'path'='hdfs://masters/test_simulated_data/ods_table_v1',
'table.type'='MERGE_ON_READ',
'hoodie.datasource.write.recordkey.field'='id',
'hoodie.datasource.write.precombine.field'='write_time',
'compaction.async.enabled'='true',
'compaction.schedule.enabled'='true',
'compaction.trigger.strategy'='time_elapsed',
'compaction.delta_seconds'='600',
'compaction.delta_commits'='1',
'read.streaming.enabled'='true',
'read.streaming.skip_compaction'='true',
'read.start-commit'='earliest',
'changelog.enabled'='true',
'hive_sync.enable'='true',
'hive_sync.mode'='hms',
'hive_sync.metastore.uris'='thrift://h35:9083',
'hive_sync.db'='test_simulated_data',
'hive_sync.table'='hive_ods_table'
);
This table, ods_table_v1, has continuous data writes. However, after three days of continuous writes, I noticed an issue with the data. When querying the table for all data, I found that the earliest batch of written data is missing. No matter what conditions I add, I cannot retrieve the earliest written data.
I am urgently seeking answers to understand the cause of this data loss. Has anyone encountered a similar issue with Hudi tables? Is there a known issue or configuration mistake that could have led to this? Any help or guidance would be greatly appreciated.
Thank you in advance for your time and assistance.

By the way, I performed the operation using Flink SQL.

what hudi release did you use then, we did fond a weird data loss issue about compaction in release 0.14.0, it is fixed in master and 1.0.x branch now.
Are you talking about streaming read data loss or batch queries?

We are using Hudi version 0.14.1, and we have tried both streaming reads and batch queries, but we cannot read the earliest written data. If this is the issue you mentioned, we will try upgrading to the master branch.

How did you write the earliest data set, are they got updated or just got lost?

The data is written through the flink-mysql-cdc method, from January 1, 2024, to March 31, 2024, with 10,000 records being written to MySQL every day. After completing one round of writing, it starts writing from the first day again and continues to cycle for several rounds. However, when I query the Hudi table, I can only find all the data for February 25, 2024, and cannot find any other data. By the way, we have configured metadata synchronization to Hive, and all the written data can be found from the Hive end.

I can only find all the data for February 25, 2024, and cannot find any other data. By the way, we have configured metadata synchronization to Hive, and all the written data can be found from the Hive end.

What engine did you use when you found the data loss?

We discovered that the incomplete data retrieval was due to the use of Flink-SQL, specifically a simple "select * from hudi_ods_table" query.

Could you please provide your contact information in China, as I have noticed you are located in Hangzhou? It would be helpful for further communication.

Upon further testing after upgrading to the new master version, we have discovered missing data. As per our testing expectations, the results for all days should be consistent and equal to the data from the first day. However, as evident from the screenshot attached, the data for subsequent days is inconsistent. I have confirmed that the entire data system has been stopped for more than half an hour, ruling out the possibility of any pending or unfinished data processing.

Did you specify the preCombine field correctly?

I appreciate the clarification. While I'm not entirely sure about the significance of preCombine, I've learned from the developer that hoodie.datasource.write.precombine.field is set to write_time. the write_time is a timestamp field representing the time when data was written, formatted like '2024-01-01 18:59:25.000000000'.
Could you elaborate on the impacts or benefits of this setting? For instance, how does it enhance data processing, query efficiency, or data consistency?

@juice411 precombine field is used as ordering field to deduplicate. For example if we have two records in source with same record key, then hudi will pick up the record with higher precombine value and skip the other one. This happens when we use upsert operation type. For bulk_insert and insert it will insert both of them.

And can you also supplement the Hudi and Flink release you use here?

@danny0405 The previous versions we were using were Hudi 0.14.1 and Flink 1.17.2. Also, we believe our issue is not related to the precombine field as we have a unique ID to identify each data entry.

Is there a domestic contact information available here?

During our testing, we've encountered an unusual issue with the Hudi stream read table. When the downstream processing system fetches data from the upstream Hudi table (designed as a stream read table) and attempts to process it, it reports that it cannot find log files. Obviously, this is expected since the data has been merged into Parquet files. However, the question remains: why is Hudi still searching for these non-existent files?
This issue is causing inconsistencies in the downstream processing results, leading us to believe that the downstream system might not be able to fully capture all the data from the upstream table. We're eager to understand the root cause of this behavior and if there are any recommended workarounds or configurations that we should be aware of.

@danny0405
As shown in the screenshot, Flink has created a job that fetches data from an upstream Hudi table and performs a count calculation. However, from the screenshot, it appears that after several minutes, the job has only fetched 6 records from the upstream table. This raises the question of where the other data might have gone.
How can I obtain the full dataset from the upstream Hudi table?

How can I obtain the full dataset from the upstream Hudi table?

Specifies the read.start-commit as earliest. By default the streaming source consumes from the latest commit of a Hudi table.

@danny0405 We set read.start-commit as earliest because it did not work as expected, and we have been very anxious about it. We have tried various solutions to obtain the full data set, but none of them worked. This issue has been plaguing us for more than three days. Is there any other approach we can take?

Can you share you source table definitions?

CREATE TABLE if not exists test_simulated_data.ods_table_v1(
id int,
count_field double,
write_time timestamp(0),
_part string,
proc_time timestamp(3),
WATERMARK FOR write_time AS write_time
)
PARTITIONED BY (_part)
WITH(
'connector'='hudi',
'path'='hdfs://masters/test_simulated_data/ods_table_v1',
'table.type'='MERGE_ON_READ',
'hoodie.datasource.write.recordkey.field'='id',
'hoodie.datasource.write.precombine.field'='write_time',
'compaction.async.enabled'='true',
'compaction.schedule.enabled'='true',
'compaction.trigger.strategy'='time_elapsed',
'compaction.delta_seconds'='600',
'compaction.delta_commits'='1',
'read.streaming.enabled'='true',
'read.streaming.skip_compaction'='true',
'read.start-commit'='earliest',
'changelog.enabled'='true',
'hive_sync.enable'='true',
'hive_sync.mode'='hms',
'hive_sync.metastore.uris'='thrift://h35:9083',
'hive_sync.db'='test_simulated_data',
'hive_sync.table'='hive_ods_table'
);

CREATE TABLE if not exists test_simulated_data.ods_table_v1(
id int,
count_field double,
write_time timestamp(0),
_part string,
proc_time timestamp(3),
WATERMARK FOR write_time AS write_time
)
PARTITIONED BY (_part)
WITH(
'connector'='hudi',
'path'='hdfs://masters/test_simulated_data/ods_table_v1',
'table.type'='MERGE_ON_READ',
'hoodie.datasource.write.recordkey.field'='id',
'hoodie.datasource.write.precombine.field'='write_time',
'compaction.async.enabled'='true',
'compaction.schedule.enabled'='true',
'compaction.trigger.strategy'='time_elapsed',
'compaction.delta_seconds'='600',
'compaction.delta_commits'='1',
'read.streaming.enabled'='true',
'read.streaming.skip_compaction'='true',
'read.start-commit'='earliest',
'changelog.enabled'='true',
'hive_sync.enable'='true',
'hive_sync.mode'='hms',
'hive_sync.metastore.uris'='thrift://h35:9083',
'hive_sync.db'='test_simulated_data',
'hive_sync.table'='hive_ods_table'
);

It should work for option 'read.start-commit'='earliest',, what is the current behavior now, comsuming from the latest commit or a very specific one?

but the issue is that we can't access older data.

If you table is ingested in streaming upsert, then you just specify the read.start-commit as the first commit instant time on the timeline, and skip the compaction. Only  instant that has not been cleaned can be consumed.
It actually depends on how you write the history dataset, because bulk_insert does not guarantee the payload sequence of one key, so if the table is boostraped with bulk_insert, the only way is to consume from earliest.

@juice411 Do you have any other help on this. Please let us know if you are good. Thanks.

@juice411 Closing this one. Feel free to reopen or create a new one for any more queries on this.


RTMP log errors when trying to use Reolink camera

Which version are you using?
v0.17.3
Which operating system are you using?

OS

[X] Linux
[ ] Windows
[ ] macOS

Architecture

[X] amd64
[ ] arm64
[ ] arm7
[ ] arm6

Describe the issue
Using rtsp-simple-server to offload my IP cameras and ive got an issue with one of my Reolink cameras that I try to use via RTMP. The Reolink camera works great with other instances such as Blue Iris etc.
Its a Reolink RLC-510A.
I am running ten IP cameras via rtsp-simple-server and they all work great, all of them except the Reolink uses RTSP.
My (simple) config (I have omitted the other working cameras in the config below)
paths:
  cam6_ch1:
    source: "rtmp://192.168.xx.xx:1935/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=xxx"
  cam6_ch2:
    source: "rtmp://192.168.xx.xx:1935/bcs/channel0_sub.bcs?channel=0&stream=0&user=admin&password=xxx"

The errors from the log (floods the logs) as soon as one or more clients connects to rtsp-simple-server
2021-09-24T09:09:00.790810949Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.790881653Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.790937313Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.790975873Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791063723Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791081429Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791103565Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791144911Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791194040Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791237847Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791302166Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791352627Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791434083Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791480926Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791536459Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791614319Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791649366Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791733750Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791753730Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791776650Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791868930Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:00.791884903Z 2021/09/24 09:09:00 W [0/15] [RTMP] [conn <IP_ADDRESS>:33062] unable to decode video track: invalid FU-A packet (non-starting)
2021-09-24T09:09:02.300346100Z 2021/09/24 09:09:02 I [0/14] [RTMP] [conn <IP_ADDRESS>:33062] ERR: write tcp <IP_ADDRESS>:1935-><IP_ADDRESS>:33062: write: broken pipe
2021-09-24T09:09:02.300379550Z 2021/09/24 09:09:02 I [0/14] [RTMP] [conn <IP_ADDRESS>:33062] closed
2021-09-24T09:09:05.006067443Z 2021/09/24 09:09:05 I [0/14] [RTSP] [conn <IP_ADDRESS>:59642] opened

Describe how to replicate the issue

Quite hard if you do not have a Reolink camera at hand.

start the server
publish with ...
read with ...

Same issue here... RTMP streams are not working...

Closed since the author didn't provide enough data.

Hello, I am having the same issue when using rtsp-simple-server built with "go build ." I do not have this issue with the prebuilt windows binary.


Bug: Bootable USB process failed on reboot step

TL,DR: Reboot step seems to work on Intel processors but not AMD.
Firstly, thanks @remay for reviving the Hudl! I did eventually get this working by using a different computer.
I initially tried this process on one computer, and all seemed to go well until the reboot step of the script where the Hudl seemed to be bricked. I retried a few times on this device with no luck. I decided to try with another computer, and the process worked perfectly! Now I was curious as to why the process had initially failed.
I tried re-flashing on a few different computers I have lying around the house, and I have found that I can only get the reboot step to work when using devices with an Intel processor (other devices were AMD). This could be a coincidence based on other hardware of the machines I had available, but I thought it might be worth pointing out in case anyone else experiences similar issues.
I might look into this further if I find some time.

The flashing and remote control is very sensitive to cheap/bad USB cables.  I've not seen evidence of anything that's specific to Intel vs AMD processors, but it's possible that the USB support is slightly different I suppose.   If you do get a chance to investigate further I'd ensure the USB cable are high quality and if it still fails I'd try putting a powered USB hub between the computer and the Hudl to see if that made any difference.
I recommend flashing using the SD Card method as it removes a lot of variables.
Thanks for the feedback, and I'm glad you got it working in the end.


React.PureComponent Implementation

Playing around with this based on Dave Doty's question in the gitter chat...
Thoughts on this approach? The tests seem to confirm behavior that aligns with the JS component, but should we interop the JS React.PureComponent instead?
The only difference is... is the shallow comparison of children a good idea? Or should children always result in updates just like the JS version? We could even disallow children in the component using propTypes...
@greglittlefield-wf @kealjones-wk @joebingham-wk

@aaronlademann-wf I assume your discussion about children is related to this comment from the React docs: Furthermore, React.PureComponent’s shouldComponentUpdate() skips prop updates for the whole component subtree. Make sure all the children components are also “pure”.
For what it's worth, I would be in favor of two implementations (or a way to toggle between them), one that assumes the children are pure and skips their re-rendering if the parent props didn't change, and another version that attempts to re-render them to be on the safe side.
I assume that the following is true of the latter case: if the children are also pure and also implementing PureComponent, then they won't ultimately be re-rendered either, but unnecessary computation will be done by calling their shouldComponentUpdate methods (unnecessary because, since they are pure, they will all return false).
The difference in "error modes" is that the former could skip necessary re-renders, resulting in bugs, whereas the latter could call children props comparison code unnecessarily, resulting in slowdowns. Although I tend to favor something that's more likely to be correct, even if slow, it's also going to throw people to have a default that differs from the ReactJS implementation.
I also have to imagine it would be rare for someone to use a PureComponent with non-pure children. (Now watch me come up with a use case for it in my own project next.)


Getting Started (README.md)

Verify Required Applications
If you aren't using node 6.11.4 and<EMAIL_ADDRESS>your project may not build correctly. If that is the case, consider using nvm or Docker as an alternative to actually installing a different version of Node on your machine.
@abettermap Could you rewrite this on how to use nvm to make this project run.
Believe You just did nvm use and it correctly grabbed the version from the .nvmrc file.

Sure thing, should have time this morning.


[SPARK-31219][YARN] Enable closeIdleConnections in YarnShuffleService

What changes were proposed in this pull request?
Close idle connections at shuffle server side when an IdleStateEvent is triggered after spark.shuffle.io.connectionTimeout or spark.network.timeout time.
Why are the changes needed?
Idle connections to YarnShuffleService could never be closed, and will be accumulating and taking up memory and file descriptors.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Existing tests.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

cc @vanzin @xuanyuanking @tgravescs who have worked on this module

Did we observe that the number of half-open connections of YarnShuffleService rises continuously? Seems the connection should be reused for different applications.

Yes, that's what we saw on our cluster. Those channels were idle as clients had closed and not used anymore.

so to clarify, you are saying that if connections timeout (from spark.shuffle.io.connectionTimeout or spark.network.timeout) then you see this build up of idle connection, but if they don't timeout then it doesn't build up?

@xuanyuanking @tgravescs
Here are detailed timelines of our investigation.

We found connections on our clusters building up continuously (> 10k for some nodes). Is that normal ? We don't think so.
We looked into the connections on one node and found there were a lot of half-open connections. (connections only existed on one node)
We also checked those connections were very old (> 21 hours). (FYI, https://superuser.com/questions/565991/how-to-determine-the-socket-connection-up-time-on-linux)
Looking at the code,  TransportContext registers an IdleStateHandler which should fire an IdleStateEvent when timeout. We did a heap dump of the YarnShuffleService and checked the attributes of IdleStateHandler. It turned out firstAllIdleEvent of many IdleStateHandlers were already false so IdleStateEvent were already fired.
Finally, we realized the IdleStateEvent would not be handled since closeIdleConnections are hardcoded to false for YarnShuffleService.

The above is based on what we've seen and know. Please correct me if any understanding is wrong or inaccurate.

thanks for the detailed explanation, that makes sense.  this is actually probably caused by the client side closing idle connections. (https://github.com/apache/spark/pull/9227). The server side never got changed.

ok to test

Test build #120428 has started for PR 27998 at commit e3b157d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25136/
Test PASSed.

Test build #120428 has finished for PR 27998 at commit e3b157d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120428/
Test PASSed.

Could you copy your investigation to the PR description?

@Ngone51 PR description has been updated.

Can one of the admins verify this patch?

ok to test

Test build #120604 has started for PR 27998 at commit e3b157d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25308/
Test PASSed.

Test build #120604 has finished for PR 27998 at commit e3b157d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120604/
Test PASSed.

merged to master and branch-3.0. thanks @manuzhang


node:15512 [DEP0137]

About the Error
I'm getting an error in the voiceStateUpdate event that I don't understand. There does not seem to be a problem with the function every time the event is triggered, but the error text comes to the console.
Code
module.exports = [{
    name: "voiceStateUpdate",
    type: "voiceStateUpdate",
    channel: "somechannelID",
    $if: "old",
    code: `
    $if[$authorID!=$clientID]
        $if[$newState[deaf]==$oldState[deaf]]
            $if[$newState[mute]==$oldState[mute]]
                $if[$voiceID[$clientID]!=]
                    $setUserVar[lastConnectedVoiceChannel;$voiceID[$authorID]]
                    $title[1;<:valentine_account_voice:1243984171399905301> Sesli Kanala Bağlantı Kuruldu]
                    $description[1;<@$authorID> **$userTag[$authorID]** bir sesli kanala katıldı.]
                    $addField[1;Bağlandığı Kanal;<#$voiceID[$authorID]> — $channelName[$voiceID[$authorID]]]
                    $addTimestamp
                    $color[1;$getGuildVar[color]]
                $else
                    $title[1;<:valentine_account_voice_off:1243984151036690492> Sesli Kanaldan Bağlantısı Sonlandı]
                    $description[1;<@$authorID> **$userTag[$authorID]** bir sesli kanaldan ayrıldı.]
                    $addField[1;Ayrıldığı Kanal;<#$getUserVar[lastConnectedVoiceChannel]> — $channelName[$getUserVar[lastConnectedVoiceChannel]]]
                    $addTimestamp
                    $color[1;$getGuildVar[color]]
                $endif
            $endif
        $endif
    $endif
    `
}];

*There are characters that are not in the English language. Just Turkish language.
*The channel parameter was filled in as an example.
 Please close FileHandle objects explicitly using FileHandle.prototype.close(). In the future, an error will be thrown if a file descriptor is closed during garbage collection.
(node:15512) Warning: Closing file descriptor 26 on garbage collection

The Warning That Comes Every Time the Event Is Triggered

it is a warning , ig I will fix this but this can be ignored as this won't cause errors

I'll be waiting for your fix.

(off topic: nothing in this repository is posted to the github-updates forum on discord)

it is a warning , I will fix this but this can be ignored as this won't cause errors

Any news with this?


iPhone 13 pro max iOS 15.4.1 (Error With JB bêta 8)

Hello,
So the problems are :
No gsm,
Freeze at any time without even touching the screen,
The restart of the user space restarts my iPhone but it is well jailbreak moreover if I do not use this functionality and that I make him a manual respring with trollstore for example it functions as if it were jailbreak and does not freeze and the gsm functions.
I want to specify that I use the beta 4 until now and I have in no way encountered any problem before no errors.
The beta 4 was very stable.
For more information do not hesitate to ask me.

Well I found the solution, when you have already jailbroken your iPhone with the old beta version you must at all costs remove .installed_fugu15max
Procedure :

jailbreak your iPhone with any version of Fugu15 from the moment you can launch a terminal with

open your terminal then go to root (for that type "su" then for the password type "alpine")

now at the end of line you should see mark root# that means that you are root

now you have to go up in the files so make " cd .. " twice until you reach /var/jb

type " rm .installed_fugu15max ".

Here you can now restart your iPhone and then rejailbreak it again that should correct a lot of other problems in passing.
I am currently with beta 8 and since I did this manipulation everything works perfectly even the reboot userspace does not cause any error, the gsm works ect


Undo closing windows doesn't work on Firefox

It happened in this commit https://github.com/adamjamesadair/manage-my-tabs/commit/7c2a7fce0c80d5d8070126662834115a83ec2625, just for reference for the following error

Error: Type error for parameter createData (Property "focused" is unsupported by Firefox) for windows.create. tabManager.js:250:5

To replicate it, close a window with the tab manager and click the "Undo" button

#135 fixes this issue.


dotnet: yield FORMAT_DOTNET before FORMAT_PE when processing .NET files

We should change the order used to yield multiple formats for .NET files so FORMAT_DOTNET is used for capa's output metadata.
https://github.com/mandiant/capa/blob/49231366f1cbb800f296c60a2cb99f97466e2e33/capa/features/extractors/dotnetfile.py#L51-L53
This order doesn't matter for matching but does matter when collecting the file's metadata because we default to the first in the list
https://github.com/mandiant/capa/blob/49231366f1cbb800f296c60a2cb99f97466e2e33/capa/loader.py#L385-L387

Alternatively, display all formats.

Hi @mr-tz , can i take this issue :)

of course


project don't run after update to netcore 1.1

Unfortunately, after upgrading my project to .net 1.1 Core project is not debugs
I click run and nothing happens
What could be the reason?
I have this error
The program '[13476] dotnet.exe' has exited with code -2147450749 (0x80008083).

@krzychu2 Try launching it as "Start Without Debugging" (Ctrl+F5).  I had a similar error and Ctrl+F5 let me see an error saying that the shared framework for version 1.1.0 was not found.  Then I ran the SDK installer from https://www.microsoft.com/net/core#windowscmd (https://go.microsoft.com/fwlink/?LinkID=835014) and everything was fine.

@krzychu2 , did that help? Can you confirm you had .NET Core 1.1 installed?

I had the same problem, .NET Core 1.1 SDK wasn't installed. You can use the command dotnet --info in the Package Manager Console to have the version of the dotnet framework your project uses (if it's not 1.1.0, the framework isn't installed)

To know if you have correctly install the .NET Core 1.1 SDK, you can also open the windows program and features and check if 1.1.0 SDK is installed.

Ok, sounds pretty unanimous that its a missing 1.1.0 install. @krzychu2 I'm going to close this. Let us know if getting 1.1.0 installed doesn't fix the issue.


Toast-timing is very inconsistent.

Before opening:

Search for duplicate or closed issues Done, couldn't find any.
Validate any HTML to avoid common problems. Done, just two warnings because I generate some stuff with JS.
Read the contributing guidelines Done.

Toast-timing is very inconsistent.
Sometimes it's the regular delay (for example 10 seconds), but sometimes the toast just disappears way quicker.

Operating system and version: Linux Mint 20.1 Cinnamon
Browser and version: Firefox
I'm not sure how to compile that code into a test-case but I took a screen capture:

https://user-images.githubusercontent.com/59611881/117564418-3c738800-b0ac-11eb-8cc8-f9ed8f1fb249.mp4

I used the version recommended in the bootstrap-dokumentation.
No idea whether that's tha alpha or not though.

I don't believe clearing the timeout in the hide method would fix this issue (though it certainly wouldn't hurt), because the timeout is already being cleared in the show method. This looks to be the same bug that was fixed with #31155 - so I'd first recommend checking if version v5.0.0-alpha2 or later is being used.

I agree, this shouldn't happen but it could if you manually close it through the dismiss button and initialize a new one without disposing the old one. IMO the problem in general is when the click handler looks something like this:
button.addEventListener('click', () => {
  const toast = new bootstrap.Toast(toastEl);
  toast.show();
});

Hard to tell because we don't know what the code looked like, but

The way i now got around this issue was by making the Toast a global variable in JS and then always doing toast.dispose() before showing it again.

suggests the OP used a code similiar to my example. In that case the timeout is never cleared and may overlap.

@alpadev Yep, that's pretty much what I did.

if you still need a fix, I've posted here https://github.com/twbs/bootstrap/discussions/37577#discussioncomment-4320814


Rendering Images from Sensor Observations

How can I render images from sensor observations and create videos of agent movement through a scene?

There is an example on the habitat-api side showing how to do this for an oracle agent: https://github.com/facebookresearch/habitat-api/blob/master/examples/shortest_path_follower_example.py


Fix side-comments on list elements Closes #852

Side comments balloon is not shown after refresh on unordered lists.
Before refresh:

After refresh:


Add support for a CredentialsProvider in GitConfigurationSource

https://github.com/cfg4j/cfg4j/issues/180

+1. Bumping this - please take a look at this.
Cfg4J is not suitable for serious use with gIt w/out this.


standalone editor in Safari: cursor off at the end of long lines

testing #19061:

change font in the standalone editor to 'courier'
open monaco-editor/test/index.html in Safari
added this line:

//012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123

zoomed in one level (View > Zoom In)
scrolled to the far right

Observe:
placing the cursor is always off by about four character positions

Many thanks!
Pushed a change that makes the editor not use the FastRenderedViewLine on Safari (same as Firefox).
Before this change:
Changing the zoom level would leave the editor positions permanently mis-aligned with the text until a browser reload occurs.
After this change:
Changing the zoom level leaves the editor positions mis-aligned for the lines that are already painted and until they are painted again (i.e. scrolled out of view and scrolled back in, or changed in any way, etc.)
After this change, the behaviour in Safari is similar to the behaviour in Firefox, transforming this issue into a duplicate of Microsoft/monaco-editor#337

Verified that this is a dupe of Microsoft/monaco-editor#337