[Roosterteeth] Update extractor to support new site.

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

[x] At least skimmed through adding new extractor tutorial and youtube-dl coding conventions sections
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Bug fix
[ ] Improvement
[ ] New extractor
[ ] New feature

Description of your pull request and other information
This updates the extractor for Roosterteeth.com to support the recent changes to the website. It now uses the officially provided API rather than scraping HTML. Addresses issue #16094.
There are some issues that need to be fixed before it can be merged.

Videos that do not require log in can be downloaded with no issue, but attempting to log in fails. I have not yet found a way to log in through the new API. The old log in URL stills exists, and can be used to log in through a browser. However, the form is now loaded through Javascript after the page loads, so youtube-dl seems unable to grab it. Logging in and downloading FIRST-only videos now works.

The changes I made don't look great, as I just wanted to get it mostly working before hopefully getting feedback on the log in issue. I'll be going through and fixing things up over the weekend. I've cleaned things up and made user-facing strings less generic. I'm open to any suggestions for further changes.

Logging in is done by POSTing username and password to an authorization URL, and an access token is returned. This access token is a cookie in the browser, so it could be saved as a cookie here, but I'm not sure how to do that, or how to check if a saved cookie is expired. I'll look more in to it.
Barring any requested changes, this seems ready to be merged.
Edit: The access token is a standard JWT that is valid for 8 hours after logging in. I have implemented saving the access token as a cookie and re-downloading it after it expires, so users who download multiple videos in a short time will not have to log in every time.

I came here looking for this precise fix and I can confirm it works for me. Is it usual for a PR like this to sit unmerged for 3 weeks?

bump because this is still unmerged on the master branch

I've (hopefully) made the requested changes. Awaiting further review.

Thanks for this, seems to be working on every video I've tried so far. One critique would be that the thumbnails it fetches are tiny (240ish), even though there are 720-1080p thumbnails available on the video page, minor thing but would be great if someone could fix.
Thanks

@RichardHancock It should now download a larger image for the thumbnail.

@ddmgy Thanks that works great. Seems to download with incorrect image file extension for some videos, but that's Rooster Teeth's end as I had this issue when manually downloading them.
Thanks for doing this, works just as good as the old website extractor now, Hopefully someone can get this merged in soon

@RichardHancock Glad to hear it works. Would you mind telling me which videos are giving the wrong extension for thumbnails? I'll see if I can fix it.

@ddmgy Seems to be pretty random, only ones I can find doing it right now are recent Camp Camp episodes. I assume one departments just uploading them wrong.
Here's one that I've checked: https://roosterteeth.com/episode/camp-camp-season-3-7
That one downloads a .png but when opened in some programs they throw an error saying that its actually a jpg (other programs just silently load it anyway).
It doesn't seem to be as bad as when I was manually downloading the thumbnails as some of those images were .webp with .png extensions which almost no program could detect and fix. I just assumed it was the same problem.
I don't really think it's anything that you need to fix, it's just their mistake. I'll post again if I find other series doing it.
Also that latest commit you made makes it download the small thumbnails again, but I assume your just testing things.
Thanks

@RichardHancock Hmm, the thumbnails returned for that Camp Camp episode are a mix of PNGs and JPGs, and I'm not sure which set of those youtube-dl is actually grabbing. I would assume the API is returning the data in the same order every time, so it should be getting PNGs, but I'll test that more tonight. Actually, I see now that even when the extension on the URL shows it is a PNG, it is sometimes actually a JPG image. The image opens fine for me in everything I'm using, but you're right: this is on RoosterTeeth's side, and there's nothing I can fix there.
Unfortunately, even if I always manage to grab the same set of thumbnails for a video, there is no way to tell what size each of the images is, and they don't always seem to be ordered correctly. When --write-thumbnail is passed to youtube-dl, it will only download the last thumbnail in the list of thumbnails (which is what my latest commit generates). You can use --write-all-thumbnails, which will download all 4 thumbnails, but then you'd have to check each one manually to see which is the size you want and then delete the other 3.
Well, you can still use --write-all-thumbnails and manually select which image you want to keep, but using --write-thumbnail should at least choose to download only the largest available thumbnail now. I wish there was a proper way to choose which thumbnail you want to download, but I'm not seeing any available options for that.
Thanks for your feedback on this. I don't use thumbnails, so it probably never would have come to my attention that I was grabbing the wrong size.

As I no longer have a FIRST subscription with roosterteeth.com, I will be unable to make any further changes to this pull request, in the unlikely event that it is reviewed again or considered for merging.
If anyone wants to take over maintenance of this pull request, leave a comment here so we can figure out a way to transfer it over.

@ddmgy I would be willing to take it over if needed.

@ddmgy Also just realized that the changes weren't marked as resolved so I just did that, hopefully that will trigger a re-review of the pull request.

The test for the extractor is failing at the moment,
'episode': 'S2:E10 - Million Dollars, But... The Game Announcement'
Needs to change to:
'episode': 'Million Dollars, But... The Game Announcement'
for it to pass the test

Hi, any ways this could support bonus episodes? For example https://roosterteeth.com/bonus-feature/rwby-adam-short (volume 6, adam character)

@dstftw
I've updated the tests for this change, and they're passing now. I can confirm that the module is working with the new Rooster Teeth site, both for public videos, as well as ones that require a FIRST subscription/login in order to download.

Any updates to this or the other PR for presumably the same problem (#17843)? It's pretty inconvenient using the API workaround described in #16094, so I'd encourage @dstftw or other maintainers to merge this...
Given that the Rooster Teeth extractor doesn't work at all in the current master branch, I would think there's not as much worry about breaking it with changes like these...

[2019-03-10 00:31:52] Rome Reginelli:

Any updates to this or the other PR for presumably the same problem (#17843)? It's pretty inconvenient using the API workaround described in #16094, so I'd encourage @dstftw or other maintainers to merge this...
Given that the Rooster Teeth extractor doesn't work at all in the current master branch, I would think there's not as much worry about breaking it with changes like these...

Not much to break and this PR works, I have it on my youtube-dl installation (thanks to gentoo allowing to just drop a patchfile) for quite a while.

FWIW I used this change to download stuff... so, it does work...

The requested changes seem pretty pedantic too - just merge it 🚀

Weren't the requested changes made anyway? Or at least, it doesn't seem to me like there's been any review since https://github.com/ytdl-org/youtube-dl/pull/16105#issuecomment-396768722.
I don't really understand why neither this nor #17843 have been merged, given that the Roosterteeth extractor is completely non-functional on master.
I've had this merged on my local installation for months.

Please merge.

I don't know - I'm still having problems:
./bin/youtube-dl --verbose https://roosterteeth.com/episode/rwby-volume-6-1
[debug] System config: []
[debug] User config: ['--no-mtime', '--write-sub', '--sub-format', 'best', '--embed-subs', '--add-metadata', '--xattrs', '--mark-watched']
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://roosterteeth.com/episode/rwby-volume-6-1']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.24
[debug] Python version 3.7.3 (CPython) - Linux-5.0.13-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Proxy map: {}
[RoosterTeeth] rwby-volume-6-1: Downloading webpage
ERROR: Unable to extract m3u8 URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/roosterteeth.py", line 115, in _real_extract
    raise ExtractorError('Unable to extract m3u8 URL')
youtube_dl.utils.ExtractorError: Unable to extract m3u8 URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[2019-05-09 12:41:13-0700] jonnyrobbie:

I don't know - I'm still having problems:
./bin/youtube-dl --verbose https://roosterteeth.com/episode/rwby-volume-6-1
[debug] System config: []
[debug] User config: ['--no-mtime', '--write-sub', '--sub-format', 'best', '--embed-subs', '--add-metadata', '--xattrs', '--mark-watched']
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://roosterteeth.com/episode/rwby-volume-6-1']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.24
[debug] Python version 3.7.3 (CPython) - Linux-5.0.13-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Proxy map: {}
[RoosterTeeth] rwby-volume-6-1: Downloading webpage
ERROR: Unable to extract m3u8 URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/roosterteeth.py", line 115, in _real_extract
    raise ExtractorError('Unable to extract m3u8 URL')
youtube_dl.utils.ExtractorError: Unable to extract m3u8 URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Weird, here it just works:
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://roosterteeth.com/episode/rwby-volume-6-1']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.30
[debug] Python version 3.6.5 (CPython) - Linux-4.19.27-gentoo-r1-x86_64-AMD_FX-tm-8350_Eight-Core_Processor-with-gentoo-2.6
[debug] exe versions: ffmpeg 4.1.1, ffprobe 4.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[RoosterTeeth] rwby-volume-6-1: Downloading video information (1/2)
[RoosterTeeth] rwby-volume-6-1: Downloading video information (2/2)
[RoosterTeeth] rwby-volume-6-1: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://rtv3-roosterteeth.akamaized.net/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/f6785b63-hls_1080p-store-777dcdfb41acc2cebc48a69a94ab4460.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 339
[download] Destination: Argus Limited-29129.mp4
[download] 100% of 864.13MiB in 02:40
[debug] ffmpeg command line: ffprobe -show_streams 'file:Argus Limited-29129.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "Argus Limited-29129.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:Argus Limited-29129.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:Argus Limited-29129.temp.mp4'

The patch I picked up for it is available at (I didn't modify it but
the code in the PR might have changed in the meantime):
https://hacktivis.me/git/etc_portage.work/patches/net-misc/youtube-dl/8b703800ecb-pull-16105-rooster-teeth.diff

Weird, here it just works:

yeah, that was my bad. I cloned the repos and switched the branches, and then just run the bin/ executable, which just imports the system vanilla version of ytdl. I should have run it with python -m youtube_dl.

Any update on if this is getting merged in to the official build?  Or if it still works?

I've tried that a few days ago and it still worked for me. But I was simply downloading single episodes by the simple episode url. Maybe some more advanced querries still cause issues? I don't know.

The URL scheme for Rooster Teeth appears to have changed from roosterteeth.com/episode/foo to roosterteeth.com/watch/foo. When using one of the new /watch/ URLs the extractor doesn’t recognize it for the RoosterTeeth extractor and then the generic one fails. Even with the new URL simply changing /watch/ to /episode/ allows it to continue to work.

I'm trying to use this version and I'm getting a 403 error:
youtube-dl --verbose https://roosterteeth.com/episode/backwardz-compatible-2019-sadism-or-masochism
[debug] System config: []
[debug] User config: [u'-o', u'~/Movies/%(title)s.%(ext)s', u'-f', u'bestvideo[height<=640]+bestaudio[height<=640]/best[height<=640]']
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://roosterteeth.com/episode/backwardz-compatible-2019-sadism-or-masochism']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.02
[debug] Python version 2.7.11 (CPython) - Darwin-18.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.1.4, ffprobe 4.1.4, rtmpdump 2.4
[debug] Proxy map: {}
[RoosterTeeth] backwardz-compatible-2019-sadism-or-masochism: Downloading video information (1/2)
[RoosterTeeth] backwardz-compatible-2019-sadism-or-masochism: Downloading video information (2/2)
ERROR: Unable to download video information (2/2): HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2019.8.2-py2.7.egg/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2019.8.2-py2.7.egg/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

I'm trying to use this version and I'm getting a 403 error:
youtube-dl --verbose https://roosterteeth.com/episode/backwardz-compatible-2019-sadism-or-masochism
[debug] System config: []
[debug] User config: [u'-o', u'~/Movies/%(title)s.%(ext)s', u'-f', u'bestvideo[height<=640]+bestaudio[height<=640]/best[height<=640]']
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://roosterteeth.com/episode/backwardz-compatible-2019-sadism-or-masochism']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.02
[debug] Python version 2.7.11 (CPython) - Darwin-18.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.1.4, ffprobe 4.1.4, rtmpdump 2.4
[debug] Proxy map: {}
[RoosterTeeth] backwardz-compatible-2019-sadism-or-masochism: Downloading video information (1/2)
[RoosterTeeth] backwardz-compatible-2019-sadism-or-masochism: Downloading video information (2/2)
ERROR: Unable to download video information (2/2): HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2019.8.2-py2.7.egg/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2019.8.2-py2.7.egg/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

It looks like that video is behind the FIRST subscription and you’ve not supplied a login so as to access FIRST content. Try supplying a username with --username and entering your password when prompted. You could also use a cookie file if the previous method doesn’t work

I'm trying to use this version and I'm getting a 403 error:
youtube-dl --verbose https://roosterteeth.com/episode/backwardz-compatible-2019-sadism-or-masochism
[debug] System config: []
[debug] User config: [u'-o', u'~/Movies/%(title)s.%(ext)s', u'-f', u'bestvideo[height<=640]+bestaudio[height<=640]/best[height<=640]']
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://roosterteeth.com/episode/backwardz-compatible-2019-sadism-or-masochism']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.02
[debug] Python version 2.7.11 (CPython) - Darwin-18.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.1.4, ffprobe 4.1.4, rtmpdump 2.4
[debug] Proxy map: {}
[RoosterTeeth] backwardz-compatible-2019-sadism-or-masochism: Downloading video information (1/2)
[RoosterTeeth] backwardz-compatible-2019-sadism-or-masochism: Downloading video information (2/2)
ERROR: Unable to download video information (2/2): HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2019.8.2-py2.7.egg/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2019.8.2-py2.7.egg/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

It looks like that video is behind the FIRST subscription and you’ve not supplied a login so as to access FIRST content. Try supplying a username with --username and entering your password when prompted. You could also use a cookie file if the previous method doesn’t work

Ah I see! Thanks for that. I have clearly misunderstood ddmgy's description - I had thought that her extractor meant that I didn't have to log in at all. I'll try this then!

It has been fixed by c72dc20d099bbe1dc4ede83e8f94a7bc42d81532 and 5efbc1366f4e4d9d4969cbfb404657349a5b3f99 , so this Pull Request can probably be closed with https://github.com/ytdl-org/youtube-dl/issues/16094 or should be rebased on current master if there is still missing pieces to be fixed (maybe the required accounts stuff).


Prepare 0.32.2 release

Hi everybody, thank you very much for working on this great gem.
The last release was more than 6 months ago and the master branch already contains a bunch of useful fixes.
So maybe it makes sense to release the new version which contains those fixes?

thanks … it is planned for this evening (CET)

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.

Coverage remained the same at 99.322% when pulling 3254c29cdf56ea7e8b2061184b2a84ecccfe6816 on usernam3:prepare-0.32.2-release into 4573a827772225df875f499ab39257822af5817e on ruby-grape:master.


ros melodic Compilation failure

The following is the error messages：
FAILED: CMakeFiles/cartographer.mapping.internal.constraints.constraint_builder_3d_test.dir/cartographer/mapping/internal/constraints/constraint_builder_3d_test.cc.o
/usr/bin/c++  -DGFLAGS_IS_A_DLL=0 -I/home/fu/cartographer_ws/src/cartographer/cartographer -I. -I/home/fu/cartographer_ws/src/cartographer -isystem /usr/src/googletest/googlemock/include -isystem /usr/include/eigen3 -isystem /usr/include/lua5.2 -O3 -DNDEBUG    -pthread -fPIC  -Wall -Wpedantic -Werror=format-security -Werror=missing-braces -Werror=reorder -Werror=return-type -Werror=switch -Werror=uninitialized -O3 -DNDEBUG -pthread -fPIC  -Wall -Wpedantic -Werror=format-security -Werror=missing-braces -Werror=reorder -Werror=return-type -Werror=switch -Werror=uninitialized -O3 -DNDEBUG -MD -MT CMakeFiles/cartographer.mapping.internal.constraints.constraint_builder_3d_test.dir/cartographer/mapping/internal/constraints/constraint_builder_3d_test.cc.o -MF CMakeFiles/cartographer.mapping.internal.constraints.constraint_builder_3d_test.dir/cartographer/mapping/internal/constraints/constraint_builder_3d_test.cc.o.d -o CMakeFiles/cartographer.mapping.internal.constraints.constraint_builder_3d_test.dir/cartographer/mapping/internal/constraints/constraint_builder_3d_test.cc.o -c /home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/constraints/constraint_builder_3d_test.cc
In file included from /home/fu/cartographer_ws/src/cartographer/cartographer/common/task.h:24:0,
from /home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/constraints/constraint_builder_3d.h:34,
from /home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/constraints/constraint_builder_3d_test.cc:17:
/home/fu/cartographer_ws/src/cartographer/cartographer/common/thread_pool.h:67:58: error: expected ‘;’ at end of member declaration
std::weak_ptr Schedule(std::unique_ptr task)
^
/home/fu/cartographer_ws/src/cartographer/cartographer/common/thread_pool.h:68:22: error: ‘mutex_’ has not been declared
LOCKS_EXCLUDED(mutex_) override;
^~~~~~
/home/fu/cartographer_ws/src/cartographer/cartographer/common/thread_pool.h:68:30: error: ISO C++ forbids declaration of ‘LOCKS_EXCLUDED’ with no type [-fpermissive]
LOCKS_EXCLUDED(mutex_) override;
^~~~~~~~
/home/fu/cartographer_ws/src/cartographer/cartographer/common/thread_pool.h:73:46: error: expected ‘;’ at end of member declaration
void NotifyDependenciesCompleted(Task* task) LOCKS_EXCLUDED(mutex_) override;
^
/home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d.h:290:71: error: expected ‘;’ at end of member declaration
void SetTrajectoryState(int trajectory_id, TrajectoryState state) override
^~~~~~~~
/home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d.h:291:34: error: ‘parent_’ has not been declared
EXCLUSIVE_LOCKS_REQUIRED(parent_->mutex_);
^~~~~~~
/home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d.h:291:41: error: expected ‘,’ or ‘...’ before ‘->’ token
EXCLUSIVE_LOCKS_REQUIRED(parent_->mutex_);
^~
/home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d.h:291:49: error: ISO C++ forbids declaration of ‘EXCLUSIVE_LOCKS_REQUIRED’ with no type [-fpermissive]
EXCLUSIVE_LOCKS_REQUIRED(parent_->mutex_);
^
/home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d.h:291:9: error: ‘int cartographer::mapping::PoseGraph2D::TrimmingHandle::EXCLUSIVE_LOCKS_REQUIRED(int)’ cannot be overloaded
EXCLUSIVE_LOCKS_REQUIRED(parent_->mutex_);
^~~~~~~~~~~~~~~~~~~~~~~~
/home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d.h:281:9: error: with ‘int cartographer::mapping::PoseGraph2D::TrimmingHandle::EXCLUSIVE_LOCKS_REQUIRED(int)’
EXCLUSIVE_LOCKS_REQUIRED(parent_->mutex_);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/fu/cartographer_ws/src/cartographer/cartographer/mapping/internal/2d/pose_graph_2d_test.cc:24:0:
/home/fu/cartographer_ws/src/cartographer/cartographer/common/internal/testing/lua_parameter_dictionary_test_helpers.h: In member function ‘virtual std::__cxx11::string cartographer::common::DummyFileResolver::GetFileContentOrDie(const string&)’:
/home/fu/cartographer_ws/src/cartographer/cartographer/common/internal/testing/lua_parameter_dictionary_test_helpers.h:42:3: error: control reaches end of non-void function [-Werror=return-type]
}
^
/home/fu/cartographer_ws/src/cartographer/cartographer/common/internal/testing/lua_parameter_dictionary_test_helpers.h: In member function ‘virtual std::__cxx11::string cartographer::common::DummyFileResolver::GetFullPathOrDie(const string&)’:
/home/fu/cartographer_ws/src/cartographer/cartographer/common/internal/testing/lua_parameter_dictionary_test_helpers.h:46:3: error: control reaches end of non-void function [-Werror=return-type]
}
^
cc1plus: some warnings being treated as errors
ninja: build stopped: subcommand failed.
<== Failed to process package 'cartographer':
Command '['ninja', '-j20', '-l20']' returned non-zero exit status 1
Reproduce this error by running:
==> cd /home/fu/cartographer_ws/build_isolated/cartographer && ninja -j20 -l20
Command failed, exiting.
I was hoping someone could help me out

哪位大佬知道或是遇到过这种错误？劳烦指点一下，感谢

This is due to an older abseil version being required. The macros are defined in abseil's common/thread_annotations.h header, but after 2019, abseil adds an ABS_ prefix to them.

Thank you very much for your answer
Thank you very much for your answer
fxx.
@.***

------------------ 原始邮件 ------------------
发件人:                                                                                "cartographer-project/cartographer_ros"                                                                                @.>;
发送时间: 2024年8月3日(星期六) 凌晨0:15
@.>;
<EMAIL_ADDRESS>主题: Re: [cartographer-project/cartographer_ros] ros melodic Compilation failure (Issue #1819)
This is due to an older abseil version being required. The macros are defined in abseil's common/thread_annotations.h header, but after 2019, abseil adds an ABS_ prefix to them.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

This is due to an older abseil version being required. The macros are defined in abseil's common/thread_annotations.h header, but after 2019, abseil adds an ABS_ prefix to them.

Thank you so much for your answer，and i will change my abseil version later


Tests pass but error JavaScript heap out of memory

Getting a crash right at the end of running my entire test suite of 246 tests
mocha-webpack node process seems hits ~800MB memory usage, before it's even started running tests. It reached ~1.10GB as it's running tests.
246 passing (485ms)

<--- Last few GCs --->

   49818 ms: Mark-sweep 923.8 (1299.4) -> 883.9 (1199.6) MB, 139.2 / 0.0 ms [allocation failure] [GC in old space requested].
   49950 ms: Mark-sweep 883.9 (1199.6) -> 883.8 (1168.6) MB, 132.0 / 0.0 ms [allocation failure] [GC in old space requested].
   50095 ms: Mark-sweep 883.8 (1168.6) -> 883.3 (1088.6) MB, 145.1 / 0.0 ms [last resort gc].
   50232 ms: Mark-sweep 883.3 (1088.6) -> 883.3 (1061.6) MB, 137.0 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x13efc2acfb51 <JS Object>
    1: DoJoin(aka DoJoin) [native array.js:~129] [pc=0x377ef0357d21] (this=0x13efc2a04381 <undefined>,w=0xe24b6cc13c1 <JS Array[182]>,x=182,N=0x13efc2a043c1 <true>,J=0x10b8be3f2eb9 <String[4]\: ,\n  >,I=0x13efc2ab46e1 <JS Function ConvertToString (SharedFunctionInfo 0x13efc2a52dc9)>)
    2: Join(aka Join) [native array.js:180] [pc=0x377eedd5e532] (this=0x13efc2a04381 <undefined>,w=0xe24b6cc13c1...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/robaldred/.nvm/versions/node/v6.9.5/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/robaldred/.nvm/versions/node/v6.9.5/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/robaldred/.nvm/versions/node/v6.9.5/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/Users/robaldred/.nvm/versions/node/v6.9.5/bin/node]
 5: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/robaldred/.nvm/versions/node/v6.9.5/bin/node]
 6: 0x377eed0092a7
 7: 0x377ef0357d21
sh: line 1: 48252 Abort trap: 6           NODE_ENV=test ./node_modules/mocha-webpack/bin/mocha-webpack --opts ./test/mocha-webpack.opts --webpack-config ./config/webpack/test.js "test/**/*.spec.js"
npm ERR! Test failed.  See above for more details.

There are a lot of issues in the webpack repo about high memory usage. I don't think that there's something we can do here.

No worries i figured it was worth a shot. I have managed to stop it crashing by increasing the allowed memory but it's rather large.

try deleting .babel.json...i had same issue..


Use global constant for channel names

In the example codes, the channel names are hard-coded in the app codes, and it should be consistent to the ones in the "setup.json".
We can mitigate this problem by defining constants for the channel names in each app code to prevent using the string-literal channel names everywhere (in which case it is hard to rename the channels).
However, we still have to define such constants in each app code consistently. For example, assume that every app uses the logging channel. Then they have to define, e.g., LOG_CHANNEL="log" in each app code.
If we use the global constant which is suggested in #185, we can tackle this issue further.
However, I am not sure yet how effective this would be. For example, even if we use the global constant namespace, we still have to use the agreed constant name and any mismatch cannot be detected statically (e.g., by IDEs).
Any opinions on this?

I also think that setting channel names as global constants is the best solution!

@kangz12345 I realized a problem while I tested several global constants.
How can I access them in each app?
In an app class, neither self._constants nor BaseApp._constants can access them well.
I think we set the constants not in BaseApp, but in each app instance directly.

Referring to #189:

For the access, BaseApp now provides a read-only property constants.

So self.constants should work. If you still have problems, please let me know.

It doesn't work well. Specifically, the constants is an empty namespace.

Could you show me the config.json file you used?

The config.json for test is as follows:
{
    "app": {
        "numgen": {
            "module": "examples.numgen",
            "cls": "NumGenApp",
            "show": true,
            "pos": "left",
            "channel": ["db"],
            "args": {
                "table": "number"
            }
        },
        "dbmgr": {
            "module": "examples.dbmgr",
            "cls": "DBMgrApp",
            "show": true,
            "pos": "right",
            "channel": []
        },
        "poller": {
            "module": "examples.poller",
            "cls": "PollerApp",
            "show": true,
            "pos": "bottom",
            "channel": ["db"],
            "args": {
                "table": "B"
            }
        },
        "logger": {
            "module": "examples.logger",
            "cls": "LoggerApp",
            "show": true,
            "pos": "bottom",
            "channel": []
        }
    },
    "constant": {
        "icon_path": "resources/icon.jpg",
        "background_path": "resources/background.jpg"
    }
}

I printed the following two properties.

constants_ in main() (qiwis.py)

ConstantNamespace(icon_path='resources/icon.jpg', background_path='resources/background.jpg')

self.constants (numgen.py)

EmptyNamespace()

I made a simple example.
test1.py
from test2 import B

class A(B):
    def show(self):
        print(self.cnt)

test2.py
class B:
    _cnt = 0

    @property
    def cnt(self):
        return B._cnt

def main():
    B._cnt = 1
    from test1 import A
    ins = A()
    ins.show()

if __name__ == "__main__":
    main()

Result
$ python test2.py
0

Isn't this the same case as that we implemented?

Hmm okay interesting 😄
I think the two B in test1.py and test2.py are distinct class objects, and therefore each of them has its own namespace.
In other words, test1.B._cnt and test2.B._cnt seem to be different variables.
I will make a bug issue about this. Thank you for reporting!

After fixing the bug in #215 and implementing this issue, I plan to release the next version.
Is it okay? @kangz12345

Oh.. actually this makes a circular import, which should be avoided.

Please continue discussion in #215 since it is out of scope of this issue.

Instead of implement it in this way, @BECATRUE and I decided to use apps' constructor arguments instead of the global constant namespace.
Constructor gives more straightforward intuition about what channel names users should provide.


new release breaks list setup I had in yaml

Describe the bug
List widget broken on new release.
The CMS was working fine 2 days ago, but I noticed there was a new release. It's loaded through CDN as <script<EMAIL_ADDRESS>The List widget that I use (find yaml below) doesn't show correctly anymore in the CMS and when I try to save I get the error:
ObjectControl.js:65 Uncaught TypeError: this.componentValidate[e.get(...)] is not a function     at ObjectControl.js:65     at it.__iterate (immutable.js:2206)     at it.forEach (immutable.js:4381)     at ObjectControl.js:63     at ListControl.js:248     at Array.forEach (<anonymous>)     at Object.galleries (ListControl.js:247)     at EditorControlPane.js:48     at it.__iterate (immutable.js:2206)     at it.forEach (immutable.js:4381)
Everything is fine when I remove that gallery list and also when I include my locally cms.js that uses the previous release.
Probable something has changed in the yaml setup?
I couldn't make out from the multiple changelogs what has changed. How best to prevent such issue in the future, breaking a live site? I guess include a local build :)
To Reproduce
I can make a further setup if needed, but I guess it should be clear from my yaml setup and what has changed in the list widget or something related to the yaml definition in the new release.
Applicable Versions:

Netlify CMS version: 2.10.45
Git provider: GitHub
OS: Linux
Browser version: any

CMS configuration
`collections:

name: 'issues' # Used in routes, e.g., /admin/collections/blog
label: 'Uitgaves' # Used in the UI
folder: 'content/issues' # The path to the folder where the documents are stored
format: 'json'
create: true # Allow users to create new documents in this collection
slug: '{{year}}-{{month}}-{{day}}-{{slug}}' # Filename template, e.g., YYYY-MM-DD-title.md
sortableFields: ['commit_date', 'title', 'order']
fields: # The fields for each document, usually in front matter

{label: Order, name: order, widget: number, valueType: int, required: false}
{label: Draft, name: draft, widget: boolean, default: false, required: false}
{label: Title, name: title, widget: string, required: true}
{label: Subtitle, name: subtitle, widget: string, required: false}
{label: Alternatieve button text, name: button_text, widget: string, required: false}
#- {label: Stripe SKU ID, name: sku, widget: string, required: false}
{label: Prijs, name: price, widget: number, valueType: float, hint: gebruik punt ipv comma voor centen, required: false}
{label: Checkout mededeling, name: checkout_text, widget: string, required: false}
{label: Content, name: body, widget: markdown, required: false}
{label: Achtergrondkleur, name: block_color, widget: string, hint: HEX code invullen, required: false}
{label: Cover, name: image, widget: image, required: false}
{label: Galleries, name: galleries, widget: list, optional: true, collapsed: true, fields: [
{label: Tekstje, name: title, widget: markdown, required: false},
{label: Draft, name: draft, widget: boolean, default: false, required: false},
{label: Only wide images?, name: wide, widget: boolean, default: false, required: false},
{label: Plaatjes, name: gallery, widget: list, optional: true, collapsed: true, fields: [
{label: Image, name: image, widget: image},
{label: Caption, name: caption, widget: string, required: false}
]}
]}`

The problem is in the last name: galleries, widget: list, item.

Probable something has changed in the yaml setup?
I couldn't make out from the multiple changelogs what has changed.

PR #3606 introduced/revealed this issue. Already have a PR that would fix it up.


[Core] Fixed cspell typos in azure-common

Description
fix https://github.com/Azure/azure-sdk-for-python/issues/22649
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

We don't plan more release of this package, and this PR has been stale for a while, closing


[5.7][CSClosure] Perform syntactic checks on rewritten statements

Cherry-pick of https://github.com/apple/swift/pull/59134

Fix an oversight where syntactic checking was not performed on
any statements in a body of a multi-statement after successful
solution application.
Resolves: rdar://94049113
(cherry picked from commit 953095a9ce041e4e913da13058c9f762c595de04)

@swift-ci please test

@swift-ci please test


Asynchronous Events

Could you show some minimal example for attaching custom async events from another thread.
Usually I use separate thread for working with inputs and timers and send events via channel. But it requires explicit dealing with boiler plates. Or if it's complicated, I'll probably just continue using own templates.
Thank you in advance for answer.

I'd look into creating the thread at start, and then putting a channel reader into a state, then reading it through a widget.
Here's a simple example of using a channel within a state.
use std::{
    error::Error,
    sync::mpsc::{channel, Receiver},
    thread,
    time::Duration,
};

use widgetui::*;

#[derive(State)]
pub struct MyEventReceiver {
    channel: Receiver<bool>,
}

fn handler(reciever: ResMut<MyEventReceiver>, mut events: ResMut<Events>) -> WidgetResult {
    if let Ok(true) = reciever.channel.try_recv() {
        events.register_exit();
    }
    Ok(())
}

fn main() -> Result<(), Box<dyn Error>> {
    let mut app = App::new(100)?;

    let (sender, reciever) = channel();

    app = app.states(MyEventReceiver { channel: reciever });

    thread::spawn(move || {
        thread::sleep(Duration::from_millis(1500));
        sender.send(true).unwrap();
    });

    app = app.widgets(handler);

    app.run()?;

    Ok(())
}

Is this what you are looking for?

probably it is!
I must check it by hands.
thank you!!

Had some practice with basic concepts of Widgetui.
After it I have re-read example and understand it. It is obviously working way.
But it leads to another question: is it possible to set off somehow internal input_handler? I'd probably like to use my own event source for input processing (and less CPU usage).

There isn't currently a way to disable the internal input handler, but if you wanted to add your own on top, you could do so with a widget and state, where the state stores those inputs.

understood.
thank you for your time. There needs to compare pros and cos. the crate is very cool. Your example works fine.


ci: Fir 29775 remove dev usage from firebolt cli ci

Use staging environment in integration tests and nightly v0

Integration tests are running but not passing: https://github.com/firebolt-db/firebolt-cli/actions/runs/7783501230/job/21222096160
They would be fixed as a part of this ticket: https://packboard.atlassian.net/browse/FIR-29929
Unit tests will be fixed in that ticket as well


Unknown directive type "exercise".

The .. exercise:: directive does not work for me.
Steps to reproduce:

sphinx-quickstart
pip install -r requirements.txt
requirements.txt
change conf.py

project = 'test-sphinx-lesson'
copyright = '2021, Kevin Sawade'
author = 'Kevin Sawade'

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx_lesson',
        'sphinx_lesson.directives',
        'sphinx.ext.autodoc',
        'sphinx.ext.napoleon',
        'sphinx.ext.todo',
        'sphinx.ext.coverage',
        'sphinx.ext.githubpages',
        'sphinxcontrib.yt',
        'sphinx.ext.mathjax',
        'sphinx_copybutton',
        'sphinx.ext.intersphinx',
        'sphinx_rtd_theme_ext_color_contrast',
        'sphinx.ext.githubpages'
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

run make html

Error:
Running Sphinx v3.5.4
making output directory... done
myst v0.13.7: MdParserConfig(renderer='sphinx', commonmark_only=False, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, update_mathjax=True, enable_extensions=['dollarmath'], disable_syntax=[], url_schemes=None, heading_anchors=None, html_meta=[], footnote_transition=True, substitutions=[], sub_delimiters=['{', '}'])
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: [new config] 1 added, 0 changed, 0 removed
reading sources... [100%] index
D:\sphinx-lesson-test\index.rst:14: WARNING: Unknown directive type "exercise".

.. exercise::

   This is an exercise
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done

Tested systems:
Ubuntu 18.04, python3.8, conda
Windows 10, python3.8, conda

I wonder if this is too new and we need a new release to make the
documentation match the PyPI version (being still testing I haven't
been too on top of making new releases).  I just released 0.8.4 - does
it work now with that?
I tried to look at git history to see when we did our big renaming, I
am not sure if it is after 0.8.3 but I think it might be.

Yeah. It's working now with 0.8.4. Thank you!


Update GsaFeedFileSender with adaptor library changes.

Update GsaFeedFileSender with the changes to support "cleanup" in
groups feeds. Add GsaFeedFileSenderTest and supporting files.

LGTM

technology's compounding interest

On Fri, Nov 17, 2017 at 5:34 PM, John Lacey<EMAIL_ADDRESS>wrote:

@wiarlawd approved this pull request.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/googlegsa/gsafeed/pull/11#pullrequestreview-77601895,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIgXe5SP36gqKtWVMROZFEvPT5KSIMb7ks5s3jQygaJpZM4QeQ7t
.


add HostSubnet.EgressCIDRs

Allow admins to specify CIDR ranges to automatically assign EgressIPs out of.
(We had talked before about implementing this as an annotation on Node, but it simplifies things to have it on HostSubnet instead, and the future solution with OVN will probably involve CRDs anyway, so there's no reason to use Node now.)
@openshift/sig-networking PTAL

/assign @smarterclayton

@smarterclayton @deads2k -- Can this go in when https://github.com/openshift/origin/pull/20258 passes tests?

@smarterclayton ok, this is green and the origin PR is green except for /approve

@smarterclayton @deads2k -- What's the status on this?  Can it go in?

The corresponding PR has passed all tests and is just waiting for API approval.

/approve

/lgtm


NavigationServer.MapSetActive() does not seem to work

Godot version
Godot 3.5-beta1_mono
System information
Windows 10, GLES3, RTX2080
Issue description
Setting a NavigationServer map to active via .MapSetActive() does not have any effect.
.MapIsActive() will always return false.
I used a pretty simple setup using C#:
    NavigationMesh navmesh;
    public override void _Ready()
    {
        navmesh = new NavigationMesh();
        navmesh.Vertices = new Vector3[]
        {
            new Vector3(0,0,0),
            new Vector3(9,0,0),
            new Vector3(0,0,9),
        };
        navmesh.AddPolygon(new int[] { 0, 1, 2 });

        NavigationServer.SetActive(true);
        var map = NavigationServer.MapCreate();
        var region1 = NavigationServer.RegionCreate();
        NavigationServer.MapSetUp(map, Vector3.Up);
        NavigationServer.RegionSetTransform(region1, Transform.Identity);
        NavigationServer.RegionSetNavmesh(region1, navmesh);
        NavigationServer.RegionSetMap(region1, map);
        NavigationServer.MapSetActive(map, true);

        GD.Print("Is active: " + NavigationServer.MapIsActive(map)); //Will return false

        var result = NavigationServer.MapGetPath(map, new Vector3(0.1f, 0, 0.1f), new Vector3(1f, 0, 1f), true);

        GD.Print("Found waypoints: " + result.Length);
    }

Steps to reproduce
Run the MRP in Godot 3.5 beta mono build.

The NavigationServer is active by default as soon as it is created by Godot, no need to set it active again.
If a Map is created through NavigationServer API alone it needs to be created / registered and set activated afterward.
Any change like registering a map, a region or an agent, changing their transform ... takes time to sync as the NavigationServer runs on a different thread. Don't expect changes to be in effect immediately or a path to return on the same frame you setup all the regions and maps. You need to wait at least 1 physics tick for the NavigationServer sync and queue flush before changes can take place.
If possible please test again and verify. I think it is not a bug but a setup, documentation and expectation issue.

Thx for the reply!
Do you have a example how to set this up in gdscript?

@Xyotic
var navmesh : NavigationMesh
func _ready():
	# use call deferred to make sure the entire SceneTree Nodes are setup
	# else if yield on 'physics_frame' in a _ready() sceneload might get stuck
	call_deferred("setup")

func setup():

	navmesh = NavigationMesh.new()
	var vertices : PoolVector3Array = PoolVector3Array([Vector3(0,0,0), Vector3(9.0,0,0), Vector3(0,0,9.0)])
	navmesh.set_vertices(vertices)
	var polygon : PoolIntArray = PoolIntArray([0, 1, 2])
	navmesh.add_polygon(polygon)

	var map : RID = NavigationServer.map_create()
	var region : RID = NavigationServer.region_create()

	NavigationServer.map_set_up(map, Vector3.UP)
	NavigationServer.map_set_active(map, true)

	NavigationServer.region_set_transform(region, Transform())
	NavigationServer.region_set_navmesh(region, navmesh)
	NavigationServer.region_set_map(region, map)

	# wait for NavigationServer sync to adapt to changes made
	yield(get_tree(),"physics_frame")

	var is_active : bool = NavigationServer.map_is_active(map)
	print("Is active: %s" % is_active)

	var path : PoolVector3Array = NavigationServer.map_get_path(map, Vector3(0.1, 0.0, 0.1), Vector3(1.0, 0.0, 1.0), true)

	print("Found waypoints:")
	print(path)

@Xyotic
Could that solve the issue?

Will try it this weekend

@smix8 waiting for the physics frame to end did the trick!
For anyone curious, this it how it would look in C#
    NavigationMesh navmesh;
    public override void _Ready()
    {
        CallDeferred(nameof(Setup));
    }

    private async void Setup()
    {
        navmesh = new NavigationMesh();
        navmesh.Vertices = new Vector3[]
        {
            new Vector3(0,0,0),
            new Vector3(9,0,0),
            new Vector3(0,0,9),
        };
        navmesh.AddPolygon(new int[] { 0, 1, 2 });

        var map = NavigationServer.MapCreate();
        var region1 = NavigationServer.RegionCreate();

        NavigationServer.MapSetUp(map, Vector3.Up);
        NavigationServer.MapSetActive(map, true);

        NavigationServer.RegionSetTransform(region1, Transform.Identity);
        NavigationServer.RegionSetNavmesh(region1, navmesh);
        NavigationServer.RegionSetMap(region1, map);

        await ToSignal(GetTree(), "physics_frame");

        var result = NavigationServer.MapGetPath(map, new Vector3(0.1f, 0, 0.1f), new Vector3(1f, 0, 1f), true);

        GD.Print("Found waypoints: " + result.Length);
        foreach (var r in result) GD.Print(r);
    }

If this is a documentation issue, it shouldn't be closed before documentation is updated.


Fix/parser

to: @chunyong-lin
cc: @airbnb/streamalert-maintainers
related to:
resolves: #1084
Background
Please see #1084
Testing

ran unit_test and pylint

Coverage increased (+0.001%) to 96.144% when pulling fda6ce6f9b30e5ba9c4959bd1c40e6429ea751a2 on jack1902:fix/parser into b87f7769c1e93f74f7072d92da1961012f928ecb on airbnb:release-3-0-0.


Add support for anariGetObjectSubtypes and anariGetObjectParameters for example library/device

For a project I'm working on, I noticed that most of the introspection methods (anariGetObjectSubtypes, anariGetObjectParameters, and anariGetParameterInfo) aren't fully implemented for the example library and device. I've added in preliminary support for the first two introspection methods (but not the last method) for all the objects within the example device that I can find.
In total, this PR adds support for the example device objects: Geometry, Material, Volume, Camera, and Spatial Field. It also fixes support for Renderer (anariGetObjectParameters always returned the same thing for every Renderer subtype despite differences between the subtypes).
This PR does not add support for Sampler or Light (both objects are implemented in the example library as placeholder objects).

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Tanner Hobson seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

There are conflicts with merged #25, which adds an introspection framework (demonstrated with the example device).

Closing as duplicate: the functionality is (now) already supported by the example device.


UUID

Umbrella issue tracking UUID

Introduced UUID with https://github.com/apple/swift-foundation/pull/1


[ZT] Audit images and edit tutorials

After removing unnecessary screenshots, the tutorials had to be rewritten. I also applied our style guide to the pages to clean them up.

Approve assets please


Playnite Game Update checker won't run because the extension thinks directory doesn't exist (after switching to portable install)

Seems like even after I deleted the config.json and the actual extension I get an error when I try to launch the manifestreader.exe in powershell (the plugin will execute and close immediately). When I manually run in powershell I can see the error spits out:
C:\Playnite\Extensions\UpdateChecker_74fe180c-7038-4908-bec1-94194b73b2e4\manifestreader>manifestreader.exe
Traceback (most recent call last):
File "manifestreader.py", line 8, in
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\((OLD INSTALL OF PLAYNITE, NOT PORTABLE))\AppData\Roaming\Playnite\ExtensionsData\74fe180c-7038-4908-bec1-94194b73b2e4\config.json'
[38856] Failed to execute script 'manifestreader' due to unhandled exception!

Not using Playnite much anymore but I'll try and help the best I can

So is the ExtensionsData located in the portable folder C:\Playnite?

Yes it is.  C:\Playnite.

Should be fixed in https://github.com/odeyity/Playnite-Update-Checker-Plugin/releases/tag/1.3.1 but please let me know if you continue to have issues.
Thanks for creating an issue :)


Pex fails to generate hermetic wheel installs on Linux hosts when the install target path is >127 characters long

For example:

Hermetic:

$ rm -rf /tmp/pex_root && pex --pex-root /tmp/pex_root mypy_protobuf==2.4 --intransitive -o mypy_protobuf.pex
$ find /tmp/pex_root/installed_wheels/ -name "protoc-gen-mypy" | while read script; do echo "$script:" && head -3 $script; done
/tmp/pex_root/installed_wheels/60974b0bc64909a12b2201d5d0093478d955e90a/mypy_protobuf-2.4-py3-none-any.whl/bin/protoc-gen-mypy:
#!python
# -*- coding: utf-8 -*-
import re

Not hermetic:

$ pex_root="/tmp/$(echo | sha512sum | cut -d' ' -f1)"
$ echo $pex_root
/tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09
$ echo $pex_root | wc -c
134
$ rm -rf $pex_root && pex --pex-root $pex_root mypy_protobuf==2.4 --intransitive -o mypy_protobuf.pex
/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/pex_bootstrapper.py:483: PEXWarning: The venv for /tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09/pip.pex/8781476b36a7a6b86d240e88ab26d74d7bc81315 at /tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09/venvs/6d965e35637dae05966e8439822c63a427c2a522/108a3ddc84230ab282ea6312e06cb68f51008ce5 has script shebangs of '#!/tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09/venvs/s/adaddbff/venv/bin/python3 -sE' with 173 characters. On some systems this may be too long and cause problems running the venv scripts. You may be able adjust PEX_ROOT from /tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09 to a shorter path as a work-around.
  pex_warnings.warn(
$ find $pex_root/installed_wheels/ -name "protoc-gen-mypy" | while read script; do echo "$script:" && head -3 $script; done
/tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09/installed_wheels/60974b0bc64909a12b2201d5d0093478d955e90a/mypy_protobuf-2.4-py3-none-any.whl/bin/protoc-gen-mypy:
#!/bin/sh
'''exec' /tmp/be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09/venvs/s/adaddbff/venv/bin/python3 "$0" "$@"
' '''

The code doing this is here:
https://github.com/pantsbuild/pex/blob/196b4cd5b8dd4b4af2586460530e9a777262be7d/pex/vendor/_vendored/pip/pip/_vendor/distlib/scripts.py#L127-L156

I think Pex will have to become console_script entry-point aware in its post-processing of installed wheel chroots. The ~spec is here: https://packaging.python.org/specifications/entry-points/ Using that, Pex could single out console_scripts and more aggressively re-write them to have #!python shebangs.

I'm almost certainly missing something...could we fork the distutils code to always use the "simple_shebang"? I think I'm not understanding the executable variable, whether it will always be python or not like the example shows.

Yeah - you're missing something, you need to read more of the pip / distlib vendored code.
Simply though, every fork makes upgrading vendored Pip harder. My strategy has been 0 forks and instead using fixups after running pip download, pip wheel or pip install. I already do pip install fixups here so this is just more of that:
https://github.com/pantsbuild/pex/blob/da25bae605407b12461180fc8701cd0e78934b8e/pex/pip.py#L931-L1065


Could you please provide more examples?

Could you please provide some examples how to use this library? Currently I want to write tests for Autocomplete input element.
Autocomplete makes requests to https://maps.googleapis.com/maps/api/place/js/AutocompletionService.GetPredictions and receives array of suggestions. How can I mock this request with js-jest-mocks?
I'm planning to:

Provide some array of suggestions
Mock user input
Expect input value should be one of my suggestions

I called initialize  in beforeEach but then I got stuck. My input obviously gets zero suggestions because they are not mocked. Where can I do this? Which method should I use to configure them?

If you would like to upvote the priority of this issue, please comment below or react on the original post above with :+1: so we can see what is popular when we triage.
@AlekseyMalakhov Thank you for opening this issue. 🙏
Please check out these other resources that might help you get to a resolution in the meantime:

Check the issue tracker - bugs and feature requests for Google Maps Platform APIs and SDKs
Open a support case - Get 1:1 support in Cloud Console.
Discord - chat with other developers
StackOverflow - use the google-maps tag

This is an automated message, feel free to ignore.

I would expect this to work when trying to change the mock return value:
vitest.spyOn(google.maps.places.AutocompleteService.prototype, 'getPlacePredictions').mockResolvedValue({
      predictions: [
        {},
        {}
      ]
    })

I understand I'm using vitest but it should work the same as jest.
However I'm getting an error: Error: getPlacePredictions does not exist which is weird because google.maps.places.AutocompleteService class does indeed have a getPlacePredictions method

Ah, ok. Now I understand what you were looking for.
The way to get access to instances that were created by the code under test is to use the mockInstances registry:
import { initialize, AutocompleteService, mockInstances } from https://github.com/googlemaps/js-jest-mocks;

beforeEach(() => {
  initialize();
});

test("my test", () => {
  // [... do something that uses the google.maps.AutocompleteService]

  const [serviceMock] = mockInstances.get(AutocompleteService);
  expect(serviceMock.getPlacePredictions).toHaveBeenCalled();
});

Note that the imported AutocompleteService and google.maps.places.AutocompleteService are the very same object. The only difference is that the former one is known to be the mocked version (so you can access the mock-interface without type-errors when writing tests in TS.
Controlling the output of e.g. getPlacePredictions isn't as easy as I'd like it, since the mocked methods are created in the constructor of the mocked AutocompleteService and you'd need to .
If the creation of the AutocompleteService happens ahead of time, this is relatively easy to do
// assuming your class under test looks like this:
class MyAutocomplete {
  constructor() { this.service = new google.maps.places.AutocompleteService(); }

  async update() {
    const res = await this.service.getPlacePredictions();
  }
}

// in a test you can overwrite the results after the constructor has been called:
test("...", async () => {
  const subject = new MyAutocomplete();

  const [serviceMock] = mockInstances.get(AutocompleteService);
  serviceMock.getPlacePredictions.mockImplementation(() => {
    return {
      // whatever you want the return-value to be
    };
  });

  // now call the method under test
  await subject.update();

  // validate the getPlacePredictions call
  expect(serviceMock.getPlacePredictions).toHaveBeenCalled();
});

The same applies if you can somehow separate the creation of the service-object from it's usage, for example by monkey-patching it in the tests:
test("...", async () => {
  const subject = new MyAutocomplete();

  // note that this is probably more of an antipattern, since it exposes
  // implementation details (like the `createAutocompleteService` method)
  // to the test, making it harder to change the implementation.

  // this is assuming that `subject.update()` will internally call a method
  // `this.createAutocompleteService()` when called.
  subject.createAutocompleteService = jest.fn(() => {
    const svc = new AutocompleteService();

    // configure mocked methods as above

    return svc;
  });

  await subject.update();
});

Lastly, you can replace the AutocompleteService entirely for full control over its behaviour in your tests, something like this:
const getPlacePredictions = jest.fn();
class MockAutocompleteService extends AutocompleteService {
  constructor() {
    super();

    this.getPlacePredictions = getPlacePredictions;
  }
}

test('...', () => {
  google.maps.places.AutocompleteService = MockAutocompleteService;
  getPlacePredictions.mockReturnValue();

  // run your test
});

Now, I agree that there has to be an easier way to do this, and I will think about how it could be done elegantly. In the meantime, it would be awesome if someone would find the time to update the documentation in the readme with some information about these cases like I did above.


tensorflow for python3.6

Motivation for this change
Right now tensorflow is unusable in release-17.09 due to #29154, why not to support default python36 then?
Things done

Built on platform(s)

[X] NixOS

[X ] Tested execution of all binary files (usually in ./result/bin/)
[X ] Fits CONTRIBUTING.md.

cc @jpbernardy

cc @jyp Could you update the handle you use in Nixpkgs?

I agree that we should move to python 3.6

@FRidh See #29263

LGTM


Measure-FleetCoreWorkload - The running command stopped...

Newly built fleet on Windows Server 2019 throwing error after write warm-up.
2021-11-08T19:29:44: WRITE WARMUP COMPLETE: took 00d.01h:54m:12s
VERBOSE: Operation '' complete.
2021-11-08T19:29:53: READ CACHE WARMUP: START
VERBOSE: 2021-11-08T19:29:53: 6 cache devices totaling 34,76 TiB capacity
VERBOSE: 2021-11-08T19:29:53: 144 VMs => 148,32 GiB cache available per VM (at 60,0% target of total)
VERBOSE: 2021-11-08T19:29:53: Profile read range: [0,00GiB - 57,50GiB] (57,50GiB total) of total 57,50GiB VM workingset
VERBOSE: 2021-11-08T19:29:53: Full range warmable
PS>TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."
PS>TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."
VERBOSE: 2021-11-08T19:29:54: pause is already set @ 11/08/2021 19:29:44
PS>TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."
PS>TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."

TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."
TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."
TerminatingError(): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Method invocation failed because [System.Management.ManagementObject] does not contain a method named 'RefreshRegParams'."

This should be fixed in 2.0.2 - can you update? Please Install-Fleet -Force after installing the new module - this places the new VM control loop script for VM pickup.


Streamline data provided by flow builder events

PHP Version
8.1
Shopware Version
<IP_ADDRESS>
Expected behaviour
The data provided in the events for the flow builder should be the same, so that there are no unexpected cases when using the data in for example e-mail templates.
Actual behaviour
For example the order data for different order related events is not the same, which is quite unexpected.
See an example here:
OrderPlacedEvent has the following order data:
https://github.com/shopware/platform/blob/6b21de1beb62e616dfbcf484152af480a512b9e5/src/Core/Checkout/Cart/SalesChannel/CartOrderRoute.php#L135
OrderStateChangedEvent has this order data:
https://github.com/shopware/platform/blob/6b21de1beb62e616dfbcf484152af480a512b9e5/src/Core/Checkout/Order/Listener/OrderStateChangeEventListener.php#L241
This leads to the unexpected error that lineItem images are present in a e-mail template when using the OrderPlacedEvent, but not when using a OrderStateChangedEvent.
How to reproduce

Have a mail template for orders where you use the lineItem-cover images
use the same template in a flow with the trigger checkout.order.placed and one with the trigger state_enter.order_transaction.state.paid
the mail that is send will contain the images in the first, but not the second flow.

Hello,
Within our company roadmap and work capacity, we try to address each bug or improvement request but admit that not each one will be resolved. To continue our culture of honesty and openness, we are closing this issue to focus on our roadmap on behalf of all Shopware users.
If you feel like this issue is still impacting you, please create a new issue and let us know.
Thank you again for your valuable feedback.


react-native-codegen 0.0.7 transitive package unset-value/1.0.0 have known vulnerability security issue

Description
react-native-codegen 0.0.7 transitive package unset-value/<IP_ADDRESS> have known vulnerability security issues.
We are using unset-value/1.0.0 transitive package under react-native-codegen 0.0.7 library, unset-value/1.0.0 transitive package having security issue ie.. unset-value is vulnerable to a prototype pollution attack. A remote attacker may be able to execute arbitrary code or cause a denial-of-service (DoS) by tricking the library into modifying or adding properties of Object.prototype. and CVE: BDSA-2021-4507
RCE
We would expect to fix BDSA-2021-4507
RCE) for unset-value/1.0.0 transitive package, upgrading react-native-codegen 0.0.7 latest version
Version
react-native-codegen 0.0.7
Output of npx react-native info
npm WARN deprecated<EMAIL_ADDRESS>See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated<EMAIL_ADDRESS>https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated<EMAIL_ADDRESS>support for ECMAScript is superseded by uglify-js as of v3.13.0
Steps to reproduce
Run the SCA using Blackduck found transitive package unset-value/<IP_ADDRESS>  vulnerable and CVE: BDSA-2021-4507
RCE
Snack, code example, screenshot, or link to a repository
NA

That's a valid report, but we'll have to bump jscodeshift to 0.14.0

looks like the version is now upgraded: https://github.com/facebook/react-native/pull/35356

Indeed so we can close this


Dror

Mosty styles, some component updates.
I think it is ready to be launched.

adding another change to this pull request:
XStory --> xStory
simple change that can break the entire project .... :(

Good stuff man, merged 💪🏻


Fix all ally/a11y warnings

#261 copes with this issue. Going through the entire codebase to fix all of these issues are too low of a priority. Closing


Better Identifier Emission & State Handling

Note that tests are indeed failing, but this is due to an oddity in roslyn that means Identifiers are created with trivia. Investigating but I'd like to merge this, ignoring the failing tests. I think it's worth tracking this oddity, but not worth blocking on.

Nevermind. I figured out why this was happening & the fix - I don't understand roslyn sometimes.


Added +lsp flag to lang/purescript

this PR merged, so I added lsp flag to PureScript.

@hlissner
fixed! 👍

Looks good to me, thanks!


Encode code points that span multiple characters

Fixes RESTEASY-2633

Tests are here: https://github.com/resteasy/Resteasy/pull/2480
Is there any reason why this did not make it into the 4.5.6 release for Quarkus?

Thanks Stuart.
As for why this didn't go into 4.5.6, it's because it was decided to make that release as minimal as possible, just fixing the CVE (plus another fix that was there since weeks).
If this and perhaps other fixes are urgent, I can arrange another micro release soon, just let me know.


Set a 5% drop threshold for code coverage

This should help with getting the Codecov checks report as success a bit
more.

Helped on #136, merging


Pending changes confirmation with PropertyGrid

How to detect change in PropertyGrid? I have a dialog similar to OtherFormInTabDialog. I made the modifications shown below hoping that changes in tab with PropertyGrid would be detected but it's not working as expected. Any insights would be greatly appreciated.
        DialogUtils.pendingChangesConfirmation(this.element, () => this.getSaveState() != this.loadedState); // primary tab
        DialogUtils.pendingChangesConfirmation(this.permitPropertyGrid.element, () => this.getSaveStatePermit() != this.loadedStatePermit); // secondary tab

......
getSaveState() {
try {
return $.toJSON(this.getSaveEntity());
}
catch (e) {
return null;
}
}
    getSaveStatePermit() {
        try {
            var saveEntity = this.permitPropertyGrid.getGridField().getSaveEntity();
            return $.toJSON(saveEntity);
        }
        catch (e) { // get an exception here
            return null;
        }
    }

    loadResponse(data) {
        super.loadResponse(data);
        this.loadedState = this.getSaveState();
        this.loadedStatePermit = this.getSaveStatePermit();
    }

I changed some code around and got it to work. The only issue remains that if changes are pending and user clicks OK (to save), only the main entity gets saved:
DialogUtils.pendingChangesConfirmation(this.element, () => (this.getSaveState() != this.loadedState) || (this.getSaveStatePermit() != this.loadedStatePermit));
.....
getSaveStatePermit() {
try {
var saveEntity = <Default.PermitRow>{};
this.permitPropertyGrid.save(saveEntity);
return $.toJSON(saveEntity);
}
catch (e) {
return null;
}
}

Got around the issue by converting the dialog to use single toolbar (as in OtherFormOneBarDialog).


I am to try that improve very simple some syntaxes again.

What kind of change does this PR introduce? (check at least one)

[ ] Bugfix
[ ] Feature
[x] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[x] It's submitted to the dev branch for v2.x (or to a previous version branch), not the master branch
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] All tests are passing: https://github.com/vuejs/vue/blob/dev/.github/CONTRIBUTING.md#development-setup
[ ] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Thanks again, I appreciate your interest in contributing, but please note that it's best to submit a pull request only when the change brings objective and substantial improvements over the original code. In this case this is mostly a style preference (the original also compresses a little better).

I see. 👀
I really love vue.
So I will try to find that is not a style preference.
Thx for your feedback. 🙏


Freshly built tfcompile core dumps

NOTE: Issues that are not bugs or feature requests will be closed. Please ask usage questions on StackOverflow.
 No
TensorFlow installed from (source or binary)?: Source
TensorFlow version: docker image: "tensorflow/tensorflow:nightly-devel" - git describe = 0.6.0-16017-ga9b7946 (aka: docker-pullable://tensorflow/tensorflow@sha256:5b568f7dd9890bb0b86101bb27779c539f608724c7d4ecf4fdfbbab289bc29de)
Bazel version (if compiling from source): Build label: 0.4.5
CUDA/cuDNN version: -
GPU Model and Memory: -
Exact command to reproduce: cd /tensorflow; bazel build tensorflow/compiler/aot:tfcompile ; bazel-bin/tensorflow/compiler/aot/tfcompile

Describe the problem clearly
I'm trying to use tfcompile, when I build it in a Pod on my kubernetes cluster, the resulting binary exits with Aborted (core dumped) after displaying the help message.
Source Code / Logs
I create a pod like this:
apiVersion: v1
kind: Pod
metadata:
  labels:
    name: tfcompile
  name: tfcompile
spec:
  containers:
  - args:
    - sh
    - -c
    - cd /tensorflow; bazel build tensorflow/compiler/aot:tfcompile ; sleep 864000
    image: tensorflow/tensorflow:nightly-devel
    name: tfcompile
    resources:
      limits:
        cpu: "1"
        memory: 16Gi
      requests:
        cpu: "1"
        memory: 16Gi

and then when the build is finished, I execute into the pod and try the resulting binary: bazel-bin/tensorflow/compiler/aot/tfcompile
full output:
root@tfcompile-1266760932-s2dht:/tensorflow# bazel-bin/tensorflow/compiler/aot/tfcompile
2017-04-06 08:08:02.798285: F tensorflow/compiler/aot/tfcompile_main.cc:136] Check failed: argc == 1 && !flags.config.empty() && (flags.dump_fetch_nodes || (!flags.graph.empty() && !flags.entry_point.empty()))
tfcompile performs ahead-of-time compilation of a TensorFlow graph,
resulting in an object file compiled for your target architecture, and a
header file that gives access to the functionality in the object file.
A typical invocation looks like this:

   $ tfcompile --graph=mygraph.pb --config=myfile.pbtxt

usage: bazel-bin/tensorflow/compiler/aot/tfcompile
Flags:
	--graph=""                       	string	Input GraphDef file.  If the file ends in '.pbtxt' it is expected to be in the human-readable proto text format, otherwise it is expected to be in the proto binary format.
	--config=""                      	string	Input file containing Config proto.  If the file ends in '.pbtxt' it is expected to be in the human-readable proto text format, otherwise it is expected to be in the proto binary format.
	--dump_fetch_nodes=false         	bool	If set, only flags related to fetches are processed, and the resulting fetch nodes will be dumped to stdout in a comma-separated list.  Typically used to format arguments for other tools, e.g. freeze_graph.
	--debug_dir=""                   	string	Specifies a directory to dump debugging information, including rewritten graphs and the XLA HLO module.
	--target_triple="x86_64-pc-linux"	string	Target platform, similar to the clang -target flag.  The general format is <arch><sub>-<vendor>-<sys>-<abi>.  http://clang.llvm.org/docs/CrossCompilation.html#target-triple.
	--target_cpu=""                  	string	Target cpu, similar to the clang -mcpu flag.  http://clang.llvm.org/docs/CrossCompilation.html#cpu-fpu-abi
	--target_features=""             	string	Target features, e.g. +avx2, +neon, etc.
	--entry_point=""                 	string	Name of the generated function.  If multiple generated object files will be linked into the same binary, each will need a unique entry point.
	--cpp_class=""                   	string	Name of the generated C++ class, wrapping the generated function.  The syntax of this flag is [[<optional_namespace>::],...]<class_name>.  This mirrors the C++ syntax for referring to a class, where multiple namespaces may precede the class name, separated by double-colons.  The class will be generated in the given namespace(s), or if no namespaces are given, within the global namespace.
	--out_object="out.o"             	string	Output object file name.
	--out_header="out.h"             	string	Output header file name.
	--xla_debug_cpu_dump_ir=""       	string	Dump IR, before optimizations to a path
	--xla_cpu_llvm_opt_level=2       	int32	The LLVM optimization level for the CPU XLA backend. Valid range is from 0 to 3 where 0 means no optimizations.
	--xla_cpu_llvm_cl_opts=""        	string	Comma-separated list of command line options to pass to LLVM.
	--xla_cpu_embed_ir=false         	bool	Embed the LLVM IR module string in the resultant CpuExecutable.
	--xla_cpu_parallel=false         	bool	Use the multi-threaded CPU backend.
	--xla_cpu_use_eigen=true         	bool	Use Eigen for matrix multiply on the CPU platform. This is a useful hack for performance comparisons against XLA's implementation.
	--xla_cpu_multi_thread_eigen=true	bool	When generating calls to Eigen for matmul and conv, should single or multi-threaded eigen be used? Only used when --xla_cpu_use_eigen is true.

Aborted (core dumped)
root@tfcompile-1266760932-s2dht:/tensorflow#

I've actually been trying this sporadically with different nightly/release builds for the past few weeks, hoping that this would be resolved, but it still does not work, so here is my GH issue ;)

The error message could probably be improved, but looking at the first line of output:
Check failed: argc == 1 && !flags.config.empty() && (flags.dump_fetch_nodes || (!flags.graph.empty() && !flags.entry_point.empty()))

it means that the tool is not being invoked correctly. As per the usage message, you need to provide the --graph and --config flags.

ok, so I do the following (per https://www.tensorflow.org/performance/xla/tfcompile):
python  tensorflow/compiler/aot/tests/make_test_graphs.py  --out_dir=.
cat > myfile.pbtxt <<EOT
# Each feed is a positional input argument for the generated function.  The order
# of each entry matches the order of each input argument.  Here “x_hold” and “y_hold”
# refer to the names of placeholder nodes defined in the graph.
feed {
  id { node_name: "x_hold" }
  shape {
    dim { size: 2 }
    dim { size: 3 }
  }
}
feed {
  id { node_name: "y_hold" }
  shape {
    dim { size: 3 }
    dim { size: 2 }
  }
}

# Each fetch is a positional output argument for the generated function.  The order
# of each entry matches the order of each output argument.  Here “x_y_prod”
# refers to the name of a matmul node defined in the graph.
fetch {
  id { node_name: "x_y_prod" }
}
EOT
bazel-bin/tensorflow/compiler/aot/tfcompile --graph=test_graph_tfmatmul.pb --config=myfile.pbtxt

and I get the same result:
root@tfcompile-1266760932-s2dht:/tensorflow# bazel-bin/tensorflow/compiler/aot/tfcompile --graph=test_graph_tfmatmul.pb --config=myfile.pbtxt
2017-04-06 08:33:28.828508: F tensorflow/compiler/aot/tfcompile_main.cc:136] Check failed: argc == 1 && !flags.config.empty() && (flags.dump_fetch_nodes || (!flags.graph.empty() && !flags.entry_point.empty()))
tfcompile performs ahead-of-time compilation of a TensorFlow graph,
resulting in an object file compiled for your target architecture, and a
header file that gives access to the functionality in the object file.
A typical invocation looks like this:

   $ tfcompile --graph=mygraph.pb --config=myfile.pbtxt

usage: bazel-bin/tensorflow/compiler/aot/tfcompile
Flags:
	--graph=""                       	string	Input GraphDef file.  If the file ends in '.pbtxt' it is expected to be in the human-readable proto text format, otherwise it is expected to be in the proto binary format.
	--config=""                      	string	Input file containing Config proto.  If the file ends in '.pbtxt' it is expected to be in the human-readable proto text format, otherwise it is expected to be in the proto binary format.
	--dump_fetch_nodes=false         	bool	If set, only flags related to fetches are processed, and the resulting fetch nodes will be dumped to stdout in a comma-separated list.  Typically used to format arguments for other tools, e.g. freeze_graph.
	--debug_dir=""                   	string	Specifies a directory to dump debugging information, including rewritten graphs and the XLA HLO module.
	--target_triple="x86_64-pc-linux"	string	Target platform, similar to the clang -target flag.  The general format is <arch><sub>-<vendor>-<sys>-<abi>.  http://clang.llvm.org/docs/CrossCompilation.html#target-triple.
	--target_cpu=""                  	string	Target cpu, similar to the clang -mcpu flag.  http://clang.llvm.org/docs/CrossCompilation.html#cpu-fpu-abi
	--target_features=""             	string	Target features, e.g. +avx2, +neon, etc.
	--entry_point=""                 	string	Name of the generated function.  If multiple generated object files will be linked into the same binary, each will need a unique entry point.
	--cpp_class=""                   	string	Name of the generated C++ class, wrapping the generated function.  The syntax of this flag is [[<optional_namespace>::],...]<class_name>.  This mirrors the C++ syntax for referring to a class, where multiple namespaces may precede the class name, separated by double-colons.  The class will be generated in the given namespace(s), or if no namespaces are given, within the global namespace.
	--out_object="out.o"             	string	Output object file name.
	--out_header="out.h"             	string	Output header file name.
	--xla_debug_cpu_dump_ir=""       	string	Dump IR, before optimizations to a path
	--xla_cpu_llvm_opt_level=2       	int32	The LLVM optimization level for the CPU XLA backend. Valid range is from 0 to 3 where 0 means no optimizations.
	--xla_cpu_llvm_cl_opts=""        	string	Comma-separated list of command line options to pass to LLVM.
	--xla_cpu_embed_ir=false         	bool	Embed the LLVM IR module string in the resultant CpuExecutable.
	--xla_cpu_parallel=false         	bool	Use the multi-threaded CPU backend.
	--xla_cpu_use_eigen=true         	bool	Use Eigen for matrix multiply on the CPU platform. This is a useful hack for performance comparisons against XLA's implementation.
	--xla_cpu_multi_thread_eigen=true	bool	When generating calls to Eigen for matmul and conv, should single or multi-threaded eigen be used? Only used when --xla_cpu_use_eigen is true.

Aborted (core dumped)
root@tfcompile-1266760932-s2dht:/tensorflow#

The error messages can be improved :), but again, looking at the Check failed line, it seems that you also need to specify --entry_point.

Ok, got it working now with the following commandline:
bazel-bin/tensorflow/compiler/aot/tfcompile --graph=test_graph_tfmatmul.pb --config=myfile.pbtxt --entry_point="test_graph_tfmatmul" --cpp_class="foo::bar::MatMulComp"
Thanks!

May I know how to generate the tfcompilebinary? I could not find it bazel-bin/tensorflow/compiler/aot/tfcompile


Added error catching logic to stop app crash when imported file is too short

In Scenario 1 of AudioCreation there is a pre-condition in the code and not messaged to the user that the imported sound file is 3 seconds or longer.
It looks like this condition exists to showcase how to trim a file in the source code.
With this added change there will now be an error message displayed in the ErrorMessage field when the imported sound file is too short for the scenario.

Thanks for the fix. It'll be in the next update of the samples repo.


Optimize LogIngestionClient.Batch yielding

Fixes the method used for flushing the last account so that it does not rely on the total count of items being known in advance (see #40379) and works even if the last item is large (#40941).
Also tweaks the size threshold calculations to account for the requirement to add [ and ] to the serialized JSON.

API change check
API changes are not detected in this pull request.

@nisha-bhatia please take a look at this.

Hi @Arithmomaniac, thank you for putting together this PR. Could you please provide a test that shows the changes you have made to the code?

I have added a test for the fix to #40941.
The #40379 - related stuff is just performance-related; if the rest of the tests work unchanged, that's good enough.
It would be harder to write a test to test exactly the size test calculations, but I can do that if required as well.

The threshold-calculating tests have been added. They require a lot of ceremony setup, so improvement suggestions welcome!

Keeping alive.
Reminder this PR fixes any issue where the SDK silently drops recoeds


grub: add support for SD card

The current grub configuration is effectively HiKey specific and supports
only eMMC boot. Since the boot device is passed to CMDLINE, set
dynamically GRUB_CFG to allow eMMC or SD card boot.
Move the cfg files to a machine specific SRC_URI.
Signed-off-by: Fathi Boudra<EMAIL_ADDRESS>

This makes it match the kernel/machine config, so I'm all for it!


DEVOPS: Fix Tag mismatch on Paloma manual build with missing v

Some exports have app_version as 1.13.1 and some have v1.13.1. This is the only difference between all the exports.
This mismatch is coming from prebuilt vs maunally built binaries. When I saw the —long version of both the binaries I saw that the prebuilt binary has this.
build_tags: ""
commit: 1143f40382ff1540dc134ceecf11b50af756428a
cosmos_sdk_version: v0.50.5
go: go version go1.22.2 linux/amd64
name: paloma
server_name: palomad
version: v1.13.1

vs.
build_tags: netgo ledger,
commit: 1143f40382ff1540dc134ceecf11b50af756428a
cosmos_sdk_version: v0.50.5
go: go version go1.22.2 linux/amd64
name: paloma
server_name: palomad
version: 1.13.1

Ensure that manual builds add the v so consistent with binary build.

Fixed with https://github.com/palomachain/paloma/pull/1140


Added Warnings to resourse

Problem
There are some cases where you just want the user to know that the import contains some data that is not now an error but may cause problems. (give a warning).
Solution
This PR add the ability to add warnings to the row result and also the display in the import view.

I think this looks really promising.  I have some thoughts as follows:

I think this would be suitable for the next major release (v4) because it will impact the UI
We could also look at the error / warnings framework in more detail, because this is something that causes confusion and frustration (e.g. #1467)
It would be very useful if we could design a pattern where users could define which exceptions constituted errors and which should be treated as warnings
We could look again and the error reporting code to ensure correct separation of concerns between the processing layer and the presentation layer.

I'll flag this as draft for v4 and happy to help move it forwards wrt the above.

Closing due to lack of activity


Nav bar on splash screen

Meizu m1 note phone on Android 4.4.4 (phonу with touch-home-button only)
cordova 5.4.1 / android platform 4.1.1
I have nav-bar on splashscreen at start and web-content deformation after hide splashscreen.
No effect from plugin(

Work, but only nav-bar, not status-bar.
Can hide status-bar manualy, by roll status-board from top of screen, and roll-up (close).