Lots of S3 improvements

These changes either fix the consistency problem or reduce the number
of API calls we make to S3.

Fix eventual consistency problem that the fragment package was finding

it was deleting an object and surprised when a folowing HEAD request
thought that it was still around.
we now mark deleted things as deleted in-memory so we can present
a coherent view to other code in the server.

Add more logging and sentry reporting for s3 request errors
Add cache to track the sizes of the objects in the given bucket. This
drastically cuts down on the number of HEAD requests
only use multipart upload for content larger than 5 MB
handle 0 length file uploads gracefully
simplify the code path for closing an S3 upload

Additionally, we adjust the fragment package

Don't add a fragment to the child list until it is successfully closed.

And cmd/bendo

Use milisecond logging

DLTP-1737

@msuhovec If there is something that manages S3 for us at this level, that would be great. I didn't see something that would work the last time I looked. That was 18 months ago; I should look again.

@msuhovec It looks like the uploader provided by the aws-sdk can now take an io.Reader. (previously, I think it required an io.ReadSeeker. This suggests maybe we could use it with an io.Pipe connecting to it.

A pipe or stream was what I was thinking of.
On Fri, Jun 26, 2020 at 1:02 PM Don Brower<EMAIL_ADDRESS>wrote:

@msuhovec https://github.com/msuhovec It looks like the uploader
provided by the aws-sdk can now take an io.Reader. (previously, I think
it required an io.ReadSeeker. This suggests maybe we could use it with an
io.Pipe connecting to it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ndlib/bendo/pull/213#issuecomment-650288776, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZUMTHP5Z3KSKX7X73K3ATRYTICXANCNFSM4OJIUG7A
.

I think I will do the pipe idea as another PR. Meanwhile I will adjust this PR to name the constant.


get 200 OK rather than 401 invalid token for expired token

Here is an issue about calling get item metadata api for root item with a expired token.
Get item metadata api responses with status code 200 for root item id.
For non-root item id, get item metadata api responses 404 Not Found.
It might lead to some security issues to distinguish root or non-root item for a given id
with a expired token.
E.g.
Request:

GET https://api.onedrive.com/v1.0/drive/items/C4A912C98CAF0C8B!267

,where C4A912C98CAF0C8B!267 is the root item id and a expired token is given.
Responses:

HTTP/1.1 200 OK
WWW-Authenticate:
Bearer realm="OneDriveAPI", error="expired_token", error_description="Auth token expired. Try refreshing."
X-MSNSERVER:
SN3302____PAP104
Date:
Wed, 13 Apr 2016 02:57:01 GMT
P3P:
CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
Content-Length:
214
OData-Version:
4.0
Via:
1.1 DM2302____PAP115 (wls-colorado)
X-AsmVersion:
UNKNOWN; <IP_ADDRESS>
Connection:
keep-alive
Content-Type:
application/json; odata.metadata=minimal; odata.streaming=true
Server:
Microsoft-HTTPAPI/2.0 Microsoft-HTTPAPI/2.0
X-WLSPROXY:
DM2302____PAP115
{
<EMAIL_ADDRESS>"https://api.onedrive.com/v1.0/$metadata#drives('me')/items/$entity",
"id": "C4A912C98CAF0C8B!267",
"name": "root",
"webUrl": "https://onedrive.live.com/?cid=c4a912c98caf0c8b",
"folder":  {
"childCount": 0
}
}

Are you still seeing this?

Dear Ryan Gregg,
yes,
A request to reproduce it:
GET https://api.onedrive.com/v1.0/drive/items/C4A912C98CAF0C8B!267
Authorization: Bearer
EwB4Aq1DBAAUGCCXc8wU/zFu9QnLdZXy+YnElFkAAbyKbL68eldZ/DKtDGN9dzYadZjqtnBwiWzNNr0qgr5/QNQy6dVKsRjEZTxt87Lr+QgBMBvQQtKsXw0SypMAVacxII1ZG0DyvVtWzZqMUH9IWhzMOdfa+pmA8ypbD44Z8SrZ3kvOzFPNAX0kYQChzAYK/j9kX84uVoEVXgn2Wd3xSxqQRHfCZW4iRNvJR8aMU7BbSejTX0/MdBHbT4vieIz6LiYF4h+zHqpF3cu06nSrYtZL7QW60ug59puj5B9AxktPrCR24gcc5EzqbJJRhUAtIDzk08EF805SH3US1LXWaXcCZHR/Aujv5IqSNcOOieuK2/BfJffH7KMqlHTjB7ADZgAACAGBhXP9vSn3SAHNo1eOj+E5uwd9Z+r40H98URvOTKbcjnDHhm4CkMC/edbO9eJmjxrZEalgHsuBFJgqps6lss253bnF5xFUCf9g/1e2YR7/hc6fVX3o3/cYDuTdhvvSDr/7fIcYN1JM4WEW5xHbd/1KsR9QDGQRpD0dnHndR+e5sAxe6bnX5VU3EDDBu0BPSbhPusH/WGIHuAKV9ndZ7xN9p3eicckF3b3y3cVX0vPBH9eseSLRU69m+zluHL8Jdbrn5JpSl7cQq2XVUPIV/eQapG6wR7XCMtG3XtBf+Fin4mJLwSTb2WyzzacX8MXrJSi9P7N8APUy4l5Z/mFXul4yUn7f91Y6bYHQR8+UwGsSyCz/uujvNaYOTDKCC1KZBTh8BzAS1KfGP1wRBzs8v3LSY3nc9nU3Z6kpKWdDoVe2jRV9i2ssuc0ZdFw04PSVngdViwE=
Response:
200 OK
{
<EMAIL_ADDRESS>"
https://api.onedrive.com/v1.0/$metadata#drives('me')/items/$entity",
"id": "C4A912C98CAF0C8B!267",
"name": "root",
"webUrl": "https://onedrive.live.com/?cid=c4a912c98caf0c8b",
"folder": {
"childCount": 0
}
}
2016-06-04 7:10 GMT+08:00 Ryan Gregg<EMAIL_ADDRESS>
Are you still seeing this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OneDrive/onedrive-api-docs/issues/353#issuecomment-223715794,
or mute the thread
https://github.com/notifications/unsubscribe/AHL8A1ugnr0pDxWO-oXunh9IoaTfvjH2ks5qILRjgaJpZM4IOtlN
.

Sorry about this very old issue going stale.  If this is still reproducing, could you please provide updated request & response information to help us investigate further?  Thanks.


Add CLI command for extracting root xprvs

Issue Number
#1316
Overview

[x] I have added cardano-wallet-jormungandr key root --type random <mnemonic words>
[x] I added unit tests for help-text and actual usage.
[x] I added a pending test making sure keys are compatible with jcli (which would fail)

Comments
$ cardano-wallet-jormungandr key --help
Usage: cardano-wallet-jormungandr key COMMAND
  Derive keys from mnemonics.

Available options:
  -h,--help                Show this help text

Available commands:
  root                     Extract root xprv as hex (64 bytes private key + 32
                           bytes chain code)
$ cardano-wallet-jormungandr key root --help
Usage: cardano-wallet-jormungandr key root --type KEYTYPE MNEMONIC_WORDS...
  Extract root xprv as hex (64 bytes private key + 32 bytes chain code)

Available options:
  -h,--help                Show this help text
  --type KEYTYPE           Any of the following:
                             random (Daedalus, 12 words)
                             icarus (15 words)
                             trezor (12, 15, 18, 21, or 24 words)
                             ledger (12, 15, 18, 21, or 24 words)
$ cardano-wallet-jormungandr key root --type random flock advance execute country leader exotic mix twenty six margin orient meat
68a0f29e6bd5d8af7ffd00a55006afa8af6fbdbded07984ddf7fb1c31c66f7460685e5d1016553fccc9724f5ee95dd8d66facd2ac1bb2f6fcd7fa5e53c97a57f50c592fcd18b67bf3393a16184d009fb25450b2de8079f870222874e804584a8

Please give PRs a proper title instead of just accepting the default. With a proper title I can see whan a PR is about without having to follow the link.

will leave the commit history as is for the moment

could split off bfe2568 to a separate pr 🤔

bors r+


Yosys uhdm frontend is creating modules on usage

Yosys uhdm frontend doesn't distinguish usage of module from a definition and it is always creating module if it can't find one. Example:
image/bin/surelog -parse -sverilog tests/ibex/ibex/build/lowrisc_ibex_top_artya7_0.1/src/lowrisc_ibex_fpga_xilinx_shared_0/rtl/fpga/xilinx/clkgen_xil7series.sv
yosys/yosys -p "read_uhdm -debug slpp_all/surelog.uhdm"

UHDM Output:
    |vpiModule:
    \_module: work@clkgen_xil7series::BUFG<EMAIL_ADDRESS>clkgen_xil7series.sv:63     : , parent:work@clkgen_xil7series
      |vpiDefName:work@clkgen_xil7series::BUFG
      |vpiName:clk_fb_bufg
<EMAIL_ADDRESS>      |vpiPort:
      \_port: (I<EMAIL_ADDRESS>        |vpiName:I
        |vpiHighConn:
        \_ref_obj: (clk_fb_unbuf), line:64
          |vpiName:clk_fb_unbuf
          |vpiActual:
          \_logic_net: (@@BAD_SYMBOL@@), line:16, parent:work@clkgen_xil7series
      |vpiPort:
      \_port: (O<EMAIL_ADDRESS>        |vpiName:O
        |vpiHighConn:
        \_ref_obj: (clk_fb_buf), line:65
          |vpiName:clk_fb_buf
          |vpiActual:
          \_logic_net: (@@BAD_SYMBOL@@), line:15, parent:work@clkgen_xil7series

Yosys AST output:
    AST_MODULE <slpp_all/surelog.uhdm:0.0-0.0> [0x5574caf67130] str='\clkgen_xil7series::BUFG' basic_prep
       AST_WIRE <slpp_all/surelog.uhdm:0.0-0.0> [0x5574caf67f60] str='\I' basic_prep port=30 range=[0:0]
       AST_WIRE <slpp_all/surelog.uhdm:0.0-0.0> [0x5574caf680a0] str='\O' basic_prep port=31 range=[0:0]

clkgen_xil7series::BUFG module was created, but it was just used in clkgen_xil7series.sv.

It is partially fixed in #64, modules are now created with correct names (and ports have correct directions), but we are still creating AST_MODULE node on usage.

I fixed creating modules on usage in https://github.com/antmicro/yosys/commit/def67af4e11acd370e554004c48ebc97fb1d3da7. This fix do not remove modules that are defined (but not used) from the design. In this kind of modules, uhdm is missing information about ranges in ports and yosys is failing (on ibex_multdiv_slow.sv) with message ERROR: Unsupported expression on dynamic range select on signal. I created separate issue for this in Surelog https://github.com/alainmarcel/Surelog/issues/892

I fixed this by removing modules, that are just defined, but not used in yosys frontend.


Avoid leaking Hadoop API to downstream libraries

Parquet currently leaks the Hadoop API by requiring callers to pass Path and Configuration instances, and by using Hadoop codecs. InputFile and SeekableInputStream add alternatives to Hadoop classes in some parts of the read path, but this needs to be extended to the write path and to avoid passing options through Configuration.
Reporter: Ryan Blue / @rdblue
Assignee: Ryan Blue / @rdblue
Related issues:

make it easy to read and write parquet files in java without depending on hadoop (is part of)

PRs and other links:

PR #429

Note: This issue was originally created as PARQUET-1142. Please see the migration documentation for further details.

Werner Daehn / @wernerdaehn:
Thanks Ryan, I believe this is one of the most important areas where Parquet can get better.
Do you happen to have some sample code on how to use that? My current approach is to extend ParquetWriter including the inner Builder class extending ParquetWriter.Builder<T, Builder>.
Where I am stuck is the Builder.WriteSupport with its Hadoop Configuration parameter. It has to be implemented but you talk about ParquetReadOptions?
And my class needs a constructor but all visible super()'s are Hadoop based.
Any hints would be greatly appreciated.
Background:
In my world there is no such thing as Hadoop but many different kinds of file storage options, HDFS being one of those. Therefore I would be in favor of removing all Hadoop related stuff from Parquet and rather have a java.io.OutputStream (=org.apache.parquet.io.OutputFile) which then can be used to store files in different locations, HDFS full client, HDFS web client, S3, local file system.
My ultimate goal for the moment is to have a AvroParquetWriter without Hadoop to save to a local file system in a streaming way. This would then be used to consume the Kafka data of the last 24 hours and write one large Parquet file.
Next step would be a matching reader that does pushdown simple filter criteria. Best would be one that allows even primary key lookups in a reasonable amount of time.

Ryan Blue / @rdblue:
The next steps for this are to get compression working without relying on Hadoop. After that, it is a matter of some fairly simple refactoring of the file writer. But that refactoring doesn't help much unless the compression implementations also don't depend on Hadoop.

Hadrien Kohl:
Hi. I've faced the same problems and I still need to drag in hadoop dependencies to be able to use ParquetWriter and ParquetReader.
I am wrapping SeekableByteChannel into InputFile/OutputFile so that I can call

ParquetWriter.builder(new Wrapper(Files.newByteChannel("/foo/bar.parquet")));

Do you plan to work on this further? I'd be happy to help.
I saw prestodb also implemented another reader/writer.

Hadrien Kohl:
Right, I think this project has dropin replacements for hadoop CompressionCodecs. It could be a good starting point.
https://github.com/airlift/aircompressor
Is anyone working on this? I could try to get into it if that is part of the roadmap? @rdblue do you know others that would like to get involved?

Ryan Blue / @rdblue:
Merged #429. Thanks to @zivanfi and @gszadovszky for reviewing!


[backport 7.59.x] Bump test-infra-definitions to 0a48ed72

Backport 32a15800d892661ef02d9c8cb2cde43fcbb9662f from #30709.

This PR was automatically created by the test-infra-definitions bump task.
This PR bumps the test-infra-definitions submodule to 0a48ed7298221d8ab5cdef39f5d50679a523411b from f3aa4172342a.
Here is the full changelog between the two commits: https://github.com/DataDog/test-infra-definitions/compare/f3aa4172342a...0a48ed7298221d8ab5cdef39f5d50679a523411b
:warning: This PR is opened with the qa/no-code-change and changelog/no-changelog labels by default. Please make sure this is appropriate

/merge


Documentation link is broken

Hello,
the current link to the documentation is broken (http://contrib.scikit-learn.org/category_encoders/). I cannot access directly to a specific page, the whole website seems down. Any plan to fix it soon?
Many thanks to the team (for the package and fix) !

Thanks for the heads up, fixed now:
http://contrib.scikit-learn.org/category_encoders/


Explorer: does not reveal file in node_modules

Steps to Reproduce:

open \node_modules\vscode-nsfw\binding.gyp
collapse all in the explorer
reload window

=> explorer does not reveal file. works fine when refreshing the explorer

Works for me. Can you repro with latest insiders, if yes what are you doing differently than me?


Include some sort of output in the VScode 'Output' pane after running shellcheck

I'm currently having an issue where I can't figure out why Shellcheck isn't running (https://github.com/vscode-shellcheck/vscode-shellcheck/discussions/907). One thing contributing to my confusion there is that explicitly invoking the shellcheck.runLint command has ... no observable effect, whatsoever. I can't tell if shellcheck isn't being found, is crashing, or isn't ever being invoked in the first place.
More debugging output would be nice — i.e. something like
[2023-01-30T19:11:09.888Z] [INFO] shellcheck (bundled) version: 0.9.0
[2023-01-30T19:11:09.888Z] [INFO] explicit shellcheck.runLint invoked by user
[2023-01-30T19:11:09.888Z] [INFO] using bundled shellcheck
[2023-01-30T19:11:09.888Z] [INFO] shellcheck response:

In ./api_annotate_dead_code.zsh line 10:
if [[ -f reanalyze.log ]]; then
   ^--------------------^ SC2169: In dash, [[ ]] is not supported.

In ./api_annotate_dead_code.zsh line 11:
...

[2023-01-30T19:11:09.888Z] [INFO] shellcheck binary exited with exit-code 1

... so on and so forth.

@ELLIOTTCABLE In #1035 a diagnostic command is provided, besides, when the logging level of shellcheck is "trace", it can show the response from shellcheck binary (though in JSON)


New Crunchyroll Beta Broke Youtube-dl on classic and beta not supported

Checklist

[ X] I'm reporting a broken site support
[ X] I've verified that I'm running youtube-dl version 2021.03.25
[ x] I've checked that all provided URLs are alive and playable in a browser
[ X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[ X] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', '/home/nick/Downloads/cookies.txt', '--write-sub', '--sub-lang', 'enUS', 'https://www.crunchyroll.com/dr-stone/episode-11-prologue-of-dr-stone-801916', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.03.25
[debug] Python version 3.9.2 (CPython) - Linux-5.10.23-1-MANJARO-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[crunchyroll] 801916: Downloading webpage
WARNING: unable to extract OpenGraph description; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.9/site-packages/youtube_dl/__init__.py", line 475, in main
    _real_main(argv)
  File "/usr/lib/python3.9/site-packages/youtube_dl/__init__.py", line 465, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2059, in download
    res = self.extract_info(
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 799, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 827, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.9/site-packages/youtube_dl/extractor/crunchyroll.py", line 456, in _real_extract
    video_title = re.sub(r'^Watch\s+', '', self._og_search_description(webpage))
  File "/usr/lib/python3.9/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', '/home/nick/Downloads/cookies.txt', '--write-sub', '--sub-lang', 'enUS', 'https://beta.crunchyroll.com/watch/GN7UDZ3VQ/PROLOGUE-OF-Dr-STONE', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.03.25
[debug] Python version 3.9.2 (CPython) - Linux-5.10.23-1-MANJARO-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[generic] PROLOGUE-OF-Dr-STONE: Requesting header
WARNING: Falling back on generic information extractor.
[generic] PROLOGUE-OF-Dr-STONE: Downloading webpage
[generic] PROLOGUE-OF-Dr-STONE: Extracting information
ERROR: Unsupported URL: https://beta.crunchyroll.com/watch/GN7UDZ3VQ/PROLOGUE-OF-Dr-STONE
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 827, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.9/site-packages/youtube_dl/extractor/generic.py", line 3506, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://beta.crunchyroll.com/watch/GN7UDZ3VQ/PROLOGUE-OF-Dr-STONE

Description

Update to the crunchyroll site has broken support. No longer works with the old format links, and does not have implementation to support beta.crunchyroll.com.

I am able to run the old links and they still work.
https://www.crunchyroll.com/dr-stone/episode-11-prologue-of-dr-stone-801916
My arguments for context:
youtube-dl.exe
 --user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36"
 --merge-output-format mkv
 --ffmpeg-location ffmpeg\bin
 --cache-dir "cache"
 --download-archive downloaded.txt
 --fixup warn
 --add-metadata
 --no-call-home
 --sub-lang enUS
 --write-sub
 --cookies cookies.txt
 -a download.txt
 -o "%%(uploader)s/%%(title)s.%%(ext)s"

I also am getting no results with those arguments. Same error as initially posted. Can we get a verbose log of it working for you?

Sure.  I've already downloaded the file in question, but here it is below:
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36', '--merge-output-format', 'mkv', '--ffmpeg-location', 'ffmpeg\\bin', '--cache-dir', 'cache', '--download-archive', 'downloaded.txt', '--fixup', 'warn', '--add-metadata', '--no-call-home', '--sub-lang', 'enUS', '--write-sub', '--cookies', 'cookies.txt', '-a', 'download.txt', '-o', '%%(uploader)s/%%(title)s.%%(ext)s', '--verbose']
[debug] Batch file urls: ['https://www.crunchyroll.com/dr-stone/episode-11-prologue-of-dr-stone-801916']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.03.25
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg 3.4.2, ffprobe 3.4.2, rtmpdump 2.3
[debug] Proxy map: {}
[crunchyroll] 801916: Downloading webpage
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-itIT information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-frFR information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-deDE information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-esES information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-ruRU information
[crunchyroll] 801916: Downloading media info
WARNING: Unable to download XML: HTTP Error 403: Forbidden
[debug] Default format spec: bestvideo+bestaudio/best
[download] Dr. STONE Season 2 Episode 11 – PROLOGUE OF Dr. STONE has already been recorded in archive

It would have errored out before this if it was going to fail.

Sure. I've already downloaded the file in question, but here it is below:
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36', '--merge-output-format', 'mkv', '--ffmpeg-location', 'ffmpeg\\bin', '--cache-dir', 'cache', '--download-archive', 'downloaded.txt', '--fixup', 'warn', '--add-metadata', '--no-call-home', '--sub-lang', 'enUS', '--write-sub', '--cookies', 'cookies.txt', '-a', 'download.txt', '-o', '%%(uploader)s/%%(title)s.%%(ext)s', '--verbose']
[debug] Batch file urls: ['https://www.crunchyroll.com/dr-stone/episode-11-prologue-of-dr-stone-801916']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.03.25
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg 3.4.2, ffprobe 3.4.2, rtmpdump 2.3
[debug] Proxy map: {}
[crunchyroll] 801916: Downloading webpage
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-itIT information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-frFR information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-deDE information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-esES information
[crunchyroll] 801916: Downloading adaptive_hls-audio-jaJP-hardsub-ruRU information
[crunchyroll] 801916: Downloading media info
WARNING: Unable to download XML: HTTP Error 403: Forbidden
[debug] Default format spec: bestvideo+bestaudio/best
[download] Dr. STONE Season 2 Episode 11 – PROLOGUE OF Dr. STONE has already been recorded in archive

It would have errored out before this if it was going to fail.

Is that not a failed run :sweat: I'm struggling to pin down the issue and it doesn't seem to have any traction.

Passing cookies or your user is the problem if you have the beta enabled. I was getting links from an un-signed in window and then using my beta-enabled account to get the episodes. This does not work because those links redirect to the beta version automatically.


flake.lock - No such file or directory

I've installed version 3.3.0 (through nix-env -if https://github.com/DavHau/mach-nix/tarball/3.3.0 -A mach-nix) and
 mach-nix gen -r requirements.txt
yields
path is '/nix/store/zm6li32ifp9by7k26ii8p340b39wmdjy-9971c5d1a40d4e9ff8159ec6595cb9e91da7d438' error: opening file '/nix/store/0k1r7yb7pc96hq8qv65vy10pmdb13fpi-mach-nix-3.3.0/lib/python3.8/site-packages/flake.lock': No such file or directory (use '--show-trace' to show detailed location information)

Same here, also happens with  nix run github:DavHau/mach-nix -- gen -r requirements.txt:
path is '/nix/store/d0c64m91jg6008yx1cglyvm1mb7z8f1b-1ca6b0a0cc38dbba0441202535c92841dd39d1ae'
error: opening file '/nix/store/nh6965qy7w4yr8n0r982cb7vfq45ywjd-mach-nix-master/lib/python3.8/site-packages/flake.lock': No such file or directory

Using stable (v3.3.0) results in the same issue nix run github:DavHau/mach-nix/3.3.0 -- gen -r requirements.txt
path is '/nix/store/zm6li32ifp9by7k26ii8p340b39wmdjy-9971c5d1a40d4e9ff8159ec6595cb9e91da7d438'
error: opening file '/nix/store/0k1r7yb7pc96hq8qv65vy10pmdb13fpi-mach-nix-3.3.0/lib/python3.8/site-packages/flake.lock': No such file or directory


Fix Lidarr build

Same as for #590, the template config file of Lidarr needs to be fixed (cf https://github.com/fastlorenzo/containers/commit/f731c834b8d5ffa895b101e67d682c16361d9f56 for the fix)
https://github.com/onedr0p/containers/blob/da1ce75a41aaadf94ebe4c1e5b155a0b1ae79393/apps/lidarr/config.xml.tmpl#L7

Thanks, I have updated it in https://github.com/onedr0p/containers/commit/92cf013820755d3e1ea15929dca3dfed693f0063 I know the stable release will fail but eventually it will start working when they push a new stable version.


Update the tags for CA and manager

Jenkins job istio/presubmit passed

Closing. This is done in #310


860 ic top navigation double reading

Summary of the changes
Added a silent prop to the ic-tooltip, for instances where the tooltip is duplicating the visible text of the child component.
Set the tooltips for icon-variant buttons and navigation-items to be silent, as they already have accessible names.
I've tested this thoroughly with Voiceover (both by tabbing through and moving the virtual cursor) on icon buttons, side/top navigation and ic-tooltip. I've also added a new "silent" story to the ic-tooltip.
I am unsure how this gels with the mozilla guidance on role=tooltip since this change ignores it. Functionally this does solve the double-reading bug and preserves functionality of the tooltip though.
Related issue
#860
Checklist

[ ] Relevant unit tests and visual regression tests added.
[x] Visual testing against Figma component specification completed.
[x] All acceptance criteria reviewed and met.
[ ] Accessibility Insights FastPass performed.
[ ] A11y unit test added and yields no issues.
[ ] A11y plug-in on Storybook yields no issues.
[x] Manual screen reader testing performed using NVDA and VoiceOver.
[ ] Page can be zoomed to 400% with no loss of content.
[ ] Screen magnifier used with no issues.
[ ] Text resized to 200% with no loss of content.
[ ] Text spacing increased as per the WCAG 1.4.12 success criterion with no loss of content.
[ ] Browser setting 'prefers reduced motion' tested. No animations or motion visible whilst this setting is on.
[ ] Windows High Contrast mode tested with no loss of content.
[x] System light and dark mode tested with no loss of content.
[x] Manual keyboard testing for keyboard controls and logical focus order.
[ ] Min/max content examples tested with no loss of content or overflow.
[x] Browser support tested (Chrome, Safari, Firefox and Edge).
[x] Correct roles used and ARIA attributes used correctly where required.
[x] Logical heading structure is maintained, and the HTML elements used for headings can be changed to fit within the wider page structure.
[ ] All prop combinations work without issue.

PR has become stale and there's significant feedback to address. Will close and return with another branch.


Implement System for Win32

Pretty simple implementation of the hostname and cpu count functions.

crystal tool format please

I think I hit all the PR comment fixes in the latest push. Let me know if there's anything else.

@markrjr This PR needs to address the review comments.

@markrjr You're initial patch wasn't bad. Keep it on! =)

Thanks for the great first patch @markrjr :tada:


Change autofill dialog corner radius

Task/Issue URL: https://app.asana.com/0/488551667048375/1202854142935496/f
Description
Change dialog corner radius
Steps to test this PR
n/a

Current dependencies on/for this PR:

develop

PR #2217   👈

This comment was auto-generated by Graphite.


Is this plugin being developed?

Would be nice to hear about current state of its development

Would be nice to hear about current state of its development

Yes, any reason to ask?
I am running this in my home setup without hiccups.

I'm buying GoodWe inverter and preparing to use it on own automations. Wondering if this plugin would be good fit or should I write a new one.

Latest mod was months ago
README.md is random (looks to be from plugin template)
Plugin is not willing to become "homebridge verified"

Hence the questions.
Thank you for quick reply.

Looks @msegzda you are new in hombridge and 1 month of last uodate is a great score. I'm using it since last year and do not care about verification however I can't wait to see your verified plugin. I can even provide some nice examples ;)


Exception on signature popup (Sublime 3107)

On Sublime 3107, Windows 8 64-bit, plugin version 4a1f6362509:
When using Alt+, to activate the signature popup, the following exception is produced in the console:
Traceback (most recent call last):
  File "C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\typescript\libs\work_scheduler.py", line 154, in on_scheduled
    job()
  File "C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\typescript\libs\work_scheduler.py", line 125, in do_work
    worker(work_done)
  File "C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\typescript\libs\popup_manager.py", line 69, in get_signature_data
    self.proxy.async_signature_help(filename, point, '', on_done)
AttributeError: 'NoneType' object has no attribute 'async_signature_help'

Note that if I set the file log mode to DEBUG, this issue disappears.

That is weird. The log mode is not supposed to affect anything. Do you have a code snippet that can repro this exception?

OK, I have a guess as to what's happening.
Here is the console output before the above exception is produced:
Packages/TypeScript/popup.html
Path of tsserver.js: C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\tsserver\tsserver.js
Path of tsc.js: C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\tsserver\tsc.js
Trying to spawn node executable from: node
ref view not found

And here it is when I move usage of get_popup_manager() inside on_text_command to inside the if block:
Path of tsserver.js: C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\tsserver\tsserver.js
Path of tsc.js: C:\Users\johng\AppData\Roaming\Sublime Text 3\Packages\TypeScript\tsserver\tsc.js
Trying to spawn node executable from: node
ref view not found
Packages/TypeScript/popup.html

Notice the location of Packages/TypeScript/popup.html in the 2nd log output.
What I think is happening is the new Sublime version or one of my plugins is issuing an on_text_command event before the typescript server is connected. When this happens, get_popup_manger is called, which creates a PopupManager instance with None for proxy because the server proxy hasn't been initialized.
Lazily initializing the PopupManager in the tooltip/on_text_command function seems to fix this, but I'm not sure if it's the best approach. I can open a PR with that change if you like.

@jgoz hijacking the event listen could very likely cause the issue. A PR will be very welcomed!


filter fetched from a remote source crashes on reading config

there's currently a lifetime issue when reading config data if the filter is initialized form a remote location.
I fixed it in here:
https://github.com/yuval-k/envoy-wasm/commit/541b2c1155fffb15ccde92b8324f3e38f7339ba6
essentially the lifetime of the config variable is as the FilterConfig ctor, and need to be copied if the callback is not called inline - which is the case when the filter is loaded from a remote location.
I can open a PR, but wasn't sure if I should do it here, or wait for this filter to be unstreamed.

cc @jplevyak

fixerd thanx.


Implement user-friendly reporting

Closes #42

This is indeed a nice way of error handling.
I noticed only one minor thing: In the printUnexpectedException function you always use the : symbol after the titles.
However, this is not presented in the printBaseUnderdocsException function. I know it is just about styling, so changing this is up to you.

Thanks for noticing that, you are absolutely right!
The styling was a bit inconsistent, because I've had a couple of versions about the styling of the output. It was probably left in from an earlier version. Now it's fixed and consistent.


Dotnet command nto working

Problem encountered on https://dotnet.microsoft.com/en-us/learn/aspnet/blazor-tutorial/install
Operating System: macos
When I tried the installation process , a 'zsh: command not found: dotnet error' kept popping up

Which version is your Mac and version of the SDK have you installed @TGRyeet?

Mac version: ventura 13.4.1
SDK: dotnet-sdk-7.0.400-osx-x64.pkg
On Mon, Aug 21, 2023 at 11:09 AM Maira Wenzel @.***>
wrote:

Which version is your Mac and version of the SDK have you installed
@TGRyeet https://github.com/TGRyeet?
—
Reply to this email directly, view it on GitHub
https://github.com/dotnet/core/issues/8699#issuecomment-1685681170, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/A4XE2N6IPB3TX6KKZISU5WTXWLYC7ANCNFSM6AAAAAA3XIWYGM
.
You are receiving this because you were mentioned.Message ID:
@.***>


RFC: add Unicode math operators

Thoughts related to the general idea should go on #362 ; comments on implementation fit here :)

LGTM.


Ntfy authentication

I have small suggestion. Could you please add authentication option for ntfy. I have my own instance of ntfy and I've configure it to only get notifications from authenticated sources. It works but I have to add header definition to NotiMail.py like ntfy has in its docs https://docs.ntfy.sh/publish/#authentication.

I've just pushed the last commits with optional ntfy authentication (via token). You can add a Token configuration option in the config.ini (for example: Url1 will use Token1 if present) and it will be used to push notifications.
It'll be a part of the 0.10 release, for now it's in the master - please test and let me know if you encounter problems.


Relax clang-format version from 8.0.1 to just 8.0.x

Description
I don't think the format changes in clang-format patch versions, and the tools are typically named without the patch version, so is 8.0.1 too much
See PR #1201  that over-prescribes the version of clang-format.
Impact analysis
Expected behavior
Any 8.0.x version of clang-format should work.
Actual behavior
Explicitly require 8.0.1 for the moment.

I believe that i run on Linux into the issue, downloaded is 8.0.0, required is 8.0.1, so have to use --no-verify. Current Debian 10.1 system clang is clang-7.

Did #1321 fix this issue?


401 after token expiry

Hi there
I've noticed that I get the following stack trace:
on

org.springframework.web.client.HttpClientErrorException$Unauthorized: 401 Unauthorized: [{"error":"HTTP 401 Unauthorized"}]
	at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:105)
	at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:170)
	at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:112)
	at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
	at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:782)
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:740)
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674)
	at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:583)
	at org.camunda.bpm.extension.keycloak.KeycloakIdentityProviderSession.getKeycloakUserID(KeycloakIdentityProviderSession.java:424)

Restarting the engine fixes the issue. It seems to be once the token has expired. I've used the example sso-kubernetes.
Any help would be appreciated.

@aminmc
Currently I sadly can't reproduce this issue. The internal token handling of the Keycloak Identity Provider plugin checks the token epxiry and will refresh it automatically if required. If this fails it will request a new token.
Do you have
a) a list of changes - if any - you implemented
b) the complete stack trace - including the lines arround the above error
c) a description what exactly I need to do to reproduce this
Thanks,
Gunnar

Hi there. Sorry I forgot to update this. It turns out it was my configuration. I came across a similar issue on this project and reset my token settings. It’s working as expected.  Thank you.

@aminmc could you point out what your issue was? I'm having similar problems

Hi there I was playing around with the access token and sso timeout which caused my issue. I reverted back to the original configuration and that resolved it.

but how was the original config different?


Zomato App UI Clone

Can I develop this UI clone in Flutter?

you can start working

Can you please assign me this issue, also add the hactoberfest 2023 label. @tanmoy27112000


Bug in count_ssm_by_region

The example in the vignette no longer runs after the migration. The affected function besides the highlighted one is GAMBLR.helpers::handle_ssm_by_region. Self-assigning this issue. Posted for tracking.
Example:
#define a region.
my_region = gene_to_region(gene_symbol = "MYC", return_as = "region")

#subset metadata.
my_metadata = get_gambl_metadata() %>%
  dplyr::filter(pathology == "FL")

#count SSMs for the selected sample subset and defined region.
fl_ssm_counts_myc = count_ssm_by_region(region = my_region, these_samples_metadata = my_metadata)

Error in vctrs::vec_size_common(string = string, pattern = pattern, replacement = replacement,  :
  argument "this_maf" is missing, with no default

This issue is being resolved with the following two PRs (in different child repos):

https://github.com/morinlab/GAMBLR.utils/pull/14
https://github.com/morinlab/GAMBLR.helpers/pull/14


Add mnemonics for base12-base17

Specification for the brighter colors can be found here: https://github.com/tinted-theming/base24/blob/master/styling.md
Also included a small fix for the base24 aliases.

Great, thanks a lot!


Correct PP deprecation notices

The PP load/save pipeline has been substantially changed for v1.10, with a number of old bits of functionality being deprecated. Unfortunately the deprecation messaged for two of the deprecated functions (being add_save_rules and reset_save_rules) actually pointed the user to go and use another function (as_pairs) that had itself been deprecated by save_pairs_from_cubes.
This PR corrects the deprecation warning messages to point to the non-deprecated function rather than the deprecated function.

Aim for 1.11.1, or not bother because 1.12 was supposed to be out two days ago?

@QuLogic let's just stick it in master and see what releases it first...

Thanks @bjlittle!


Motivation for this change

A few terraform provider updates
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 106657 run on x86_64-linux 1

3 packages built:

terraform-full
terraform-providers.aws
terraform-providers.postgresql


Include .command.sh of each process in provenance tracking

.command.sh in each work folder contains the command executed for each module. These are only a few bytes and should be saved as part of provenance tracking, thus allowing for historic inspection of the pipeline commands even after the work directory is deleted.

Add .command.log to the provenance tracking as well.


Set Temperature on Oventrop R-Tronic RT B

Hello,
i have integrated my Oventrop R-Tronic RT B (Enocean) and it seems to work fine. In the next step i tried to send the setpoint temperature from iobroker to the device but it is not working. I have used the script as described in the devices.json file an started the teach in process again.
After that i have created a script that sets the value of the SP after the RSSI is updated (as it says in the description that it is possible to send the temperature within 1 second). The value is set correctly in iobroker but it is not transferred to the device.
Can you please give me a hint what i can/must do ?
Thanks in advance!

Hi,
did you setup your RT B as A5-10-06 + RPS? Check the device configuration at the RT B.

Yes i did.

I think it is possible that i did something wrong as i just started with iobroker because of the possibilty to pair the rt b so any newbie hint is welcome.

on({id: \"enocean.0.DeviceID.rssi\"}, () => { setState(\"enocean.0.DeviceID.CMD\", 1) })
Did you replace DeviceID with the real ID of the device?

Yes, also did that. my script looks like this
on({id: "enocean.0.059499FD.rssi"}, () => { setState("enocean.0.059499FD.CMD", 1) })

i had to delete the \ because i got errors before.
after an RSSI update i can see the change of the CMD State from 0 to 1 so the script seems to work.
How should the script for sending the setpoint look ?
Mine looks like this at the moment (i created it with blockly and for better reading here i copied the js code)
on({ id: 'enocean.0.059499fd.rssi' /* Signal Strength */, change: 'any' }, async (obj) => {
  let value = obj.state.val;
  let oldValue = obj.oldState.val;
  setState('enocean.0.059499fd.SP' /* Sollwert */, '105', true);
  setState('enocean.0.059499fd.CMD' /* Befehl */, '0', true);
});

in this script i tried 0 and also 1 at the end for the CMD state.

The script its self looks good, but the values are written as string which has to be a number.
I am not sure actual if the set point value is 0-255 or in °C. Test it with a value between 0 and 30°C

i tried both, setting a number instead of a string and use °C instead of 0-255 but it doesn`t work.
is it possible that it takes more than 1 second until my command is sent back to the rt b ? is there any possibility to check that ?

Should there be a sender id ? if yes, how can i achieve that ?

You can change the log level for the instance from info to debug. Then all telegrams will be printed to the log.
Go to config of the instance and change it where the version is shown near the play (pause) symbol is shown.
No Sender id is ok for this device.

I think it is finally working.
Thanks a lot for your hints. Just for documentation reasons if anyone comes along this issue:
In the Logs i saw, that no message was sent. After some trying i found out that i had to chenge the blockly command from "refresh state ..." to "control state ...". It now looks like this:

After that i saw in the Logs that the message was sent.
This lead me to the idea that i had to change it in the teach-in script too. So i changed that and started the teach-in on my rt b again. For the first time i got the message "Successfull" on my rt b. Before it was only "Telegram sent". I have also seen that in the Diagnostics of the Device there is now also the Gateway ID of my USB300 Stick listed.
After the next update message from the rt b the script was executed and the target temperature was set as i configured it in the script.
Again, thanks for your help!

Thanks for explaning your problem and the solution


kymotrack/kymotrackgroup: allow plotting the localization fit

Why this PR?
It is good for users to be able to verify their localization fits, to make sure nothing went wrong. This PR adds two methods to be able to plot them quickly.

Why is it important to plot this? Well, if a user chooses a window that's too small, they might inadvertently find themselves in the situation where the baseline eats up a large chunk of the amplitude, such as here:

A similar situation would happen if you take a window that is too large, but fail to track one of the peaks inside the window.
I intend to add something to the tutorial about this feature (and how to verify that your fits are good etc), but I will keep that for the next PR since there are two PRs with changes that impact the plots in that section.

Very nice feature! I tried it and it works well. I never gave the refinement much thought, it is really insightful to see how different choices affect the gaussian fit.

👍

I don't really like frame_idx as frame is used for a 2D scan Frame. Line has already been used for the tracks. How about time_idx?

Ahh, I see.
The reason I went with frame on the KymoTrackGroup is because it is actually analogous to a Scan frame. It is just a 1D scan frame. We also use the notion of frames for Kymos in a few other docstrings (see for example fit_binding_times).
For the KymoTrack, I wonder if time_idx could lead to confusion. A track may skip time points (e.g. time being 0, dt, 2dt, 3dt, 5dt for instance). In that case, time_idx=4 would be time point 5dt. This is actually also an issue for the frame_idx. So maybe on the KymoTrack it should be called node_idx to make explicit that it's just the track node we're talking about.
For KymoTrackGroup we could go with frame_idx or time_idx. If you feel strongly about time_idx I don't mind changing it for the group specifically.

It would indeed be important to add an explanation to the docs of what you are looking at exactly, what a good fit looks like, and what you can change in the refinement to improve the fit. And also explain how a bad fit affects the downstream analysis.

Yeah. That's a good point. I think we should probably have dedicated section for localization methods (the kymotracking chapter is too big already as it is). I think it would be good to do that in a separate PR, since it would be nice to expand it a little with some simulations as well. I can make a ticket for it.

Very nice feature! I tried it and it works well. I never gave the refinement much thought, it is really insightful to see how different choices affect the gaussian fit.
I don't really like frame_idx as frame is used for a 2D scan Frame. Line has already been used for the tracks. How about time_idx?
It would indeed be important to add an explanation to the docs of what you are looking at exactly, what a good fit looks like, and what you can change in the refinement to improve the fit. And also explain how a bad fit affects the downstream analysis.

I don't really mind frame_idx since I think of the scan lines as analogous to frames of a Scan but I see your point. Also got me thinking more about the different behaviors between KymoTrack and KymoTrackGroup.
For the track the index is referring to the specific node, so if you scroll through you don't see plots for frames that aren't localized. Whereas for KymoTrackGroup you will see the data for frames even if there are no tracks.  So semantically it might be best to use something like node_idx for KymoTrack and any(frame_idx, time_idx, scan_line_idx) for KymoTrackGroup


macOS build crashes if product_string is not nwjs

Issue Type

[X] Bug Fix
[ ] Feature
[ ] Other

Current/Missing Behaviour
When attempting to use "osx" as the platform and building in macOS Ventura, the build fails. As shown below, no errors are reported but the resulting directory and ZIP file are missing application content.
Building package for osx...
 ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 100% | ETA: 0s | 123635907/123635907
[ DEBUG ] 2022-11-22T03:05:46.247Z Manifest file does not exist locally
[ DEBUG ] 2022-11-22T03:05:46.248Z Downloading latest manifest file under /Users/zombie/Downloads/nw-react-nw-build/cache
[ DEBUG ] 2022-11-22T03:05:47.092Z Succesfully cached manifest metadata
[ DEBUG ] 2022-11-22T03:05:47.095Z Store manifest metadata in memory
[ DEBUG ] 2022-11-22T03:05:47.096Z Convert manifest data into JSON
[ DEBUG ] 2022-11-22T03:05:47.097Z Search for 0.70.1 specific release data
[ DEBUG ] 2022-11-22T03:05:47.098Z Remove any files at /Users/zombie/Downloads/nw-react-nw-build/dist/nw-react-example-0.1.1-osx directory
[ DEBUG ] 2022-11-22T03:05:47.098Z Copy /Users/zombie/Downloads/nw-react-nw-build/cache/nwjs-v0.70.1-osx-x64 files to /Users/zombie/Downloads/nw-react-nw-build/dist/nw-react-example-0.1.1-osx directory
[ DEBUG ] 2022-11-22T03:05:47.432Z Copy /Users/zombie/Downloads/nw-react-nw-build/dist/app files to /Users/zombie/Downloads/nw-react-nw-build/dist/nw-react-example-0.1.1-osx directory
[ DEBUG ] 2022-11-22T03:05:47.436Z Get NW's package.json as a buffer
[ DEBUG ] 2022-11-22T03:05:47.436Z Convert package.json buffer into JSON
[ DEBUG ] 2022-11-22T03:05:47.436Z Starting platform specific config steps for osx
Finished building package for osx

Expected/Proposed Behaviour
Additional Info

Operating System: macOS Ventura 13.0.1
Node version: 18.12.1
NW.js version: 0.70.1
Repro link:

Part 1 of this issue is because the directory name is incorrectly being set to nw.app instead of app.nw: https://github.com/nwutils/nw-builder/search?q=nw.app
This prevents the actual app content from being included in the final package.
There is a secondary issue, where the various metadata modification steps are breaking the nwjs.app bundle. Even when manually creating the missing app.nw directory, the app crashes when launched. If everything in bld/osxCfg.js is commented out except for the following line, the app launches without crashing:
await fs.rename(`${outDir}/nwjs.app`, `${outDir}/${pkg.name}.app`);

Okay, the app only fails to launch if the helper apps are renamed (or product_string is added to package.json). So, I've pushed a PR that fixes everything else. We'll need to follow-up on the mailing list to determine why the helper apps cannot be renamed.

I did some digging with respect to renaming the helpers. Chrome hardcoded the helper app paths which we can see in nw22.  As of nw71, a product_string property is used to rename the helper apps. Since I don't have a Mac or a MacOS compiled version of NW.js, my guess is that the linked code does not apply to current releases with the current app structure:

As of nw71, a product_string property is used to rename the helper apps. Since I don't have a Mac or a MacOS compiled version of NW.js, my guess is that the linked code does not apply to current releases with the current app structure leading to crashes. The zombie process issue could be related to this.

@rogerwang any insight on renaming helper apps? I asked on the mailing list but haven't seen any responses.

Tracking in upstream https://github.com/nwjs/nw.js/issues/7253

Yikes, this ticket should not have been closed! The issue is not fixed... In the latest MacOS builds (0.79.1) I'm still seeing the app crash if product_string is set to anything other than nwjs.

It’s a bug with NW.js, not nw-builder. The issue for tracking is linked above.


Improper team set

I currently don't have access to this repository.
@ndaniels

I added the clam team with admin rights,


Keyboard shortcuts are not preserved between sessions

I love codespaces, but am finding the keyboard shortcuts are not preserved between sessions. The setting file's path is reported as /User/keybindings.json, but this doesn't exist in the container. For now I'm saving a copy in .vscode and then copying it over when the bindings disappear. Note that I'm using an in-private window for codespaces, as otherwise it gets very confused between my personal and organization microsoft accounts (separate issue).

@sumitsumit is it possible that you are using different browsers for different sessions, e.g. home and work?

Oh, this explains it:

Note that I'm using an in-private window for codespaces

I think settings are stored in IndexDB or localStorage.

I would appreciate if you could report this issue to https://github.com/MicrosoftDocs/vsonline/issues

will do!


Package no longer solves the problems portrayed in the readme.

I realize i'm years late to this but the readme is incorrect now that the package has been changed to be less safe with the changes to how it stores things and more so the support for initialization via literals.
Aside from the fact that "A compile-time guarantee that a collection contains a value." is no longer true, the readme also presents multiple issues in code that illustrate the usefulness of this package, but a few of them are no longer valid.
The example in the section "Better interface with 3rd party APIs" is no longer correct. It states that we can prevent the problematic code from ever happening in the first place by using non empty sets instead but this is no longer true. Now the problematic code is not only still possible but also causes a crash.

The example in the section "More expressive data structures" is also no longer correct. It shows that trying to initialize with an empty array gives a compiler error and while it does, it is only because the initializer that takes a collection is fail-able and returns an optional, not because initializing with an empty array generates a compiler error. If instead it was .invalid([]) or .invalid(.init()) it would compile just fine and then crash at runtime.

Again I know this is years late to the party but I was really excited to have a compile-time guarantee of non emptiness after watching the original video in the pointfree series about NonEmpty, I included it in my project and started using it, only to later discover that it is no longer safe.
I feel the risk is too high that eventually even someone who is intimately familiar with this package will be dealing with an optional NonEmpty collection and accidentally use ?? [] without thinking about it and cause a crash in production in a scenario that isn't usually tested.
I understand that using [] with NonEmpty is being considered a programmer error, but force unwrapping .first! on a regular array when that array is empty is also a programmer error, and I thought the purpose of this package was to make that impossible in the first place.
Also completely understand if I don't like it I don't have to use it or I can modify it or whatever but I still wanted to voice that I feel the move away from the safety seems counter to what I thought the original goal of the package was, and point out that the readme is misleading in the current state.

Hi @Helam24, the readme should definitely be updated, but we do believe that the package solves the problems it originally set out to.
First, some history. The library started exactly in the way that was built in episodes, but after about 2 years of using it we realized there are some problems with it. It just wasn't ergonomic or performant at all. So, we ultimately decided to refactor it to be "technically" compile-time safe, though the internals are a bit unsafe: https://github.com/pointfreeco/swift-nonempty/pull/26. Because we constrained how NonEmpty can be used with its public API, it was just as safe as before the PR.
This is a common trick to employ in the hopes of making something more ergonomic and performant. You often need to resort to unsafe APIs under the hood and present a safe API externally. And we're taking the compromise that if we try our hardest to make the external API safe, then we are ok with calling the library "safe". Otherwise any library using a single pointer under the hood cannot be thought of as "safe".
Further, the screenshots you pointed out are definitely very old, and should be updated. But we did make another concession to the ergonomics of NonEmpty by allowing it to conform to ExpressibleByArrayLiteral. This means you can technically do non-sensical things like this:
var xs: NonEmptyArray<Int> = []

Our view was that the ergonomic win of being able to use array syntax is greater than the safety concern, because array literal syntax is something very local, and hopefully it would be clear that is not valid to use an empty array.
However, the ExpressibleByArrayLiteral conformance is the only safety escape hatch in the library. So, before adding that the library was just as safe to use as it was when we modeled things as (head: A, tail: [A]).
We could re-consider the ExpressibleByArrayLiteral conformance. Maybe it wasn't the right choice after all. But, it's a very difficult problem to handle correctly. Currently it is our opinion that basically no one is going to want to use a NonEmpty type without that conformance. It's just too much of a pain. But we may be wrong!
I am curious: have you used a NonEmpty type in many places across a code base before? Have you not been annoyed by some of its ergonomics quirks?

@mbrandonw thank you for your response. I acknowledge the difficulty of the task you're trying to solve, and my lack of experience with it.
I have not used it at scale before so I have not yet had the chance to run into the quirks you're talking about, I was just starting to add it across a project before I realized that initialization via literals was possible and could cause a crash and am now trying to decide if I want to redo the work and add back in all the empty checks or if I want to make a separate version of NonEmpty that does not support initialization via literals.
I agree that this is largely non-sensical and relatively easy to avoid:
var foo: NonEmptyArray<Int> = []

The scenarios that scare me involve nil coalescing to an empty array, which is something I believe to be fairly common to do for regular arrays. Imagine the following scenarios where .ingredients is a non empty array.
let ingredientNames = product?.ingredients.map(\.name) ?? []

or
checkAvailability(for: product?.ingredients ?? [])

The above scenarios feel less non-sensical to me. The words NonEmpty do not show up on the same line and may not even be in the same file. It looks and acts largely like a normal array, it seems reasonable that someone might not realize or remember what they're dealing with and mistakenly use ?? [] and introduce a crash waiting to happen. This would be difficult to catch in code reviews as well when viewing a diff that has no indicators that a non empty array is being used.
A compiler error in this case seems extremely desirable.

Hi @Helam24, after discussing with @stephencelis we are more of the mind to sunset and archive this project rather than keeping it going in its current form. It's just a very difficult problem to solve, and we're not sure Swift is the right language for this tool.
Alternatively, I could see us releasing a 1.0 of the library that drops the literal conformance, and that would probably be the last release we ever do for the library.

@mbrandonw well shoot, it certainly wasn't my intention to spur something like that. I had watched the original video and felt all motivated to try to use it all over the place. I imagined that you must use it in your projects regularly, is that not the case? If you have a bunch of experience with it and have found it not to be worth the hassle maybe I should "take your word for it" and stick to handling empty the good old fashioned way.
Regardless of what direction you go with this, I want to say thank you for your videos. I've learned a lot from them and feel I have improved as a developer because of them.


Can you add alternative of winsound for linux users?

Hey, I am trying to use this script for Linux but due to unavailability of winsound for Linux distro, I am unable to run the script.
Here are some of the solution you can try,

https://stackoverflow.com/questions/12354586/python-what-are-the-nearest-linux-and-osx-equivalents-of-winsound-beep
https://stackoverflow.com/questions/13150267/convert-winsound-to-a-linux-platform

Sure, will try

Almost every solution I see brings in a new third-party package dependency - which I really don't want to do. And so for now, I have tabled this. Also, I myself have to first setup a Linux VM to test it out.
Meanwhile, please feel free to fork this and edit it. If you do edit it in a cross-platform way, I would be glad to merge that into this.

I have incorporated cross platform support. Could you please try running the python file now?
Please note, you will need to install the beep package on linux to run the beep command. You can install by giving the command
sudo apt-get install beep

@pallupz Thanks for the efforts, Can you please tell me source of beep package?
I found this (https://github.com/johnath/beep) but I am not sure.
I am asking this because I am an arch Linux user and apt install doesn't work for me.
Thanks again for the efforts :)

Hi - I have no idea. I am primarily a Windows user and just modified the code in a way that should be 'runnable' on Linux. Sorry I couldn't be of more help.

No worries, Thanks to your script, my first dose is now booked :)


fix: Add ChakraMenu to provide context for MenuIcon dependency

This is required because of the issue in the latest Chakra UI release where useMenuStyles will break the app.
See example from Storybook

Thanks for pushing this, and bringing it to my attention! There were a couple issues with your original push though. The comment you included wasn't in a valid spot. Also, you were including a Menu for every option, where as there's no need to include more than one. I ended up moving it to the react-select Menu component to reduce duplication.


[ISSUE] Zeebe Gateway SERVER_PORT is using the incorrect port (affects readiness probes)

Describe the issue:
During the switch to application-style configurations for each component, we accidentally changed the SERVER_PORT (or server.port)  option to the Zeebe gateway REST API port rather than the normal port (9600). This does not normally matter, since both the zeebe gateway rest api  port and the 9600 port should both have actuator endpoints, however, there are some install-flows that are affected.  Those who use version 8.5 (helm 10.0.2) along with v8.4.6 of zeebe will experience readiness probe failures due to the REST API port not being available on 8.4.6 of zeebe.
The application teams have informed us that they do not want us referring to the REST API endpoint for readiness probes, so it should be changed back.

Actual behavior:
Readiness probes fail for zeebe-gateway pod on v10.0.2  with 8.4.6 of zeebe gateway.

Expected behavior:
Readiness probes should succeed for  zeebe-gateway pod.

How to reproduce:

Logs:

Environment:
Please note: Without the following info, it's hard to resolve the issue and probably it will be closed.

Platform:
Helm CLI version:
Chart version:
Values file:

I'm closing this issue since that case is not supported (Camunda 8.4 cannot be used with chart v10.x.x since it supports Camunda 8.5 only).
helm search repo camunda/camunda-platform --versions | head -n 5
NAME                    	CHART VERSION	APP VERSION	DESCRIPTION
camunda/camunda-platform	10.0.2       	8.5.x      	Camunda 8 Self-Managed Helm charts. Camunda's p...
camunda/camunda-platform	9.3.3        	8.4.x      	Camunda 8 Self-Managed Helm charts. Camunda's p...
camunda/camunda-platform	9.3.2        	8.4.x      	Camunda 8 Self-Managed Helm charts. Camunda's p...
camunda/camunda-platform	9.3.1        	9.3.x      	Camunda 8 Self-Managed Helm charts. Camunda's p...


Implement MPS plugin serving models using the high-level v2 API

Implemented in https://github.com/modelix/modelix/tree/mps/2020.3/mps/org.modelix.model.server.mpsplugin


Fail fast if JCK_GIT_REPO and JDK_VERSION has a mismatch

While there is no guarantee that everyone using our scripts use the recommended name, JCKxx-unzipped for JCK_GIT_REPO, but if they do, we can add a check for whether the version in use matches the JDK_VERSION under test.  This was identified as a common user error for some people when running Grinders (who are not launching Grinders from the quick links provided in the original test job) during the retrospective.
If there is a mismatch, we should fail immediately instead of trying to run tests.

I think we should use build.txt in the JCK repo (i.e., under runtime) to verify JCK version/build, the expected JDK_VERSION, and the actual JDK_VERSION. For example:
java full version openjdk "21-ea+32"
JDK_VERSION=21
JDK_BUILD_ID=32

The build should fail if the expected JDK_VERSION > and the actual JDK_VERSION do not match. And maybe a warning message if the JDK_VERSION is the same, but expected JDK_BUILD_ID > actual Java build. For example, JDK_VERSION=21, but the expected JDK_BUILD_ID=32, actual Java build 30. It means an older JCL level is used.

Note: Needs to use the environment variable for TCK location to be user-agnostic


add theming

remove hardcoded inline styles on components and replace with themed, styled components.
React-native-paper supports this out of the box

this is hard.


Pykit ideal syntax

from itoolkit.transport.http import connect
from itoolkit import ProgramCall, Command, Parameter, DataStructure, Char, Decimal

connection = connect()
toolkit = connection.toolkit()

pc = ProgramCall(program='x', library='y', function='z');
user = itoolkit.Char(10, 'value')
income = itoolkit.Decimal(10, 3, 456.787)
description = itoolkit.VarChar(100)

pc.addParameter(user, direction='both')
pc.addParameter(income, direction='both')
pc.addParameter(description, direction='out')

class MyDS(itoolkit.DataStructure):
   field1 = itoolkit.Char(5)
   field2 = itoolkit.Int()

ds = MyDS()
pc.addParameter(ds, direction='both')

toolkit.add(pc)
toolkit.execute()

cm = Command('CALL CLPGM PARM(\'anything\')')
toolkit.add(cm)
toolkit.execute()

FYI: If you import from you can't prefix it:
either:
from itoolkit import Char
user = Char(10, 'value')

or
import itoolkit
user = itoolkit.Char(10, 'value')

but this won't work:
from itoolkit import Char
user = itoolkit.Char(10, 'value')

@jordiwes is there a particular reason for getting the toolkit object from the connection:
connection = connect()
toolkit = connection.toolkit()

Since the transport is REST, I don't see Connect having any other purpose but to test that the DB2Sock REST transport exists and the proper credentials have been set.

It's true, connect won't make sense for REST, but will make sense for future transports.

I think this has been decided enough to be closed.


update write pages to reflect new partial batch success writes

Originally: https://github.com/influxdb/influxdb.com/issues/436
From https://github.com/influxdb/influxdb/pull/4587:
"HTTP line protocol handler will now write all successfully parsed points instead of failing the entire batch if one failed. When some points fail to parse and the remaining are written successfully, the endpoint returns a partial write error with the failed lines in the response. This makes the behavior more consistent with the other inputs."

hasn't been a source of confusion since the change, closing.


visualize_collider doesn't visualize

Not sure when this started happening.
Repro in newest version of examples/basics/physics.

Right, tracking it down.
The gizmo renderer is invoked, though there are no gizmos primitives to draw
4c17805 works


Add browser warning message

If user is using IE10 or below, we should display a message telling them to upgrade their browser.

https://github.com/lancedikson/bowser

Added browser notification if user is using IE10 or below. It shows up on IE10 and IE9. Here's how it looks on IE10:


SSD Object Detection parsing is broken. parseSSDMobileNet returns erroneous and/or invalid results

I successfully load up my model and start running model on frames from the camera controller.  I notice results come in when my object appears in camera field of view (water test strips).  However, the parsed parameters from the output tensors are all inaccurate which leads to incorrect bounding and class detection.
For example, in parseSSDMobileNet here are some of the outputs for locations, classes, scores, and num detection float arrays.  Why are classes and num detections float arrays first off?  The results of these outputs:
  float* output_locations = TfLiteInterpreterGetOutputTensor(interpreter, 0)->data.f;
  float* output_classes = TfLiteInterpreterGetOutputTensor(interpreter, 1)->data.f;
  float* output_scores = TfLiteInterpreterGetOutputTensor(interpreter, 2)->data.f;
  float* num_detections = TfLiteInterpreterGetOutputTensor(interpreter, 3)->data.f;

Results:
when I log first 8 entries for float pointers:
  for (int d = 0; d < 8; d++) {
      NSLog(@"output_classes loc %d %f", d, output_classes[d]);
  }

I get the following:
output_classes loc 0 0.590460
output_classes loc 1 0.468586
output_classes loc 2 0.628208
output_classes loc 3 0.514471
output_classes loc 4 0.355844
output_classes loc 5 0.468039
output_classes loc 6 0.392715
output_classes loc 7 0.510365
output_locations loc 0 0.707110
output_locations loc 1 0.689438
output_locations loc 2 0.580224
output_locations loc 3 0.514742
output_locations loc 4 0.494435
output_locations loc 5 0.494395
output_locations loc 6 0.344939
output_locations loc 7 0.124733
num_detections loc 0 5.000000
num_detections loc 1 2.000000
num_detections loc 2 3.000000
num_detections loc 3 1.000000
num_detections loc 4 4.000000
num_detections loc 5 0.000000
num_detections loc 6 0.000000
num_detections loc 7 1.000000
output_scores loc 0 10.000000
output_scores loc 1 -0.015256
output_scores loc 2 0.140742
output_scores loc 3 0.009112
output_scores loc 4 0.334685
output_scores loc 5 0.185375
output_scores loc 6 0.033109
output_scores loc 7 -0.257905
So these are obviously incorrect.  For instance, why the negative scores?  Why is the first score regularly either 10 or 0? Why would the locations yield negative widths and heights.  num_detections looks interesting, but I can't piece out what it may represent.
My model works perfectly in python, and following the jupyter notebooks in the following tutorials:
https://github.com/nicknochnack/TFODCourse
Though when I bring in the transfer model after exporting to tensor flow lite, I notice the model yields these kinds of results.
here's my training config:
model {
  ssd {
    num_classes: 8
    image_resizer {
      fixed_shape_resizer {
        height: 320
        width: 320
      }
    }
    feature_extractor {
      type: "ssd_mobilenet_v2_fpn_keras"
      depth_multiplier: 1.0
      min_depth: 16
      conv_hyperparams {
        regularizer {
          l2_regularizer {
            weight: 4e-05
          }
        }
        initializer {
          random_normal_initializer {
            mean: 0.0
            stddev: 0.01
          }
        }
        activation: RELU_6
        batch_norm {
          decay: 0.997
          scale: true
          epsilon: 0.001
        }
      }
      use_depthwise: true
      override_base_feature_extractor_hyperparams: true
      fpn {
        min_level: 3
        max_level: 7
        additional_layer_depth: 128
      }
    }
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    matcher {
      argmax_matcher {
        matched_threshold: 0.5
        unmatched_threshold: 0.5
        ignore_thresholds: false
        negatives_lower_than_unmatched: true
        force_match_for_each_row: true
        use_matmul_gather: true
      }
    }
    similarity_calculator {
      iou_similarity {
      }
    }
    box_predictor {
      weight_shared_convolutional_box_predictor {
        conv_hyperparams {
          regularizer {
            l2_regularizer {
              weight: 4e-05
            }
          }
          initializer {
            random_normal_initializer {
              mean: 0.0
              stddev: 0.01
            }
          }
          activation: RELU_6
          batch_norm {
            decay: 0.997
            scale: true
            epsilon: 0.001
          }
        }
        depth: 128
        num_layers_before_predictor: 4
        kernel_size: 3
        class_prediction_bias_init: -4.6
        share_prediction_tower: true
        use_depthwise: true
      }
    }
    anchor_generator {
      multiscale_anchor_generator {
        min_level: 3
        max_level: 7
        anchor_scale: 4.0
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        scales_per_octave: 2
      }
    }
    post_processing {
      batch_non_max_suppression {
        score_threshold: 1e-08
        iou_threshold: 0.6
        max_detections_per_class: 100
        max_total_detections: 100
        use_static_shapes: false
      }
      score_converter: SIGMOID
    }
    normalize_loss_by_num_matches: true
    loss {
      localization_loss {
        weighted_smooth_l1 {
        }
      }
      classification_loss {
        weighted_sigmoid_focal {
          gamma: 2.0
          alpha: 0.25
        }
      }
      classification_weight: 1.0
      localization_weight: 1.0
    }
    encode_background_as_zeros: true
    normalize_loc_loss_by_codesize: true
    inplace_batchnorm_update: true
    freeze_batchnorm: false
  }
}
train_config {
  batch_size: 4
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
  data_augmentation_options {
    random_crop_image {
      min_object_covered: 0.0
      min_aspect_ratio: 0.75
      max_aspect_ratio: 3.0
      min_area: 0.75
      max_area: 1.0
      overlap_thresh: 0.0
    }
  }
  sync_replicas: true
  optimizer {
    momentum_optimizer {
      learning_rate {
        cosine_decay_learning_rate {
          learning_rate_base: 0.08
          total_steps: 50000
          warmup_learning_rate: 0.026666
          warmup_steps: 1000
        }
      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  fine_tune_checkpoint: "Tensorflow/workspace/pre-trained-models/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8/checkpoint/ckpt-0"
  num_steps: 50000
  startup_delay_steps: 0.0
  replicas_to_aggregate: 8
  max_number_of_boxes: 100
  unpad_groundtruth_tensors: false
  fine_tune_checkpoint_type: "detection"
  fine_tune_checkpoint_version: V2
}
train_input_reader {
  label_map_path: "Tensorflow/workspace/annotations/label_map.pbtxt"
  tf_record_input_reader {
    input_path: "Tensorflow/workspace/annotations/train.record"
  }
}
eval_config {
  metrics_set: "coco_detection_metrics"
  use_moving_averages: false
}
eval_input_reader {
  label_map_path: "Tensorflow/workspace/annotations/label_map.pbtxt"
  shuffle: false
  num_epochs: 1
  tf_record_input_reader {
    input_path: "Tensorflow/workspace/annotations/test.record"
  }
}

my label file:
ch
tc
fc
ph
ta
cya
ccv1
ccv2
Is it possible that things are not being parsed to the TensorFlowLiteC library specs?

Wow, well in writing this, I got a better at the numbers, and switched out the tensor parsing as follows:
  float* output_locations = TfLiteInterpreterGetOutputTensor(interpreter, 1)->data.f;
  float* output_classes = TfLiteInterpreterGetOutputTensor(interpreter, 3)->data.f;
  float* output_scores = TfLiteInterpreterGetOutputTensor(interpreter, 0)->data.f;
  float* num_detections = TfLiteInterpreterGetOutputTensor(interpreter, 2)->data.f;

as apposed to:
  float* output_locations = TfLiteInterpreterGetOutputTensor(interpreter, 0)->data.f;
  float* output_classes = TfLiteInterpreterGetOutputTensor(interpreter, 1)->data.f;
  float* output_scores = TfLiteInterpreterGetOutputTensor(interpreter, 2)->data.f;
  float* num_detections = TfLiteInterpreterGetOutputTensor(interpreter, 3)->data.f;

I am getting great results now!!  I need to figure out the scaling now to present the bounding boxes...the scale seems to be off by a factor of 2 or so.  I'll let you know what works.
I'm super glad I noticed this as I was getting a bit hopeless.  I also don't think the number of detections is still parsed correctly and will let you know if I find the correct parsing for the num_detections.

This worked on iOS, but is not working on android.  I tried to make the same changes, but it's getting hung up on the error others have complained about:
Unhandled Exception: PlatformException(Failed to run model, Cannot copy from a TensorFlowLite tensor (StatefulPartitionedCall:1) with shape [1, 10] to a Java object with shape [1, 1]., java.lang.IllegalArgumentException: Cannot copy from a TensorFlowLite tensor (StatefulPartitionedCall:1) with shape [1, 10] to a Java object with shape [1, 1].

I think it's complaining about the numDetections map, and if I could figure out where to get the number of detections from one of the output tensors, things may work.

Well, I got android working through various hacks.
I take a photo on a periodic timer for android now, rather than streaming the image, unlike iOS...Something is wrong with flutter's YUV 420 conversion
So, for android I do:
        XFile tempFile = await photoController!.takePicture();
        Uint8List imageBytes = File(tempFile.path).readAsBytesSync();
        List<int> values = imageBytes.buffer.asUint8List();
        imglib.Image? photo = imglib.decodeImage(values);

        ProcessedImage p = ProcessedImage();

        var recs = await Tflite.detectObjectOnImage(
          path: tempFile.path,
          threshold: 0.25,
          model: "SSDMobileNet",
          numResultsPerClass: 1,
          asynch: false,
        );
        p.recognitions = recs?.map((e) => Recognition.fromData(e)).toList();

Then I modified TFlitePlugin.java to match what I did to the iOS TFlitePlugin.mm:
RunSSDMobileNet(HashMap args, ByteBuffer imgData, int numResultsPerClass, float threshold, Result result) {
      super(args, result);
      this.num = 10; //tfLite.getOutputTensor(2).shape()[0][0];
      this.numResultsPerClass = numResultsPerClass;
      this.threshold = threshold;
      this.outputLocations = new float[1][num][4];
      this.outputClasses = new float[1][num];
      this.outputScores = new float[1][num];
      this.inputArray = new Object[]{imgData};
      Log.v("T0", " " + tfLite.getOutputTensor(0).numBytes());
      Log.v("T1", " " + tfLite.getOutputTensor(1).numBytes());
      Log.v("T2", " " + tfLite.getOutputTensor(2).numBytes());
      Log.v("T3", " " + tfLite.getOutputTensor(3).numBytes());

      outputMap.put(1, outputLocations);
      outputMap.put(3, outputClasses);
      outputMap.put(0, outputScores);
      outputMap.put(2, numDetections);

      startTime = SystemClock.uptimeMillis();
    }

For whatever reason, the number that it expects is 10 for the size of the tensor conversion, so I just hardcode that, which is ugly and I hate it, but it works
I have to rotate the image after the bitmap is read to look like.  I tried to do this in flutter and send to the detectObjectOnBinary, but detectObjectOnBinary does not seem to accept flutter's byte fetching:
ByteBuffer feedInputTensorImage(String path, float mean, float std) throws IOException {
    InputStream inputStream = new FileInputStream(path.replace("file://", ""));
    Bitmap bitmapRaw = BitmapFactory.decodeStream(inputStream);
    Matrix matrix = new Matrix();
    matrix.postRotate(90);
    bitmapRaw = Bitmap.createBitmap(bitmapRaw, 0, 0, bitmapRaw.getWidth(), bitmapRaw.getHeight(), matrix, true);
    return feedInputTensor(bitmapRaw, mean, std);
  }

Now object detection works for my model on android as it does in iOS.  I will have to look into other libraries if I find this not working across all devices as it seems like this lib is getting a bit rusty.

Derp, wish I'd checked this comment before spending a few days solving the same problem. Here's what worked for me:
https://github.com/shaqian/flutter_tflite/pull/235/files

Derp, wish I'd checked this comment before spending a few days solving the same problem. Here's what worked for me: https://github.com/shaqian/flutter_tflite/pull/235/files

I have tried this fix, but I get:
error: cannot find symbol
        String thisTensorName = tfLiteObjectRecognition.getOutputTensor(outputMapLocationIterator).name();
                                ^
  symbol:   variable tfLiteObjectRecognition
  location: class TflitePlugin.RunSSDMobileNet

Did I miss something?


Change wording for EU citizens

What
Change the wording of the employing EU citizens in the business finder smart answer.
Why
Research found that people were confused by the terminology of these questions.
Background
https://trello.com/c/uNWqplBr/165-change-wording-of-question-about-employing-eu-citizens
Screenshots
Before

After

Demo
http://finder-frontend-pr-1228.herokuapp.com/prepare-business-uk-leaving-eu?business_activity-yesno=yes&sector_business_area[]=electronics-parts-machinery&page=3
https://trello.com/c/WDUH4kll/186-change-wording-for-eu-citizens

@leenagupte all issues have been addressed. Please let me know if you want further changes


Splitting compaction changes stage 4 - Remove concept of splitting compaction jobs

Background
Split from #966
Description
In stage 3 (#1367), we changed splitting compactions so they no longer read or write files, they just create/delete references to existing files.
We can now remove the concept of a splitting compaction job and perform this splitting step before compaction jobs are created. We can also remove a lot of the splitting compaction infrastructure in the process.
Analysis
Ordered sub-issues have been created here:

#1669
#1672
#1673
#1670

Closing as all split issues have been completed.


Database

There's no database file in this project
dbConnection.php on line 6
mysqli_error

Hi, did you get the database file from the author?


Galactic Navigation Parameters

With the default parameters on galactic robot is not navigate on the map. However if you change the parameters with the foxy (controller server parameters) it can navigate.
// foxy parameters
controller_server:
  ros__parameters:
    # controller server parameters (see Controller Server for more info)
    use_sim_time: True
    controller_frequency: 50.0
    min_x_velocity_threshold: 0.001
    min_y_velocity_threshold: 0.5
    min_theta_velocity_threshold: 0.001
    goal_checker_plugin: "goal_checker"
    controller_plugins: ["FollowPath"]
    goal_checker:
      plugin: "nav2_controller::SimpleGoalChecker"
      xy_goal_tolerance: 0.25
      yaw_goal_tolerance: 0.25
      stateful: True
    # RegulatedPurePursuitController Parameters
    FollowPath:
      plugin: "nav2_regulated_pure_pursuit_controller::RegulatedPurePursuitController"
      desired_linear_vel: 1.2
      max_linear_accel: 0.8
      max_linear_decel: 0.8
      lookahead_dist: 0.6
      min_lookahead_dist: 0.3 #0.3
      max_lookahead_dist: 0.9 #0.9
      lookahead_time: 1.5 #1.5
      rotate_to_heading_angular_vel: 0.9
      transform_tolerance: 0.1
      use_velocity_scaled_lookahead_dist: false
      min_approach_linear_velocity: 0.01
      use_approach_linear_velocity_scaling: true
      max_allowed_time_to_collision: 1.0
      use_regulated_linear_velocity_scaling: true
      use_cost_regulated_linear_velocity_scaling: false
      regulated_linear_scaling_min_radius:  0.9
      regulated_linear_scaling_min_speed: 0.25
      use_rotate_to_heading: true
      rotate_to_heading_min_angle: 0.785
      max_angular_accel: 10.0
      allow_reversing: false

//galactic parameters
controller_server:   ros__parameters:     # controller server parameters (see Controller Server for more info)     use_sim_time: True     controller_frequency: 50.0     min_x_velocity_threshold: 0.001     min_y_velocity_threshold: 0.5     min_theta_velocity_threshold: 0.001     goal_checker_plugin: "goal_checker"     controller_plugins: ["FollowPath"]     goal_checker:       plugin: "nav2_controller::SimpleGoalChecker"       xy_goal_tolerance: 0.25       yaw_goal_tolerance: 0.25       stateful: True     # RegulatedPurePursuitController Parameters     FollowPath:       plugin: "nav2_regulated_pure_pursuit_controller::RegulatedPurePursuitController"       desired_linear_vel: 0.5       max_linear_accel: 0.2       max_linear_decel: 0.2       lookahead_dist: 0.6       min_lookahead_dist: 0.3 #0.3       max_lookahead_dist: 0.9 #0.9       lookahead_time: 1.5 #1.5       rotate_to_heading_angular_vel: 0.6       transform_tolerance: 0.1       use_velocity_scaled_lookahead_dist: false       min_approach_linear_velocity: 0.01       use_approach_linear_velocity_scaling: true       max_allowed_time_to_collision: 1.0       use_regulated_linear_velocity_scaling: true       use_cost_regulated_linear_velocity_scaling: false       regulated_linear_scaling_min_radius:  0.9       regulated_linear_scaling_min_speed: 0.25       use_rotate_to_heading: true       rotate_to_heading_min_angle: 0.785       max_angular_accel: 0.6       allow_reversing: false

Thanks for the report. I’ll check it and get back to you tomorrow

could you please retry with the new commits and let me know if it works for you ?

Could you please post the complete los?

oops sorry for closing it! should have been a mistake


Embed local file/dir values as objects when calling remote node

Today when I start an op on a remote node and I want to pass it a local file or directory as an arg, I have to write code to embed the files/dirs as objects. I would like whatever mechanism is needed to pass local files/dirs as args to an op to be performed automatically so I don't have to do it myself and so that as future enhancements like content defined chunking (CDC) and dedupe  are exposed via our data API, they will automatically be leveraged.

I'd like to see a little more context as to why we're doing this in the PR description, for future historical reasons.

I'd like to see a little more context as to why we're doing this in the PR description, for future historical reasons.

Done! LMK if that doesn't cover it


[native_toolchain_c] Bump deps

native_assets_builder's tests use native_toolchain_c, so we need to roll this first.

https://github.com/dart-lang/native/issues/142

coverage: 98.178%. remained the same when pulling eef3b828011bc52afd9d01b9f874ab7ffc8e0f7b on roll-cli into ec0a7c9edf2164047d0f86ab1e27eb017b4c3947 on main.


fix: also split on delimited double quote

Before this PR, given the string "class=\"bg-white\"", the default split extractor extracted bg-white\ instead of bg-white.
This becomes problematic when extracting from SolidJS SSR output, which becomes an issue when using SolidJS with Astro.
This PR fixes the issue by including \" in the split token list.

I agree that this can be somewhat opinionated. I fully respect your opinion and if you don't like it, I'll just use it as a custom extractor in my projects :D

Okay I guess \ was not allowed in the rules name anyways XD

Ah, we do have some cases that need backslash 🤔

 ❯ test/preset-mini.test.ts  (7 tests | 1 failed) 272ms
   ❯ test/preset-mini.test.ts > preset-mini > targets
     → expected [ '[content:attr(attr\_content)]' ] to deeply equal []

Looks like we should strictly check for \\["']?

I guess we could do /\\?[\s'";=]+/g

Can you add a test for your case "class=\"bg-white\"" so we won't break the usage accidentally in the future


Patch for copy and copy-if-different scripts with whitespace problem #1023 - CLA: trivial

Small "CLA: trivial" fix related to #1023

You may need to add CLA: trivial in your commit message or sign a CLA at: https://www.openssl.org/policies/cla.html, to avoid cla-check failure.

Simon, we need you to put "CLA: trivial" in your commit text (and push -f) or sign our CLA.
I agree this is trivial.
Ping @levitte

@richsalz is that ok?

Seems you should put the CLA: trivial line into the commit text, not the commit title:
patch for copy and copy-if-different whitespace problem

CLA: trivial

What @InfoHunter said.

commit b832656 in 1.0.2-stable, thank you!

Thanks and no problem


AdapterFusion | AttributeError: 'NoneType' object has no attribute 'dropout_prob'

Environment info

- `transformers` version: 4.35.2
- Platform: Linux-5.15.0-73-generic-x86_64-with-glibc2.35
- Python version: 3.10.13
- Huggingface_hub version: 0.20.1
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: DEEPSPEED
        - use_cpu: False
        - debug: True
        - num_processes: 4
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - deepspeed_config: {'deepspeed_config_file': 'ds_config/ds_config-llama-7b.json', 'zero3_init_flag': True}
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
        - dynamo_config: {'dynamo_backend': 'INDUCTOR', 'dynamo_mode': 'max-autotune', 'dynamo_use_dynamic': True, 'dynamo_use_fullgraph': True}
- PyTorch version (GPU?): 2.1.1+cu121 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

adapters version: 0.1.0
Platform:
Python version: 3.10.13
PyTorch version (GPU?):

Details
Hello, I met this error when I migrated from adapter-transformers`` to adapters``` package
Traceback (most recent call last):
  File "/home/farid/adapters-lid/fusionbert_exp/adapterfusion-languagefusion.py", line 250, in <module>
    model.add_adapter_fusion(Fuse(*fused), config=fusion_config)
  File "/home/mambaforge/envs/venvadapt/lib/python3.10/site-packages/adapters/model_mixin.py", line 652, in add_adapter_fusion
    self.apply_to_adapter_layers(lambda i, layer: layer.add_fusion_layer(adapter_names))
  File "/home/mambaforge/envs/venvadapt/lib/python3.10/site-packages/adapters/model_mixin.py", line 443, in apply_to_adapter_layers
    fn(i, module)
  File "/home/mambaforge/envs/venvadapt/lib/python3.10/site-packages/adapters/model_mixin.py", line 652, in <lambda>
    self.apply_to_adapter_layers(lambda i, layer: layer.add_fusion_layer(adapter_names))
  File "/home/mambaforge/envs/venvadapt/lib/python3.10/site-packages/adapters/methods/bottleneck.py", line 126, in add_fusion_layer
    dropout_prob = fusion_config.dropout_prob or getattr(self.model_config, "attention_probs_dropout_prob", 0)
AttributeError: 'NoneType' object has no attribute 'dropout_prob'

my code:
for lang in tqdm(train_langs):
    lang_simple = lang.split("-")[0]

    lang_adapter_config = AdapterConfig.load("pfeiffer", reduction_factor=2)

    print(lang_adapter_config)

    if lang_simple == 'zhc':
        model.load_adapter(adapter_name_or_path="zh_yue/wiki@ukp", set_active=True, with_head=False)
    elif lang_simple == 'zht':
        model.load_adapter(adapter_name_or_path="zh/wiki@ukp", set_active=True, with_head=False)
    else:
        model.load_adapter(adapter_name_or_path=lang_simple+"/wiki@ukp", set_active=True, with_head=False)

fused = deepcopy(simple_train_langs)
fused.remove("zh_yue")

model.add_adapter_fusion(Fuse(*fused))
model.set_active_adapters(Fuse(*fused))

model.add_classification_head("massive", num_labels=60)

adapter_setup = Fuse(*fused)
model.train_adapter_fusion(adapter_setup)

Is this a bug or do I make mistake in my implementation?
Thank you for your help!

Hey @faridlazuarda, on first glance, the code you shared looks correct. I've tried to reproduce the issue using your snippet with some example model/ adapters, but wasn't able to do so. Could you possibly share a minimal reproducible example that throws this error to help us investigate? Thanks!

This issue has been automatically marked as stale because it has been without activity for 90 days. This issue will be closed in 14 days unless you comment or remove the stale label.


Update property value on BulkInsertOrUpdate

Is het possible to change a property value of an item only if it is an update on BulkInsertOrUpdate?

No, what you could do is separate it into BulkInsert and BulkUpdate.
And if you don't know which ones are for update BulkRead can be used to read them first.

Ok, thanks.
I will use BulkRead first.


Addition of processing logic when null is included in Double type result

Description
Addition of processing logic when null is included in Double type result.
Previously, JsonNullNode was treated as String, but changed to treat as null.
Related Issue :
How Has This Been Tested?

Create Dashboard and chart
The dimension value whose group by result is null is put on the cross shelf and the data containing null in the result is drawn on the chart.

Need additional checks?
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.

Additional Context

deploy to 1

deploy to 1


Fix case when readDirSync patch returns object

Will fix

readDirSync did not work in the new Svelte Kit because it expects returned Dirent[] instead of string[].

Thx for your PR @janvotava LGTM

Thanks for taking care of this PR. Could be probably better - some generic function for handling of arrays of string[] | Buffer[] | Dirent[], but I think for the monkey patch it's kind of all right.

And actually maybe I just instantiate Dirent properly through the constructor like:
new fs.Dirent('+page.js', fs.constants.UV_DIRENT_FILE)


Stop concatenate using all memory

Does iris.cube.CubeList.concatenate() need to load the data payload? If it can be changed so that it doesn't then this would be very useful. If not, then it would be good if a safeguard could be added to abort the concatenate when the memory usage is getting critically high. You don't know before you perform a cat if iris will swallow 100% of the memory, leading to a very messy and painful abort for the user.

This one is fixed in v1.8.0, concatenate() no longer loads the data payload.


Images are not restricted to 100% body width

We need an
img {
 width: 100%
}

somewhere so that on mobile it doesn't blow out the screen with overflow

This issue can be closed. Fixed by #38


Missing requirements.txt

I tried to reproduce the results locally but was unable to do so as I got stuck in an endless loop of missing dependencies. Can the authors of the paper please push the same.

Did you follow PFLlib to create the python environment?


Porting selected changes from master to Releases/m91

Hi @KrishnaAdityaB, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Krishna Aditya). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;


remove cx_direction from O3 if symmetric

Summary
This removes the calls to CXDirection pass in O3 passmanager if the coupling map is symmetric.  This reduces the compile time of QV circuits by a factor of two in this mode.
Details and comments

makes sense but what happens if half are symmetric.

Not sure what half-symmetric means?  Symmetric here refers to the adjacency graph, which is symmtric iff you can perform cx gates in both directions for all pairs.

ok I take my comment back. I did not realize we always take the subgraph. Thanks @nonhermitian I go read your tutorial in more detail. I am good with this PR.


add DensityMatrix.to_statevector() for pure states

This is handy when you have a circuit in a product state and you want the reduced state as a statevector (e.g. discarding an uncomputed ancilla).

@chriseclectic @jaygambetta addressed your comments.
Chris I restricted it to trace-1 density matrices for now as that is the is_valid definition currently.


Value objects do not have equality when using Virtus.value_object

Given:
class Value
  include Virtus.value_object
  attribute :number, Integer
end

We expect: Value.new(:number => 7) == Value.new(:number => 7)
This behavior works as expected when using the deprecated include Virtus::ValueObject but is not preserved with the newer include. The equality behavior seems to be provided by ValueObject::ClassMethods which does not get extended when a value object class is constructed using the builder.
I took a quick crack at correcting the issue but I'm pretty sure I'm going down the wrong path. Including the class methods fixes equality but seems to introduce conflicting ideas about how to set options on attributes. Can you provide some guidance?

You need to wrap attribute definitions in a values do .. end block. That's horrible I have to admit so it's going away in Virtus 2.0. Sorry about that.

Oof. Thanks.