Refactors script/setup_docker_prereqs

This PR refactors the script/setup_docker_prereqs to allow toggling of certain packages to be installed. It also adds support for OpenALPR cli, which support was added in #5166.
The main reason for these changes is so that other people can maintain their own Docker image for other platforms where some of these features are not available, i.e. Tellstick does not provide arm64 (aarch64) binaries and the phantonjs just downloads the x86 binary, without having to maintain their own build scripts.
I feel like this is a good compromise without architecting a proper build system. Ideally, all of these build scripts, Docker, and Vagrant files would be in a separate repository.

Shouldn't it make more sense to have it install everything unless you specify a ENV? This way, you don't need to change the Dockerfile for a full install.

Something like a ENV INSTALL_3THPARTY "ffmpeg openlpr tellstick" could be used to pass the options to the script?

@michaelarnauts: Yes, quite possibly. However, the environment variables are defaulted to "yes". The line additions in the Dockerfile are there for verbosity.
I would like to reiterate that this approach was an explicit compromise and that I believe there are more proper, long term, solutions to some of these issues.

@mathewpeterson sorry, missed it that they were yes by default. I would comment them out however so the Dockerfile doesn't change and the existing layer cache can still be used.
I think would be a good idea to defaulting them to no, so they can be uncommented to disable some dependencies.
FROM python:3.5
MAINTAINER Paulus Schoutsen<EMAIL_ADDRESS>
# Uncomment any of the following lines to disable the installation.
#ENV INSTALL_TELLSTICK no
#ENV INSTALL_OPENALPR no
#ENV INSTALL_FFMPEG no
#ENV INSTALL_PHANTOMJS no
...

@michaelarnauts I've updated the Dockerfile with your recommendation.

Just a question, shouldn't we do the same for openzwave and libcec then?

Possibly, however I think that would be out of scope for this PR because those have build scripts that are cpu architecture agnostic.

@michaelarnauts I have moved the echo up a few lines. Sorry about that.

Hi @mathewpeterson,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!

Hi @mathewpeterson,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!

@michaelarnauts Good catch. I've made the correction and rebased the branch.

Looks good! Thanks!

Awesome! Thank you for all your help!

This has broken our Docker builds
script/setup_docker_prereqs: line 110: script/install_phantomjs: No such file or directory

This seems to be due to the openalpr changing the path. I'll take a look.


The implementation of nms seems wrong.

The implementation of nms in layers/box_utils.py picks the top-k scores and the corresponding boxes, and then filter by IOU among the above boxes. Therefore nms is only applied to the corresponding boxes of the top-k scores in the code. However, the normal implementation filters among boxes and returns the top-k boxes with biggest scores and IOU less than the threshold.

I practice it wouldn't make much difference.

Good to know! Thanks @gurkirt !

Good to know! Thanks @gurkirt !


Make Timezone and Regional Settings variable

The current config is a static configuration for a Dutch region and Timezone. This should be a configurable item

done


Missing inspection for pf.ko and /etc/pf.conf

dockerbox start should do the following:

Inspect if pf.ko has been loaded, and kldload pf.ko if not
Inspect if /etc/pf.conf exist, and touch it if not

Resolved with commit 318f6d4


Paths query

Evening
Not been able to find the answer to this for sure.
The "paths" variable in the config, should this be the path inside the Sonarr / Radarr docker?  Rather than full actual system path?  I am just getting "Configured paths do not exist" in logs, but have tried both ways and not had any joy as yet.  I currently have it matched to Sonarr/Radarr configured "Local Path"
Tried leaving blank also to see if it picked up itself, but it fails on that also
Top of logs:
2021/03/03 19:44:03.321337 start.go:235: Unpackerr v0.7.2-275 Starting! (PID: 1) 2021-03-03 19:44:03.31945678 +0000 UTC m=+0.029422215,
2021/03/03 19:44:03.321370 logs.go:177: ==> GoLift Discord: https://golift.io/discord <==,
2021/03/03 19:44:03.321381 logs.go:178: ==> Startup Settings <==,
2021/03/03 19:44:03.321425 sonarr.go:41:  => Sonarr Config: 1 server: https://sonarr.blah.org/, apikey:true, timeout:10s, verify ssl:false, protos:torrent, paths:["/downloads/complete"],
2021/03/03 19:44:03.321444 radarr.go:41:  => Radarr Config: 1 server: https://radarr.blah.org/, apikey:true, timeout:10s, verify ssl:false, protos:torrent, paths:["/downloads/complete"],
2021/03/03 19:44:03.321455 lidarr.go:45:  => Lidarr Config: 0 servers,
2021/03/03 19:44:03.321473 readarr.go:45:  => Readarr Config: 0 servers,
2021/03/03 19:44:03.321489 folder.go:57:  => Folder Config: 0 paths, event buffer: 20000,
2021/03/03 19:44:03.321503 logs.go:184:  => Parallel: 1,
2021/03/03 19:44:03.321529 logs.go:185:  => Interval: 5m0s,
2021/03/03 19:44:03.321544 logs.go:186:  => Delete Delay: 1s,
2021/03/03 19:44:03.321565 logs.go:187:  => Start Delay: 1m0s,
2021/03/03 19:44:03.321581 logs.go:188:  => Retry Delay: 5m0s,
2021/03/03 19:44:03.321598 logs.go:189:  => Debug / Quiet: true / false,
2021/03/03 19:44:03.321612 logs.go:190:  => Directory & File Modes: 0755 & 0644,
2021/03/03 19:44:03.321626 webhook.go:200:  => Webhook Configs: 0 URLs,
2021/03/03 19:44:03.699641 sonarr.go:72: [Sonarr] Updated (https://sonarr.blah.org/): 0 Items Queued,
2021/03/03 19:44:03.754520 radarr.go:72: [Radarr] Updated (https://radarr.blah.org/): 229 Items Queued,
2021/03/03 19:44:03.754588 handlers.go:182: [DEBUG] Radarr: Configured paths do not exist; could not find alternative path in error message for.............

The 229 items isn't correct, one is pending unpacking, after this it lists everything from Transmission.  Presume after it gets the right paths, it will sort itself out on the numbers
Config:
##      Unpackerr Example Configuration File      ##
## The following values are application defaults. ##
####################################################

# [true/false] Turn on debug messages in the output. Do not wrap this in quotes.
# Recommend trying this so you know what it looks like. I personally leave it on.
debug = true

# Disable writing messages to stdout. This silences the app. You should set a log
# file below if you set this to true.
quiet = false

# Write messages to a log file. This is the same data that is normally output to stdout.
# This setting is great for Docker users that want to export their logs to a file.
# The alternative is to use syslog to log the output of the application to a file.
# Default is no log file; this is unset. log_files=0 turns off auto-rotation.
# Default files is 10 and size(mb) is 10 Megabytes; both doubled if debug is true.
# Windows paths must use two backslashes: "C:\\Some\\Path\\Unpacker.log" (true everywhere).
#log_file = "/downloads/unpackerr.log"
log_files = 10
log_file_mb = 10

# How often to poll sonarr and radarr.
# Recommend 1m-5m. Uses Go Duration.
interval = "5m"

# How long an item must be queued (download complete) before extraction will start.
# One minute is the historic default and works well. Set higher if your downloads
# take longer to finalize (or transfer locally). Uses Go Duration.
start_delay = "1m"

# How long to wait before removing the history for a failed extraction.
# Once the history is deleted the item will be recognized as new and
# extraction will start again. Uses Go Duration.
retry_delay = "5m"

# How many files may be extracted in parallel. 1 works fine.
# Do not wrap the number in quotes. Raise this only if you have fast disks and CPU.
parallel = 1

# Use these configurations to control the file modes used for newly extracted
# files and folders. Recommend 0644/0755 or 0666/0777.
file_mode = "0644"
dir_mode = "0755"

##-Notes-######################################################################
## The following sections can be repeated if you have more than one Sonarr,   #
## Radarr or Lidarr, Readarr, Folder or Webhook.                              #
## You must uncomment the [[header]] and api_key at a minimum for Starr apps. #
###############################################################################

################
### Webhooks ###
################
# Sends a webhook when an extraction starts and again when it finishes.
# Created to integrate with discordnotifier.com.
# Can possibly be used with other services. Don't forget to uncomment [[webhook]].
#[[webhook]]
#  url        = "https://discordnotifier.com/notifier.php?api=abcdef-ghijklmn-op"
#  name       = ""    # Set this to hide the URL in logs.
#  silent     = false # do not log success (less log spam)
#  events     = [0]   # list of event ids to include, 0 == all.
#  exclude    = []    # list of apps to exclude, ie. ["radarr", "lidarr"]

################
### Episodes ###
################
[[sonarr]]
url = "https://sonarr.blah.org/"
api_key = "blah"
## File system path where downloaded Sonarr items are located.
  paths = ["/downloads/complete"]
## Default protocols is torrent. Alternative: "torrent,usenet"
#  protocols = "torrent"
## How long to wait for a reply from the backend.
#  timeout = "10s"
## How long ago a file must have been imported before deletion. The file must
## also not be part of an active queue item. Set this to "1m" to make sure files
## are deleted quickly after being imported. Recommend "5m". Uses Go Duration.
## Setting this to any negative number `-1` disables deleting extracted files.
#  delete_delay = "5m"
## If you use this app with NZB you may wish to delete archives after extraction.
## General recommendation is: do not enable this for torrent use.
## This setting is not affected by `delete_delay`.
#  delete_orig = false

################
#### Movies ####
################
[[radarr]]
url = "https://radarr.blah.org/"
api_key = "blah"
## File system path where downloaded Radarr items are located.
  paths = ["/downloads/complete"]
## Default protocols is torrents. Alternative: "torrent,usenet"
#  protocols = "torrent"
## How long to wait for a reply from the backend.
#  timeout = "10s"
## How long ago a file must have been imported before deletion. The file must
## also not be part of an active queue item. Set this to "1m" to make sure files
## are deleted quickly after being imported. Recommend "5m". Uses Go Duration.
## Setting this to any negative number `-1` disables deleting extracted files.
#  delete_delay = "5m"
## If you use this app with NZB you may wish to delete archives after extraction.
## General recommendation is: do not enable this for torrent use.
## This setting is not affected by `delete_delay`.
#  delete_orig = false

################
#### Albums ####
################
#[[lidarr]]
#  url = "http://<IP_ADDRESS>:8686"
#  api_key = "0123456789abcdef0123456789abcdef"
## File system path where downloaded Lidarr items are located.
#  paths = ["/downloads"]
## Default protocols is torrent. Alternative: "torrent,usenet"
#  protocols = "torrent"
## How long to wait for a reply from the backend.
#  timeout = "10s"
## How long ago a file must have been imported before deletion. The file must
## also not be part of an active queue item. Set this to "1m" to make sure files
## are deleted quickly after being imported. Recommend "5m". Uses Go Duration.
## Setting this to any negative number `-1` disables deleting extracted files.
#  delete_delay = "5m"
## If you use this app with NZB you may wish to delete archives after extraction.
## General recommendation is: do not enable this for torrent use.
## This setting is not affected by `delete_delay`.
#  delete_orig = false

################
# Publications #
################
#[[readarr]]
#  url = "http://<IP_ADDRESS>:8787"
#  api_key = "0123456789abcdef0123456789abc"
## File system path where downloaded Readarr items are located.
#  paths = ["/downloads"]
## Default protocols is torrent. Alternative: "torrent,usenet"
#  protocols = "torrent"
## How long to wait for a reply from the backend.
#  timeout = "10s"
## How long ago a file must have been imported before deletion. The file must
## also not be part of an active queue item. Set this to "1m" to make sure files
## are deleted quickly after being imported. Recommend "5m". Uses Go Duration.
## Setting this to any negative number `-1` disables deleting extracted files.
#  delete_delay = "5m"
## If you use this app with NZB you may wish to delete archives after extraction.
## General recommendation is: do not enable this for torrent use.
## This setting is not affected by `delete_delay`.
#  delete_orig = false

##-Folders-####################################################################
## This application can also watch folders for things to extract. If you copy a
## subfolder into a watched folder (defined below) any extractable items in the
## folder will be decompressed. This has nothing to do with Starr applications.
###############################################################################
#[[folder]]
## Windows paths must use two backslashes: "C:\\Some\\Folder\\To\\Watch"
#  path = "/some/folder/to/watch"
## Delete extracted or original files this long after extraction. Set to 0 to disable all deletes. Uses Go Duration.
#  delete_after = "10m"
## Delete extracted files after successful extraction? true/false, no quotes. Honors delete_after.
#  delete_files = false
## Delete original items after successful extraction? true/false, no quotes. Honors delete_after.
#  delete_original = false
## Move extracted files into original folder? If false, files go into an _unpackerred folder.
#  move_back = false

Many thanks indeed

The error message you "blurred out" should list the path it's looking in, probably /downloads/complete/Thing.You.Downloaded. The question is, where is Thing.You.Downloaded if not in /downloads/complete? Perhaps it's in /downloads/complete/tv-sonarr or perhaps you forgot to mount your download location to /downloads using -v in your docker run command? Sharing the docker run command will help me visualize the rest of your configuration.

The 229 items isn't correct,

Yes, it is. Go into Sonarr, click Activity, then click Options in the top right and check the box to show all your items. Do this in Radarr too..

@davidnewhall Really appreciate the reply, thanks
Sorry for cutting the logs early, bit more before it starts repeating for each file:
2021/03/04 08:57:23.584970 handlers.go:182: [DEBUG] Radarr: Configured paths do not exist; could not find alternative path in error message for TV_show_name_here,
2021/03/04 08:57:23.585051 handlers.go:184: [DEBUG] Radarr: Errors encountered looking for TV_show_name_here path: ["stat /downloads/complete/TV_show_name_here: no such file or directory"],
2021/03/04 08:57:23.585092 handlers.go:83: [Radarr] Waiting for Start Delay: TV_show_name_here (1m0s remains),
2021/03/04 08:57:23.585110 radarr.go:89: [DEBUG] Radarr: (https://radarr.blah.org/): Completed (torrent:100%): TV_show_name_here,

Sonarr docker command:
docker run -d \
--name=sonarr \
-e PUID=1000 \
-e PGID=100 \
-e TZ=Europe/London \
--net=proxy \
-v /sharedfolders/dockerdata2/sonarr:/config \
-v /sharedfolders/media/Videos/Series:/tv \
-v /sharedfolders/downloaded/transmission/downloads:/downloads \
--restart unless-stopped \
linuxserver/sonarr:preview

Is this the same folder mount on Sonarr?
-v /sharedfolders/downloaded/transmission/downloads:/downloads

This seems correct. Does /sharedfolders/downloaded/transmission/downloads contain TV_show_name_here ?

Yes, Sonarr and Radarr both setup identically, apart from their destination folders
Yes, Transmission downloads to that folder, well actually to /sharedfolders/downloaded/transmission/downloads/complete, but did since change that in what has been copied above.  Triple checked the path and yes, the files/folders mentioned exist there.
So in the conf, paths= should be /sharedfolders/downloaded/transmission/downloads/complete ?  Just tried and restarted, but same messages in the logs
I did read one report somewhere saying they were having a permissions issue, but didn't see any env options for the unpackerr docker to run with certain GUID / UUID.  Just thinking of a possible cause
Latest logs:
2021/03/04 09:20:04.369206 start.go:235: Unpackerr v0.7.2-275 Starting! (PID: 1) 2021-03-04 09:20:04.367762723 +0000 UTC m=+0.000939687,
2021/03/04 09:20:04.369235 logs.go:177: ==> GoLift Discord: https://golift.io/discord <==,
2021/03/04 09:20:04.369251 logs.go:178: ==> Startup Settings <==,
2021/03/04 09:20:04.369277 sonarr.go:41:  => Sonarr Config: 1 server: https://sonarr.blah.org/, apikey:true, timeout:10s, verify ssl:false, protos:torrent, paths:["/sharedfolders/downloaded/transmission//downloads/complete"],
2021/03/04 09:20:04.369307 radarr.go:41:  => Radarr Config: 1 server: https://radarr.blah.org/, apikey:true, timeout:10s, verify ssl:false, protos:torrent, paths:["/sharedfolders/downloaded/transmission/downloads/complete"],
2021/03/04 09:20:04.369317 lidarr.go:45:  => Lidarr Config: 0 servers,
2021/03/04 09:20:04.369328 readarr.go:45:  => Readarr Config: 0 servers,
2021/03/04 09:20:04.369340 folder.go:57:  => Folder Config: 0 paths, event buffer: 20000,
2021/03/04 09:20:04.369350 logs.go:184:  => Parallel: 1,
2021/03/04 09:20:04.369368 logs.go:185:  => Interval: 5m0s,
2021/03/04 09:20:04.369381 logs.go:186:  => Delete Delay: 1s,
2021/03/04 09:20:04.369389 logs.go:187:  => Start Delay: 1m0s,
2021/03/04 09:20:04.369397 logs.go:188:  => Retry Delay: 5m0s,
2021/03/04 09:20:04.369406 logs.go:189:  => Debug / Quiet: true / false,
2021/03/04 09:20:04.369414 logs.go:190:  => Directory & File Modes: 0755 & 0644,
2021/03/04 09:20:04.369424 webhook.go:200:  => Webhook Configs: 0 URLs,
2021/03/04 09:20:04.530301 radarr.go:72: [Radarr] Updated (https://radarr.blah.org/): 230 Items Queued,
2021/03/04 09:20:04.552403 sonarr.go:72: [Sonarr] Updated (https://sonarr.blah.org/): 0 Items Queued,
2021/03/04 09:20:04.552463 handlers.go:182: [DEBUG] Radarr: Configured paths do not exist; could not find alternative path in error message for tv_show_here,
2021/03/04 09:20:04.552525 handlers.go:184: [DEBUG] Radarr: Errors encountered looking for tv_show_here path: ["stat /sharedfolders/downloaded/transmission/downloads/complete/tv_show_here: no such file or directory"],
2021/03/04 09:20:04.552549 handlers.go:83: [Radarr] Waiting for Start Delay: tv_show_here (1m0s remains),

Yes, Sonarr and Radarr both setup identically, apart from their destination folders

Setup identically (same /downloads mount) as Unpackerr?

So in the conf, paths= should be /sharedfolders/downloaded/transmission/downloads/complete ?

No, I believe it should be ["/downloads"]. Just make sure you also add this to your docker run for unpackerr:
-v /sharedfolders/downloaded/transmission/downloads:/downloads

You can also try ["/downloads", "/downloads/complete"] just for good measure. One of them should work.

I did read one report somewhere saying they were having a permissions issue, but didn't see any env options for the unpackerr docker to run with certain GUID / UUID. Just thinking of a possible cause

If you run golift/unpackerr then it runs as root and should not have permissions issues. If you run the hotio docker container then I believe it has GUID / UUID you can set with env variables.
If none of this gets it working, try changing to this:
-v /sharedfolders/downloaded/transmission/downloads/complete:/downloads

with paths = ["/downloads"]. I don't suspect this should work any different, but changing things may reveal the problem.
Now working, thanks so much for your time and sorry for wasting it!  Didn't see on the readme page, but do let me know if I can contribute in anyway, buy you a coffee etc.  Been a big help!

I know it doesn't help now, but it was totally one of the first things I mentioned. :D

perhaps you forgot to mount your download location to /downloads using -v in your docker run command?

Very happy you got it working!

oh dear, that makes the shame worse.  I work in IT and would be calling my users all sorts of names if they did this to me
Way to confident in my own abilities it seems, humbled by being on the other end for a change.


Remove PR labeler bot

RIP

[x] Closes #8524

Wait why did it still run on this PR??

Wait why did it still run on this PR??

It's in pull-request-target, so it runs based on main. Merge and it'll be gone...

It's in pull-request-target, so it runs based on main. Merge and it'll be gone...

Ah ha! My github-foo needs some work, clearly.


forked

Hey there, I see you've created a repo for this now. Would you like to take over what I'd done so far? https://github.com/timabell/jquery.stickytabs
There's also a nuget package that I published from it https://www.nuget.org/packages/jquery.stickytabs/

Hey timabell, would you like commit access or are there any changes that are just in your version?

I don't think I need commit access, but thanks.
You might want to grab the readme and modify it to suit as it's more extensive than what you currently have.
The only other thing is the nuget package. I don't know if that's of interest to you or if it's something you'd want to take ownership of. I could maintain that independently but base it on your repo instead of mine if that suits. I do ASP.NET contracting for a living so that's why the nuget package is of interest to me.
Oh and thanks for publishing this in the first place, very nice piece of work, and neatly solved a problem in my current project.

What do I need to do for the nuget thing? If you could maintain it that's fine .. do I just add your nuget file to my repo?

I think it probably only makes sense to put the nuget file in your repo if you're going to maintain the published nuget package. If not I'll keep the nuget elsewhere and track your repo.
To publish it I use a windows machine and the nuget tool you can download from their site, though it's not the only way of doing it.
So it looks like the best approach is for me to look after the nuget bit and just package whatever you publish here then. If you could tag releases that would be handy as then I'll know when it's stable enough to update the package rather than just tracking master.

Alright I have no interest in windows so I'll leave it to you :)


Refactor handshake

This is intended to simplify the handshake code.
It should simplify reusing much of the handshake code in order to resolve #75 in a future PR.
I also changed a few small things i came across while working on this. All should be documented within the commit messages.
I hope i did not miss any subtleties why async_handshake was a coroutine? I could not think of a good reason with the current code.
Feedback would be much appreciated :)

@jens-diewald First of all thanks a lot for this PR. I really appreciate it.
Regarding the implementation of the handshake as a coroutine I had a lot of help from the nice guys from boost::beast, namely @vinniefalco and @madmongo1.
To be honest I can't remember all the details but I would rather avoid removing the coroutine part without understanding the full implications of that. We definitely don't want the async parts of the code to be blocking which is why a coroutine was needed in the first place as far as I remember.
Maybe we can get @vinniefalco and/or @madmongo1 to help a bit with this?
I definitely appreciate it if this could be simplified so I hope this can be merged in one way or another.
Thanks once again.

@laudrup Whether async_handshake is implemented as a coroutine or not should not be a problem with regard to asynchronity by itself. (Of course i could have gotten something wrong.)
Anyhow, i thought about this some more and i found, that if we remove the state member and enum, we can actually make use of the function being a coroutine. I have reverted the original change and added a new commit removing the state and making use of the coroutine instead. Maybe this is how it was originally intended?
I have then made another commit re-adding the comment regarding posting self on the first call to the function and a small simplification, so that the respective code is needed only once.
I have also thought about this some more, but i am afraid i still do not really get it. I will reply to my comment on the code to elaborate my understanding so far.

I have moved the self-posting logic into a seperate file, added a case distinction on BOOST_VERSION and added some comments.
This is somewhat over-the-top, but i could not think of another reasonable way to document what is goind onb and how it should be done better in newer boost versions.
For the case distinction i added boost 1.80 and boost 1.81 to the CI matrix.
@klemens-morgenstern, if you find the time to look at this and possibly confirm that it makes sense to you, i would appreciate that a lot.
@laudrup, sorry that this went somewhat off-topic from my originally intended changes. The self-posting stuff would have deserved a seperate issue. From my side, the PR could be merged like this now. (Assuming the tests go through and there are no other objections.)

@jens-diewald So sorry for the late reply and thanks once again for your pull request.
I would definitely like this to work with boost versions earlier than the latest. A somewhat new version could be required. Assuming there's tests ensuring it works on different versions then adding it to the CI is definitely the right thing to do. Thanks for that.
@klemens-morgenstern Thanks a lot for your input here. It seems like you're a lot more experienced with the details of boost::asio than I am so if you think these changes make sense then I'll feel fairly confident merging them.
So if we get the tests to pass and @klemens-morgenstern approves then so will I, although I'll probably look through the code again a final time before merging.
Thanks a lot once again.

Hi @jens-diewald,
So sorry for not having looked at this for ages. I haven't really looked at this library for quite some time.
Is this still something you'd like me to merge at some point?
As you might have noticed I've update the pipeline to test newer boost versions and made a new release so it might be a good time for me to revisit this.
The main issue I have with the change is that I would very much like someone else to have a look at it as well. If you're still interested I'll see if I can ping some relevant people.
Thanks a lot.

Hi @laudrup,
yes I think this pull request does improve the code.
I worked on this as a preparation of #78, but all the changes are independent of that issue.
A second opinion regarding the post_self part would be nice.

@madmongo1 has said he'll have a look at this when time permits.
Hope we can get this merged.


Error when proto package name contains dot(s)

https://github.com/WireMock-Net/WireMock.Net/issues/1077

https://github.com/StefH/ProtoBufJsonConverter/pull/11


Swift 4.1 compatibility

Thanks Zev!
On Apr 3, 2018, 10:11 AM -0400, Zev Eisenberg<EMAIL_ADDRESS>wrote:

Merged #56.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

❤ going to do a minor release soon to push this out


stock requests only meet 2/3 of data points

Any time is ask for a month of data, it will give around 21 days, and a year will give around 200 days of data. Is there anyway to fix this?

The data points probably refer to workdays only.


Refactoring

TODO

[ ] Move manifest related functions to its own space
[ ] Move Cache handling to the Store
[ ] Add static "factory" method to create an orbit instance

[ ]  Make it async and check if ipfs is ready, throw otherwise

[ ] After the Store refactoring is done, move it to orbit-db package and inject it into other stores (orbitdb/orbit-db-store#31)

@thiagodelgado111 I'd love to revisit this at some point with you and the Colony team! Leaving open

Closing as obsolete. A lot of refactoring was done for v1.0.0.


Substrate: Write assumed environment configuration into metadata

In ink!, certain types are configurable. The latest version of the metadata crate stores this information in the ABI file.

What would this test be good for? I trust the EnvironmentSpec builder to do its job correctly

What would this test be good for? I trust the EnvironmentSpec builder to do its job correctly

If one refactors the code and moves everything around, one might mistakenly remove this portion of your code. As refactoring PRs are usually large, we might miss this during review. We trust our tests to detect regressions. Because there is no test for your modification, its removal will go unnoticed.
The rule of thumb is that if you remove your additional lines of code and no test fail, you need a test!

It's not a bad idea to have some code which does some asserts on the output metadata, both to ensure that the metadata looks like we expect now and that there are no regressions in future. EnvironmentSpec surely works but we're interested in that the metadata looks like we want, have we made all the right calls to EnvironmentSpec etc.

Having said that @xermicus  is the maintainer for Substrate, so it is up entirely up to him

@LucasSte in general I agree with your sentiment. However, for this case here it would be incredibly hard for someone to "move everything aroung", additionally forgetting to write the Environment spec, and then us reviewing the code not catch this.
This environment data is just for frontends. The way this should be tested is to have integration that ensures we output compatible metadata. But right now I'm not aware on any frontend validating this.

Thanks.
Yes we still miss coverage in some places. For example we don't have any metadata unit tests at all. So before splitting hairs on this rather unimportant (at least at the moment) detail I'd rather spend my time implementing tests for the more important parts of the metadata first. Or other more important parts of the compiler.


Improve address conversion docs

In #1765 we added a couple of new functions.

Patch 1: Fix mis-documented function.
Patch 2: Do trivial rustdocs fix.

CI fails appear to be unrelated, I think master is in a bit of a state at the moment.


[JavaScript] var is colored as a keyword even if it is a part of variable name

what theme are you using @vladimir-kotikov ?

This particular theme is Monokai from VS Code insiders. But it is displayed this way with a couple of other themes (incl. Dark+, Red and Solarized light)

There is no such problem with Dark (Visual Studio) theme.

Duplicate of https://github.com/Microsoft/TypeScript-TmLanguage/issues/96


Add CsvAbnornalReplacing alerts

Signed-off-by: Ben Ye<EMAIL_ADDRESS>
Description of the change:
This pr adds two alerts CsvAbnormalReplacingOver30Min and CsvAbnormalReplacingOver4Hr, porting from https://github.com/openshift/managed-cluster-config/blob/master/deploy/sre-prometheus/100-csv-abnormal.PrometheusRule.yaml#L20.
Motivation for the change:
https://issues.redhat.com/browse/OSD-4435
https://issues.redhat.com/browse/OSD-4436
Reviewer Checklist

[ ] Implementation matches the proposed design, or proposal is updated to match implementation
[ ] Sufficient unit test coverage
[ ] Sufficient end-to-end test coverage
[ ] Docs updated or added to /docs
[ ] Commit messages sensible and descriptive

This PR failed tests for 1 times with 1 individual failed tests and 0 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 1
flaketestcount: 1
skippedtestcount: 0
flaketests:

classname: End-to-end
name: 'BeforeSuite'
counts: 2
details:

count: 2
error: |4-
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/e2e_test.go:63
Test Panicked
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/ctx/must.go:13

meandurationsec: 211.349999
skippedtests: []

This PR failed tests for 1 times with 1 individual failed tests and 0 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 1
flaketestcount: 1
skippedtestcount: 0
flaketests:

classname: End-to-end
name: 'BeforeSuite'
counts: 2
details:

count: 2
error: |4-
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/e2e_test.go:63
Test Panicked
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/ctx/must.go:13

meandurationsec:<PHONE_NUMBER>
skippedtests: []

Seems the E2E test failure is not related. PTAL!

This PR failed tests for 1 times with 1 individual failed tests and 4 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 1
flaketestcount: 1
skippedtestcount: 4
flaketests:

classname: End-to-end
name: 'Subscription can reconcile InstallPlan status'
counts: 1
details:

count: 1
error: |4-
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/subscription_e2e_test.go:880
Unexpected error:
    <*errors.errorString | 0xc0001f02c0>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/subscription_e2e_test.go:920

meandurationsec: 300.554875
skippedtests:
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should have created the expected
prometheus objects'
counts: 1
details: []
meandurationsec: 2.177452
classname: End-to-end
name: 'Catalog image update'
counts: 1
details: []
meandurationsec: 0.334473
classname: End-to-end
name: 'Subscription updates existing install plan'
counts: 1
details: []
meandurationsec: 0.024637
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should install the operator
successfully'
counts: 1
details: []
meandurationsec: 2.226671

/lgtm

/ok-to-test

/retest

This PR failed tests for 1 times with 1 individual failed tests and 4 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 1
flaketestcount: 1
skippedtestcount: 4
flaketests:

classname: End-to-end
name: 'Operator Group cleanup csvs with bad owner operator groups'
counts: 1
details:

count: 1
error: |4-
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_groups_e2e_test.go:1800

	Error Trace:	operator_groups_e2e_test.go:2040
	            				runner.go:113
	            				runner.go:64
	            				it_node.go:26
	            				spec.go:215
	            				spec.go:138
	            				spec_runner.go:200
	            				spec_runner.go:170
	            				spec_runner.go:66
	            				suite.go:62
	            				ginkgo_dsl.go:226
	            				ginkgo_dsl.go:214
	            				e2e_test.go:54
	Error:      	Received unexpected error:
	            	timed out waiting for the condition

/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1005

meandurationsec: 609.192154
skippedtests:
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should install the operator
successfully'
counts: 1
details: []
meandurationsec: 2.205553
classname: End-to-end
name: 'Subscription updates existing install plan'
counts: 1
details: []
meandurationsec: 0.021273
classname: End-to-end
name: 'Catalog image update'
counts: 1
details: []
meandurationsec: 0.013766
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should have created the expected
prometheus objects'
counts: 1
details: []
meandurationsec: 2.117018

This PR failed tests for 1 times with 1 individual failed tests and 4 skipped tests. A test is considered flaky if failed on multiple commits.

totaltestcount: 1
flaketestcount: 1
skippedtestcount: 4
flaketests:

classname: End-to-end
name: 'CSVs with a Webhook when Installed in a SingleNamespace OperatorGroup
Fails if webhook intercepts Admission Webhook resources'
counts: 1
details:

count: 1
error: |4-
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/webhook_e2e_test.go:115
Unexpected error:
    <*errors.StatusError | 0xc00038eaa0>: {
        ErrStatus: {
            TypeMeta: {Kind: "", APIVersion: ""},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            },
            Status: "Failure",
            Message: "etcdserver: request timed out",
            Reason: "",
            Details: nil,
            Code: 500,
        },
    }
    etcdserver: request timed out
occurred
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/csv_e2e_test.go:3183

meandurationsec: 8.960258
skippedtests:
classname: End-to-end
name: 'Catalog image update'
counts: 1
details: []
meandurationsec: 0.096283
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should install the operator
successfully'
counts: 1
details: []
meandurationsec: 2.162268
classname: End-to-end
name: 'Subscriptions create required objects from Catalogs Given a Namespace
when a CatalogSource is created with a bundle that contains prometheus objects
creating a subscription using the CatalogSource should have created the expected
prometheus objects'
counts: 1
details: []
meandurationsec: 2.474733
classname: End-to-end
name: 'Subscription updates existing install plan'
counts: 1
details: []
meandurationsec: 0.586313

/retest

/retest

@yeya24 you still following up here?

/retest

/retest

Hi, sorry for the late reply. Yes, I can continue following up this pr.  Do I need to change something to pass the CI? Maybe @ecordell or @njhale know?

flaky tests
/retest

Tests faliing are unrelated. Anything else holding us back here?

/retest
/assign @awgreene
anything we can do to get this to proceed?

@awgreene @ecordell Please review and give this PR a lgtm and approve labels if it is ready to go. Thanks.
@yeya24 Can you please create a BZ for this. Thanks.

/assign @RiRa12621

/assign @RiRa12621

/retitle Bug 1916624: Add CsvAbnornalReplacing alerts

/retitle Bug 1916624: Add CsvAbnornalReplacing alerts

@dinhxuanvu I can't retitle but created the bug

@dinhxuanvu I can't retitle but created the bug

/retitle Bug 1916624: Add CsvAbnornalReplacing alerts

/bugzilla refresh

/lgtm

/approve


Use better icons for "armed away" and "armed vacation"

Checklist

[x] I checked for similar existing requests (both open and closed) before posting.
[x] My request is generic, other users may benefit from it too.

Proposal
"armed away" is currently represented as a car driving away and "armed vacation" as a plane taking off. Both are too specific for my taste. I propose using more generic icons:

"armed away" something like: https://fontawesome.com/icons/house-person-leave?s=regular&f=classic
"armed vacation" something like: https://fontawesome.com/icons/island-tropical?s=solid&f=classic

Additional info
Ok, I admit it, I was mainly triggered by the plane. It is bad for the environment ;-)

Being able to change icons would be apricated. Now its also a inconsistency between the Alarmo pnel and icons appearing in Lovelace Tile card.


Searching by tags with "ot" shows no matching notes.

Hi! First of all, thanks for an amazing workflow. :)
I'm having some troubles with using an ot command. I've installed the latest release and both needed plugins; ran osetup.
For some reason ot shows me all of the tags correctly yet after selecting anything it shows nothing:

Smart search works well with tags:

I've tried to debug it, seems like there is a script missing: 🤔  /Users/vitkarpenko/Documents/Alfred/Alfred.alfredpreferences/workflows/user.workflow.D18A8F31-D730-4750-9DB6-147C026C7817/scripts/ot-helper.sh:cd:4: no such file or directory: /Users/vitkarpenko/Library/Application Support/Alfred/Workflow Data/de.chris-grieser.shimmering-obsidian
I'm using Mac M1 and I've installed the workflow through drag'n'drop (double-click gives me an error for some reason) if that helps. Do you have any idea what could be going wrong? :)
Full debug log:
[16:06:47.915] Logging Started...
[16:06:53.191] Shimmering Obsidian[Script Filter] Queuing argument '(null)'
[16:06:53.274] Shimmering Obsidian[Script Filter] Queuing argument '(null)'
[16:06:53.390] Shimmering Obsidian[Script Filter] Script with argv '(null)' finished
[16:06:53.392] Shimmering Obsidian[Script Filter] {"items":[{"title":"#dev/reading","subtitle":"1x","match":"dev reading dev/reading #dev reading dev/reading nested child","uid":"dev/reading","arg":"dev/reading","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23dev%2Freading"}}},{"title":"#ukulele","subtitle":"2x","match":"ukulele ukulele #ukulele ukulele","uid":"ukulele","arg":"ukulele","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23ukulele"}}},{"title":"#dev","subtitle":"4x","match":"dev dev #dev dev","uid":"dev","arg":"dev","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23dev"}}},{"title":"#interview","subtitle":"3x","match":"interview interview #interview interview","uid":"interview","arg":"interview","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23interview"}}},{"title":"#db","subtitle":"1x","match":"db db #db db","uid":"db","arg":"db","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23db"}}},{"title":"#travel","subtitle":"1x","match":"travel travel #travel travel","uid":"travel","arg":"travel","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23travel"}}},{"title":"#python","subtitle":"1x","match":"python python #python python","uid":"python","arg":"python","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23python"}}},{"title":"#go","subtitle":"1x","match":"go go #go go","uid":"go","arg":"go","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23go"}}},{"title":"#lamoda","subtitle":"4x","match":"lamoda lamoda #lamoda lamoda","uid":"lamoda","arg":"lamoda","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23lamoda"}}},{"title":"#self-development","subtitle":"1x","match":"self development self-development #self development self-development","uid":"self-development","arg":"self-development","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23self-development"}}},{"title":"#home","subtitle":"1x","match":"home home #home home","uid":"home","arg":"home","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23home"}}},{"title":"#wishlist","subtitle":"1x","match":"wishlist wishlist #wishlist wishlist","uid":"wishlist","arg":"wishlist","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23wishlist"}}},{"title":"#lamoda/tooling","subtitle":"2x","match":"lamoda tooling lamoda/tooling #lamoda tooling lamoda/tooling nested child","uid":"lamoda/tooling","arg":"lamoda/tooling","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23lamoda%2Ftooling"}}},{"title":"#games","subtitle":"1x","match":"games games #games games","uid":"games","arg":"games","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23games"}}},{"title":"#lamoda/catalog","subtitle":"1x","match":"lamoda catalog lamoda/catalog #lamoda catalog lamoda/catalog nested child","uid":"lamoda/catalog","arg":"lamoda/catalog","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23lamoda%2Fcatalog"}}},{"title":"#backlog","subtitle":"1x","match":"backlog backlog #backlog backlog","uid":"backlog","arg":"backlog","mods":{"cmd":{"arg":"obsidian://search?vault=Notes&query=tag%3A%23backlog"}}}]}
[16:06:54.873] Shimmering Obsidian[Script Filter] Processing complete
[16:06:54.874] Shimmering Obsidian[Script Filter] Passing output 'ukulele' to Run Script
[16:06:54.892] STDERR: Shimmering Obsidian[Run Script] /Users/vitkarpenko/Documents/Alfred/Alfred.alfredpreferences/workflows/user.workflow.D18A8F31-D730-4750-9DB6-147C026C7817/scripts/ot-helper.sh:cd:4: no such file or directory: /Users/vitkarpenko/Library/Application Support/Alfred/Workflow Data/de.chris-grieser.shimmering-obsidian
[16:06:54.893] Shimmering Obsidian[Run Script] Processing complete
[16:06:54.894] Shimmering Obsidian[Run Script] Passing output '' to Arg and Vars
[16:06:54.894] Shimmering Obsidian[Arg and Vars] Processing complete
[16:06:54.895] Shimmering Obsidian[Arg and Vars] Passing output '' to Script Filter
[16:06:54.910] Shimmering Obsidian[Script Filter] Queuing argument '(null)'
[16:06:55.065] Shimmering Obsidian[Script Filter] Script with argv '(null)' finished
[16:06:55.073] Shimmering Obsidian[Script Filter] {"items":[]}

ah yes, this is the good old case of "forgot to create the data folder, before writing into it" 🙈
This line should fix it, download to 2.20.1 and it should work again :)

Thank you. :)


syntax error in format string

line 20 in cypher.py
' -> "

fix is very simple:
return f"[{', '.join(escaped_list)}]" instead of
return f'[{', '.join(escaped_list)}]'
thanks for making this great tool!

My first thought was "how can this happen, I tested it".
Turns out, while you can reuse quotation marks inside format string placeholders with Python 3.12, you cannot on older versions. Pushed a fix. Thank you for reporting it!

thanks for making this great tool!

Cool :) I was a bit surprised that someone is already using it :grin:


Cherry-pick of upstream deps bump

This is to fix GO-2024-2687
See https://github.com/kedacore/keda-olm-operator/pull/224
Checklist

[X] Commits are signed with Developer Certificate of Origin (DCO)

/retest

/retest


make install gives error related to fidesctl distribution not available

Bug Description
A description of what the bug is.
Steps to Reproduce
I have setup the pre-requisites as required for fidesdemo.
When I execute the make install command, it gives me the below error while building the environment.
waghsac@WAGHSAC:~/fidesdemo$ make install
Bringing up docker containers...
WARN[0000] Found orphan containers ([fidesdemo-fidesops-1 fidesdemo-redis-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Running 4/4
⠿ Container fidesdemo-db-1        Started                                                                                1.2s
⠿ Container fidesdemo-fidesctl-1  Started                                                                                2.6s
⠿ Container fidesdemo-redis-1     Started                                                                                1.0s
⠿ Container fidesdemo-fidesops-1  Started                                                                                1.3s
localhost:5432 - accepting connections
Creating virtual environment ./venv...
Installing project dependencies...
Collecting black>=21.9b0 (from -r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/2e/ef/a38a2189959246543e60859fb65bd3143129f6d18dfc7bcdd79217f81ca2/black-22.3.0-py3-none-any.whl
Collecting click>=7.0.0 (from -r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/4a/a8/0b2ced25639fb20cc1c9784de90a8c25f9504a7f18cd8b5397bd61696d7d/click-8.0.4-py3-none-any.whl
Collecting fidesctl==1.2.0 (from -r requirements.txt (line 3))
Could not find a version that satisfies the requirement fidesctl==1.2.0 (from -r requirements.txt (line 3)) (from versions: )
No matching distribution found for fidesctl==1.2.0 (from -r requirements.txt (line 3))
Makefile:32: recipe for target 'install' failed
make: *** [install] Error 1
I'm using Ubuntu 18.04 LTS on Windows.

@conceptualshark can you please help?

@waghsac thank you for reporting this error!  We want to get this routed to the fidesctl team, can you also provide the version of make you're using?

@iamkelllly thanks, will watch. Make version details as below.
waghsac@WAGHSAC:~$ make --version
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
waghsac@WAGHSAC:~$

@iamkelllly thanks, will watch. Make version details as below.
Make version - GNU Make 4.1.

That's very curious, because the fidesctl version it's looking for is right here: https://pypi.org/project/fidesctl/1.2.0/
The install procedure should create a venv and then use the pip version from there. Can you share the output of ./venv/bin/pip --version, e.g.
~/git/fidesdemo% ./venv/bin/pip --version
pip 21.3.1 from /Users/neville/git/fidesdemo/venv/lib/python3.9/site-packages/pip (python 3.9)

can confirm i was able to pip install fidesctl==1.2.0 and pip install -r requirements.txt

@NevilleS
waghsac@WAGHSAC:~/fidesdemo$ ./venv/bin/pip --version
pip 9.0.1 from /home/waghsac/fidesdemo/venv/lib/python3.6/site-packages (python 3.6)

Tried the new version and got the same error.
waghsac@WAGHSAC:~/fidesdemo$ pip install fidesctl[postgres,aws]==1.6.0
Collecting fidesctl[aws,postgres]==1.6.0
Could not find a version that satisfies the requirement fidesctl[aws,postgres]==1.6.0 (from versions: )
No matching distribution found for fidesctl[aws,postgres]==1.6.0

I see- your pip version is quite old (2016), I think it's probably suggesting an upgrade? Let me double-check what the command is for that

OK, you can use pip install --upgrade pip to get a more recent release. I'm not 100% sure that's the problem, but it's a good next step

Updated the pip version but same error. Updated the python version as well but its not reflecting somehow.
Creating virtual environment ./venv...
Installing project dependencies using pip version:
pip 21.3.1 from /home/waghsac/fidesdemo/venv/lib/python3.6/site-packages/pip (python 3.6)
./venv/bin/pip install -r requirements.txt
Collecting black>=21.9b0
Using cached black-22.3.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB)
Collecting click>=7.0.0
Using cached click-8.0.4-py3-none-any.whl (97 kB)
ERROR: Could not find a version that satisfies the requirement fidesctl[aws,postgres]==1.6.0 (from versions: none)
ERROR: No matching distribution found for fidesctl[aws,postgres]==1.6.0
Makefile:65: recipe for target 'install' failed

Thanks @NevilleS @ThomasLaPiana upgrading the pip and python version worked.


Optimize final fields that are inferred as primitives

i.e. this should not generate change detection code:
@Component(
  selector: 'comp',
  template: '{{message}},
)
class Comp {
  final message = 'Hello World';
}

Closed with https://github.com/dart-lang/angular/commit/f819f19f4a93cbc588b897164f4afe0b05e1fc3d


test issue for the thanks bot

This is an issue to test the Azure Function in production

Thank you @dpnl87 you for submitting this issue. We will get back to you shortly!


Add MAGMA implementations of Torch LAPACK functions

This adds MAGMA binding for a number of LAPACK functions. If MAGMA is not installed, the functions simply raise an error.

this looks largely good to go. It has merge conflicts, because I just merged in a PR that refactors the CMakeLists. Look at this PR: https://github.com/torch/cutorch/pull/174

I think all cmake MAGMA logic has to go to lib/THC and FindMAGMA has to be in lib/THC/cmake/CMakeLists.txt
btw should there be a doc how to install it?

I've fixed the bug @dominikgrewe pointed out where a large input could result in the block dim overflowing.
I also rebased and THC should still be separately buildable. The FindMAGMA is required by both cutorch and THC since THCGeneral.h.in defines USE_MAGMA based on whether the package is found.

Are there any more reservations in merging this? Sam addressed dominik's and sergey's comments.

@colesbury merge conflicts again :(

I rebased


Optuna DDP with Slurm Cluster

I was wondering if there is any example for optuna hyperparameter tuning with multi gpu ddp for hyper parameter optimization in Slurm Cluster bacause I could not find one.

As you can see in this repo, no. Optuna's github action does not use GPUs, so slum with GPUs would not be available soon.


"Added some more keywords"

Description of PR

Added some more keywords related to covid19 for safety and precautions

Type of PR

[x] Keywords
[ ] Bugfix
[ ] New feature

Relevant Issues
Fixes #...
Checklist

[ ] Compiles and passes lint tests
[x] Properly formatted
[ ] Tested on desktop
[ ] Tested on phone

Screenshots

https://github.com/covid19india/covid19india-react/issues/662


Satelite service logs hard to locate

If the service is down, you get a dialog that looks like this:

It just says to open the logs, but doesn't say where they are.  In the documentation, I found this:

If the satellite service doesn't work as expected, please use the open service logs folder to locate its logs, and attach the latest one when sending a bug report.

What I eventually realized is that it wants me to click the "open service logs folder" button in the "Satellite Service" section, but that wasn't very obvious.
I think the docs could be improved here to be more clear, and it'd be really nice if the error page had a button that would just open the logs folder too.

My bad; I opened this in the wrong repository.  New issue filed in the right place here.


Cameras duplicate in Home??

Hi ,
I must be doing something really wrong as i can see any cameras in the ui ( but all offline)
The weirdest thing of all is that now i have duplicates in my HOME app
Heres my config
{
"name": "CameraUI",
"port": 3333,
"auth": "none",
"language": "en",
"debug": false,
"reset": false,
"mqtt": {
"active": false
},
"http": {
"active": false,
"localhttp": false
},
"ssl": {
"active": false
},
"cameras": [
{
"name": "Master Bedroom",
"videoConfig": {
"source": "-rtsp_transport tcp -i rtsp://XXXXXXXX/unicast",
"stillImageSource": "-i rtsp://XXXXXXXX/unicast",
"audio": true,
"socketPort": 8100
}
},
{
"name": "Aeglis Room",
"videoConfig": {
"source": "-rtsp_transport tcp -i rtsp://XXXXXXXX/unicast",
"stillImageSource": "-i rtsp://XXXXXXXX/unicast",
"audio": true,
"socketPort": 8100
}
},
{
"name": "Office",
"videoConfig": {
"source": "-rtsp_transport tcp -i rtsp://XXXXXXXX/unicast",
"stillImageSource": "-i rtsp://XXXXXXXX/unicast",
"audio": true,
"socketPort": 8100
}
}
],
"platform": "CameraUI"

Duplicates? Remove your cache folder (cachedAccessories) and restart homebridge

Hi Seydx,
Do you mean this file?

This is what i meant by duplicates

Yes, but this will delete all your homebridge accessories so you need to bind it again to homekit

So that means i will need to recreate automations relevant to Homebridge etc?
Is there another way maybe?

@Mikellas
Do you using a another plugin next to camera ui? Maybe homebridge-camera-ffmpeg?
You should be able to only remove the cameras by tapping on them > details > Remove

Cant remove camera no option for it
![Uploading A6CF0EE1-319F-4BAD-9075-C65F3D2F51D9.png…]
Yes i do use ffmpeg for the same cameras

With ffmpeg do you the homebridge camera plugin (homebridge-camera-ffmpeg) ?
If yes, pls remove it from your config and from homekit

So i understand clearly do you want me to remove ffmpeg completely ?

Yes that means Seydx. You don't need the homebridge-ffmpeg plugin for using Homebridge-camera-ui.

If you using HOMEBRIDGE-CAMERA-FFMPEG, pls remove it
NOT FFMPEG

Ok cool didnt know
I removed it now cameras display correctly but im getting error 1 from ffmpeg now and cameras arent responding in the GUI or in Homekit

Can you trie to reinstall ffmpeg.
sudo apt-get install ffmpeg
sudo reboot

Reinstalled FFmpeg
Still getting the below error when trying to view cameras in Homekit
[CameraUI] ffmpeg exited with code: 1 and signal: null (Error) Aeglis Room
[10/19/2020, 10:04:36 PM] [CameraUI] Aeglis Room: Stopped video stream.

@Mikellas
Can u set audio to false pls and try again

@SeydX Cool that did the trick in Homekit.Live video and snapshots work now
Camera Ui still says that all cameras are offline though,no errrors in debug

@Mikellas can u post your config.json pls??

{
"name": "CameraUI",
"port": 3333,
"auth": "none",
"language": "en",
"debug": true,
"reset": false,
"mqtt": {
"active": false
},
"http": {
"active": false,
"localhttp": false
},
"ssl": {
"active": false
},
"cameras": [
{
"name": "Master Bedroom",
"videoConfig": {
"source": "-rtsp_transport tcp -i rtsp://<IP_ADDRESS>:8554/unicast",
"stillImageSource": "-rtsp_transport tcp -i rtsp://<IP_ADDRESS>:8554/unicast",
"forceMax": true,
"vcodec": "copy",
"audio": false,
"socketPort": 8100
}
},
{
"name": "Aeglis Room",
"videoConfig": {
"source": "-rtsp_transport tcp -i rtsp://<IP_ADDRESS>:8554/unicast",
"stillImageSource": "-rtsp_transport tcp -i rtsp://<IP_ADDRESS>:8554/unicast",
"forceMax": true,
"vcodec": "copy",
"audio": false,
"socketPort": 8200
}
},
{
"name": "Office",
"videoConfig": {
"source": "-rtsp_transport tcp -i rtsp://<IP_ADDRESS>:8554/unicast",
"stillImageSource": "-rtsp_transport tcp -i rtsp://<IP_ADDRESS>:8554/unicast",
"forceMax": true,
"vcodec": "copy",
"audio": false,
"socketPort": 8300
}
}
],
"platform": "CameraUI"
}

the problem is the socketPort you use in your config.json
each camera must have an UNIQUE socketPort, you have set the same socketPort for all cams

@SeydX You probably saw my previous json post? Update above shows the actual ones i use now (8100,8200,8300)

Hmmm.. Can u show me your browser console pls? Example (Google Chrome)

Ok thats good so far
Can you put DEBUG=CameraUI* into your homebridge environment file?
If you are using Config UI X you can do it like below

Or you can edit the environment file manually, the file shoud be located under
/etc/default/homebridge
# Defaults / Configuration options for homebridge
# The following settings tells homebridge where to find the config.json file and where to persist the data (i.e. pairing and others)
HOMEBRIDGE_OPTS=-I -U /var/lib/homebridge

#To enable web terminals via homebridge-config-ui-x
HOMEBRIDGE_CONFIG_UI_TERMINAL=1

# If you uncomment the following line, homebridge will log more
# You can display this via systemd's journalctl: journalctl -f -u homebridge
DEBUG=CameraUI

I did it with the fist way you told me but no logs turn up for your plugin while i use the gui
I can see the rest plugins reports i can send but i believe it wont help you in any way
One thing i forgot to mention though is that i have authentication mode to none because i tried using the form and the default admin credentials didnt work even after resetting

That the authentication form is on "none" should not be the problem. But more important is why the default login data (admin:admin) did not work.
How did you set log for config-ui-x? systemd or file?

Nah thats wrong, as you seen in the GIF, in config-ui-x you must define ONLY CameraUI* (without DEBUG=)

omg sorry about that..
2020-10-19T21:14:01.129Z CameraUIInterface {
userID: false,
message: 'session redirect',
originalUrl: '/',
authenticated: false,
noAuth: false
}
2020-10-19T21:14:01.129Z CameraUIInterface {
userID: false,
message: 'ensure authenticated',
originalUrl: '/',
authenticated: false,
noAuth: false
Below once i try to log in

Could you try first without authentification pls to debug the not working livestreams

2020-10-19T21:18:40.248Z CameraUIInterface Master Bedroom: Offline
2020-10-19T21:18:40.531Z CameraUISocket admin disconnected!
2020-10-19T21:18:40.531Z CameraUISocket { admin: { devices: 0 } }
2020-10-19T21:18:40.753Z CameraUISocket admin connected!
2020-10-19T21:18:40.753Z CameraUISocket { admin: { devices: 1 } }
2020-10-19T21:18:40.764Z CameraUISocket {
username: 'admin',
currentUrl: '/camera/Master%20Bedroom',
targetUrl: false,
back: false
}

2020-10-19T21:20:40.998Z CameraUIInterface {
userID: '3be9895a-be63-44a9-8b75-4495ef5d9b7e',
message: 'session redirect',
originalUrl: '/camviews',
authenticated: false,
noAuth: true
}
2020-10-19T21:20:40.998Z CameraUIInterface {
userID: '3be9895a-be63-44a9-8b75-4495ef5d9b7e',
message: 'ensure authenticated',
originalUrl: '/camviews',
authenticated: false,
noAuth: true
}
2020-10-19T21:20:40.998Z CameraUIInterface admin (Master): Authentication skipped!
2020-10-19T21:20:40.998Z CameraUIApp {
userID: '3be9895a-be63-44a9-8b75-4495ef5d9b7e',
message: 'locals',
url: '/camviews',
authenticated: false,
noAuth: true
}
2020-10-19T21:20:40.998Z CameraUIInterface Trying to ping Master Bedroom
2020-10-19T21:20:41.526Z CameraUIInterface Master Bedroom: Offline
2020-10-19T21:20:41.528Z CameraUIInterface Trying to ping Aeglis Room
2020-10-19T21:20:42.042Z CameraUIInterface Aeglis Room: Offline
2020-10-19T21:20:42.043Z CameraUIInterface Trying to ping Office
2020-10-19T21:20:42.550Z CameraUIInterface Office: Offline
2020-10-19T21:20:42.740Z CameraUISocket admin disconnected!
2020-10-19T21:20:42.740Z CameraUISocket { admin: { devices: 1 } }
2020-10-19T21:20:42.740Z CameraUISocket admin disconnected!
2020-10-19T21:20:42.740Z CameraUISocket { admin: { devices: 0 } }
2020-10-19T21:20:42.961Z CameraUISocket admin connected!
2020-10-19T21:20:42.961Z CameraUISocket { admin: { devices: 1 } }
2020-10-19T21:20:42.967Z CameraUISocket admin connected!
2020-10-19T21:20:42.967Z CameraUISocket { admin: { devices: 2 } }
2020-10-19T21:20:43.037Z CameraUISocket {
username: 'admin',
currentUrl: '/camviews',
targetUrl: false,
back: false
This when i press on CamViews

hm as i thought, the plugin can not ping the cameras, i will check this thanks!

No problem if you need anything else let me know camera models etc

I fixed the not found error, looking now for the ping issue

v3.0.9 is online
could you try pls?

Hi,
I have tried and all is good!!
Doesn this mean i can enable sound as well?
Thanks

It was the camera settings after all
All seems ok now


plan to support FaceNet or OpenFace?

Great utility!
Do you have a plan to support FaceNet or OpenFace also?
The models have nn.SpatialConvolutionMM and nn.Inception, which are not supported yet :(
Thank you!

nn.SpatialConvoltuionMM can drop in replacement for nn.SpatialConvoltion.
Modify the lines:
if name == 'SpatialConvolution':
to
if name == 'SpatialConvolution' or name == 'SpatialConvolutionMM' :
nn.Inception is not a torch standard layer. PyTorch can not load it, this script is hard to support it.


merge package from arlac77/npm-package-template

package.json

chore(package<EMAIL_ADDRESS>chore(scripts): cover@#overwrite c8 --temp-directory build/coverage ava && c8 report -r lcov --temp-directory build/coverage
chore(package): add nyc from template
docs(package): remove keyword null
chore(package): set $.ava.require='esm' as in template
chore(package): set $.ava.files='tests/-test.js,tests/-test.mjs' as in template
chore(package): set $.ava.extensions='js,mjs' as in template

Codecov Report

Merging #663 into master will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #663      +/-   ##
==========================================
+ Coverage   92.91%   92.96%   +0.04%
==========================================
  Files           7        7
  Lines        1073     1066       -7
==========================================
- Hits          997      991       -6
+ Misses         76       75       -1

Impacted Files
Coverage Δ

src/service-config.js
92.59% <0%> (-0.14%)
:arrow_down:

src/service.js
90.73% <0%> (-0.02%)
:arrow_down:

src/endpoints-mixin.js
97.96% <0%> (-0.02%)
:arrow_down:

src/service-logger.js
100% <0%> (ø)
:arrow_up:

src/interceptor-provider-mixin.js
100% <0%> (ø)
:arrow_up:

src/service-consumer-mixin.js
100% <0%> (ø)
:arrow_up:

src/service-provider-mixin.js
89.77% <0%> (+0.39%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2776253...f69b177. Read the comment docs.

Coverage increased (+0.02%) to 89.761% when pulling f69b177866c9990f738a8d682fade51222821d4d on template-sync-1 into 2776253d5c9cdb779278351ebd828c83c331fefe on master.

:tada: This PR is included in version 5.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[Odnoklassniki/ok.ru]Formats detection does not work on some videos

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that a supported site is broken
[X] I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
France
Provide a description that is worded well enough to be understood
Hi,
Prior to playing a video with mpv --ytdl-format, I always chose among the SD formats listed by the command yt-dlp --no-config --list-formats https://ok.ru/video/4794078595693. But it does not work on this url.
Thanks a lot for this great tool and for all your work and help you have done so far :+1:
God bless you.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', '--no-config', '-F', 'https://ok.ru/video/4794078595693']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[392389b7d] (pip)
[debug] Python 3.8.10 (CPython x86_64 64bit) - Linux-5.15.0-60-generic-x86_64-with-glibc2.29 (OpenSSL 1.1.1f  31 Mar 2020, glibc 2.31)
[debug] exe versions: ffmpeg 4.4.1-static (setts), ffprobe 4.4-1ppa1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2020.12.05, mutagen-1.45.1, secretstorage-2.3.1, sqlite3-2.6.0, websockets-10.0
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>yt-dlp is up to date<EMAIL_ADDRESS>[Odnoklassniki] Extracting URL: https://ok.ru/video/4794078595693
[Odnoklassniki]<PHONE_NUMBER>693: Downloading desktop webpage
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for<PHONE_NUMBER>693:
ID EXT RESOLUTION │ PROTO │ VCODEC  ACODEC
───────────────────────────────────────────
hd mp4 unknown    │ https │ unknown unknown

Is your problem that you are only being offered a format called hd with unknown resolution, possibly because that's the only available format?
Or that this format is not available using mpv --ytdl-format=hd ...? WFM, but I have no idea whether mpv is calling yt-dl or yt-dlp under the hood.

There are other formats available in the website player that are not being extracted. This should fix:
diff --git a/yt_dlp/extractor/odnoklassniki.py b/yt_dlp/extractor/odnoklassniki.py
index 4b73eed37..8462e7573 100644
--- a/yt_dlp/extractor/odnoklassniki.py
+++ b/yt_dlp/extractor/odnoklassniki.py
@@ -367,12 +367,17 @@ def _extract_desktop(self, url):
             'format_id': f['name'],
         } for f in metadata['videos']]

-        m3u8_url = metadata.get('hlsManifestUrl')
+        m3u8_url = traverse_obj(metadata, 'hlsManifestUrl', 'ondemandHls')
         if m3u8_url:
             formats.extend(self._extract_m3u8_formats(
                 m3u8_url, video_id, 'mp4', 'm3u8_native',
                 m3u8_id='hls', fatal=False))

+        mpd_url = metadata.get('ondemandDash')
+        if mpd_url:
+            formats.extend(self._extract_mpd_formats(
+                mpd_url, video_id, mpd_id='dash', fatal=False))
+
         dash_manifest = metadata.get('metadataEmbedded')
         if dash_manifest:
             formats.extend(self._parse_mpd_formats(

Duplicate of #4462


build: add CMake option LIBUV_BUILD_SHARED to enable/disable shared library

Fix #3637

bump?

On a related note, would it be possible to build the static library as just uv (instead of uv_a) when LIBUV_BUILD_SHARED=OFF? This would allow applications like Neovim to statically link both LibUV and LUV (which currently is only possible with major contortions).

Not from our experience.
We are successfully able to apply this PR via a patch. See https://github.com/neovim/neovim/pull/19632

Not from our experience.
We are successfully able to apply this PR via a patch. See neovim/neovim#19632

Looks related to BUILD_TESTING config.

Bad bot. Bump?

when turned off, default build will fail.
/workspace/libuv/test/blackhole-server.c:22:10: fatal error: uv.h: No such file or directory 22 | #include "uv.h" | ^~~~~~ compilation terminated.

@clason

Again, we build LibUV fine with with LIBUV_BUILD_SHARED=OFF, see https://github.com/neovim/neovim/blob/37a71d1f28e5d7fd13f0ede69b4d2558157a9e4b/cmake.deps/cmake/BuildLibuv.cmake#L9
So there's something else going on. Can you share a complete build invocation?

Again, we build LibUV fine with with LIBUV_BUILD_SHARED=OFF, see<EMAIL_ADDRESS>So there's something else going on. Can you share a complete build invocation?

Just set line 31 to OFF. And build
mkdir build
cd build
cmake -G Ninja ..
cmake --build .

console
gitpod /workspace/libuv/build (cmake-build-shared-option) $ cmake --build .
[26/408] Building C object CMakeFiles/uv_run_tests.dir/test/test-getsockname.c.o
FAILED: CMakeFiles/uv_run_tests.dir/test/test-getsockname.c.o
/usr/bin/cc -DUSING_UV_SHARED=1 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200112  -fvisibility=hidden -Wstrict-prototypes -Wextra -Wall -Wno-unused-parameter -fno-strict-aliasing -std=gnu90 -MD -MT CMakeFiles/uv_run_tests.dir/test/test-getsockname.c.o -MF CMakeFiles/uv_run_tests.dir/test/test-getsockname.c.o.d -o CMakeFiles/uv_run_tests.dir/test/test-getsockname.c.o -c /workspace/libuv/test/test-getsockname.c
/workspace/libuv/test/test-getsockname.c:22:10: fatal error: uv.h: No such file or directory
   22 | #include "uv.h"
      |          ^~~~~~
compilation terminated.
[43/408] Building C object CMakeFiles/uv_run_benchmarks_a.dir/test/benchmark-fs-stat.c.o
ninja: build stopped: subcommand failed.

That looks like a test that fails, not the library itself? (Which is not as surprising that those would need to be adapted.)

@gengjiawen Do I understand you correctly that it would be acceptable if LIBUV_BUILD_SHARED disabled the tests?

Merged in https://github.com/libuv/libuv/pull/3762


Fix reference tracking when using nested dictionaries/objects under py/state

Fixes #500
This PR causes a dict returned from __getstate__ (and stored in py/state) to no longer be restored twice, which previously caused any objects in it to be incorrectly duplicated in the Unpickler's list of object instances.

Thanks for this enhancement. I'll wait until @Theelx has a chance to review. Also, we're going to want to tag and release v3.1.0 before merging this (unless Theelx thinks otherwise).
Let me know what you think about the restore_key() thing and then we can start planning the next release and subsequent merge.

Thanks for this enhancement. I'll wait until @Theelx has a chance to review. Also, we're going to want to tag and release v3.1.0 before merging this (unless Theelx thinks otherwise).
Let me know what you think about the restore_key() thing and then we can start planning the next release and subsequent merge.

This seems like something that we'd want in 3.1.0 rather than 3.1.1 (i.e. sooner rather than later). I think this could fix at least a few other open issues, so delaying it possibly a few weeks/months until 3.1.1 might not be the best. Also, if it does end up somehow accidentally breaking someone's code (unlikely), then we could still revert it and quickly release a 3.1.1 until we can figure out how to resolve it.

This seems like something that we'd want in 3.1.0 rather than 3.1.1 (i.e. sooner rather than later).
...
Also, if this PR does break something, I think it's better to have it in a minor release rather than a bugfix one

Out of extra caution I tagged v3.1.0 before this merge and I'll tag v3.2.0 now with just this merge in it after writing up a changelog entry for this one.
I should have read my older emails first ;-) Thanks for reviewing!


DOM renderer can get into state where rows aren't redrawn

bootstrap-window.js:212 [uncaught exception]: TypeError: Cannot read property 'length' of undefined
onUnexpectedError @ bootstrap-window.js:212
bootstrap-window.js:215 TypeError: Cannot read property 'length' of undefined
    at e.createRow (DomRendererRowFactory.ts:50)
    at t.renderRows (DomRenderer.ts:356)
    at t._renderRows (RenderService.ts:91)
    at e._renderCallback (RenderService.ts:46)
    at e._innerRefresh (RenderDebouncer.ts:11)
    at RenderDebouncer.ts:52
onUnexpectedError @ bootstrap-window.js:215
DomRendererRowFactory.ts:50 Uncaught TypeError: Cannot read property 'length' of undefined
    at e.createRow (DomRendererRowFactory.ts:50)
    at t.renderRows (DomRenderer.ts:356)
    at t._renderRows (RenderService.ts:91)
    at e._renderCallback (RenderService.ts:46)
    at e._innerRefresh (RenderDebouncer.ts:11)
    at RenderDebouncer.ts:52

Source maps appear to be messed up, it's probably talking about lineData?

Repro on mac:
ls
ll

cmd+shift+up

I think the bug here is actually related to markers/command tracking, rather than the dom renderer.

Don't think this was a regression, just always happened when terminal rows were odd 🤦♂


Remove the cache from the provider.

This removes the cache added in #94. It's possible that there are use cases that I don't see (and which aren't covered by tests) where it's used, but it looks like we shouldn't call the provider multiple times for the same request?

I imagine this was necessary before your item building refactoring, where resolution happened at different stages and the same provider could be called multiple times, and it is not needed anymore. But I suspect @BurnzZ might have a better view here (or/and I might be mixing different caches in my mind).

This coincides with https://github.com/scrapy-plugins/scrapy-zyte-api/pull/161/ where we still kinda need the cache, albeit replaced with the new one in https://github.com/scrapinghub/scrapy-poet/pull/184.

Ahh I think the lines like self.update_cache(request, {BrowserHtml: html}) aren't needed anymore. Perhaps it's best to have this after https://github.com/scrapy-plugins/scrapy-zyte-api/pull/161/ is merged?

Do we only need the scrapy-poet one then?

Is this still valid?

Done in #161


Website design improvement/changes

This is a Feature Request
I have some ideas for changes/improvements to the website design.

What would you like to be added

Changing the hero section, this is just an idea for a background for visual improvement.
Adding the project logo and adding some text information about the project ( 2-3 lines ) to the hero-section would make a good viewpoint when the website loads.
Also along with the 'learn k8s basics' button, we can add other important links as well like github, slack etc. Currently those buttons are at the bottom of the page, adding them at the top as well can make it easier to access important links.

- another idea, adding a card like structure to some sections would improve the visualisation and provide structure as well.
Other ideas
Some more changes i can think of are:

improving the video-section for better visibility and look
adding card format like features(above) to case studies and  'we are a graduated project' section
improving the kubeweekly subscribe section to utilise space and look better, we can also add a background like the hero-title section to this section as well, that can make it look better.

Why is this needed

better usability of the website
improved design and look of the website
better utilisation of space and providing structure to sections

Comments

/assign

/sig-docs

Here's my opinion on this issue:

There's no need to add something to the hero section (or modify anything until you come across broken links/ bad redirecting link) because it's already working fine. However, the blue color looks good.
I don't think there's need to add a separate link mentioning about slack, GitHub because these links are already visible at the end of the page. Even if you go through the CNCF website, it also follows the same approach for mentioning a link at the end of the introduction page.
For adding a card like structure, it may or may not help in solving this issue

/area web-development

I like the card-like structure idea. How about splitting that into its own issue?

Here's my opinion on this issue:

There's no need to add something to the hero section (or modify anything until you come across broken links/ bad redirecting link) because it's already working fine. However, the blue color looks good.
I don't think there's need to add a separate link mentioning about slack, GitHub because these links are already visible at the end of the page. Even if you go through the CNCF website, it also follows the same approach for mentioning a link at the end of the introduction page.
For adding a card like structure, it may or may not help in solving this issue

ok we can let the hero section remain as it is.
Yeah the blue color looks good. We can add the blue background to footer as well, that will maintain consistency.
For the card structure, i couldn't understand the reference to the issue you mentioned.

Here's my opinion on this issue:

There's no need to add something to the hero section (or modify anything until you come across broken links/ bad redirecting link) because it's already working fine. However, the blue color looks good.
I don't think there's need to add a separate link mentioning about slack, GitHub because these links are already visible at the end of the page. Even if you go through the CNCF website, it also follows the same approach for mentioning a link at the end of the introduction page.
For adding a card like structure, it may or may not help in solving this issue

ok we can let the hero section remain as it is. Yeah the blue color looks good. We can add the blue background to footer as well, that will maintain consistency. For the card structure, i couldn't understand the reference to the issue you mentioned.

Go with the suggestions of @sftim and break down it into issues.
Mentioning about this, the idea in that issue is to create a page to list some blogs. It'd be better to implement (something like a card structure) once that page has been created. Long way to go before that I guess.

Here's my opinion on this issue:

There's no need to add something to the hero section (or modify anything until you come across broken links/ bad redirecting link) because it's already working fine. However, the blue color looks good.
I don't think there's need to add a separate link mentioning about slack, GitHub because these links are already visible at the end of the page. Even if you go through the CNCF website, it also follows the same approach for mentioning a link at the end of the introduction page.
For adding a card like structure, it may or may not help in solving this issue

ok we can let the hero section remain as it is. Yeah the blue color looks good. We can add the blue background to footer as well, that will maintain consistency. For the card structure, i couldn't understand the reference to the issue you mentioned.

Go with the suggestions of @sftim and break down it into issues.
Mentioning about this, the idea in that issue is to create a page to list some blogs. It'd be better to implement (something like a card structure) once that page has been created. Long way to go before that I guess.

ok yeah i'll make new issues.
for the card structure, i think we can implement that on the homepage initially and on some other pages like community etc after that and then we can add them to the blogs page whenever it is created. while working on that, i'll create a css class for the card structure which we can then simply add to the sections required along with some modification if needed.

Yes, please go ahead! Let's see what you bring to the discussion table:)

I have made a PR for the background changes.

Apart from the background and card structure changes, i had mentioned a couple of other ideas as well.

improving the video section. As shown in the ss below, the heading is not clearly visible and readable. Also the button size is too large which overpowers the text, ideally the buttons work as call-to-action and should be smaller in size comparable to the heading and text. So that's one thing we should sort out.
--> The solution to the visibility part can be making some design changes to the section, like adding a shadow behind the text( or dimming the image) or choosing another image which is less bright and then making changes to the heading and buttons accordingly.

increasing the 'we are a cncf graduated project' part for better emphasis on that particular part ( just a small change suggestion)
improving the sign-up for kubeweekly section. We can add a background like the one in hero part or something similar to the section and changing the button, input box size along with minor style changes.

After working on the homepage, improvements can be made on other pages as well like community, case studies, partners etc. Improvements like adding card structure where suitable, making style and design changes. These can be discussed after the current changes are worked upon.

Also regarding the background changes, i have received positive feedback for that so we can discuss upon that as well if any changes are required for that.

/sig docs

I'd still recommend splitting out the ideas into more than one issue. Then, we (SIG Docs) can triage and accept the ideas that folks are happy with, and also different people can work on implementations in parallel.

Ok. Let's keep this issue as the background change. I'll create new issues for the other ideas.

/retitle Alter background for page headings

I'm happy with the existing pattern that we use as a background for headings (when there is no banner active). Why should we change it?

@reveurguy, if you can explain the reason to change, that'll help people work out what design to adopt.


Same code local work, but not on server ?

I have write some simple code to send notification to IOS device as the guide, and it work on my local mac machine, but when i upload it to ubuntu server it did not work, the debug log seems nothing wrong, but can not receive the message
apn Initialising connection +0ms
  apn Loading Credentials +3ms
  apn 1 left to send +5ms
  apn Connection established +939ms
Connected
  apn Socket drained +103ms 0
Notification transmitted to:6e6650ass2bafff7bf729ba2bb1337e23fdc7279e004b67dc01c3f4e2b71e0d6
  apn 0 left to send +1ms
  apn 0 left to send +1ms

does any body have come across the same issue ?

Have you ensured that the process is running long enough to check for errors received from Apple?

On 25 Aug 2015, at 18:32, Wang<EMAIL_ADDRESS>wrote:
I have write some simple code to send notification to IOS device as the guide, and it work on my local mac machine, but when i upload it to ubuntu server it did not work, the debug log seems nothing wrong, but can not receive the message
apn Initialising connection +0ms
apn Loading Credentials +3ms
apn 1 left to send +5ms
apn Connection established +939ms
Connected
apn Socket drained +103ms 0
Notification transmitted to:6e6650ass2bafff7bf729ba2bb1337e23fdc7279e004b67dc01c3f4e2b71e0d6
apn 0 left to send +1ms
apn 0 left to send +1ms
—
Reply to this email directly or view it on GitHub https://github.com/argon/node-apn/issues/302.

Now i can get the notification, not do any changing, i think maybe there are some issue when we build the app, or one failed notification send has block the later ones.


BUG: cant' stop saving after runing GpChatNew

with all default configuration
https://github.com/Robitx/gp.nvim/assets/62557596/f4f43491-2e9e-4047-a28b-e716ec4349db

Okay, I think I figure where the bug is out...
In this line
https://github.com/Robitx/gp.nvim/blob/607f94d361f36b8eabb148d95993604fdd74d901/lua/gp/init.lua#L1370
we call the function to imitate we hit the <ESC> key after event BufEnter.
Meanwhile, I have this keymap
nmap("<ESC>", "<CMD>w|e|redraw<CR>")

which do a lot including reenter  the buffer. So we are trapped in an endless loop.... 😢

I made a PR to resolve the bug


--securitygroup flag for disco_aws.py

Pushed to master rather than branch. So opening a PR to view change in reverse pattern.. so we can get it code reviewed.

Can you elaborate on why this is required?

This is not required. Its a creature comfort. It aids in troubleshooting problems. You want to correlate problematic instances to other configs. This gives you some insight on sg and metanetwork that the instance is in. Previously you could infer this from IPs (although even this was bothersome), today with dynamic IP allocation to metanetworks its not easy.

👍

Since this is already on master I'm just closing this PR.


SFTTrainer device error even though it doesn't take device as an argument

As the title suggests. I was trying to fine-tune gpt2 model, the pipeline should be pretty standard (https://github.com/zyzhang1130/agentscope/blob/conversation_with_agent_with_finetuned_model/examples/conversation_with_agent_with_finetuned_model/huggingface_model.py). Got the following error:
Exception has occurred: TypeError
device() received an invalid combination of arguments - got (NoneType), but expected one of:
 * (torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType)
 * (str type, int index)
  File "/home/zy1130/agentscope/examples/conversation_with_agent_with_finetuned_model/huggingface_model.py", line 553, in _fine_tune_training
    trainer.train()
  File "/home/zy1130/agentscope/examples/conversation_with_agent_with_finetuned_model/huggingface_model.py", line 110, in __init__
    self.model = self._fine_tune_training(
  File "/home/zy1130/agentscope/src/agentscope/models/__init__.py", line 117, in load_model_by_config_name
    return _get_model_wrapper(model_type=model_type)(**kwargs)
  File "/home/zy1130/agentscope/src/agentscope/agents/agent.py", line 195, in __init__
    self.model = load_model_by_config_name(model_config_name)
  File "/home/zy1130/agentscope/src/agentscope/agents/dialog_agent.py", line 45, in __init__
    super().__init__(
  File "/home/zy1130/agentscope/examples/conversation_with_agent_with_finetuned_model/FinetuneDialogAgent.py", line 46, in __init__
    super().__init__(
  File "/home/zy1130/agentscope/src/agentscope/agents/agent.py", line 82, in __call__
    instance = super().__call__(*args, **kwargs)
  File "/home/zy1130/agentscope/examples/conversation_with_agent_with_finetuned_model/conversation_with_agent_with_finetuned_model.py", line 78, in main
    dialog_agent = FinetuneDialogAgent(
  File "/home/zy1130/agentscope/examples/conversation_with_agent_with_finetuned_model/conversation_with_agent_with_finetuned_model.py", line 123, in <module>
    main()
TypeError: device() received an invalid combination of arguments - got (NoneType), but expected one of:
 * (torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType)
 * (str type, int index)

It is strange, as SFTTrainer does not have device as its argument, so there seems to be nothing I can do about it..

Hey, thanks for sharing. How do you run the provided code? I can't reproduce the error. If you still encounter the error, please provide a minimal code to reproduce it.

Hey, thanks for sharing. How do you run the provided code? I can't reproduce the error. If you still encounter the error, please provide a minimal code to reproduce it.

currently, I'm not sure how to get a minimal code that gives the same error, as it is part of a pipeline. What I have found out, however, is this error only occurs if I use QLoRA (LoRA, i.e., without quantizing the model to 4 bit does not give this error). The QLoRA config used is pretty standard:
 "bnb_config": {
                         "load_in_4bit": True,
                         "bnb_4bit_use_double_quant": True,
                         "bnb_4bit_quant_type": "nf4",
                         "bnb_4bit_compute_dtype": "torch.bfloat16",
                     },

were you able to figure this one out . I am having the same issue


"Automower Connect API" not reachable. (ERR_#003)

The problem
Hello
I found this adapter yesterday and it instantly worked after setup - great stuff.
Today, I only get the following error:
"Automower Connect API" not reachable. (ERR_#003)
Tried to manually test the api using Postman which returns "Invalid client: cannot retrieve client credentials"
Am I doing something wrong or is there a general problem with the connect api?
Thanks
Dominik
Version of Node.js
v16.14.0
Version of ioBroker js-controller
4.0.23
Version of "husqvarna-automower"-adapter
0.2.0
Version of "admin"-adapter
6.1.9
Operating system incl. version running ioBroker
Linux (Docker) on Synology
Checklist of files to include below

[X] iobroker.current.log (Please share the logs in debug mode and check that there are no personal information)
[ ] Contents of the JavaScript browser console (always include in cases of issues with the user interface)

Additional information & file uploads

husqvarna-automower.0
2022-07-04 08:47:29.699
error
Error: "Automower Connect API" not reachable. (ERR_#003)

husqvarna-automower.0
2022-07-04 08:47:29.423
info
Mowerdata initially saved...

husqvarna-automower.0
2022-07-04 08:47:29.064
info
"Husqvarna Authentication API Access token" received.

husqvarna-automower.0
2022-07-04 08:47:28.844
info
starting adapter "husqvarna-automower"...

husqvarna-automower.0
2022-07-04 08:47:28.813
info
starting.  Version 0.2.0 (non-npm:  ice987987/ioBroker.husqvarna-automower#7adf27a0eff3eb8fd12783a03774f4540f9bdaeb)  in /opt/iobroker/node_modules/iobroker.husqvarna-automower, node:  v16.14.0, js-controller: 4.0.23

Sorry it is now working.
I recreated the API secrets and it solved the problem.


Encryption support

Hi, by using this library, wondering if there's any option to encrypt the entire sqlite at rest?

Duplicate of https://github.com/craftzdog/react-native-sqlite-2/issues/32


Fixed wrong location of gflags license file

With the latest master it seems gflags tries to pick up the license file in the wrong spot.
cc @ras0219-msft

Hi @willyd, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

Good catch. Fixing other portfiles to use the SOURCE_PATH variable to avoid issues like this.


Swift 4.2 Compilation errors trying to build in Debug

In the latest swift migration there are two lies of code in #IF DEBUG blocks that weren't migrated along with the rest. This causes compilation errors when trying to build in debug.
I have the fix done locally for normal development, but it would be nice to fix for first pod installs and the like.
The problems are in MemberQuery.swift on line 48 and PredicateBuilder.swift on line 150. Both require changing String(type) to String(describing: type)

Thanks @NerdBird! I'll merge this right now and submit a release

Quick update: Apparently, past Hector decided it would be a good idea to convert the library to support type safe key paths and got a new computer since. It's almost done but I'll keep you updated!

Alright cool it's in master now @NerdBird! I need to run the upload tool for publishing to Cocoapods but it should already start working for Swift Package Manager and Carthage if you're using either of those

Awesome! Thanks, pulling from master now and its all good for me 👍


Support devices with multiple interfaces

A few fixes that should hopefully resolve #4, see the last patch. Devices with multiple interfaces should now work properly.
With these changes, I'm able to proxy the K12 keyboard without issues.
I think devices that switch between configurations/alt settings will still glitch: endpoint handling threads are not joined properly as they are blocked on receive_data(). But this issue is present in the current code and will require another fix.

Hi @xairy, thank you very much for the PR! I just came back from a business trip, will try to find some time to review it, thanks again!

hi @xairy I test the code https://github.com/AristoChen/usb-proxy/pull/6, there two issues.

my mouse is a low speed device, and it response the MaxPageSize0 is 0x08, it should change the MaxPackageSize0 0x40, 0x08 make the raw_gadget which use device musb-hdrc.4.auto dead loop,
ignore usb_raw_ep0_stall when SET_IDLE request to interface 1，it like in linux don't support this request, in proxy.cpp:499
event: control, length: 8
bRequestType: 0x21 (OUT), bRequest: 0x0a, wValue: 0x0000, wIndex: 0x0001, wLength: 0
type = USB_TYPE_CLASS
req = unknown = 0xa
ep0: stalling

I add this code in ep0_loop
index e3dbd57..d703445 100644
--- a/proxy.cpp
+++ b/proxy.cpp
@@ -368,6 +368,12 @@ void ep0_loop(int fd) {
break;
}
}

                          if ((event.ctrl.bRequestType & USB_TYPE_MASK) == USB_TYPE_STANDARD &&

                                  event.ctrl.bRequest == USB_REQ_GET_DESCRIPTOR &&

                                  (event.ctrl.wValue >> 8) == USB_DT_DEVICE) {

                                  struct usb_device_descriptor* pdata = (struct usb_device_descriptor*)&io.data;

                                  pdata->bMaxPacketSize0 = 64;

                          }

                          if (verbose_level >= 2)
                                  printData(io, 0x00, "control", "in");

@@ -495,7 +501,14 @@ void ep0_loop(int fd) {
printf("ep0: transferred %d bytes (out)\n", rv);
}
else {

                                  usb_raw_ep0_stall(fd);

                                  if (event.ctrl.bRequestType == 0x21

                                          && event.ctrl.bRequest == 0x0a

                                          && event.ctrl.wIndex != 0) {

                                          continue;

                                  }

                                  else {

                                          usb_raw_ep0_stall(fd);

                                  }
                          }
                  }
          }

Hi @imatespl,
Are these issues reproducible with Dummy HCD?
Which device are you trying to proxy? Please share an Amazon link or something like that.
Which Linux board do you use for proxying?
I'll be busy until June, but after that I can try to look into this.
Thanks for testing!

Hi @xairy,
My mouse is logitech m170 https://www.amazon.com/Logitech-Wireless-Mouse-M170-Grey/dp/B019RPOZLI/ref=sr_1_2?keywords=logitech%2Bmouse%2Bm170%2Bwireless&qid=1684203813&sprefix=logitech%2Bmouse%2Bm170%2Caps%2C450&sr=8-2&th=1, and test on orangepc.
I'm not test Dummy HCD, I'm not sure. I will try it, and response here.

Hi @xairy,
I test the Dummy HCD, it work ok, I think musb-hdrc make the bug, when I change usb_raw_init(fd, USB_SPEED_LOW, driver, device) or usb_raw_init(fd, USB_SPEED_FULL, driver, device), use driver musb-hdrc and device musb-hdrc.4.auto, return ioctl(USB_RAW_IOCTL_RUN): Invalid argument, and dmesg has error
udc musb-hdrc.4.auto: failed to start USB Raw Gadget: -22
misc raw-gadget: fail, usb_gadget_probe_driver returned -22
when use Dummy HCD it work ok,

With the changes proposed by @imatespl (https://github.com/AristoChen/usb-proxy/pull/6#issuecomment-1545455139) my Logitech Superlight mouse now proxies without issues :-)

Thanks @imatespl  and @KernelKrusha for spending time testing this PR!


Cloudbuild updates

make some binaries available to all users
build controller server
update to golang 1.17

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 64.905%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1646

Relevant Lines:
2536

💛  - Coveralls


Multiple galleries on a single page

Need to replace this code:
var $thumbwrapper = angular.element(document.querySelectorAll('.ng-thumbnails-wrapper'));
var $thumbnails = angular.element(document.querySelectorAll('.ng-thumbnails'));

With this:
var $thumbwrapper = element.find('.ng-thumbnails-wrapper');
var $thumbnails = element.find('.ng-thumbnails');

This allows to use multiple galleries on a single page and it should be faster to search element only inside scope.

Hi. Can you submit a PR?

No, sorry, i don't use gitlab.

Fixed with #27

This PR introduced  #30 if jQuery isn't included.
The angularJS documentation says "find() - Limited to lookups by tag name" so this doesn't work with the class attribute.