replace libarchive dependency with Python standard library

Pulled out of #10 and rebased to current main.

Just to confirm; does this produce exactly the same archives as before? It looks like it would, but I want to ask to make sure.

Now that you mentioned it, the original variant passed directory entries on for ZIP files ...
I've force pushed the PR to fix this. Results match now for the current set of default wheels:
% ls
main  pr
% for variant in *; do (cd $variant; pdm run make_wheels.py; cd ..;)
# ...
% sha1sum main/dist/* pr/dist/*
25140e06d536e389a6f7738ddc6fbc5dfabfdab4  main/dist/ziglang-0.11.0-py3-none-macosx_10_9_x86_64.whl
c61864725ca16ec16c50612499772d7856bd2f44  main/dist/ziglang-0.11.0-py3-none-macosx_11_0_arm64.whl
0151ef48df78a794598d65d3d35084f257d4ccff  main/dist/ziglang-0.11.0-py3-none-manylinux_2_12_i686.manylinux2010_i686.musllinux_1_1_i686.whl
0a0c6665cf0f1c5464c126d7d0eeeccf40726df0  main/dist/ziglang-0.11.0-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl
1d9930dca0663e55b19b0458b2d6ff99007d2a4d  main/dist/ziglang-0.11.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl
685090e3a3dd00b01f9258b18ad14e65a4b0703b  main/dist/ziglang-0.11.0-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_1_armv7l.whl
36e59e098746644b89cd64d65db0bc78e79558ae  main/dist/ziglang-0.11.0-py3-none-win_amd64.whl
25140e06d536e389a6f7738ddc6fbc5dfabfdab4  pr/dist/ziglang-0.11.0-py3-none-macosx_10_9_x86_64.whl
c61864725ca16ec16c50612499772d7856bd2f44  pr/dist/ziglang-0.11.0-py3-none-macosx_11_0_arm64.whl
0151ef48df78a794598d65d3d35084f257d4ccff  pr/dist/ziglang-0.11.0-py3-none-manylinux_2_12_i686.manylinux2010_i686.musllinux_1_1_i686.whl
0a0c6665cf0f1c5464c126d7d0eeeccf40726df0  pr/dist/ziglang-0.11.0-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl
1d9930dca0663e55b19b0458b2d6ff99007d2a4d  pr/dist/ziglang-0.11.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl
685090e3a3dd00b01f9258b18ad14e65a4b0703b  pr/dist/ziglang-0.11.0-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_1_armv7l.whl
36e59e098746644b89cd64d65db0bc78e79558ae  pr/dist/ziglang-0.11.0-py3-none-win_amd64.whl

Glad I asked then! That's good for me.

Thank you!


permission denied when build

❓ Questions and Help
Hi guys!
I am trying to install it based on the instructions from INSTALL.md.
I get the error at this step : python setup.py build develop. It shows the error of permission denied. If I add sudo to this command, it will not use conda as the environment.. Any ideas will be appreciated.
I tried the solution at https://stackoverflow.com/questions/22551461/how-to-avoid-permission-denied-while-installing-package-for-python-without-sudo by adding --user at the end of this command, while it did not work..
environment:
ubuntu 16
conda
cuda 9.0
python 3.6
pytorch nightly 1.0.0 dev20190409
torchvision from the master branch

use this command will work..
sudo /home/xxxx/anaconda3/envs/maskrcnn_benchmark/bin/python setup.py build develop --user

is there any way to bypass it if I don't have the sudo permission?


Fix(config): Handled trim correclty on ubuntu

I added a handler for the trim method call in the config instance, it shouldn't throw an error now. And now the unit test run on some mock configuration data that has been set up.

This was a cloned branch, the mask pull request has been updated to handle this changes


HTTP input plugin failed to parse array of json when there is "time" property that is not a number

Fluentd Version: 1.2.1
Environment: Windows 10
Configuration:
<source>
  @type http
  port 8887
  <parse>
    @type json
    time_key nil
  </parse>
</source>

<match **>
  @type stdout
</match>

Problem:
Running this command curl -X POST -d '[{"foo":"bar", "time": "2018-06-12T18:35:48.1488850Z"},{"abc":"def"},{"xyz":"123"}]' http://localhost:8887/app.log  gives me this error can't convert String into an exact number
This parsing logic should respect the time_key nil in the setting. It's not a problem if I'm only sending one json object like curl -X POST -d '{"foo":"bar", "time": "2018-06-12T18:35:48.1488850Z"}' http://localhost:8887/app.log

Sorry, I missed this issue.
Just created the patch: https://github.com/fluent/fluentd/pull/2025

@repeatedly Yes, verified the fix to be working. Thank you for the quick turn around.


Ignore any IDs generated by the DB

This is causing us issues as we need to use MySQL directly and from Vitess for a transition period.
This could possibly be a command line flag but I think it's fine to just ignore the MySQL generated ID. That's after all what MySQL does, if you provide an ID when inserting it just ignored the autoinc column.

Alright, comments addressed please have another look.

LGTM


Remove title fallback for start nodes

Previously if not title was specified for a start node, we fell back to using
a human-friendly version of the node key. This was a carry over from code
originally in the NodePresenter and needlessly complicates the code.
Ideally we'd fail fast if no title was found (as in #2075), but given that the
regression tests render the title of the landing page for each flow, I think
this step is a safe one and it will make something else I'm working on easier.

Looks good to me.

I've rebased this against master and force-pushed in preparation for merging.


Revert "Merge pull request #638 from alphagov/add-definition-list"

This reverts the merge into master of https://github.com/alphagov/govuk-frontend/pull/638
Why? After some more discussions, it appears there's a need for a variant of the definition list from the above PR that reverses the bold styles. It makes sense to make definition list into a thing of its own that doesn't use govuk-list. We want to get this version of FE out so pulling this out of master and we'll investigate definition lists in more detail in due course.

Thanks @hannalaakso.
I think the changelog entry will need reverting too?

Thanks @36degrees, that's now done.


View never gets re-assigned once presenter is detached

Currently attach method of presenter is not counter to detach. In detach method view is nullified but it never gets re-assigned in attach method which raises NPE at most of the places.
Attaching - detaching behavior of presenters is need to be modified.
I am working on it.

App was debugged and found view is getting re-assigned correctly in presenter's constructor on activity/fragment restart after presenter is detached. Hence it is just a name issue that attach is not exactly opposite of the detach.
closing this issue.


failed building wheel for pillow error

How can I resolve this error? I'm trying to install the requirement.txt

Pillow==9.5.0

I got the same error so I removed the environment and created a new one with the specific Python version they suggest

conda create --name envname python=3.9.12


canal/calico-node - ipset v7.1: Kernel and userspace incompatible

Rancher Server Setup

Rancher version: 2.6.5
Installation option (Docker install/Helm Chart): helm, rke1

Information about the Cluster

Kubernetes version: v1.22.9
Cluster Type (Local/Downstream): Custom

Information about underlying OS

OS: Flatcar Linux
Release: Beta Channel / 3227.1.0
Kernel: 5.15.43
ipset: ipset v7.11, protocol version: 7

Describe the bug
Canal container calico-node fails to run properly, it's failing to get ready and keeps spamming the same error.
To Reproduce
On Flatcar Linux 3227.1.0 upgrade your Kubernetes cluster to v1.22.9.
Result
Canal pods never enter a "Running" state.
Expected Result
Canal pods should enter "Running" state.
Screenshots
Logs from the calico-node container
2022-06-16 10:20:15.978 [WARNING][2189649] felix/ipsets.go 322: Failed to resync with dataplane error=exit status 1 family="inet"
2022-06-16 10:20:16.237 [ERROR][2189649] felix/ipsets.go 574: Bad return code from 'ipset list'. error=exit status 1 family="inet" stderr="ipset v7.1: Kernel and userspace incompatible: settype hash:ip,port with revision 6 not supported by userspace.\n"
2022-06-16 10:20:16.237 [WARNING][2189649] felix/ipsets.go 322: Failed to resync with dataplane error=exit status 1 family="inet"
2022-06-16 10:20:16.754 [ERROR][2189649] felix/ipsets.go 974: Failed to read IP sets error=exit status 1 family="inet"
2022-06-16 10:20:16.754 [PANIC][2189649] felix/ipsets.go 355: Failed to update IP sets after multiple retries. family="inet"
panic: (*logrus.Entry) 0xc000fb00f0
goroutine 269 [running]:
github.com/sirupsen/logrus.Entry.log(0xc000140180, 0xc000a49500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/go/pkg/mod/github.com/projectcalico/logrus@v1.0.4-calico/entry.go:128 +0x6c5
github.com/sirupsen/logrus.(*Entry).Panic(0xc0007cd130, 0xc001723b80, 0x1, 0x1)
/go/pkg/mod/github.com/projectcalico/logrus@v1.0.4-calico/entry.go:173 +0xfa
github.com/projectcalico/felix/ipsets.(*IPSets).ApplyUpdates(0xc0006292c0)
/go/pkg/mod/github.com/projectcalico/felix@v0.0.0-20211111175437-b56ed543a3fb/ipsets/ipsets.go:355 +0x37a
github.com/projectcalico/felix/dataplane/linux.(*InternalDataplane).apply.func1(0xc0005b4c00, 0xc000ba6290, 0x370df90, 0xc0006292c0)
/go/pkg/mod/github.com/projectcalico/felix@v0.0.0-20211111175437-b56ed543a3fb/dataplane/linux/int_dataplane.go:1808 +0x35
created by github.com/projectcalico/felix/dataplane/linux.(*InternalDataplane).apply
/go/pkg/mod/github.com/projectcalico/felix@v0.0.0-20211111175437-b56ed543a3fb/dataplane/linux/int_dataplane.go:1807 +0x4ae
W0616 10:20:17.025597 2189717 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
W0616 10:20:17.048016 2189717 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
2022-06-16 10:20:17.376 [ERROR][2189717] felix/ipsets.go 574: Bad return code from 'ipset list'. error=exit status 1 family="inet" stderr="ipset v7.1: Kernel and userspace incompatible: settype hash:ip,port with revision 6 not supported by userspace.\n"
2022-06-16 10:20:17.376 [WARNING][2189717] felix/ipsets.go 322: Failed to resync with dataplane error=exit status 1 family="inet"
2022-06-16 10:20:17.380 [ERROR][2189717] felix/ipsets.go 574: Bad return code from 'ipset list'. error=exit status 1 family="inet" stderr="ipset v7.1: Kernel and userspace incompatible: settype hash:ip,port with revision 6 not supported by userspace.\n"

Additional context
I was able to work around this error by editing the canal DaemonSet.
Simply replace all images that has version v3.21.1 with v3.22.2.
calico-node v3.21.1 has ipset v7.1, v3.22.2 has ipset v7.11

I believe that to avoid this, there should be some mechanism for running specific canal versions based on what kernel/ipset version you have on your hosts.

Is there an update on this?  I'm running into the same thing trying to run metalLB.  After installing metallb, it appears there's a routing issue, and I can't get any responses on the ip pool I have configured.  It does work when accessing it directly from a node.  The resolution is supposed to be have kubeproxy use itvs mode instead, however that causes issues with Canal.
kubeproxy:
  extra_args:
    ipvs-strict-arp: 'true'
    proxy-mode: ipvs

`
2022-09-26 18:23:57.444 [WARNING][15138] felix/ipsets.go 322: Failed to resync with dataplane error=exit status 1 family="inet"
2022-09-26 18:23:57.704 [ERROR][15138] felix/ipsets.go 574: Bad return code from 'ipset list'. error=exit status 1 family="inet" stderr="ipset v7.1: Kernel and userspace incompatible: settype hash:ip,port with revision 6 not supported by userspace.\n"
2022-09-26 18:23:57.705 [WARNING][15138] felix/ipsets.go 322: Failed to resync with dataplane error=exit status 1 family="inet"
2022-09-26 18:23:58.221 [ERROR][15138] felix/ipsets.go 974: Failed to read IP sets error=exit status 1 family="inet"
2022-09-26 18:23:58.221 [PANIC][15138] felix/ipsets.go 355: Failed to update IP sets after multiple retries. family="inet"
panic: (*logrus.Entry) 0xc00091cf00

goroutine 236 [running]:
github.com/sirupsen/logrus.Entry.log({0xc00012e180, 0xc0003e8060, {0x0, 0x0, 0x0}, 0x0, {0x0, 0x0}, 0x0}, 0x0, ...)
/go/pkg/mod/github.com/projectcalico/logrus@v1.0.4-calico/entry.go:128 +0x56c
github.com/sirupsen/logrus.(*Entry).Panic(0xc0006ccf00, {0xc000a6fb80, 0x1, 0x1})
/go/pkg/mod/github.com/projectcalico/logrus@v1.0.4-calico/entry.go:173 +0xfb
github.com/projectcalico/calico/felix/ipsets.(*IPSets).ApplyUpdates(0xc000b05900)
/go/src/github.com/projectcalico/calico/felix/ipsets/ipsets.go:355 +0x737
github.com/projectcalico/calico/felix/dataplane/linux.(*InternalDataplane).apply.func1({0x324f590, 0xc000b05900})
/go/src/github.com/projectcalico/calico/felix/dataplane/linux/int_dataplane.go:1822 +0x3d
created by github.com/projectcalico/calico/felix/dataplane/linux.(*InternalDataplane).apply
/go/src/github.com/projectcalico/calico/felix/dataplane/linux/int_dataplane.go:1821 +0xd88
`


In scankeys, add all synced secret key, not just one at random

Non-determinism came from here: https://github.com/keybase/client/blob/master/go/libkb/sync_secret.go#L182
ss.keys.PrivateKeys is a map of strings and iterating over it is randomized.
I added new methods like GetSyncedSecretKeys that have "signular" ~Key methods equivalent because there are multiple flows that assumed only one key - including PGP provisioning :( - and I don't want to change all of this right now.

Fixes https://github.com/keybase/keybase-issues/issues/3373

Any idea when this might land in a stable release channel?

I also just encountered this.  Using version 4.7.2-20191026044208+6fc2e969b4

Hey, is this still broken for you @StephanX ? Can you send a log using keybase log send command?

along with more info e.g. what action have you tried and which key you expected to work. If it's private, and you are willing to only share with keybase, you can put it in as "log feedback". thank you.

Sure, logid: 2be52c7ca739c21bf406b61c
I created a gpg key 5D030726DF141AE3 a few days ago, using the mac version of gpgtools.org , version:
gpg (GnuPG) 2.2.17
libgcrypt 1.8.5

Perhaps of note, I created a new UID version of the original key to change the name and password (ref: https://coderwall.com/p/tx_1-g/gpg-change-email-for-key-in-pgp-key-servers )
I then installed keybase for the first time today, also mac client version, and imported my pgp key.  Finally, I attempted to use keybase to decrypt a fyle with keybase pgp decrypt and received the error ERROR decrypt error: unable to find a PGP decryption key for this message
Happy to offer more debugging info, thanks for the quick response!

Hey, sorry for not getting back to you, I was out of office.
Looks like this key was revoked and another one was added: https://keybase.io/stephanx/sigchain#42f610201df16afe14daa8bc838a009511bf031625c166f59a6dae194a8e8b8d0f
What's your status now? PGP commands are only available for keys that are published on your Keybase user. If you have a key that's not attached to Keybase profile, it's better to just use GPG program.
Let me know if I misunderstood what's going here.

Hi @zapu
I have the same issue as @StephanX I think.
For example running this gets the same error as mentioned above:
keybase pgp encrypt -m "message" | keybase pgp decrypt
▶ ERROR decrypt error: unable to find a PGP decryption key for this message

How can I figure out where it's going wrong?
Thanks
Joe

@joe-niland likely you don't have the private key on the local machine.

Thanks @maxtaco
It is in my GPG keychain but I can't remember if/how I imported it into keybase.
Is it a case of running keybase select --import?
I will try this later.


treewide: Remove some more crossAttrs

Motivation for this change
Progress towards #33302. Packages here picked so as to not cause a mass-rebuild.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

Success on x86_64-linux (full log)
Attempted: curl, kmod, libxml2, newt, sharutils
Partial log (click to expand)
these paths will be fetched (0.24 MiB download, 1.37 MiB unpacked):
  /nix/store/834gppliwhn90aspxywl8zk71r8fa1hd-sharutils-4.15.2
copying path '/nix/store/834gppliwhn90aspxywl8zk71r8fa1hd-sharutils-4.15.2' from 'https://cache.nixos.org'...
/nix/store/ghbljn615xjslw8xp36j0yrxnhs7bhcn-curl-7.61.0-bin
/nix/store/k6pn35lzq6k5d6clzbkf2rkl08hfkr4b-kmod-25
/nix/store/ykxqvw7340aqvrj5vv71f845y1rb74z4-libxml2-2.9.8-bin
/nix/store/4vjgszyqr72z565dwnx6pzswy3cdqc92-newt-0.52.20
/nix/store/834gppliwhn90aspxywl8zk71r8fa1hd-sharutils-4.15.2

Success on aarch64-linux (full log)
Attempted: curl, kmod, libxml2, newt, sharutils
Partial log (click to expand)
these paths will be fetched (0.22 MiB download, 1.34 MiB unpacked):
  /nix/store/3nmy2gjj1pjd3nqvzkr28k18jl3l60a7-sharutils-4.15.2
copying path '/nix/store/3nmy2gjj1pjd3nqvzkr28k18jl3l60a7-sharutils-4.15.2' from 'https://cache.nixos.org'...
/nix/store/228s4k6k1dyhfanlmz7r9k585kaybv13-curl-7.61.0-bin
/nix/store/6009qbgng3rvqrgrssgfvsyz4zz2h9av-kmod-25
/nix/store/v6zql9axnwh0dz9qjjj57mkfr3v8dxzx-libxml2-2.9.8-bin
/nix/store/1cnglihsh1qyw34rz3864v1nxmws5lms-newt-0.52.20
/nix/store/3nmy2gjj1pjd3nqvzkr28k18jl3l60a7-sharutils-4.15.2

Success on x86_64-darwin (full log)
Attempted: curl, libxml2, sharutils
The following builds were skipped because they don't evaluate on x86_64-darwin: kmod, newt
Partial log (click to expand)

these paths will be fetched (0.28 MiB download, 1.47 MiB unpacked):
  /nix/store/5f41a0b7j8p68wnpm1lcba0mk1a9825g-sharutils-4.15.2
  /nix/store/ql8m0v44i00kyrrs64fv6slffcxqs6jj-curl-7.61.0-bin
copying path '/nix/store/ql8m0v44i00kyrrs64fv6slffcxqs6jj-curl-7.61.0-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/5f41a0b7j8p68wnpm1lcba0mk1a9825g-sharutils-4.15.2' from 'https://cache.nixos.org'...
/nix/store/ql8m0v44i00kyrrs64fv6slffcxqs6jj-curl-7.61.0-bin
/nix/store/5wsv6yj86r8wfjaqivfq6xcb3vi9mmzz-libxml2-2.9.8-bin
/nix/store/5f41a0b7j8p68wnpm1lcba0mk1a9825g-sharutils-4.15.2


Sweep Line Algorithm for Convex Hull in C

issue resolved :#1562
Pull Request: Sweep Line Algorithm for Convex Hull in C
Description
This PR introduces a C implementation of the Sweep Line Algorithm to compute the convex hull for a set of 2D points. The convex hull is a minimal convex boundary that can enclose all given points. This sweep line approach enables efficient computation by processing points in sorted order and maintaining the hull dynamically, making it well-suited for large sets of points.
Features Implemented

Sorting Points: Points are sorted by x-coordinate, enabling efficient left-to-right processing.
Convex Hull Construction:

The algorithm uses the sweep line method to iteratively build both the lower and upper hulls.
Cross Product: This function helps determine the orientation of three points, essential for ensuring the hull's convexity.

Memory-Efficient Design: This implementation uses a linked list for dynamically managing points, reducing memory constraints compared to array-based approaches.

Files Added

sweep_line_convex_hull.c: Contains the main C code implementing the sweep line approach for convex hull.
README.md: Documentation covering the algorithm, usage, and example scenarios.

Code Overview

Data Structures:

Point struct to store x and y coordinates.
Linked list used for handling dynamically increasing points on the hull.

Key Functions:

compare: Helper function to sort points by x and y coordinates.
cross_product: Calculates the cross product of three points to determine left or right turns.
sweep_line_convex_hull: Computes the convex hull using the sweep line method.

Main Function:

Initializes a set of points and calls sweep_line_convex_hull to calculate and print the vertices of the convex hull.

Example
Consider the input points:
{ (0, 3), (2, 2), (1, 1), (2, 1), (3, 0), (0, 0), (3, 3) }

Expected output (convex hull vertices in counter-clockwise order):
(0, 0)
(3, 0)
(3, 3)
(0, 3)

Testing
The code has been tested with:

Basic configurations to verify output correctness.
Edge cases such as collinear points, minimal input (2-3 points), and duplicate points.
Larger datasets to ensure efficiency and correctness.

Checklist

[x] Implemented Sweep Line Algorithm for convex hull.
[x] Verified against known test cases for accuracy.
[x] Included comprehensive documentation in README.md.
[x] Code is formatted and commented for readability.

Additional Notes
The Sweep Line Algorithm provides a foundation for computational geometry tasks that require efficient hull calculations. This PR supports integer coordinates; future extensions could add floating-point support or graphical visualizations.

@pankaj-bind kindly review the issue and possibly merge it


help: Devtools WebSocket disconnected

📚 What are you trying to do?
Hi, I'm trying to use devtools Nux 3 project which extend a layer but it does't work propely.
DevTools works but it shows Connecting to server... for a few time then the message turn in "WebSocket disconnected".

🔍 What have you tried?
I've spent a lot of time trying to solve it, I'he read all documentation avaiable online.
here my last configuration:
devtools: {
enabled: true,
},
extends: ['../core-layer'],
devServer: {
host: '<IP_ADDRESS>',
port: 80,
},
ℹ Additional context
No response

We temporarily closed this due to the lack of enough information. We could not identify whether it was a bug or a userland misconfiguration with the given info.
Please provide a minimal reproduction to reopen the issue.
Thanks.

Why reproduction is required


Fixed #31621 -- Add support for '--parallel auto' to test command

ticket

@lothemar This PR intersects with your project as well.

This PR changes the current behavior, e.g. on Jenkins we specify DJANGO_TEST_PROCESSES=1 and with this PR it runs tests in 9 parallel processes instead of 1.

Right sorry - forgot to copy-paste some stuff from the runner back into runtests.py
Imo though, runtests.py should default parallel to "auto". Why can't Jenkins invoke it with --parallel 1 rather than the env var? I'm not even sure why the env var exists - none of the other options can be overridden with env vars.

@adamchainz Do you have time to keep working on this?

Closing due to inactivity.

I've rebased the branch and updated the docs for 4.0.
I believe I already fixed the concern with Jenkins back in the June 2020 push. When I run locally with DJANGO_TEST_PROCESSES=1 ./runtests.py test_runner --parallel (plus a patch for macOS to use fork), I get one process.

Oh GitHub doesn't allow reopening a PR after the branch has been force-pushed. Recreated PR here: #14614

Thanks


BHBC-1571

BHBC-1583 DRAFT - DO NOT MERGE
Overview

BHBC-1571
BHBC-1583

This PR contains the following changes

support multiple contacts and contact types
support project species

This PR contains the following types of changes

[ ] New feature (change which adds functionality)
[ ] Enhancement (improvements to existing functionality)
[ ] Bug fix (change which fixes an issue)
[ ] Misc cleanup / Refactoring / Documentation
[ ] Version change

Checklist
A list of items that are good to consider when making any changes.
Note: this list is not exhaustive, and not all items are always applicable.
General

[ ] I have performed a self-review of my own code

Code

[ ] New files/classes/functions have appropriately descriptive names and comment blocks to describe their use/behaviour
[ ] I have avoided duplicating code when possible, moving re-usable pieces into functions
[ ] I have avoided hard-coding values where possible and moved any re-usable constants to a constants file
[ ] My code is as flat as possible (avoids deeply nested if/else blocks, promise chains, etc)
[ ] My code changes account for null/undefined values and handle errors appropriately
[ ] My code uses types/interfaces to help describe values/parameters/etc, help ensure type safety, and improve readability

Style

[ ] My code follows the established style conventions
[ ] My code uses native material-ui components/icons/conventions when possible

Documentation

[ ] I have commented my code sufficiently, such that an unfamiliar developer could understand my code
[ ] I have added/updated README's and related documentation, as needed

Tests

[ ] I have added/updated unit tests for any code I've added/updated
[ ] I have added/updated the Postman requests/tests to account for any API endpoints I've added/updated

Linting/Formatting

[ ] I have run the linter and fixed any issues, as needed
See the lint commands in package.json
[ ] I have run the formatter and fixed any issues, as needed
See the format commands in package.json

SonarCloud

[ ] I have addressed all SonarCloud Bugs, Vulnerabilities, Security Hotspots, and Code Smells

How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Screenshots
Please add any relevant UI screenshots, if applicable.

I reviewed the changes and it looks good to me. Also testing on BHBC-1570 by fixing the error that was popping. I think @cgarrettjones we can close this branch and PR.

Merged new changes from this PR into https://github.com/bcgov/restoration-tracker/pull/18. So this PR and Branch can be now closed.


Edit "Working with Containers" page for readability

Fix some wording/grammar issues
Already reviewed at https://github.com/docker/docker/pull/13905, submitting this duplicate because it was easier for me than rebasing.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "patch-2" git@github.com:airandfingers/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Thanks @airandfingers !
This was already LGTM-ed in https://github.com/docker/docker/pull/13905, so merging

cherry-picked


[Decorator] MultipartFile

@Controller('/')
class MyCtrl {
     @Post('/file')
     private uploadFile(@MultipartFile() file: Multer.File) {

     }

     @Post('/files')
     private uploadFile(@MultipartFile() files: Multer.File[]) {

     }
}

Hi Romakita, how can Multer configuration be passed? I see that the only config available is the uploadDir

Hi @m0uneer ,
Actually, global settings let you to customize the uploadDir. For more, options you need to give that to the MultipartFile decorators.
@MultipartFile(multerOptions)

See you

@MultipartFile still not recognized, copied the code from the documentation:
import { Controller, Get, Post } from '@tsed/common'
import Multer from 'multer'

type MulterFile = Express.Multer.File

export interface ViewerModel {
  name: string
  file: string
}

@Controller('/models')
export class ViewerModelCtrl {
  @Get('/')
  async renderModels() {
    return 'Hello World'
  }

  @Post('/')
  private uploadeModel(@MultiplartFile() file: MulterFile) {}
}

I get 'Cannot find name 'MultipartFile'

Solved by installing @tsed/multipartfiles and
import { MultipartFile } from '@tsed/multipartfiles'

In the documentation, @tsed/multipartfile is mentioned instead of @tsed/multipartfiles (plural)
And it looks like it should be used only for multiple files upload.
https://romakita.github.io/ts-express-decorators/#/tutorials/upload-files-with-multer?id=example

Hum ok. I'll fix this typo :) Thanks @naoric


purge redundant mappings from official repo

since losing sort.py a lot has changed upstream in SDL including

Android maps changing format
alternative maps for controllers with Nintendo-style layout (using hint:SDL_GAMECONTROLLER_USE_BUTTON_LABELS)
myriad new devices added (this db is now long behind SDL's)

since it doesn't seem like there has been any movement toward authoring a new CI check to handle parity, i think would best to consider this DB purely additive and remove maps that are redundant with upstream data
this should help prevent a duplication of effort, and encourage people who have issues with upstream SDL maps to take them upstream
to prevent further barriers to contribution, no need to force an automatic check or similar -- an occasional manual purge of matching upstream GUIDs  should suffice. if @gabomdq's fine with this direction i'll author a PR that removes all present GUIDs that match upstream, and make note of this change in the readme

We'll keep the "redudant" mappings for now, my intention is that this DB is a superset of what SDL includes in their binaries, useful for people that want to update mappings without having to rebuild or update SDL. Also, it seems that these mappings are not just used by SDL but also parsed independently by Godot, so there's no gain in removing mappings.

A number of projects do indeed use this DB without SDL, but if it's to be a superset the mapping list needs maintaining since it's fallen significantly behind the SDL mappings. Issues like the recent PR that breaks official PS3 controllers on Windows should also presumably be avoided?


Fix PRs from forks failing CI due to lack of Github secrets

This should only be merged after @terror changes the OAuth secret to the dev one.
This has some security implications, because it means the CI will be ran on our resources. Github action codes from forks should be verified to ensure no malicious code before running any actions.

Resolves #527

I can't really test this lmao but I'll make a dummy PR from a fork after this gets merged to check if it works well

@terror feel free to merge this once you swapped the secret

wait i dont think this is right checks arent running (?

ehhhhhhhhhhhh need more research before proceeding


Is it possible to install fastAPI on Raspberry Pi without docker?

Installing the fastapi package with pip will throw this error:
pi@raspberrypi:~/Desktop/ $ pip install fastapi

DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: Could not find a version that satisfies the requirement fastapi (from versions: none)
ERROR: No matching distribution found for fastapi

Install a newer python version

I'm on python 3.8 and pip 20.2.3.

Make sure you're using the right pip on your system - try pip3.8

@moritzsalla, please run these and post the outputs here.
python -V

pip -V

python3 -V

pip3 -V

I had an SSL certificate issue with my python install. Thanks for the help!

Thanks for the help everyone! 🍰
And thanks for coming back to close the issue @moritzsalla ☕

Sorry for the long delay! 🙈  I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.


autoscaling instances will not get deleted on "terraform destroy"

autoscaling instances will not get deleted on "terraform destroy"
happened two times, so probably a small bug. nothing urgent I guess since I deleted it manually

@ifeulner Just FYI.
I agree with @captnCC that for now, manual deletion seems the only option. But good to keep that issue in the back of our minds.

Yes, absolutely. The only way I do see here is to do some work around (e.g. via hcloud cli, labels), but terraform does not know about the dynamically created resources.

Exactly!

The only possible way would be to deploy a the ccm/auto scaler via terraform resources, so they could be destroyed by terraform. But that adds it's own set of problems

Exactly, so not an option.
Folks, I actually still never used the autoscaler, so I need some guidance here. In order to clarify the "Takedown" section in the readme, what is the optimal way to remove all autoscaled nodes?
Would changing the autoscaler nodepool definitions like setting min nodes and max nodes to 0 and applying again work? Would that successfully instruct the Kube autoscaler to drain the nodes and then remove them?


CMPT Remove redundant and admin fields from XML

Fixes #6501

Test repo: https://github.com/abapGit-tests/CMPT

Thanks for the test repo 👍


Add the same image to different table faild

Bug Description
I tried to add the same style object (just set bg_image) to two different tables, and the script generated the numbers file without an error, but the numbers file could not be opened properly.
Code
from numbers_parser import Document,BackgroundImage

def test_add_bg_image(configurable_save_file):

    doc = Document()
    doc.sheets[0].add_table()
    table1 = doc.sheets[0].tables[0]
    table2 = doc.sheets[0].tables[1]

    image_filename = "cat.jpg"
    image_data = open(image_filename, mode="rb").read()
    bg_image = BackgroundImage(image_data, image_filename)
    cats_bg_style = doc.add_style(bg_image=bg_image)

    table1.write(0, 0, "cats", style=cats_bg_style)

    table2.write(0, 0, "cats", style=cats_bg_style)

    doc.save(configurable_save_file)

test_add_bg_image("test1.numbers")

In addition, for two style objects with different bg_image, the numbers file can be opened correctly.
from numbers_parser import Document,BackgroundImage

def test_add_bg_image(configurable_save_file):

    doc = Document()
    doc.sheets[0].add_table()
    table1 = doc.sheets[0].tables[0]
    table2 = doc.sheets[0].tables[1]

    image_filename = "cat.jpg"
    image_data = open(image_filename, mode="rb").read()
    bg_image = BackgroundImage(image_data, image_filename)
    cats_bg_style1 = doc.add_style(bg_image=bg_image)

    image_filename = "test.png"
    image_data = open(image_filename, mode="rb").read()
    bg_image = BackgroundImage(image_data, image_filename)
    cats_bg_style2 = doc.add_style(bg_image=bg_image)

    table1.write(0, 0, "cats", style=cats_bg_style1)

    table2.write(0, 0, "cats", style=cats_bg_style2)

    doc.save(configurable_save_file)

test_add_bg_image("test2.numbers")

In Console, I see an assertion from Numbers like this before it crashes:
#Assert *** Assertion failure #0: -[TSPDataManager addData:] /Library/Caches/com.apple.xbs/Sources/iWorkDependenciesMacOS/shared/persistence/src/TSPDataManager.mm:1735 Should not have two TSPData instances with the same digest 99291ca00549136f04ff70d04c1ca7c26e53d37b: <private> (<private>) and <private> (<private>).

Which is certainly true. Would be good to confirm you see this too but I'll proceed with a fix that avoids this.

Give v4.10.3 a try.


Address for stalking and SA petitions

From Oakland (11/6/24):
-Respondent’s address and/or telephone number is required; however, it can be the address of their job. If ex parte is denied and the matter is set for a hearing, the respondent cannot be verbally notified of the hearing, they must be served the documents and NOH personally; therefore, we will accept any address where respondent can be served.
-If petitioner doesn’t know respondent’s address, we typically tell them we can accept it without the address; however, if the matter gets set for hearing they MUST have an address to serve respondent. Usually, they can find the address online. If they try service at that address and service fails, they can motion for alternate service at that point.

Temporary note: this is just documentation, nothing needs fixed. Leaving this as-is for now because I need to see if I can give Amy access to add tags.


Not compatible with scikit-learn 1.1 and above

When using this library with scikit-learn version 1.1 or above, the following exception is thrown:
evolutionary_search/cv.py", line 12, in <module>
    from sklearn.model_selection._search import BaseSearchCV, check_cv, _check_param_grid
ImportError: cannot import name '_check_param_grid' from 'sklearn.model_selection._search'

This happens because the (private) method ParameterGrid._check_param_grid() of scikit-learn was deprecated, and its functionality is now included within the ParameterGrid.__init__() method.

The following PR aims to fix this issue: https://github.com/rsteca/sklearn-deap/pull/80


Executor trying to copy from base image when using trailing slash in volumeMounts[*].volumePath

Summary
Having declared a template with:
volumeMounts:
- name: out
  mountPath: /tmp/out/

When running it, just before completion we'd run into the error (executor error: CopyFile() is not implemented in the k8sapi executor.)
After a bit of digging into the code, I have tried with
volumeMounts:
- name: out
  mountPath: /tmp/out

And it worked. It turns out that when checking if the artifact is from the base image or a mount dir, eventually this method is called:
https://github.com/argoproj/argo-workflows/blob/d717865716ea399284c6193ceff9970e66bc5f45/workflow/common/util.go#L40
The above linked line adds a trailing slash to the path, therefore ending up with 2 when there is already present.
Diagnostics
What Kubernetes provider are you using?
GKE
What version of Argo Workflows are you running?
2.12.3
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: output-artifact-bug
spec:
  entrypoint: mytemplate
  serviceAccount: <redacted>
  templates:
  - name: mytemplate
    outputs:
      artifacts:
      - name: test-out-artifact
        path: "/tmp/out/myartifact.tar"
        gcs:
          bucket: <redacted>
          key: "mykey.tar"
          serviceAccountKeySecret:
            name: <redacted>
            key: <redacted>
    volumes:
    - name: out
      emptyDir: {}
    script:
      image: bash:5.1
      command: [bash]
      source: |
        touch /tmp/out/myartifact.tar
      volumeMounts:
      - name: out
        mountPath: /tmp/out/

Paste the logs from the workflow controller:
kubectl logs -n argo $(kubectl get pods -l app=workflow-controller -n argo -o name) | grep ${workflow}

time="2021-03-26T15:08:59.072Z" level=info msg="Processing workflow" namespace=argo workflow=output-artifact-bug-dtmcw
time="2021-03-26T15:08:59.073Z" level=info msg="Updated phase  -> Running" namespace=argo workflow=output-artifact-bug-dtmcw
time="2021-03-26T15:08:59.083Z" level=info msg="Updated phase Running -> Failed" namespace=argo workflow=output-artifact-bug-dtmcw
time="2021-03-26T15:08:59.083Z" level=info msg="Updated message  -> invalid spec: templates.mytemplate.outputs.artifacts.test-out-artifact: k8sapi executor does not support outputs from base image layer.  Use an emptyDir: https://argoproj.github.io/argo/empty-dir/" namespace=argo workflow=output-artifact-bug-dtmcw
time="2021-03-26T15:08:59.083Z" level=info msg="Marking workflow completed" namespace=argo workflow=output-artifact-bug-dtmcw
time="2021-03-26T15:08:59.083Z" level=info msg="Checking daemoned children of " namespace=argo workflow=output-artifact-bug-dtmcw
time="2021-03-26T15:08:59.092Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=1436328119 workflow=output-artifact-bug-dtmcw

Changing the workload to the following (removing trailing slash in the volumeMounts.mountPath), will work.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: output-artifact-bug
spec:
  entrypoint: mytemplate
  serviceAccount: <redacted>
  templates:
  - name: mytemplate
    outputs:
      artifacts:
      - name: test-out-artifact
        path: "/tmp/out/myartifact.tar"
        gcs:
          bucket: <redacted>
          key: "mykey.tar"
          serviceAccountKeySecret:
            name: <redacted>
            key: <redacted>
    volumes:
    - name: out
      emptyDir: {}
    script:
      image: bash:5.1
      command: [bash]
      source: |
        touch /tmp/out/myartifact.tar
      volumeMounts:
      - name: out
        mountPath: /tmp/out

Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

Want to submit a PR to fix this? @rccrdpccl

Sure, I'll give it a go


Fix @Published

#97 gets too big, so I decided to fix @Published in a separate PR.
Fixes #102.
This needs tests.

LGTM

  Generated by :no_entry_sign: Danger Swift against e45e8e813ad2b7b0008a20e5d6b41b3c256b89b8


[Broken] AMCNetworks

[x] I'm reporting a broken site support
[x] I've verified that I'm running yt-dlp version 2021.03.15
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log
[debug] User config file: C:\lib\yt-dlp.conf
[debug] User config: ['--ap-mso', 'PRIVATE', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE']
[debug] Home config file: yt-dlp.conf
[debug] Home config: ['--ap-mso', 'PRIVATE', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE']
[debug] Command-line config: ['--verbose', 'https://www.amc.com/shows/talking-dead/episodes/season-10-episode-24-on-the-walking-dead-1019--54270']
[debug] Loading archive file None
[debug] Encodings: locale cp1252, fs utf-8, out cp1252, pref cp1252
[debug] yt-dlp version 2021.03.15 (exe)
[debug] Python version 3.8.7 (CPython 64bit) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg 4.3.1-2020-10-01-full_build-www.gyan.dev, ffprobe 4.3.1-2020-10-01-full_build-www.gyan.dev, phantomjs 2.1.1
[debug] Proxy map: {}
[AMCNetworks] shows/talking-dead/episodes/season-10-episode-24-on-the-walking-dead-1019--54270: Downloading JSON metadata
[AMCNetworks] shows/talking-dead/episodes/season-10-episode-24-on-the-walking-dead-1019--54270: Downloading JSON metadata
[AMCNetworks] MdqNc4oS_P6P: Downloading Provider Redirect Page
[AMCNetworks] MdqNc4oS_P6P: Downloading Provider Login Page
[AMCNetworks] MdqNc4oS_P6P: Logging in
[AMCNetworks] MdqNc4oS_P6P: Confirming Login
[AMCNetworks] MdqNc4oS_P6P: Retrieving Session
[AMCNetworks] MdqNc4oS_P6P: Retrieving Authorization Token
[AMCNetworks] MdqNc4oS_P6P: Retrieving Media Token
[AMCNetworks] MdqNc4oS_P6P: Downloading SMIL data
[AMCNetworks] MdqNc4oS_P6P: Downloading m3u8 information
ERROR: No video formats found; please report this issue on https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1024, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1045, in __extract_info
  File "yt_dlp\extractor\common.py", line 554, in extract
  File "yt_dlp\extractor\amcnetworks.py", line 92, in _real_extract
  File "yt_dlp\extractor\common.py", line 1663, in _sort_formats
yt_dlp.utils.ExtractorError: No video formats found; please report this issue on https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.

Description
Some videos from amc.com download fine but others don't, such as this one:
https://www.amc.com/shows/talking-dead/episodes/season-10-episode-24-on-the-walking-dead-1019--54270
For this video, the m3u8 file downloaded by yt-dlp uses Fairplay protection. However, if I inspect the webpage in a browser, the m3u8 file doesn't use Fairplay.  The URLs for the two m3u8 files are of the form
https://amc-dai-amd-theplatform.akamaized.net/sp/AMC_Networks_-_AMC/567/91/hls_fp/1615316725297/1871051843874_1615313060919_1871069251586_mp4_video_1920x1080_0_primary_audio_und_6-master.m3u8?hdnea=...
and
https://amc-dai-amd-theplatform.akamaized.net/sp/AMC_Networks_-_AMC/567/91/hls_e/1615316713798/1871051843874_1615313060919_1871069251586_mp4_video_1920x1080_0_primary_audio_und_6-master.m3u8?hdnea=...

did you try signing in with your TV provider and then try using the cookies command ??

did you try signing in with your TV provider and then try using the cookies command ??

Doing that results in a different error:
Traceback (most recent call last):
File "yt_dlp\YoutubeDL.py", line 1024, in wrapper
File "yt_dlp\YoutubeDL.py", line 1045, in __extract_info
File "yt_dlp\extractor\common.py", line 554, in extract
File "yt_dlp\extractor\amcnetworks.py", line 87, in _real_extract
File "yt_dlp\extractor\adobepass.py", line 1544, in _extract_mvpd_auth
File "yt_dlp\extractor\adobepass.py", line 1370, in post_form
File "yt_dlp\extractor\common.py", line 1050, in _html_search_regex
File "yt_dlp\extractor\common.py", line 1041, in _search_regex
yt_dlp.utils.RegexNotFoundError: Unable to extract post url; please report this issue on https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.

In the _real_extract in the amcnetworks.py extractor, data comes from the JSON info from the URL
https://content-delivery-gw.svc.ds.amcn.com/api/v2/content/amcn/amc/url/shows/talking-dead/episodes/season-10-episode-24-on-the-walking-dead-1019--54270
The function uses the value of videoPid to create a URL that leads to Fairplay protection-
https://link.theplatform.com/s/M_UwQC/media/z_KGAHuv2JhE? ...
If one uses the value of releasePid instead to create the URL
https://link.theplatform.com/s/M_UwQC/vD5punENmVLh? ...
then the resulting video doesn't have Fairplay and yt-dlp has no trouble with the download. Note the removal of "media/" from the URL.
I haven't yet tested this change for other videos on the site.

@2ShedsJackson Since you already have a solution, a PR would be appreciated and would save me so much time

@2ShedsJackson Since you already have a solution, a PR would be appreciated and would save me so much time

Will do. I first looked at the code literally only a couple of days ago and I'm checking some other stuff. It's annoying that the different extractors don't follow a common template but that's inherited from youtube-dl.
But here's a question- should a thumbnail image match the video resolution or should the highest resolution be downloaded or all the resolutions?  The extractor currently downloads a 1920x1080 image for the video in this issue although a 2560x1440 version is also available.

should a thumbnail image match the video resolution or should the highest resolution be downloaded or all the resolutions

If there are multiple thumbnails available, ideally all the thumbnails should be extracted to the thumbnails key. Then yt-dlp will automatically select the best one to download. Currently, there is no option for the user to choose another thumbnail than the default, but it is a planned feature

It's annoying that the different extractors don't follow a common template but that's inherited from youtube-dl.

Due to how the different extractors work very differently, there is no way for them to be too much similar. But the overall structure is actually the same. See the documentation of InfoExtractor class in extractor/common.py

Due to how the different extractors work very differently, there is no way for them to be too much similar. But the overall structure is actually the same. See the documentation of InfoExtractor class in extractor/common.py

Some extractors don't return thumbnails or series titles. I found another one that doesn't return season and episode info. Beyond that, I'm being picky about the lack of consistency in variable names, error reporting, etc.


Add step extension definition for set-header (KaotoIO/kaoto-ui#1080)

sorry the URL has to be changed to upstream


Fixed yrotation and zrotation for mpl overlay plots.

Maybe I shouldn't be trying to rotate axis labels, but I was, and I got quite confused...

Thanks, clearly a bug.


Cannot generate IREE benchmarking module for Vulkan backend

I'm following this documentation (https://google.github.io/iree/developing-iree/profiling-vulkan-gpu) to benchmark an IREE module for vulkan backend.  I'm not able to generate the iree-run-module-vulkan-gui through bazel (no BUILD in /iree/testing/vulkan) as well as using cmake command. I have vulkan sdk installed. Here's my cmake command to build IREE:
cmake -G Ninja -DIREE_BUILD_COMPILER=ON -DIREE_BUILD_TESTS=ON -DIREE_ENABLE_MLIR=ON -DIREE_ENABLE_TENSORFLOW=ON -DIREE_BUILD_PYTHON_BINDINGS=ON -DCMAKE_INSTALL_PREFIX=/iree/install -DIREE_BUILD_SAMPLES=ON -DIREE_HAL_DRIVER_VULKAN=ON .. && ninja install

I inspected the CMakeLists.txt and I suspect that I maybe missing imgui. Any information on how to integrate imgui with IREE would also be helpful.
Steps to reproduce the behavior:
Just follow the steps in the above link and you can see there's no executable generated after build.
I'm trying to bechmark IREE's performance on GPU (vulkan) and tensorflow.

Will let scott talk about the specific gui issue, but if you just want to run something then you don't need any of the gui stuff - it's all just demos - iree-run-module should work for executing things and doesn't have dependencies on imgui/SDL/etc.

iree-run-module works but it only provided total time of run and CPU time. I'm trying to benchmark inference performance on GPU (vulkan) and I thought iree-run-module-vulkan-gui would probably give that information. After building IREE with the above cmake command, iree-run-module-vulkan-gui binary was not generated. Inspecting the CMakeLists in iree/testing/vulkan seemed like the executable wouldn't build if the two dependencies aren't presend (imgui and SDL).

First, regarding building iree-run-module-vulkan-gui: https://github.com/google/iree/pull/4130 has some info that can help with the build configuration, but the documentation at https://google.github.io/iree/developing-iree/profiling-vulkan-gpu is out of date. Specifically, you'll need CMake's find_package to locate imgui and SDL2: https://github.com/google/iree/blob/84b0c2cadd165f3dd88fc6eb79e82b66af330512/iree/testing/vulkan/CMakeLists.txt#L29-L41
The most robust way I've found for getting C++ dependencies like those is using vcpkg with the install commands mentioned in that CMake file and on that PR. If you'd rather not install yet another package manager, you can 'install' those dependencies on their own (e.g. https://wiki.libsdl.org/Installation, not sure about imgui...).

Second, regarding GPU benchmarking in general: iree-run-module-vulkan-gui won't give you more detailed information on its own. That program is just a GUI wrapper around iree-run-module in the hopes of getting external developer tools (like NVIDIA Nsight Graphics) to recognize that the program is actually submitting commands worth instrumenting. Lots of the GPU tools were built with a graphics focus, where there is a render loop (100 API calls used to generate frame 1, 250 API calls to generate frame 2, etc.)... but we just use compute, so there isn't as clear a place to start/stop measuring. That being said, Nsight has new support for compute applications (source):

GPU Trace supports graphics compute applications with no frame boundaries (Beta)

It is now possible for you to profile graphics applications which use D3D Compute, DirectML and Vulkan Compute but do not have a traditional frame begin/end/present.
To use this feature, set the ‘Capture Type’ to ‘One-shot [Beta]’

Broadly speaking, an apples to apples comparison between IREE's GPU execution and another framework like TensorFlow's would either need to operate at a very coarse granularity (entire model dispatch including overheads) or would require both a vendor tool like Nsight and a trained eye to understand which parts of program execution to compare.
As Ben mentions, iree-run-module does "work" with GPU execution, with the usual caveats about measuring anything involving a GPU/accelerator/remote system:

the reported CPU time includes overheads from building command buffers, submitting those to the GPU, uploading data, synchronizing when complete, reading data back, etc.
depending on the computation (e.g. batch size, image dimensions, computational complexity) those overheads could totally eclipse what's actually happening in the kernels
reported measurements may be difficult to interpret, particularly if conditions are not controlled (e.g. other programs are running, network has variable latency, thermal throttling on mobile systems)

That all being said, we are very interested in benchmarking, profiling, comparing with similar libraries, etc. and would be happy to help figure out ways to make these sorts of investigations easier.
cc @antiagainst (did I miss anything? xD)

Oh, I can also give a bit more context on why we'd like to be specific about what is measured and compared, particularly for GPU execution.
As ML models get increasingly complex (by supporting things like dynamic flow control, dynamic shapes, and variables), the overheads from building command buffers, scheduling, and moving data grow. Additionally, for inference on mobile/edge devices, small batch sizes and varying workloads can make it difficult to fully saturate the compute resources on a device. Together, this means that we're expecting to run in scenarios where the overheads are already fairly high compared to actual kernel execution costs. IREE is designed (though not exclusively) for those cases where model complexity is high and resources are limited. For example, we'd expect to see pretty different results analyzing the performance on an arithmetic heavy convolutional model compared to on a model including tree-based search algorithms.
We're also designing for streaming cases, where applications would do something like asynchronously push data into a model (e.g. frames of audio) then periodically poll for outputs (e.g. text transcriptions). The performance metrics such applications care about aren't as easily captured by a benchmark that could introduce artificial synchronization points.
See https://www.usenix.org/conference/osdi20/presentation/ma for similar ideas in a different project.
(Sorry if this feels like excess/unrelated detail, I may have got nerd sniped here...)

Thank you @ScottTodd for the very detailed explanation, this is super useful. As @antiagainst  suggested, I'll use iree_hal_semaphore_wait_with_deadline duration that I got from Tracy to compare against Tensorflow execution and keep an eye out for upcoming patches. Thank you

@nick-furry something else you'll want to make sure of is that you set the --vulkan_validation_layers=false flag - it's probably time we flipped it off by default and enabled it on the CI's/tests explicitly.


Dimension Data - Enable api version

Changes Title (replace this with a logical title for your changes)
Description
Allow specifying api version to use. such as
driver = cls('schong_platcaas', 'T3stst@r!', region='dd-au', api_version='2.4')
if no api version is provided, it will use the default version (usually the latest).
Validation is included to check for the oldest and latest version supported.
Status
Replace this: describe the PR status. Examples:

done, ready for review

Checklist (tick everything that applies)

[X] Code linting (required, can be done after the PR checks)
[X] Documentation
[X] Tests
[ ] ICLA (required for bigger changes)

thanks @samuelchong won't the API behaviours available affect the choice of API version? What's the other benefit of using this override?

Thanks @tonybaloney for the review.
We are in a situation where the data contract changes from version 2.3 and 2.4. In view of support and maintainability, I think it's better to have version control.
The ideas behind this is

if no api version is specified, then the latest version is used. This will imply breaking changes if it occurs. But use will benefit from the latest release
Having specify a version, consistency is assured with no changes required until the version is out dated and no longer supported.
We will avoid breaking changes as much as we can.

An example of use of this version control for backward compatibility
https://github.com/samuelchong/libcloud/commit/9cb7785e1707b7c688820a7a7a6a8a1df9c19597
Some other benefit:

oldest_api_version = 2.2
latest_api_version = 2.4
active_api_version = 2.3

Deprecate a version when it is outdated, or give warning for a period of time warning user to upgrade before it fall out of support.
Avoid complex logic to support X number of versions which will get messy when more versions are being released.
Allow testing of beta version by using latest_api_version
Default version (stable version) is set to active_api_version unless user specify one

Further feedback and comment welcome.

ah i see, yes if you're also planning on building logic into the driver around the models and behaviours of the API.
the distutils package has a LooseVersion class which has a __cmp__ method for comparing values, this would be cleaner than using the float( converter all over the place
http://epydoc.sourceforge.net/stdlib/distutils.version.LooseVersion-class.html
I don't know if it's supported in 2.6, but work checking. Otherwise implement a separate version class method to do __cmp__ comparison within libcloud.common.dimensiondata

also implement __lt__ and __eq__ for python 3. https://docs.python.org/3/whatsnew/3.0.html#ordering-comparisons

thx @tonybaloney. I will make the changes and resubmit for review.

Tested  LooseVersion('2.2') > LooseVersion('2.1') with python 2.6, 2.7 and 3.5.

Tested with python version 2.6, 2.7, 3.5.
LooseVersion('2.4') > LooseVersion('2.2')

Hi @tonybaloney, I have used a customised version of LooseVersion in DD common class. A workaround to unsupported disutils.version issues in travis ci pylint.

ok :+1:


Add Manastorm to the games list

Hello, would you add my game Manastorm to your list in the README?  Steam page is http://store.steampowered.com/app/548560/
Thanks.

Added this in this PR https://github.com/thestonefox/VRTK/pull/729


Row selection documentation is broken

I want to look at the row selection documentation but the page is broken -> https://vincjo.fr/datatables/examples/row-selection
Can you fix it or explains how to add a row selection please ?

Thx for notifying, just fixed the documentation page:
https://vincjo.fr/datatables/examples/row-selection
Bellow is a basic setup:
// stores:
const selected = handler.getSelected()
const isAllSelected = handler.isAllSelected()

<!-- selectAll checkbox: -->
<input type="checkbox"
    on:click={() => handler.selectAll()}
    checked={$isAllSelected}
/>

<!-- rows markup -->
<tbody>
    {#each $rows as row}
        <tr class:active={$selected.includes(row)}>
            <td>
                <input type="checkbox"
                    on:click={() => handler.select(row)}
                    checked={$selected.includes(row)}
                />
            </td>
            [...]

Thanks a lot for your answer and your lib !
Nice work :)


Unable to set Pact Broker Context path

We are trying to set the env variable PACT_BROKER_BASE_URLS to set a context path for running the broker at a custom endpoint, but looks like the variable is misconfigured in the helm chart. It should be PACT_BROKER_BASE_URL.
By setting PACT_BROKER_BASE_URLS to a custom path, the css & js gets loaded from that path, but it's still accessible on /
Also, in ingress there is no way to set any other Prefix other than /, so its difficult to configure pact to a custom path, considering we can't customize ingress or pass custom env variables to the helm chart

@ChrisJBurns Can you take a look at it? Is that something planned to be implemented soon?

Hi @manali14 so I'd raise the Ingress issue and custom env vars separately if that's ok? Just so we can attach the right PR to them.
With regards to the PACT_BROKER_BASE_URLS vs PACT_BROKER_BASE_URL, I'm not sure if I understood what you're asking, are you saying that we have to rename PACT_BROKER_BASE_URLS to PACT_BROKER_BASE_URL?

@ChrisJBurns Yes, should be fine
With regards to the PACT_BROKER_BASE_URLS vs PACT_BROKER_BASE_URL, the pact official docker image accepts the latter env variable as mentioned here: https://hub.docker.com/r/pactfoundation/pact-broker#:~:text=PACT_BROKER_BASE_URL but the helm chart isn't compatible with this change & not allowing to set the context path

@manali14  I have resolved the environment variable issue, would you be able to raise the separate issues for the ingress as the extra env vars?

Hmm, if the variable is being set, not sure if there's much we can do in the Helm chart :( you may have to raise it on the Pact Slack? There may be some common issues people have with it?


make doc gen repro

Jenkins, retest this please

[ ] Is it mergeable?
[ ] Did it pass the tests?
[ ] If it introduces new functionality in scripts/ is it tested?
Check for code coverage with make clean diff-cover
[ ] Is it well formatted? Look at make pep8, make diff_pylint_report,
make cppcheck, and make doc output. Use make format and manual
fixing as needed.
[ ] Did it change the command-line interface? Only additions are allowed
without a major version increment. Changing file formats also requires a
major version number increment.
[ ] Is it documented in the ChangeLog?
http://en.wikipedia.org/wiki/Changelog#Format
[ ] Was a spellchecker run on the source code and documentation after
changes were made?

@camillescott @luizirber @ctb Ready for review and merge

LGTM, just need to fix Changelog for merging.

Thank you @luizirber


JSX or TSX is not compiled

Hi, I'm having a packaging problem,.tsx is not compiled, do I need to do additional configuration or is there a reason for this

The mkdist builder does not support JSX or TSX compilation. I am not sure if the rollup builder supports it either 🤔

TSX support for mkdist added  by https://github.com/unjs/mkdist/pull/71