Allow Xpoa-block-txs-selection-max-time to be greater than 100%

PR description
In #6044 Xpoa-block-txs-selection-max-time options has been introduced to limit the block creation time to a fraction of the genesis block time, but there are cases where it is useful to let the block creation time last more, so with this PR is it possible to specify value greater than 100%
Fixed Issue(s)

@pinges because there could be existing networks that are used to have txs that take more than the genesis block time to process, and/or do not care too much about a fixed block time, but more about a target block time, allowing outliers.

Close in favor of #6423


Add docker-compose

It's about time I made a production-ready one-click setup for CM.
CM can be run as-is, without docker-compose, because it uses sensible defaults and can generate dependencies as plain files (sqlite db) or use memory (memory cache) when none is provided.
However, to run in production, CM really should have proper databases and cache dependencies to in order to scale properly. DB should be mysql/postgres and cache should be Redis.
Additionally, adding (optional) reverse image/full text search would require elasticsearch as a dependency #101 without a substitute available.
A docker-compose file would make all these production dependencies available with one-click setup and I can make sure they are configured correctly.

Implemented in e11a1d2a52abec1de50efa76bb3f9262168f1685


Fix clblas and libgpuarray on darwin

Motivation for this change
This enables the builds for clblas and libgpuarray, which consequently should also lead to
working python2Packages.Theano/python3Packages.Theano.
This fixes the issue #33073.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build python3Packages.Theano python2Packages.Theano python27Packages.libgpuarray python3Packages.libgpuarray clblas

@GrahamcOfBorg build python3Packages.Theano python2Packages.Theano python27Packages.libgpuarray python3Packages.libgpuarray clblas

Failure on aarch64-linux (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
  ^~~~~~~
acquired default
acquired Global_A
building of '/nix/store/g9ccyb85m5kxbfb3gy26840d90qxsf9k-python2.7-Cython-0.28.1.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/qvqv66l4irjx0q8is7r5s8gqas31wx44-python2.7-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vvq3v4s85dyrv8qwz7ajw2d9inaqcwvq-python2.7-Theano-1.0.1.drv': 1 dependencies couldn't be built
building of '/nix/store/l60jan1mgs2yd0lnkmsprzjapx4rynv9-python3.6-Cython-0.28.1.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/sa9kks3cph0dl1ak04azzzp9ln0m9hsv-python3.6-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/grwj7zyg7nwvwwgglgr6w4jgcrpfk9ac-python3.6-Theano-1.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/grwj7zyg7nwvwwgglgr6w4jgcrpfk9ac-python3.6-Theano-1.0.1.drv', '/nix/store/qvqv66l4irjx0q8is7r5s8gqas31wx44-python2.7-libgpuarray-0.7.5.drv', '/nix/store/sa9kks3cph0dl1ak04azzzp9ln0m9hsv-python3.6-libgpuarray-0.7.5.drv', '/nix/store/vvq3v4s85dyrv8qwz7ajw2d9inaqcwvq-python2.7-Theano-1.0.1.drv' failed

Failure on aarch64-linux (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
  ^~~~~~~
acquired default
acquired Global_A
building of '/nix/store/g9ccyb85m5kxbfb3gy26840d90qxsf9k-python2.7-Cython-0.28.1.drv' timed out after 3600 seconds
building of '/nix/store/l60jan1mgs2yd0lnkmsprzjapx4rynv9-python3.6-Cython-0.28.1.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/qvqv66l4irjx0q8is7r5s8gqas31wx44-python2.7-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/sa9kks3cph0dl1ak04azzzp9ln0m9hsv-python3.6-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vvq3v4s85dyrv8qwz7ajw2d9inaqcwvq-python2.7-Theano-1.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/grwj7zyg7nwvwwgglgr6w4jgcrpfk9ac-python3.6-Theano-1.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/grwj7zyg7nwvwwgglgr6w4jgcrpfk9ac-python3.6-Theano-1.0.1.drv', '/nix/store/qvqv66l4irjx0q8is7r5s8gqas31wx44-python2.7-libgpuarray-0.7.5.drv', '/nix/store/sa9kks3cph0dl1ak04azzzp9ln0m9hsv-python3.6-libgpuarray-0.7.5.drv', '/nix/store/vvq3v4s85dyrv8qwz7ajw2d9inaqcwvq-python2.7-Theano-1.0.1.drv' failed

Failure on aarch64-linux (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
memoryview_inplace_division.cpp:1599:28: note: in expansion of macro ‘PyMem_NEW’
         op->data.ob_item = PyMem_NEW(char, nbytes);
                            ^~~~~~~~~
building of '/nix/store/g9ccyb85m5kxbfb3gy26840d90qxsf9k-python2.7-Cython-0.28.1.drv' timed out after 3600 seconds
building of '/nix/store/l60jan1mgs2yd0lnkmsprzjapx4rynv9-python3.6-Cython-0.28.1.drv' timed out after 3600 seconds
cannot build derivation '/nix/store/qvqv66l4irjx0q8is7r5s8gqas31wx44-python2.7-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/sa9kks3cph0dl1ak04azzzp9ln0m9hsv-python3.6-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vvq3v4s85dyrv8qwz7ajw2d9inaqcwvq-python2.7-Theano-1.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/grwj7zyg7nwvwwgglgr6w4jgcrpfk9ac-python3.6-Theano-1.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/grwj7zyg7nwvwwgglgr6w4jgcrpfk9ac-python3.6-Theano-1.0.1.drv', '/nix/store/qvqv66l4irjx0q8is7r5s8gqas31wx44-python2.7-libgpuarray-0.7.5.drv', '/nix/store/sa9kks3cph0dl1ak04azzzp9ln0m9hsv-python3.6-libgpuarray-0.7.5.drv', '/nix/store/vvq3v4s85dyrv8qwz7ajw2d9inaqcwvq-python2.7-Theano-1.0.1.drv' failed

Failure on x86_64-darwin (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
pep448_test_extcall.cpp:12571:26: warning: code will never be executed [-Wunreachable-code]
                module = PyImport_ImportModuleLevelObject(
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
building of '/nix/store/x9ff903kpnmmvf39cdw4pbp08rl30mmh-python3.6-Cython-0.28.1.drv' timed out after 1800 seconds
cannot build derivation '/nix/store/33fsl0n5nmyiwvkzq4mh80v1nzb55r14-python3.6-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/snhqdl4ihcba1cr9ky1rwavlv052mi5q-python3.6-Theano-1.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/33fsl0n5nmyiwvkzq4mh80v1nzb55r14-python3.6-libgpuarray-0.7.5.drv', '/nix/store/snhqdl4ihcba1cr9ky1rwavlv052mi5q-python3.6-Theano-1.0.1.drv' failed

Failure on x86_64-darwin (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
r_vree_1.cpp:1732:26: warning: code will never be executed [-Wunreachable-code]
                module = PyImport_ImportModuleLevelObject(
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
building of '/nix/store/x9ff903kpnmmvf39cdw4pbp08rl30mmh-python3.6-Cython-0.28.1.drv' timed out after 1800 seconds
cannot build derivation '/nix/store/33fsl0n5nmyiwvkzq4mh80v1nzb55r14-python3.6-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/snhqdl4ihcba1cr9ky1rwavlv052mi5q-python3.6-Theano-1.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/33fsl0n5nmyiwvkzq4mh80v1nzb55r14-python3.6-libgpuarray-0.7.5.drv', '/nix/store/snhqdl4ihcba1cr9ky1rwavlv052mi5q-python3.6-Theano-1.0.1.drv' failed

Failure on x86_64-darwin (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
                module = PyImport_ImportModuleLevelObject(
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
clang-5.0: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
building of '/nix/store/x9ff903kpnmmvf39cdw4pbp08rl30mmh-python3.6-Cython-0.28.1.drv' timed out after 1800 seconds
cannot build derivation '/nix/store/33fsl0n5nmyiwvkzq4mh80v1nzb55r14-python3.6-libgpuarray-0.7.5.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/snhqdl4ihcba1cr9ky1rwavlv052mi5q-python3.6-Theano-1.0.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/33fsl0n5nmyiwvkzq4mh80v1nzb55r14-python3.6-libgpuarray-0.7.5.drv', '/nix/store/snhqdl4ihcba1cr9ky1rwavlv052mi5q-python3.6-Theano-1.0.1.drv' failed

@Mic92: The ofBorg builds timed out, because of an unrelated Cython rebuild. I just rebased locally on upstream/master and could get cython from cache.nixos.org. Would a new "ofBorg build" pick up the now cached cython or do I need to force-push the rebase?

Success on x86_64-linux (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/lib  /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/bin
patching script interpreter paths in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1
/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/lib/python3.6/site-packages/theano/misc/check_blas_many.sh: interpreter directive changed from "/bin/bash" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/bash"
head: error writing 'standard output': Broken pipe
checking for references to /build in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1...
wrapping `/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/bin/theano-nose'...
wrapping `/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/bin/theano-cache'...

Success on x86_64-linux (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
/build/Theano-1.0.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/lib  /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/bin
patching script interpreter paths in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1
/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/lib/python3.6/site-packages/theano/misc/check_blas_many.sh: interpreter directive changed from "/bin/bash" to "/nix/store/xn5gv3lpfy91yvfy9b0i7klfcxh9xskz-bash-4.4-p19/bin/bash"
checking for references to /build in /nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1...
wrapping `/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/bin/theano-cache'...
wrapping `/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1/bin/theano-nose'...

Success on x86_64-linux (full log)
Attempted: python3Packages.Theano, python2Packages.Theano, python27Packages.libgpuarray, python3Packages.libgpuarray, clblas
Partial log (click to expand)
/nix/store/9af4c4hfrphrqdsl1gad9gss6d8vabfg-python3.6-Theano-1.0.1
/nix/store/jygnxkykpy37dk1vs22wyx0bfk3f2zva-python2.7-Theano-1.0.1
/nix/store/32qadyyrp9r4cm4hwz0zpgdp61ah70ff-python2.7-libgpuarray-0.7.5
/nix/store/l464agyffh5qq2pr1jw92h2fdsx6qc9x-python3.6-libgpuarray-0.7.5
/nix/store/qagkks3v46fha3x0jm198ws359jg6ym5-clblas-2.12

@Mic92 yes, unsupported packages are usually just never tested so backporting that is pretty safe.
Backported to 18.03 in 3fd2b19d39a.


net/http: request context is not propagating deadlines

Consider the following program:
go func() {
    req, _ := http.NewRequest("GET", "http://localhost:8080", nil)
    ctx, cancel := context.WithDeadline(req.Context(), time.Now().Add(5*time.Second))
    defer cancel()

    req = req.WithContext(ctx)
    _, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Println(err)
    }

}()

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    fmt.Println(r.Context().Deadline())
    select {
    case <-r.Context().Done():
        fmt.Fprintln(w, "done")
    }
})
http.ListenAndServe(":8080", nil)

The output of this program is:
0001-01-01 00:00:00 +0000 UTC false
2017/03/15 10:44:55 Get http://localhost:8080: context deadline exceeded

Instead, I expected to see a non-zero deadline and "done" written to the response body after 5 seconds.
Any plans to inject the deadline in a preserved header to the outgoing requests, so the handler can automatically extract them and construct the incoming request context from there.
/cc @bradfitz

I am unaware of any standards in this area and am reluctant to invent a Go-specific one.

Agreeing with not inventing one if there is not an established standard.


Add a privacy policy and cookie acceptance banner

I have no clue if this is GDPR compliant. The website might need one of those accept cookies banners at the bottom of the screen.

Checked. GDPR isn't about the Cookie Banners by the way, that's the ePrivacy Directive, but as you don't set any cookies you're fine with that as soon as the Google Tag Manager is removed.
What you'd need to comply with the GDPR would be a Privacy Policy on which data you collect and for what reasons and how long (eg. IP addresses are saved for 7 days to protect against attacks), as well as a point of contact for users to request their information (see https://en.wikipedia.org/wiki/General_Data_Protection_Regulation#Rights_of_the_data_subject)

Checked. GDPR isn't about the Cookie Banners by the way, that's the ePrivacy Directive, but as you don't set any cookies you're fine with that as soon as the Google Tag Manager is removed.
What you'd need to comply with the GDPR would be a Privacy Policy on which data you collect and for what reasons and how long (eg. IP addresses are saved for 7 days to protect against attacks), as well as a point of contact for users to request their information (see https://en.wikipedia.org/wiki/General_Data_Protection_Regulation#Rights_of_the_data_subject)

Checked. GDPR isn't about the Cookie Banners by the way, that's the ePrivacy Directive, but as you don't set any cookies you're fine with that as soon as the Google Tag Manager is removed.
What you'd need to comply with the GDPR would be a Privacy Policy on which data you collect and for what reasons and how long (eg. IP addresses are saved for 7 days to protect against attacks), as well as a point of contact for users to request their information (see https://en.wikipedia.org/wiki/General_Data_Protection_Regulation#Rights_of_the_data_subject)

Oh great, thankyou very much for helping me out with that! Do you mean that I'd need to remove the Google Analytics to comply with the ePrivacy Directive? I'll have a better look into getting a proper privacy policy and hosting it. Thanks again for the help!

Checked. GDPR isn't about the Cookie Banners by the way, that's the ePrivacy Directive, but as you don't set any cookies you're fine with that as soon as the Google Tag Manager is removed.
What you'd need to comply with the GDPR would be a Privacy Policy on which data you collect and for what reasons and how long (eg. IP addresses are saved for 7 days to protect against attacks), as well as a point of contact for users to request their information (see https://en.wikipedia.org/wiki/General_Data_Protection_Regulation#Rights_of_the_data_subject)

Oh great, thankyou very much for helping me out with that! Do you mean that I'd need to remove the Google Analytics to comply with the ePrivacy Directive? I'll have a better look into getting a proper privacy policy and hosting it. Thanks again for the help!

Either remove Google Analytics, or add the Cookie Banner which has to be Opt-In to be valid.
As for the Privacy Policy, it doesn't have to be legalese. Article 12 (1)[even states that]

The controller shall take appropriate measures to provide any information referred to in Articles 13 and 14 and any communication under Articles 15 to 22 and 34 relating to processing to the data subject in a concise, transparent, intelligible and easily accessible form, using clear and plain language, in particular for any information addressed specifically to a child.

So just write up some bullet points for each data point you collect, why you collect them, and how long they're saved. Add them after a paragraph explaining the rights of the user (right to access under Article 15, rectification Article 16, …) and you're good.

Either remove Google Analytics, or add the Cookie Banner which has to be Opt-In to be valid.
As for the Privacy Policy, it doesn't have to be legalese. Article 12 (1)[even states that]

The controller shall take appropriate measures to provide any information referred to in Articles 13 and 14 and any communication under Articles 15 to 22 and 34 relating to processing to the data subject in a concise, transparent, intelligible and easily accessible form, using clear and plain language, in particular for any information addressed specifically to a child.

So just write up some bullet points for each data point you collect, why you collect them, and how long they're saved. Add them after a paragraph explaining the rights of the user (right to access under Article 15, rectification Article 16, …) and you're good.

Either remove Google Analytics, or add the Cookie Banner which has to be Opt-In to be valid.
As for the Privacy Policy, it doesn't have to be legalese. Article 12 (1)[even states that]

The controller shall take appropriate measures to provide any information referred to in Articles 13 and 14 and any communication under Articles 15 to 22 and 34 relating to processing to the data subject in a concise, transparent, intelligible and easily accessible form, using clear and plain language, in particular for any information addressed specifically to a child.

So just write up some bullet points for each data point you collect, why you collect them, and how long they're saved. Add them after a paragraph explaining the rights of the user (right to access under Article 15, rectification Article 16, …) and you're good.

Perfect, thankyou very much!
For future reference, we collect:

User IP addresses, time of accessing the page, and their user agent in the Apache server logs
Any data that the Google Analytics stores, although it might be good to remove this!

Either remove Google Analytics, or add the Cookie Banner which has to be Opt-In to be valid.
As for the Privacy Policy, it doesn't have to be legalese. Article 12 (1)[even states that]

The controller shall take appropriate measures to provide any information referred to in Articles 13 and 14 and any communication under Articles 15 to 22 and 34 relating to processing to the data subject in a concise, transparent, intelligible and easily accessible form, using clear and plain language, in particular for any information addressed specifically to a child.

So just write up some bullet points for each data point you collect, why you collect them, and how long they're saved. Add them after a paragraph explaining the rights of the user (right to access under Article 15, rectification Article 16, …) and you're good.

Perfect, thankyou very much!
For future reference, we collect:

User IP addresses, time of accessing the page, and their user agent in the Apache server logs
Any data that the Google Analytics stores, although it might be good to remove this!

I think we should switch to using an analytics tool such as https://usefathom.com/, which doesn’t set cookies, and collects much less information which is good because we don’t need that much.

I think we should switch to using an analytics tool such as https://usefathom.com/, which doesn’t set cookies, and collects much less information which is good because we don’t need that much.

As of 93b991b we now use self-hosted Matomo analytics, and the Google Analytics has been removed from the site.

A privacy policy has now been added as of e4563b0, which is hosted at https://royalur.net/privacy.
@pludi, if you had a spare moment I'd really appreciate it if you could look over it and make sure its all good :)

Looks good as far as I can tell!

Looks good as far as I can tell!

Awesome, thanks!


Middleware to cache the body for multiple passes

This keeps coming up. (e.g., #5190).  I don't recall that we've ever written a middleware that caches all the bytes and passes a re-readable body downstream.  If we haven't, let's stop handwaving at it and provide it.

it would be nice if this middleware is also aware of http cache headers...

it would be nice if this middleware is also aware of http cache headers...

I don't think that's applicable here? This is memoizing an individual message body as an implementation detail for handling it correctly, not caching in the http sense.

it would be nice if this middleware is also aware of http cache headers...

I don't think that's applicable here? This is memoizing an individual message body as an implementation detail for handling it correctly, not caching in the http sense.

Still, knowledge of the fact a response hasn't changed (because it has been cached / memoized) is something the transport layer (HTTP) is able to share....
I agree there are situations where you still want the client to revalidate / query again, etc.. but curiously enough that is also in the HTTP cache spec.... :D

Perhaps it's possible to reuse ContextRequest here like with AuthedRequest:
type CachedRequest[F[_]] = ContextRequest[F, Array[Byte]]

This way the user still has the ability to compile the body again, but at the same can use the cached one.

A second compilation is not going to work for a socket-backed stream, and whoever goes second might not know that they're going second.  I think we want to make it transparent.

Well, I did some thinking and came to this dead simple solution for the server caching middleware. It literally fits in one snippet:
import cats.data.Kleisli
import cats.effect.Concurrent
import fs2.Stream
import org.http4s.HttpApp
import cats.implicits._

  def cachingMiddleware[F[_]: Concurrent]: HttpApp[F] => HttpApp[F] =
    app =>
      Kleisli {
        req =>
          for {
            body <- req.body.compile.toList
            cachedReq =
              req.withBodyStream(Stream.evalSeq[F, List, Byte](body.pure[F]))
            resp <- app(cachedReq)
          } yield resp
      }

Of course, it needs some work done yet (docs, performance, etc.) but I think you get the idea. Would it be sufficient? If so, then I could make a PR.

I was playing with a transparent caching fs2 pipe as well....
https://gist.github.com/kryptt/e20e861df414a643503112b408abde38

@FunFunFine @kryptt so, who's going to PR :)

Nothing about @kryptt's looks specific to HTTP.  Something around it that understood HTTP caching directives would make it a better candidate for http4s, but the current gist might be more appropriate for something upstream.
@FunFunFine's integrates with the request and would definitely be in scope for http4s.  I have some feedback on it that's probably easier done on a live PR.

Yes, what I was doing was a general caching of fs2 streams... unfortunately scalacache is having a hard time getting any updates it seems...


Feature/upgrade to Material-UI version 5

Description
Update Material-ui from v.4 to v.5
Related issues
Closes https://github.com/CSCfi/metadata-submitter-frontend/issues/542
Type of change

[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update (this needs a follow up PR)

Changes Made

The changes follow these steps: https://mui.com/guides/migration-v4/#update-mui-version
Main packages are renamed from:

@material-ui/core to @mui/material
@material-ui/lab to @mui/lab
@material-ui/icons to @mui/icons-material

Related codes in all files are updated by using codemod: npx @mui/codemod v5.0.0/preset-safe <path | folder>
Important:

MUI v.5 uses emotion as the style library and the support for JSS utility like makeStyles and withStyles will be deprecated from the next version (still in discussion). MUI v.5 recommends other ways to style the components which are styled or sx. More info here: https://mui.com/customization/how-to-customize/
Since we are in the slow progress of switching to the new UI, I would suggest not to apply the new ways of styling (using sx or styled) for all the files now, instead, making changes when we implement each new UI's component/feature. This way we could avoid time consuming and the risk of bugs.
Document for new MUI will need to be updated afterwards.

Testing

[x] Unit Tests
[x] Integration Tests

Regarding the header "Changes Made" Is @mui/lab in use at the moment? At least Alert is at "@mui/material/Alert" older version got path "@material-ui/lab/Alert" and  with "@mui/material/Autocomplete" older was "@material-ui/lab/Autocomplete".

Old stepper:

MUI 5 stepper:

I'm not sure if it makes any sense to try to fix these to match original styles at this point as we're going to do whole design overhaul?

In uploading file the text is white and the "Submit" button is white.

#544 is  needed for tests to pass

Regarding the header "Changes Made" Is @mui/lab in use at the moment? At least Alert is at "@mui/material/Alert" older version got path "@material-ui/lab/Alert" and with "@mui/material/Autocomplete" older was "@material-ui/lab/Autocomplete".

Good point! With the new MUI, Alert and Autocomplete have been moved from "@mui/material", but we still need the @mui/lab. One of my developing branch is using it.

Old stepper:
MUI 5 stepper:
I'm not sure if it makes any sense to try to fix these to match original styles at this point as we're going to do whole design overhaul?

I think it might be unnecessary to fix design related issues (if issue doesn't prevent developing new features) before doing the new layout.

--> Yes, I think it would be better to fix the design related issues while doing the new UI as it would probably change totally.


Full sentence needs a period

Full sentences need a period
Browser metadata
Path:      /create
Browser:   Chrome <IP_ADDRESS> on Mac OS 10.15.7
Viewport:  1720 x 717 @1x
Language:  en-US
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved
Originally posted by @netlify[bot] in https://github.com/decentdao/fractal-interface/issues/1578#issuecomment-2101801880

This is purely personal preference, but I disagree.
This is purely personal preference, but I disagree

Something I read a while back - that made sense to me in retrospect - about communication via text and the difference in perception between single-sentence texts sent with and without a period: One of them feels... abrupt, almost standoff-ish. The other more, I dunno, friendly? Mentally parseable?
Or maybe it's just me. Not a hill I'm willing to die (or even stay) on.

I think full sentences need periods too.

Closed via https://github.com/decentdao/fractal-interface/pull/1760


A alphabet not able to type in vs code

Type: Feature Request
A alphabet not able to type in vs code. Showing error like "Missing Label predicates no folder opened text"
VS Code version: Code 1.75.1 (441438abd1ac652551dbe4d408dfcec8a499b8bf, 2023-02-08T21:32:34.589Z)
OS version: Windows_NT x64 10.0.22000
Modes:
Sandboxed: No

Thanks for creating this issue! We figured it's missing some basic information or in some other way doesn't follow our issue reporting guidelines. Please take the time to review these and update the issue.
Happy Coding!

Maybe this is from an extension, you can use the "Start Extension Bisect" command to figure out which one

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Return row count when inferring schema from JSON

Which issue does this PR close?
Closes #5007.
What changes are included in this PR?
This PR changes the return type of infer_json_schema{,from_seekable}() from Schema to (Schema, usize).  These functions are public, so this is a user-facing change.  The docs have been updated.

Yes, good call.

Thank you


Incorporate custom direct methodology

This change is

Current dependencies on/for this PR:

main

PR #1226   👈

This comment was auto-generated by Graphite.


Fix issue #171 calculating checksum before any controls

Moved super().__init__ before calling _calculate_checksum() to check if the string contains any letters to prevent value errors that can happen when calculating checksum. I tested the new version with the input<PHONE_NUMBER> and the produced barcode before and after the commit is the same  as<PHONE_NUMBER>.

Is the error because the checksum contains letters?

This merely changes the error raised to be the correct one, if I understand correctly, right?

This merely changes the error raised to be the correct one, if I understand correctly, right?

Yes instead of getting invalid literal for int() with base 10:  now it returns IllegalCharacterError('EAN code can only contain numbers.')

Makes sense, thanks


Video element turns very small when a participant leaves

When layout() is called on event streamDestroyed the video element turn into a tiny square

Can you link me to an example and give me steps to repro?

I am not how to share my entire code here, but I can share privately with you. I am using the latest Angular and calling layout each time a new stream is added(which works fine and re arranges it nicely) but when a stream is destroyed and the layout function is called the element width and height becomes 0. One thing that is different from the example is I don't call layout when my own publisher is initialized and neither have I included the publisher div in the layout container. The application I am trying to make is video conferencing and the person's own video is always small on top right corner.


"Save to Git" button not opening modal

Description
"Save to Git" button not opening Modal
Steps to reproduce

Open start app and open web app
click "Start new family tree"
click "Save to Git"

Actual result
Nothing happens on page.
console error as follows:
vue.js:634 [Vue warn]: Error in v-on handler: "TypeError: Cannot read properties of null (reading 'open')"

(found in <Root>)
warn @ vue.js:634
logError @ vue.js:1893
globalHandleError @ vue.js:1888
handleError @ vue.js:1848
invokeWithErrorHandling @ vue.js:1871
invoker @ vue.js:2188
original._wrapper @ vue.js:7547
vue.js:1897 TypeError: Cannot read properties of null (reading 'open')
    at Vue.openGitModal ((index):449)
    at Vue.saveGit ((index):509)
    at click (eval at createFunction (vue.js:11649), <anonymous>:3:6790)
    at invokeWithErrorHandling (vue.js:1863)
    at HTMLAnchorElement.invoker (vue.js:2188)
    at HTMLAnchorElement.original._wrapper (vue.js:7547)

Expected result
Open git save modal.

I'm not sure what is exactly the problem but new-person modal is breaking git one. I moved <new-person tag after modalGit div and both work

Sorry, I havent seen your PR. This is exactly that Ive found

This was still an issue on master, PR'd @Zarzuelo's changes in #10

I was about to file a bug report that the save to git button doesn't do anything when I saw this.  Hopefully it will be fixed soon.

As of November 2023, this issue appears to still be a problem.

I hope someone still maintains this because this would be a great feature.  I would love to commit a Dockerfile to make this trivial to deploy for people.

@Leopere feel welcome to commit the dockerfile to https://gitlab.com/norcivilian-labs/genea as well. We maintain a fork with vue3 and bugfixes while we wait for this to get a new maintainer.


Aggregate es6 modules in workbox cli and build

For example;
import {a, b , c,  d } from './lib'

by re-exporting modules in an index.ts in a root folder with many modules.

Hello @mayeedwin!
The public entry point for workbox-build already does this: https://github.com/GoogleChrome/workbox/blob/v6/packages/workbox-build/src/index.ts
(There is no public interface for workbox-cli, as it's meant to be run from the command line and doesn't export anything.)
The modules within the src/lib/ subfolders for each project are just imported internally, and are effectively "private". Could you explain the advantage of restructuring them in the manner you propose?

For the src/lib, no significant perf or related improvement but it's more of a formatting improvement. Thanks.

I'd prefer not to go ahead with this change, as we've just focused on exporting from and index for our public interfaces.
Thanks for the suggestion, though!

Sounds good! Thanks!


Add logs over commands

I'm tackling #372 with a twist:

Long running command is at INFO
Commands per se are at DEBUG level

I think it gives enough feedback for day-to-day use, but allows to have a more verbose if needed.
Regarding design, it introduces a lot of burden, I wonder if I'd better to integrate it in Cradle.
WDYT?

Sorry for the long delay! I just kept putting off the work to review :sweat_smile:

The proposed changes are breaking changes, while there is nothing wrong with breaking changes in hie-bios, I would like to keep the breaking change as minimal as possible.
It seems like you do a lot of parameter passing just to add it to runGhcCmd. How about we add it as a paramater to runGhcCmd, because otherwise it feels redundant and weird to pass a log function to runCradle and pass the log action for runGhcCmd via the cradle generation. I think that'd make it more consistent. That, or making runCradle to also use the log action passed in to the Cradle creation function.
Otherwise, lgtm, thank you for your contribution!

No worries, thanks for your feedback, I have pushed a new version integrating them.


how many pdfs were supported?

i have many pdf files , it's series of tech books , about 1000 or more files . thanks

Hello, theoretically IncarnaMind can support an unlimited number of documents. However, in the current version, we have included all file names in the prompt. The supported number of documents depends on the different LLMs' token limits. We have planned to improve this issue in future versions. If you want to upload a large quantity of text now, the simplest method is to comment out line 330 in docs2db.py:
# file_names.append(file_name_with_extension)


.NET 6 and Function v4

Description
Upgrade to .NET 6 and Function v4.
References
https://app.zenhub.com/workspaces/mighty-ducks---the-outlaws-6193fe815d79fc0011e741b1/issues/energinet-datahub/the-outlaws/3

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Dan Stenrøjl seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Add transpilation and optimization (minification, etc) to build pipeline

px-data-grid currently serves the development versions of polymer components to component consumers.
To be consistent with other components, and to reduce integration effort for px-data-grid users on legacy browsers, we should transpile px-data-grid JS down to ES5 so that it works on all target browsers out-of-the-box.

CC @sebastiantb, I understand you've had to transpile the component JS on your project for IE11 users. If there's interest in deferring that to the component (ie. making it so that component always serves transpiled/IE11 friendly code) let me know.

That's a bad idea... minification and transpilation are an app concern, not that of the component.

Hey evan thanks for the help,  solving this issue really helped us out.
Right now, we have a transpilation gulp task that grabs the bundled elements.html and transpile the javascript. we are serving 2 versions of the javascript one in es6 for browsers that supports it and the other in  es5 for browsers that do not support that feature. We also had to add webpack css post processing plugins in order to make the elements display styling correctly.  maybe we can add some a wiki page at the main page showcasing how to successfully transpile these components. steven davie told me you need to add support to ie 11 as well. Let me know if you need any help.

there are more components that are written in es6, I think changing all of them will be a lot of work

@mdwragg Hi! Fair call. I guess the issue here is the inconsistency with how we distribute components. Some transpile, some don't. I'm fine with just picking one and standardising on that.

Yeah, I’m surprised the maintainers did this, it certainly wouldn’t have happened back in the day. Thoughts maintainers?

No comments from the broader group, closing as I agree it's a bad idea. Have asked for this to be raised at next PDS meeting.


python3Packages.cassandra-driver: 3.26.0 -> 3.28.0

Description of changes
This technically adds a breaking change for the column encryption feature, but seeing as that feature was only added in 3.27.0 which we missed, it shouldn't affect nixpkgs users.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

@ofborg build python311Packages.cassandra-driver

Result of nixpkgs-review pr 266334 at 0981d534 run on aarch64-darwin 1

2 packages failed to build:

python310Packages.cassandra-driver
python311Packages.cassandra-driver


Berlin 24-Hour Hackathon

Solana Foundation will be hosting an IRL overnight hackathon in Berlin on 21-22nd May during Berlin Blockchain Week.
65 developers of all skill levels will have the opportunity to compete for a prize pool of over $15,000 from the Solana Foundation and our ecosystem projects. This is a unique opportunity to chew glass, learn from some of the best Solana developers in the ecosystem, win prizes and have fun!
AGENDA
The Hackathon will start at 12:00 on May 21st and continue through the night until May 22nd at 13:00 when teams will present their pitches.
May 21st:
12:00 | Welcome & Lunch
13:30 | Speed Networking & Team Building
14:00 | Hacking Starts
15:00 - 16:00 | Technical Office Hours
16:00 -18:00 | Pitch VC Feedback
19:00 | Dinner
22:00 - 23:00 | Technical Office Hours
24:00 - 1:00 | Disco Rave at the Hackathon
Hacking continues through the nightmorning
May 22nd:
7:00 | Breakfast
9:00 - 10:00 | Pitch VC Feedback
10:00 - 11:00 | Technical Office Hours
12:00 | Lunch
13:00 | Pitches begin
Winner Announcements & Closing

WHAT TO EXPECT
The 24-hour hackathon will be hosted on two floors of the w3 hub in Berlin. The 3rd floor is equipped with adjustable workstations, monitors, and office rooms for technical office hours and VC pitch sessions—everything you need for a productive hackathon! The 4th floor will include food stations, sleeping areas for when you need a late-night rest, and fun activities to let off steam. Lunches, Dinner, and Breakfast will be served at the venue, and snacks, coffee, and Red Bulls will always be in full supply!
Technical office hours: Two veteran Solana Developers will be onsite to help answer questions and their time can be booked through the two days for technical support.
VC pitch feedback sessions: VC’s will be available during two time slots where you can get 1:1 direct feedback on your pitches.
Final Pitch: You will have a chance to pitch your project to the panel of judges (5 min pitch per team). Judges will vote to select a winning team in each of 3 categories.
PRIZES & TRACKS:
$15k+ prize pool split across three tracks:
-Decentralized Finance & Payments
-Consumer Apps & NFTs
-Public Goods
Who can apply?
You can apply as an individual or also join with a pre-existing team. Teams & individuals with pre-existing projects are welcome to join, but judges will only review the work that is developed during the 24 hours.
Inexperienced devs are welcome and encouraged to apply, there will be ample technical support and experienced devs to join forces with.
Applications will be approved on a rolling basis and spaces are limited, make sure to apply ASAP.
https://lu.ma/berlin-24hour-hackathon

Can you please use the template?
https://github.com/blockchainweek/data/issues/new/choose

Hey could you guide me trough what i did wrong? This template is a bit
confusing to me.
Thanks in advance, best
Carlo
On Tue 30. Apr 2024 at 14:03, Afri @.***> wrote:

Can you please use the template?
https://github.com/blockchainweek/data/issues/new/choose
—
Reply to this email directly, view it on GitHub
https://github.com/blockchainweek/data/issues/37#issuecomment-2085148716,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A53DUTJQYHDJ2TURMQW245TY76B7ZAVCNFSM6AAAAABG36YWD6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBVGE2DQNZRGY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Ok, no worries, I'll add it.


More Details about how to the use the loginErrorDisplayConfig

Describe the desired behavior
Add a comment or some additional information about how to use this property.  For example, the default error message of LOGIN.INVALID_CREDENTIALS doesn't work with this property enabled.
Describe the current behavior
There's little information around this.
Is this request related to a current issue?
No
Additional Context

for next sprint


Missing section for webhooks with secret_scanning_alert or secret_scanning_alert_location

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads
What part(s) of the article would you like to see updated?
There's no documentation for webhook with X-GitHub-Event: secret_scanning_alert or X-GitHub-Event: secret_scanning_alert_location!
Additional information
This blog post https://github.blog/changelog/2022-04-05-secret-scanning-webhooks-for-alert-locations/ tries to link to https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#secret_scanning_alert_location but the secret_scanning_alert_location section doesn't exist!
We are trying to add support for secret scans a fork of this repo https://github.com/github/ghas-jira-integration, but we are missing crucial information, like the valid actions for that event type!  For example, a code_scanning_alert can have the fixed action, but in secret_scanning_alert we have resolved.  It would be nice to have this documented clearly instead of reading webhook recent deliveries to find out!

@tempuslabs-devops @mstandley-tempus Thanks so much for opening an issue! I'll triage this for the team to take a look :eyes:

Hi @mstandley-tempus - thanks for raising this omission in the docs and including a link to the blog post. It's really helpful to have the extra context ✨
I'll follow up on this with the engineering team.

@mstandley-tempus - I spent some time looking into this and can see what's happened.
Secret scanning only generates alerts for users of an enterprise version of GitHub. So the webhooks are only useful to people using enterprise versions of GitHub. Consequently, the webhooks are only listed in enterprise versions of the article, for example: GitHub Enterprise Cloud.
It's clear that the link in the blog post is to the wrong version of the documentation 🤦🏻♀
I'll look at getting the blog post link updated asap and highlight this to the product manager for this feature so that we avoid this mistake in the future.
Thank you so much for taking the time to let us know about this error. I hope that the link above gives you the information you're missing 🤞🏻

@mstandley-tempus - we've corrected the link from the changelog to the article about web hooks.
I don't think that there are any other actions to take here, but feel free to reopen this issue if I've missed something and we can continue the discussion.
Wishing you the best of luck with your integration.

Understood, thank you!


[Bug Report] GLib-Error: creating thread 'mpv_render_context_set_update_callback': Resource temporarily unavailable

It's running out of memory gradually, even though I see dispose messages for every controller.

I am on Ubuntu 22.04 LTS, media_kit master, mpv 0.34.1, FFmpeg version: 4.4.2-0ubuntu<IP_ADDRESS>.
I am having a controller for each player. Gradually, over the period of 1-2 minutes, the app crashes with
GLib-ERROR **: 20:30:54.533: creating thread 'mpv_render_context_set_update_callback': Error creating thread: Resource temporarily unavailable

Duplicate #68.
GNU/Linux Flutter shell is most unoptimized.
Another GNU/Linux specific issue: #64.

You need to free'd Player as well, not just the VideoController.


add "@esbuild/darwin-arm64"

Codecov Report
Patch and project coverage have no change.

Comparison is base (af2041b) 42.13% compared to head (b9c104a) 42.13%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #642   +/-   ##
=======================================
  Coverage   42.13%   42.13%
=======================================
  Files          19       19
  Lines        1329     1329
=======================================
  Hits          560      560
  Misses        709      709
  Partials       60       60

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Outdated so closing


default.theme: add theme.json to aid in theme resolution

To allow projects to use the LaxarJS UiKit default.theme without additional configuration, a descriptor must be placed in the default.theme directory.

Implemented on master (v2.0.0).


podcast-dl is hanging on download

The app is hanging on podcast download:
RSS: https://www.cbc.ca/podcasting/includes/whitehothate.xml
LOG_LEVEL="static" podcast-dl --file "/downloads/White Hot Hate/Metadata/White Hot Hate.rss" --out-dir "/downloads/White Hot Hate" --episode-template "{{podcast_title}} - {{release_year}}-{{release_month}}-{{release_day}} {{title}}" --include-meta --threads 4 --add-mp3-metadata
White Hot Hate
A violent, white supremacist ideology is spreading around the world, with believers trying to create chaos, in order to seize control.

When recruitment posters for a group calling itself The Base start to appear in Manitoba, Winnipeg Free Press journalist Ryan Thorpe decides to go undercover...and infiltrate.

Ryan meets with a Canadian Armed Forces reservist trying to establish a local neo-Nazi cell, and discovers that he is planning attacks and destruction abroad.

This six-part series follows the extraordinary case through Ryan’s perspective as an infiltrator - while host Michelle Shephard, a veteran national security journalist, explores the rise of white supremacist accelerationism.

For the best in true crime from CBC, ad-free, visit apple.co/cbctruecrime.

Downloading podcast image...
https://www.cbc.ca/radio/podcasts/images/whitehothate-promo-v2.jpg | Starting download...

I've let it be for a few hours--its not progressing past this step.
I am running podcast-dl with a bash script:
#!/bin/bash
LOG_LEVEL="static" npx podcast-dl "$@"

Thanks for the report, @xmoforf! Looks like cbc.ca was blocking the default user agent when trying to download the podcast images. I've updated the sent user agent and was able to get it working in the latest release.

nice one! thanks.


Natura Furnaces + Steam Heater = Game crashes as soon as I try to access the Furnace

I have posted the Crashlog on Pastebin. Only happens with the Nether Furnace it seems.
http://pastebin.com/0XUyKQuv

Ok so since the actual bug has been fixed, I'm closing this.


Add files via upload

@parnaljoshi I have added the Rmd file with the exploratory plots here. Feel free to pull down this branch to access the file.


No method to create operation from NSURLRequest

Is there any specific reason that you have not provided a method to create task using dataTaskWithRequest:completionHandler. My issue is that I want to use your operation and I have NSURLRequest already with HTTPBody set in that. Now using the API that you provided, I cannot set HTTPBody in request. So I want to use dataTaskWithRequest:completionHandler as it accepts parameter of type NSURLRequest directly.
Please let me know if you faced any issues in implementing it or it is by choice that you did not provide an interface to accept request directly.

It is there. I suspect you're looking at AFHTTPSessionOperation, but refer to AFURLSessionOperation.

For example:
AFURLSessionOperation *operation = [AFURLSessionOperation dataOperationWithManager:manager request:request uploadProgress:nil downloadProgress:nil completionHandler:^(NSURLResponse * _Nonnull response, id  _Nullable responseObject, NSError * _Nullable error) {
    if (responseObject == nil || error != nil) {
        NSLog(@"error = %@", error);
        return;
    }

    // do something with `responseObject` here
}];

[queue addOperation:operation];


Fix error where the same function in different files would clobber nodes in cprofiles

The previous version was using the function name as a key for the name_to_dict in cProfile profiles -- this had the effect that identically-named functions would collide, clobbering the previous entry.
Using the u_fn_name prevents these collisions.

@JBlaschke Can you provide some context for when do you get multiple files from cprofile? And are those functions in the different files representing the same function and calling context?
cc: @cscully-allison

I'm sorry @bhatele and @cscully-allison , I think I wasn't being clear: cProfile is producing only one file. However the program defines a function by the same name (different methods) in different  source code files. I will attach an example where __getitem__ is overloaded in several places. This creates missing nodes (because the second definition of __getitem__ clobbers the first). You then run into a KeyError when doing:
print(gf.tree())

Here is an example profile that runs into this:
https://bitbucket.org/jpblaschke/hatchet-cprofile-error/downloads/cpu_0.prof

I think this is the gist, these two u_fn_name have the same fn_name:
Foo:bar.py:123
Foo:spam.py:234

Thanks for clarifying. This looks good to me.

thanks  @bhatele  and @jrmadsen

This looks good to me as well. Thanks for the clarifying points @JBlaschke and @jrmadsen

This looks good, we may want to come up with some better alias down the line though, because my original u_fn_name is a little verbose.

I don't know if the verbosity can be helped. I am working on a mip-aware version, which would add rank as well. I'll @cscully-allison you when I submit the PR


Implement file quota

State the following policy on a policies page in pages.hydroshare.org, and implement it on the backend.
Each user receives automatically an allocation of 20 GB of space for their resources when they sign up.  Users who need more space should contact one<EMAIL_ADDRESS>and explain who they are and why they need more space.  This request can be quite short and all reasonable requests will be approved subject to system capacity limitations, with preference given to users from CUAHSI member institutions and NSF funded hydrology research projects.  Large requests for space may lead to negotiation over an appropriate contribution towards the costs of providing the space requested, or configuration of capability for use of space at your own institutions server for your HydroShare resources.
Implementation.  An admin page is needed where an admin can go in and set the quota for a user.

@dtarb @mjstealey FYI: This is related to the ResourceFile reorganization #1687 , in the sense that it allows us to measure -- in Django -- the size of all resource files owned by a user.

@hyi and @mjstealey I assume you both are working on this.

@dtarb @horsburgh @alvacouch Can one of you draft an email message for sending to users who have used up 90% or above of their allocated quota? I am close to get the first pass of quota implementation done on Django side. Then @mjstealey and I can work together to integrate @mjstealey's nightly script run with Django. I am writing an email_users management command to be called by @mjstealey's script, so it'd be great if you can draft an email message I can use to put into code before opening a PR for code review.

I will write some text below.  But, I think we need an admin interface that allows administrators to change the texts of these emails.  The text should be pulled from a field in a database that is editable by an administrator, rather than in the code.  That way as policies and messaging changes, the text can be easily updated.
Message to be transmitted by email to user when their quota crosses the 90% threshold.  Send this email once triggered by the action that put them over quota.  Unless they go under quota and then over again, in which case they should get the message again.  The message should also appear at the top of any HydroShare page when they are logged in as long as they are over quota, similar to the way other messages from Django appear.
Your quota for HydroShare resources is X GB.  You currently have resources that consume Y GB, Z% of your quota.  Once your quota reaches 100% you will no longer be able to create new resources in HydroShare.  To request additional quota please contact<EMAIL_ADDRESS> We will try accommodate reasonable requests for additional quota.  If you have a large quota request you may need to contribute towards the costs of providing the additional space you need.  See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.
Message to be transmitted by email and put at the top of the page in Django following an action that is refused due to being over quota.
Your action to add content to HydroShare was refused due to being over your HydroShare quota.  Your quota for HydroShare resources is X GB.  You currently have resources that consume Y GB, Z% of your quota.  To request additional quota please contact<EMAIL_ADDRESS> We will try accommodate reasonable requests for additional quota.  If you have a large quota request you may need to contribute towards the costs of providing the additional space you need.  See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.

@dtarb @hyi We had a detailed discussion on the issue of quotas on the developer call.

Other current practical quotas (e.g., how Linux does things) leverage a "soft limit" above which one can create files for a short time, which is usually 7 days. This is called a "grace period". At the end of the 7 days, the soft limit becomes hard and one cannot create any more files until one goes beneath the soft limit, which resets the grace period clock.

The hard limit stops everything, even if the grace period has not expired.

At Tufts, my soft limit is 75% of my hard limit. We were thinking that for HydroShare, the soft limit should be about 80% of the hard limit.

The function of the soft limit is that it starts quota warnings. At Tufts, I receive two kinds:
a. email warnings for things to which I don't have direct access, e.g., my email inbox.
b. Per-login warnings for things to which I do have access, e.g., logging into a Tufts server.

Analogously, this means that we should have:
a. An email reminder -- per day -- whenever someone is over the soft quota, and an email warning when the soft quota becomes hard.
b. Warnings when logging in, on the initial landing page, that one is over the soft limit and what one's grace period is.

Soft limits are a really good idea for several reasons:

They warn the user ahead of time that a problem will occur.
They give ample time to recover and clean up before shutting off service.

There was also significant discussion on how quotas should interact with federated resources. It is the feeling of CUAHSI that these can potentially represent an unacceptable management burden if the only way to change quotas for a federated site is through our admin interface. This is especially true if a third party comes to us with much more storage than we own and wants it to be managed aggressively by giving people quotas incrementally according to need. We don't want to administer those.
So, this means that we either need:

An administrative interface specifically for federated sites or
A REST call that can set quotas for federated users.
(These are essentially the same amount of trouble.)

CUAHSI doesn't have the staff to handle quotas for federated resources, and we would rather not advertise the service until there is a self-service option for this.

@alvacouch @dtarb I believe I have implemented everything @alvacouch mentioned above except for grace period in that PR. The underlying database schema takes into account the third party federated zone by making UserQuota class a foreignkey relationship to User class rather than OneToOne relationship so that more UserQuota records/instances can be added for a single User instance with each UserQuota instance corresponding to hydroshare_internal zone vs. external third party federated zone. @alvacouch You will know how I implemented this after you are done with code review. My implementation allows the hydroshare admin to manage third party federated zone if desired, but not enforce this, so if you don't add a separate UserQuota instance from admin interface to manage a separate federated zone, the quota management will only apply to the one single hydroshare_internal zone (i.e., hydroshareZone and hydroshareuserZone). My current implementation set the soft limit percent value to be 90% as @dtarb indicated above, but can be changed to 80% or any other value from admin interface. However, enforcement only happens for the hard limit as indicated by @dtarb above. If a decision is made to implement grace period clock and use soft limit for enforcement after grace period is over, it can be implemented but definitely introduce complexity especially with grace period count down. I will work with @mjstealey to integrate current implementation with Michael's script to close the loop and deploy it to alpha for you to test and get a feel on how it works, and then go from there.

I agree with @alvacouch suggestions on softquotas and grace period.  With respect to resources stored on a federated zone I do not see a way to avoid some administrative burden, but agree we need to keep it practical and manageable.  Here are some suggestions

All resources and initial quota is by default in the HydroShare zone.
When an organization creates a federated resource they designate a user or set of users who can set quotas for other HydroShare users on their zone
Quotas on each zone (federated zones and the HydroShare zone) be managed through a table that for each user holds Quota (GB), Soft limit % (default 75 or 80%), Grace period (days, default 7).
The quota table for each zone be held in a HydroShare resource.  That way the sharing of it and management of who can access or edit it can be managed by HydroShare access control mechanisms, and the administrator of a federated zone does not have to be a special user.
The HydroShare resource that holds the quota table could also hold messages to be used inform users about quota exceedances.
For owners that have quota on multiple zones the landing page on each resource should have a link giving an owner the option to move to different zone (this is something only an owner can do)
For resources with multiple owners, the first owner that creates the resource is set as the initial quota holder when the resource is created, but any owner can change the quota holder to any other owner (a friendly owner trusts owner model, with quota only charged to one owner).
The profile page of each user should summarize the quota status for each zone on which they have any quota.

There are probably other details to work out that I'd be happy to discuss and elaborate on if going in this direction seems a good idea.

@dtarb @alvacouch It appears grace period is desired, so I will try to implement grace period, which means we have to keep track of when the used quota starts reaching the soft limit so count down can be achieved. Also the user may clean up their storage so count down needs to be reset if the used quota falls below the soft limit after count down starts. It definitely introduces a lot of complexity, but I will try to implement it. @dtarb On your points No. 4 and No. 5, I am not sure I understand your point, but I don't want to make quota table subject to access control, currently it is only editable by admin. Quota warning and enforcement messages are already editable by admin. For your point No. 6, since we don't have external federated zone yet, I don't think it is worth the trouble to implement the option that allows users to move resources between zones, not at this point at least. For your point No. 7, I am not sure whether it is a good idea to allow any owner to change quota holder to any other owner, but if we decide it is a good idea to do so, it can be implemented. point No. 8 can be implemented. So in short, I will work on No. 8, and grace period implementation with No. 7 on hold until a decision is reached. This is enough functionality for the first pass implementation given the short time we have left.

Points 4 and 5 deal with where the information about what quota's are are stored and managed.  I am suggesting that instead of setting up separate mechanisms that only administrators can access, which seems complex to me and requires us to designate administrators who do this, and manage these administrators when we get to federated zones, that we instead just use what we have already developed with HydroShare.  HydroShare resources can be shared, or not shared with users according to the access control system we have and like.  I am suggesting we keep the quota table in a HydroShare resource, and then we can just use what we already work with to manage any changes.
While we do not have external federated zones yet, the whole reason federated zones are being set up is to allow external expansion.  Perhaps 6 can be deferred for initial implementation, but not too long, and lets keep this in mind in the design.
On 7, we want a system that is easy to use.  We have a model where owners are empowered.  Owners have to be people you trust.  If you make another user an owner of your resource it is like giving them the keys to your house or your car.  They can do anything.  I think changing the quota holder is quite consistent with this model.

@dtarb Thanks for your clarification on points 4 and 5. I think my current implementation with UserQuota table with many-to-one relations with User table works well without much administrative burden. You can test it out when we have alpha ready for testing. I agree your point No. 6 is useful, but let's defer it to the next phase and keep this in mind in the design. On 7, I see your point, but also see some quota-specific subtle differences. For example, the primary owner who is also a quota holder for a resource added owner A, then the primary owner changed quota holder to owner A. Does the primary owner needs to get approval from owner A before changing quota holder to owner A? Owner A may not want to be the owner of that resource if he gets changed to quota holder since he is pretty tight on quota already. I think this introduces a major risk factor for people so they may be reluctant to be put as owner of a resource. However, we don't have invitation, notification, and acceptance feedback mechanism implemented for adding owners, so I think we should be very cautious on introducing this extra risk factor, which may not be what we want.

Owners have to work it out among themselves socially, not programmatically.  I would like the system to be simple.  An owner can delete a resource if they need to get it off their quota so if another owner forces it on to them against their will, they just delete it.  This has to be sorted out by owners talking among themselves, not us building complicated checks in the system.  You should not make someone an owner of a resource you care about if you do not trust them.  The biggest risk factor is us building too complicated a system that it never gets built and confuses users so it never gets used.

@dtarb @alvacouch I am currently adding grace period into quota implementation and will have to modify the default warning and enforcement messages. Note these messages can be edited from admin interface, so getting them right initially is not critically important, but it'd be nice to get the default messages approved by you initially. Below are the modified messages. Let me know if you want to make any edits.
Warning message:
Your quota for HydroShare resources is X GB in A zone. You currently have resources that consume
Y GB, B% of your quota which has reached the soft quota limit. You have a grace period of 7 days to take actions to get your quota below the soft quota limit (80%). Otherwise, when your grace period is over, you will no longer be able to create new resources in HydroShare. To request additional quota, please contact<EMAIL_ADDRESS>We will try to accommodate reasonable requests for additional quota. If you have a large quota request, you may need to contribute towards the costs of providing the additional space you need. See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.

The enforcement message:
Your action to add content to HydroShare was refused because you are over 80% of your quota and your grace period of 7 days have passed. Your quota for HydroShare resources is X GB. You currently have resources that consume Y GB, Z% of your quota. To request additional quota, please contact<EMAIL_ADDRESS>We will try accommodate reasonable requests for additional quota. If you have a large quota request, you may need to contribute towards the costs of providing the additional space you need. See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.

@hyi, I think there needs to be a warning before the soft limit, a soft limit (that we call the quota) and a hard limit that is 125% of the quota.  I suggest the following messages.
Warning (when over 80% of quota)

Your quota for HydroShare resources is X GB in A zone. You currently have resources that consume
Y GB, B% of your quota. Once your quota reaches 100% you will no longer be able to create new resources in HydroShare. To request additional quota, please contact<EMAIL_ADDRESS>We will try to accommodate reasonable requests for additional quota. If you have a large quota request, you may need to contribute towards the costs of providing the additional space you need. See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.

Grace period over quota (when over quota within grace period and less than 125% of quota)

You have exceeded your HydroShare quota of X GB in A zone.  Your quota for HydroShare resources is X GB. You currently have resources that consume Y GB, Z% of your quota. You have a grace period until (insert date) to reduce your use to below your quota, or to acquire additional quota, after which you will no longer be able to create new resources in HydroShare.  To request additional quota, please contact<EMAIL_ADDRESS>We will try to accommodate reasonable requests for additional quota. If you have a large quota request, you may need to contribute towards the costs of providing the additional space you need. See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.

After grace period or more than 125%

Your action to add content to HydroShare was refused because you have exceeded your quota. Your quota for HydroShare resources is X GB. You currently have resources that consume Y GB, Z% of your quota. To request additional quota, please contact<EMAIL_ADDRESS>We will try accommodate reasonable requests for additional quota. If you have a large quota request, you may need to contribute towards the costs of providing the additional space you need. See https://pages.hydroshare.org/about-hydroshare/policies/quota/ for more information about the quota policy.

@dtarb OK. I like your suggestions and will implement as such. The used percentage of quota (i.e., 80%) is editable from admin interface, but the 125% hard limit will be hard-coded (i.e., not editable from admin interface). Let me know whether this is OK with you. @alvacouch See if @dtarb's suggestion is in line with your thought.

@dtarb @alvacouch I decide to make hard limit 125% editable from admin interface in the similar way as the soft limit 80% as there may be needs to adjust these percentages over time.

@dtarb @alvacouch I have implemented allowing quota holder to change quota holder to another owner from resource landing page. However, if this quota holder owner is removed from access control by another owner, I assume the only reasonable action is to transfer the quota holder to this owner who is removing the quota holder owner. Do you agree?

@dtarb @alvacouch I took my suggestion above back. I think it'd be more reasonable just to disallow owners to delete a quota holder owner from manage access panel, in the same way as the only remaining owner cannot be removed. This makes more sense to me, so I will go with that for implementation.

I am OK with all of this and look forward to a demo.

@hyi @dtarb @pkdash @mseul @aphelionz
There is a generally better way to handle invitation/response than we are doing now. If we look at how Google Drive and DropBox do things, it should inspire us toward a different approach:

An invitation is emailed to the user. There is no automatic UI generated for responding to the invitation. The email is the only notification.
The email contains accept and reject links that link to REST calls.
These links tell the user that the action has been accomplished.
The links have a set timeout, e.g., one week. After that the invitation is deleted. It can of course be re-issued.
Re-issuing a current invitation does nothing except email the user again.
There's no "retract invitation" function. Invitations simply time out.

It strikes me that:

This is much lower commitment than building a UI for accept/reject. This UI does not exist for DropBox or Google Drive. Only the email invite exists.
All we need is a UI for "invite" and for "request access". These are just buttons.
This is in general a superior and faster way to manage group invitations as well and -- for that matter -- any kind of invitation.  Much less UI work is involved, and much faster notification of the user results.

@alvacouch

All we need is a UI for "invite" and for "request access". These are just buttons.

For group the above UI is what we have (buttons) currently.

@pkdash As I understand things, the responding user has to deal with a list of invitations. That is what I suggest we might do away with.

@alvacouch @dtarb I think invite/response model for changing quota holders introduces delayed action and thus may involve a lot of more work than it appears for our current quota implementation. Specifically, we allow any owner to change quota holder to another owner which is based on "owners should trust each other and can do anything" principle. If we introduce invite/response for changing quota holder, consider this scenario: owner A tries to transfer quota holder to owner B, before owner B approves this, owner C tries to transfer quota holder to owner D, and owner D approves it before owner B, so the quota holder is transferred to D, then owner B approves it also, then owner B gets the quota holder status. We can add in more constraints such as not allowing changing quota holders when a invite request is pending, but as you can see, implementing this invite/response model is more involved in order to accommodate more potential scenarios to make it work. Given that, I'd go with simple email notification at least for now given the time constraint.
@dtarb Can you draft email notifications for adding owners and changing quota holders, especially for changing quota holders. I think we should tell the user in the notification email that owner A has transferred quota holder to you for a resource, and if you don't approve, you can contact the owner A, or  ...  I'll add email notifications after I see your draft.

@dtarb @alvacouch I decided to only send email notification when a user has been changed to a quota holder for a resource, but not to send email notification when a user is added as an owner of the resource for the first pass for the sake of consistency. I think the notification for access control needs to be thought out as a whole including add an owner to a resource, demoting an owner, adding a user as editor, demoting an editor, etc. It does not make sense to me to only send email notification for adding an owner, but not for other access control changes. So I think for the first pass, I'll just implement email notification for quota holder change, but leave other email notifications to a separate work that needs to handle access control notifications as a whole including adding authors and contributors which has already been detailed in this issue: https://github.com/hydroshare/hydroshare/issues/975

@hyi @pkdash I would rather that all of the logic for quota were inside hs_access_control rather than depending on fields in *Resource.  So far, there are no dependencies in access control upon fields in other places. This would allow me to continue to debug access control without having to load the resource model to do so, and will keep access control a plug-in module that can be reused in other places the same way.  I estimate that debugging access control in this way is 100 time faster than debugging it in situ inside HydroShare.
The way it's written now, with policy logic references to BaseResource.creator, hs_access_control is now inextricably coupled to the resource model ... which is exactly what I wanted to avoid.
Fixing this would mean adding a field quota_user that is a Foreign key to User inside the ResourceAccess class.  Then the logic for maintaining this field could be embedded inside access control without having to know the details of *Resource. Then, the logic for all of this -- including email notifications, REST responders, etc -- could be embedded in access control where it seems to belong, without coupling data from access control with data from other places. User.email would be the only coupling....
I can assist with this if you wish.
In general, I have tried to keep access control "orthogonal" to the other systems, in the sense that there is no coupling except through the API. This makes it much, much easier to update it without messing up other things. This is part of why access control is implemented through an external class system rather than being embedded inside BaseResource.

@alvacouch I understand your point trying to keep access control "orthogonal" to the other systems, however, in this special quota case, accessing resource to retrieve quota holder is needed since the quota holder information is only kept as an AVU pair on the resource collection in iRODS, not in Django, which is intended to avoid duplicate information on Django and iRODS side. Since this quota holder info is needed by @mjstealey's script, it needs to reside in AVU pair in iRODS for the resource collection, and the method get_quota_holder() and set_quota_holder() on BaseResource instance are just convenience methods to get/set AVU pairs in iRODS. Since access control system is for applying access control to some target, in our case, HydroShare resources, attaching quota holder to the access control target may be an acceptable model. For example, if access control system is used for other system target, that new target just need to provide two convenience methods, get_quota_holder() and set_quota_holder(), which encapsulate the internal implementation details from access control system. In our case, we use iRODS AVU, but for another target, whatever implementation can be used without depending on iRODS, access control system just depends on the two API convenience methods. That said, if you want to improve it further, you are welcome to do so in the next iteration after this quota implementation is deployed. Hope this makes sense to you.

@alvacouch and @hyi

Fixing this would mean adding a field quota_user that is a Foreign key to User inside the ResourceAccess class. Then the logic for maintaining this field could be embedded inside access control without having to know the details of *Resource. Then, the logic for all of this -- including email notifications, REST responders, etc -- could be embedded in access control where it seems to belong, without coupling data from access control with data from other places. User.email would be the only coupling....

I agree there may be some value using quota_user field in ResourceAccess class. However, if we move the responsibility of maintaining this field to access control that would create more coupling between access control and hs_core in particular unnecessary dependency  related to iRODS. I am also not sure about access control being responsible for email notification - more coupling. I would prefer hs_core to maintain the quota_user field in ResourceAccess class and access control to only act (allow or disallow certain actions) depending on the status of this field.

@hyi @pkdash I am willing to settle for changes to the quota_user from outside, because that does indeed satisfy my concerns. My main concern is that the deep logic of access control needs to be based upon fields contained inside access control. It's difficult to debug future features if that control reaches outside the module.
However, there is an argument for having the state machines for various invite-accept protocols -- especially those that involve email/REST exchanges -- inside access control, because it actually has the knowledge to implement these transactions completely and in isolation from the rest of the system. Thus these state transitions are easier to write and debug :) Tests take a few seconds rather than up to 40 minutes, because I don't have to fire up and populate the resource models in order to test them!

@hyi it is going to be quite difficult for me to move the quota_user from Resource to ResourceAccess after the fact. This will require a data migration involving a live iRODS.

@alvacouch The current implementation does not have quota_user field on Resource model in Django, so there will not be data migration involved on Django side. I prefer to have quota user stored as AVU metadata on iRODS side, and retrieve quota user on Django side when needed to avoid duplication and to have single point of truth.

Suggesting an update:  What about something shorter and more direct.
Development Update: User storage is 20 GB. Visit our Quota Policy Help page to learn more.

@phuongdm Please close or comment.

Yes, it's done


Enchantment Tables - Unusable

Hello,
I'm using the latest version <IP_ADDRESS>.4 with the latest Spigot 1.12 jar.
I have an issue with Enchantment Tables not working (vanilla style).
I had tested and removed every plugin possible and it never occurred to me that this plugin would be the cause (since I never had trouble with this plugin in the past)
Here is a link to my post regarding this issue for a preview of what the problem is, it seems like a inventory click event getting canceled: https://www.spigotmc.org/threads/spigot-1-12-enchantment-table-not-working.256730/
I was looking around the config for some sort of enchantment table option but couldn't find one relative to the problem.
Please help, thanks!

Strange, I can't seems to replicate it. Try setting this option to false.
https://github.com/kicjow/Crazy-Enchantments/blob/master/config.yml#L249
If you don't see that option then add it in and set it to false.
If that doesn't fix it then pastebin the config and send the link.


Tweak verbiage for launch

This commit tweaks some of the page verbiage for form notices and
the <h1> for type-os

lgtm


Feature/2108 move multi select from canary to core

Summary of the changes
Moved the multi-select component from the canary package to the core package (merged the code into the select code).
This means that the multi-select variant now has bug fixes and functionality which were previously only available in the single and searchable select variants - which are mainly the following:

Focus on multi-select no longer triggered with keydown event - PR #1382
icClose +  icOpen events added - PR #1643
Escape + menu focus issue fixed - PR #1966
icBlur trigger prevention added - PR #1652
Console errors fixed - PR #1658 + #1816
Text spacing alignment issue fixed - PR #1917
Attribute mutation observers added - PR #1767
Functionality for displaying icon added - PR #2071

I looked through a lot of PRs which changed ic-select + ic-menu code to help check that the select + multi-select changes had been merged together correctly (quite a few had already been done) - hopefully everything is there and looks ok 😅
The overall multi-select functionality was all reviewed and approved when it went into canary, so reviewing this PR should mainly just involve checking that everything looks good after merging all the functionality together.
Related issue
#2108
Checklist
General

[x] Changes to docs package checked and committed.
[x] All acceptance criteria reviewed and met.

Testing

[x] Relevant unit tests and visual regression tests added.
[x] Visual testing against Figma component specification completed.
[x] Playground stories in React Storybook up to date, with any prop changes and additions addressed.
[x] Compare performance of modified components against develop using Performance addon in React Storybook.

Accessibility

[x] Accessibility Insights FastPass performed.
[x] A11y unit test added and yields no issues.
[x] A11y plug-in on Storybook yields no issues.
[x] Manual screen reader testing performed using NVDA and VoiceOver.
[x] Manual keyboard testing for keyboard controls and logical focus order.
[x] Correct roles used and ARIA attributes used correctly where required.
[x] Logical heading structure is maintained, and the HTML elements used for headings can be changed to fit within the wider page structure.

Resize/zoom behaviour

[x] Page can be zoomed to 400% with no loss of content.
[x] Screen magnifier used with no issues.
[x] Text resized to 200% with no loss of content.
[x] Text spacing increased as per the WCAG 1.4.12 success criterion with no loss of content.

System modes

[x] Browser setting 'prefers reduced motion' tested. No animations or motion visible whilst this setting is on.
[x] Windows High Contrast mode tested with no loss of content.
[x] System light and dark mode tested with no loss of content.
[x] Browser support tested (Chrome, Safari, Firefox and Edge).

Testing content extremes

[x] Min/max content examples tested with no loss of content or overflow.
[x] All prop combinations work without issue.
[x] Tested for FOUC (Flash of Unstyled Content) in both SSR (Server-Side Rendering) and SSG (Static Site Generation) settings.
[x] Controlled and uncontrolled input components tested.
[x] Props/slots can be updated after initial render.

Web components build
React build


iss-81

Coverage increased (+0.2%) to 95.042% when pulling d2e2fe92411016d22a6a8ee435b6828b30a6ab08 on iss-81 into ebe0e7222c3e3949a5314ab96227cc3400cffa45 on dev.


Change DispatchQueue.after() to take a DispatchTimeInterval

Change DispatchQueue.after() to take a DispatchTimeInterval instead of an absolute time
Rename DispatchQueue.after() that takes an absolute time to DispatchQueue.at(time:)

I couldn't find a way to build/test these changes on OSX. All I know is that this API really bugs me. Especially that DispatchQueue.after() takes an absolute time. That has always bugged me, even with the C API.
I would much rather write DispatchQueue.main.after(.seconds(1)) { /* some stuff */ }

this should not take a time interval, the deadline is specific to the chosen clock, TimeInterval does not contain that information.

@das how about:

Adding a clock parameter, with a default, or
Adding a convenience function

instead?

What is the advantage from
DispatchQueue.main.after(when: .now() + .seconds(1))
DispatchQueue.main.after(walltime: .now() + .seconds(1))

the number of keystrokes isn't significantly more and it is properly typed

@MadCoder It would be:
DispatchQueue.main.after(.seconds(1))
DispatchQueue.main.at(time: .now() + .seconds(1))

It bugs me because on the relatively infrequent occasions that I need this API, it always involves a trawl through the interfaces, cross-checking types and finding out how to construct something.
The words don't even make grammatical sense: "after when" -- which is it; an "after" (i.e. relative time) or a "when" (i.e. absolute time)? If you look at the level of nuance at which they're picking apart the standard library over on swift-evo, I'm pretty sure the "swifty" way is to disambiguate these cases.

I agree, I don't see a huge advantage over the changes we were already thinking about:
func asyncAfter(deadline:qos:flags:work:)
func asyncAfter(wallDeadline:qos:flags:work:)

Most of these examples can even be shortened to:
DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) { /* block */ }

@mwwa Even if it was renamed to asyncAfter, it should still take an interval rather than an absolute time. If you wanted a looser guarantee about timing, it should be:
DispatchQueue.main.by(dealine: ...)

Or asyncBy(deadline:), or some other variant which is more descriptive about what it's doing. But for as long as it's after, IMO it should take a duration (and optionally a point in time that it's relative to).

Also, wallTime is not necessary. I would refer everybody to the Swift API Design Guidelines. Specifically points:

Take advantage of defaulted parameters when it simplifies common uses. Any parameter with a single commonly-used value is a candidate for a default.
Omit needless words. Every word in a name should convey salient information at the use site. More words may be needed to clarify intent or disambiguate meaning, but those that are redundant with information the reader already possesses should be omitted. In particular, omit words that merely repeat type information.
Name variables, parameters, and associated types according to their roles, rather than their type constraints.
Prefer method and function names that make use sites form grammatical English phrases.
Clarity at the point of use is your most important goal. Entities such as methods and properties are declared only once but used repeatedly. Design APIs to make those uses clear and concise. When evaluating a design, reading a declaration is seldom sufficient; always examine a use case to make sure it looks clear in context.

Also, wallTime is not necessary

Yes it is, again, the DispatchTimeInterval type doesn't express which clock is used and this cannot work. There is a difference between deadline (monotonic clock) and wallDeadline which is calendar clock. One takes system sleep into account and not the other, these are quite important differences.
We know the swift guidelines alright, your proposal fails the first test: there's no good default clock to use here, so you can't omit the deadline / wallDeadline.

Well I continue to disagree. I'm pretty sure that if it actually went through a swift-evo discussion this entire new Dispatch API would get torn apart. It doesn't fit Swift at all - it's full of direct translations from C, even when you literally feel like you're speaking gobbledegook (such as "after when").
And no, wallTime really is not necessary.  Look at the 2nd bullet point. wallTime literally just repeats type information. Sure, there is a distinction between the wall and monotonic clocks, but you actually made that distinction at the type level - look at src/swift/Time.swift, literally at the very start of the file. You don't need to carry it around with you every time you describe it - you've moved above C now; we know that's a wall time by virtue of the fact that it's declared as taking a type DispatchWallTime. Do you understand me yet?
Also, I believe the wallclock issue is a perfect example of one where there is an obvious default. Is it really such a very common situation that relative times I specify (likely to be on the duration of a handful of seconds) are likely to intersect a sleep event? It's the very definition of a edge-case, and translates in to cryptic and undiscoverable boilerplate that (I would seriously bet) >90% of GCD users don't understand or care about. I would argue that my approach, with an explicit default clock, is far more obvious and clearer at the point of use.
Are the swift corelibs under the domain of swift-evo? I have to apologise, but I disagree so strongly I'm not going to drop it. I have to use this API, it's open-source, so I'm going to insist on opening this discussion to a wider audience.

dispatch is part of corelibs which is discussed on swift-evo and this proposal was. We have a few adaptations to make for obvious actual bugs in the interface (around DispatchWorkItems), but a pull request is indeed not the right forum to discuss an API surface change.

@MadCoder Okay I've posted an evo thread about it. While writing it down, the splitting of the clocks at the type level started to feel very strange and potentially error-prone. I added a bit about possibly unifying them and adding a bool/enum field for the clock (embedded types make this quite elegant IMO). It's also a bit more like the C API in that way (which just has a single dispatch_time_t).

we care about distinct types. how dispatch_time_t was built in C is considered a mistake because it makes the type not Comparable. We don't want to repeat that mistake


Update git_homework_1.md

Hi @simeon-demo,
I have completed the git homework for review.
I'm not sure if this would work. But thank you so much, Simeon!
Best regards,
Caroline

uoft-dsi-c3-homework-20240426


Can we get a new release?

Lots of commits to master since 2.5.1 (i am particularly interested in getting the telegram icon).
I know i can just use git in Gemfile but it always looks better if there's a tagged release :)

La la la, i'm just writing shit here so the bot doesn't close this issue

Duplicate of #411


Update MD file on new branch 2017-25-16_11-11-41

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestCon.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add references about new stylesheet architecture in README

Based on lastest changes, @jjaderg can you add references about this new architecture in README?

Certainly!


Video Dataset / Model refactor + framework for action tests

Description
basic framework for testing for action recognition
Checklist:

[x] I have followed the contribution guidelines and code style for this project.
[x] I have added tests covering my contributions.
[x] I have updated the documentation accordingly.
[x] This PR is being made to staging and not master

Looks great, feel free to merge.
ps. I found one probable minor bug (test for '1') and please set the number of workers in dataset db_num_workers() in a future PR (I believe this will work)

Regarding setting num_workers == db_num_workers() - this is the comment i found regarding why it is set to 0: # Torch 1.2 has a bug when num-workers > 0 (0 means run a main-processor worker)
As you said, in a later PR i will test out whether there is a better fix for this bug (or if the bug is even still an issue)


Add documentation for loading from json

Add documentation in README.md and other if needed demonstrating how to import a langchain agent.

Th


The certificate chain is incomplete

After fixing the DH group size issue, ssllabs is still showing this error to us:

This server's certificate chain is incomplete. Grade capped to B.

Pretty self-explaining and easy to fix. We don't only have to send our certificate, but the ones for all the intermediates CA too: we'll have to concatenate everything into one file and give it to nginx. Let' reach the A o/
@whit537 Maybe the complete certificate already exists. Can you give me the output of a ls -alh /etc/nginx/certs/, please?

root@grtp:~# ls -alh /etc/nginx/certs/
total 32K
drwx------ 2 root root  4.0K Mar 10  2015 .
drwxr-xr-x 7 root root  4.0K Jul 14 14:11 ..
-rw------- 1 root root  4.9K Jun 13  2013 bundle.pem
-rw-r--r-- 1  501 staff 2.7K Mar  9  2015 grtp.co.crt
-rw-r--r-- 1  501 staff 3.2K Mar  9  2015 grtp.co.key
-rw-r--r-- 1 root root  4.4K Mar 10  2015 old.tbz
root@grtp:~#

I'd go for:
cp grtp.co.crt grtp.co.crt.old
wget https://startssl.com/certs/sca.server2.crt
cat sca.server2.crt >> grtp.co.crt
service nginx restart

I think it's the right intermediate certificate. Can't find the suggested one displayed in the ssllabs records on startssl's website and never used anything else than Let's Encrypt 😞 Or maybe it's in the .zip you got when creating the certificated.

root@grtp:~# cd /etc/nginx
root@grtp:/etc/nginx# cd certs
root@grtp:/etc/nginx/certs# l
bundle.pem  grtp.co.crt  grtp.co.key  old.tbz
root@grtp:/etc/nginx/certs# cp grtp.co.crt grtp.co.crt.bak
root@grtp:/etc/nginx/certs# wget https://startssl.com/certs/sca.server2.crt
--2016-07-14 16:03:42--  https://startssl.com/certs/sca.server2.crt
Resolving startssl.com (startssl.com)... <IP_ADDRESS>
Connecting to startssl.com (startssl.com)|<IP_ADDRESS>|:443... connected.

HTTP request sent, awaiting response... 200 OK
Length: 2140 (2.1K) [application/x-x509-ca-cert]
Saving to: ‘sca.server2.crt’

100%[==========================================================================>] 2,140       --.-K/s   in 0s

2016-07-14 16:03:42 (242 MB/s) - ‘sca.server2.crt’ saved [2140/2140]

root@grtp:/etc/nginx/certs# cat sca.server2.crt >> grtp.co.crt
root@grtp:/etc/nginx/certs# service nginx restart
nginx stop/waiting
nginx start/running, process 22976
root@grtp:/etc/nginx/certs#

Looks like that wasn't sufficient.

That would be too easy… Let's go for a cp grtp.co.crt.old grtp.co.crt && service nginx reload while I try to find the right one. The only website linking to the supposed correct file is https://ssl-tools.net/subjects/b9b2d56db021b36e42f627245806c4a9a6979aeb, but I'm not considering this as a good solution.

I logged back into StartSSL and have re-retrieved a ZIP file for NGINX. Configuring ...

Looks like that worked! :-)
https://www.ssllabs.com/ssltest/analyze.html?d=grtp.co

@nashe Can I let you follow up on H1?

Sure! Updating the Radar accordingly. Thanks!


Add AssertReplyContain

Hi @baruchiro, thank you for your contribution, I requested some changes in my review, please check them out and let me know if you have any question.
Also, please provide detailed comments in the PR the next time you submit one, it will help us triage the request faster.
Thanks again.

@gabog I believe all of your concerns were addressed. Could you revisit your review?


Rename OSS build packages

This PR will rename built oss packages as pharos-cluster-linux-amd64-${version}-oss.
The motivation behind this is that we could then use ${version}-oss as version for the binary.  It would make more straight-forward to install and resolve binaries, for example:
$ chpharos list-remote
2.0.0
2.0.0-oss
1.3.2
1.3.1
...
$ chpharos install 2.0.0-oss

I think + would be more semver (2.0.0-rc1+oss), but I'm fine with this too.

yes, that +oss might be better. I Will change it.


 Try with a real cat picture, I think it won't be detected as face.
so I'd say this is a success not a bug.. ?