Configurable grace period support after a SIGTERM signal caught

Description
This PR introduces a new --grace-period option that happens right after a SIGTERM in order to delay server's graceful showdown.
-q, --grace-period <grace-period>
Defines a grace period in seconds after a `SIGTERM` signal is caught which will delay the server before to
shut it down gracefully. 255 seconds maximum [env: SERVER_GRACE_PERIOD=]  [default: 0]

Related Issue

No
Motivation and Context

Feature request #79
How Has This Been Tested?

Using the following command and options
static-web-server -p 8787 -d docker/public/ -g trace -q 10

Note: The logs below have been split out for better understanding.

Server starts with a grace period of 10 secs

2022-02-01T15:21:41.222866Z  INFO static_web_server::server: bound to TCP socket [::]:8787
2022-02-01T15:21:41.222950Z  INFO static_web_server::server: runtime worker threads: 4
2022-02-01T15:21:41.222973Z  INFO static_web_server::server: security headers: enabled=false
2022-02-01T15:21:41.222991Z  INFO static_web_server::server: auto compression: enabled=true
2022-02-01T15:21:41.223017Z  INFO static_web_server::server: directory listing: enabled=false
2022-02-01T15:21:41.223042Z  INFO static_web_server::server: directory listing order code: 6
2022-02-01T15:21:41.223058Z  INFO static_web_server::server: cache control headers: enabled=true
2022-02-01T15:21:41.223100Z  INFO static_web_server::server: basic authentication: enabled=false
2022-02-01T15:21:41.223150Z TRACE mio::poll: registering event source with poller: token=Token(1), interests=READABLE | WRITABLE
2022-02-01T15:21:41.223194Z TRACE mio::poll: registering event source with poller: token=Token(2), interests=READABLE | WRITABLE
2022-02-01T15:21:41.223402Z TRACE mio::poll: registering event source with poller: token=Token(3), interests=READABLE | WRITABLE
2022-02-01T15:21:41.223544Z  INFO Server::start_server{addr_str="[::]:8787" threads=4}: static_web_server::server: close time.busy=0.00ns time.idle=11.3µs
2022-02-01T15:21:41.223599Z  INFO static_web_server::server: listening on http://[::]:8787
2022-02-01T15:21:41.223622Z  INFO static_web_server::server: press ctrl+c to shut down the server

Server receives one request

2022-02-01T15:21:50.015015Z TRACE mio::poll: registering event source with poller: token=Token(4), interests=READABLE | WRITABLE
2022-02-01T15:21:50.015353Z TRACE hyper::proto::h1::conn: Conn::read_head
2022-02-01T15:21:50.015442Z TRACE hyper::proto::h1::io: received 93 bytes
2022-02-01T15:21:50.015617Z TRACE parse_headers: hyper::proto::h1::role: Request.parse bytes=93
2022-02-01T15:21:50.015714Z TRACE parse_headers: hyper::proto::h1::role: Request.parse Complete(93)
2022-02-01T15:21:50.015854Z TRACE parse_headers: hyper::proto::h1::role: close time.busy=264µs time.idle=25.0µs
2022-02-01T15:21:50.015939Z DEBUG hyper::proto::h1::io: parsed 3 headers
2022-02-01T15:21:50.015970Z DEBUG hyper::proto::h1::conn: incoming body is empty
2022-02-01T15:21:50.016070Z TRACE static_web_server::static_files: dir? base="docker/public/", route="/assets/main.js"
2022-02-01T15:21:50.016262Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
2022-02-01T15:21:50.016379Z TRACE static_web_server::static_files: dir: "docker/public/assets/main.js"
2022-02-01T15:21:50.016464Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
2022-02-01T15:21:50.016794Z TRACE encode_headers: hyper::proto::h1::role: Server::encode status=200, body=Some(Unknown), req_method=Some(HEAD)
2022-02-01T15:21:50.016869Z TRACE encode_headers: hyper::proto::h1::role: server body forced to 0; method=Some(HEAD), status=200
2022-02-01T15:21:50.016932Z TRACE encode_headers: hyper::proto::h1::role: close time.busy=140µs time.idle=15.4µs
2022-02-01T15:21:50.016999Z TRACE hyper::proto::h1::dispatch: no more write body allowed, user body is_end_stream = false
2022-02-01T15:21:50.017122Z DEBUG hyper::proto::h1::io: flushed 223 bytes
2022-02-01T15:21:50.017158Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: Init, writing: Init, keep_alive: Idle }
2022-02-01T15:21:50.017377Z TRACE hyper::proto::h1::conn: Conn::read_head
2022-02-01T15:21:50.017438Z TRACE hyper::proto::h1::io: received 0 bytes
2022-02-01T15:21:50.017470Z TRACE hyper::proto::h1::io: parse eof
2022-02-01T15:21:50.017506Z TRACE hyper::proto::h1::conn: State::close_read()
2022-02-01T15:21:50.017532Z DEBUG hyper::proto::h1::conn: read eof
2022-02-01T15:21:50.017565Z TRACE hyper::proto::h1::conn: State::close_write()
2022-02-01T15:21:50.017588Z TRACE hyper::proto::h1::conn: State::close_read()
2022-02-01T15:21:50.017616Z TRACE hyper::proto::h1::conn: State::close_write()
2022-02-01T15:21:50.017650Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: Closed, writing: Closed, keep_alive: Disabled }
2022-02-01T15:21:50.017727Z TRACE hyper::proto::h1::conn: shut down IO complete
2022-02-01T15:21:50.017774Z TRACE mio::poll: deregistering event source from poller

Server receives an intentional ctrl+c
Note the Grace period of 10 second(s) after the SIGTERM log entry

^C2022-02-01T15:21:58.756116Z  INFO static_web_server::signals: SIGTERM, SIGINT or SIGQUIT signal caught
2022-02-01T15:21:58.756301Z  INFO static_web_server::signals: Grace period of 10 second(s) after the SIGTERM

Server receives another request completed sucessfully

2022-02-01T15:22:02.685889Z TRACE mio::poll: registering event source with poller: token=Token(16777220), interests=READABLE | WRITABLE
2022-02-01T15:22:02.686097Z TRACE hyper::proto::h1::conn: Conn::read_head
2022-02-01T15:22:02.686159Z TRACE hyper::proto::h1::io: received 94 bytes
2022-02-01T15:22:02.686322Z TRACE parse_headers: hyper::proto::h1::role: Request.parse bytes=94
2022-02-01T15:22:02.686393Z TRACE parse_headers: hyper::proto::h1::role: Request.parse Complete(94)
2022-02-01T15:22:02.686501Z TRACE parse_headers: hyper::proto::h1::role: close time.busy=199µs time.idle=28.4µs
2022-02-01T15:22:02.686561Z DEBUG hyper::proto::h1::io: parsed 3 headers
2022-02-01T15:22:02.686584Z DEBUG hyper::proto::h1::conn: incoming body is empty
2022-02-01T15:22:02.686658Z TRACE static_web_server::static_files: dir? base="docker/public/", route="/assets/main.css"
2022-02-01T15:22:02.686810Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
2022-02-01T15:22:02.686954Z TRACE static_web_server::static_files: dir: "docker/public/assets/main.css"
2022-02-01T15:22:02.687026Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
2022-02-01T15:22:02.687292Z TRACE encode_headers: hyper::proto::h1::role: Server::encode status=200, body=Some(Unknown), req_method=Some(HEAD)
2022-02-01T15:22:02.687356Z TRACE encode_headers: hyper::proto::h1::role: server body forced to 0; method=Some(HEAD), status=200
2022-02-01T15:22:02.687411Z TRACE encode_headers: hyper::proto::h1::role: close time.busy=120µs time.idle=12.9µs
2022-02-01T15:22:02.687469Z TRACE hyper::proto::h1::dispatch: no more write body allowed, user body is_end_stream = false
2022-02-01T15:22:02.687583Z DEBUG hyper::proto::h1::io: flushed 210 bytes
2022-02-01T15:22:02.687638Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: Init, writing: Init, keep_alive: Idle }
2022-02-01T15:22:02.687816Z TRACE hyper::proto::h1::conn: Conn::read_head
2022-02-01T15:22:02.687872Z TRACE hyper::proto::h1::io: received 0 bytes
2022-02-01T15:22:02.687897Z TRACE hyper::proto::h1::io: parse eof
2022-02-01T15:22:02.687918Z TRACE hyper::proto::h1::conn: State::close_read()
2022-02-01T15:22:02.687937Z DEBUG hyper::proto::h1::conn: read eof
2022-02-01T15:22:02.687955Z TRACE hyper::proto::h1::conn: State::close_write()
2022-02-01T15:22:02.687974Z TRACE hyper::proto::h1::conn: State::close_read()
2022-02-01T15:22:02.687990Z TRACE hyper::proto::h1::conn: State::close_write()
2022-02-01T15:22:02.688011Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: Closed, writing: Closed, keep_alive: Disabled }
2022-02-01T15:22:02.688074Z TRACE hyper::proto::h1::conn: shut down IO complete
2022-02-01T15:22:02.688116Z TRACE mio::poll: deregistering event source from poller

Server's 10 secs grace period elapsed so delegate server's graceful shutdown

2022-02-01T15:22:08.757722Z  INFO static_web_server::signals: Grace period has elapsed
2022-02-01T15:22:08.757845Z  INFO static_web_server::signals: Delegating server's graceful shutdown
2022-02-01T15:22:08.757948Z TRACE mio::poll: deregistering event source from poller
2022-02-01T15:22:08.758197Z DEBUG hyper::server::shutdown: signal received, starting graceful shutdown
2022-02-01T15:22:08.758347Z TRACE mio::poll: deregistering event source from poller
2022-02-01T15:22:08.758844Z TRACE mio::poll: deregistering event source from poller
2022-02-01T15:22:08.759118Z  WARN static_web_server::server: termination signal caught, shutting down the server execution

Screenshots (if appropriate):
No

@jtackaberry what should happen after the tests pass is two Docker images.

joseluisq/static-web-server:devel
joseluisq/static-web-server:devel-alpine

So I will let you know to do the tests.

Thanks Jose. Looking forward to tryingit out.

@joseluisq I got impatient and built a custom image from source. :)
Great news: everything is working well.  Whereas with the previous build, or with the new build with a grace period of 0, I can reliably experience "502 Bad Gateway" failures during a kubectl rollout restart of the sws Deployment, with a grace period of 5 it's completely stable.  I've been curling URLs in a tight loop and have done a number of restarts without curl reporting any failures at all. 👍

Great news: everything is working well. Whereas with the previous build, or with the new build with a grace period of 0, I can reliably experience "502 Bad Gateway" failures from the upstream nginx-ingress during a kubectl rollout restart of the sws Deployment, with a grace period of 5 it's completely stable. I've been curling URLs in a tight loop and have done a number of restarts without curl reporting any failures at all.

Great to heard that! then it works as expected.
From my side I would like to do few more tests and just polish a bit the current changes.
I will do it right now so this PR should be ready in next minutes. Stay tuned.

BTW I'm using an u8 (8-bit unsigned integer type) for grace period in seconds which can host until 255 value max.
I hope it is something fine since I don't think most use cases will exceed the 4.25 min delay. Or?

@joseluisq  I am sure that's fine.  I can't imagine a grace period longer than that would be needed even for the largest of clusters.

The PR should be ready now. I have enhanced a bit the feature with some info logs as well as added the missing support and tests on Windows.
The devel Docker images should be right now available with these changes on Docker Hub for more tests from your side.
If everything is working as expected. There is no more stuff to do.

@joseluisq  devel image tested.  Everything looks good over here!

FYI: Binaries of devel  Docker images for Linux amd64 were compiled using normal production release procedure.
cargo build --release --target x86_64-unknown-linux-musl

So since upcoming v2.5.0 will take some time, it can be perfectly fine for you to use that devel Docker images until next release or feel free to use your own builds if you prefer so.

Sounds good to me.  Thanks very much for doing this @joseluisq.


Add 'authPassword' length limits to sepmac-cnf-xml.html

The password length limits probably the same for all models as it will be whatever CUCM uses. Otherwise the table needs to contain at least the limits for 6900, 7800, 7900 and 8800 series phones.
The limits for the other string fields (authName, featureLabel) should be documented at the same time.
The limit in the PR is for 6700 which does not appear to be a valid phone series.

Appoligies, typo with the model number, should read 7900.
Agree that the password length limit is likely universal, independent documentation I've come across also suggests this.
I only have a 7962 available for testing, but will report back on length limits for authName and featureLabel

I am not sure how these limits are being tested as those numbers seem odd.
Cisco publish a list of the tables used by CUCM called the Data Dictionary: https://developer.cisco.com/docs/axl/15-cucm-data-dictionary/
The devicenumplanmap table seems to be the source for line keys. The validation regexp shows that it accepts up to 30 characters.
Use that limit for all the fields.
The limit text needs to exactly match that used by <phoneLabel>.
<name> also needs the length limit text.

Testing by simply adding a character at a time until the phone fails to register. (Except featureLabel which was until the phone automatically added an ellipse, it continued to accept characters beyond that but they were meaningless)

It looks like the field length limit for authUsername is affected by the address of the SIP server. As this is appended to complete the authentication ie. @. In my case this allowed a username up to ~70 characters, but milage may vary. Diving into the validation rules however, it looks like the maximum hostname is ~50 characters, add this to the username validation at ~30 characters gets close to the practical limit I was reaching.
For some reason most of those validation rules are 31 characters instead of 30 (or a nice round number, but +1), not sure how to approach that...
Should we also document the 50 character length limit on the processNodeName being implied by the Data Dictionary?

Looks good to me.


Add South Bay Meetup

Hi - here are South Bay Meetup changes from<EMAIL_ADDRESS>Note the videos are on a Cisco public site.  Its not the same pattern you have now where they are on one you tube site.   I do have the original source .mpg files if someone needs for a you tube site but they are huge files so not sure how easy to transfer.

LGTM


[Suggestion] EMC for Basalt and Xtone Tiles

Both are nice building materials, and not much else.  Basalt only crafts into andesite (1 basalt + 1 nether quartz).  Xtones tiles chisel back and forth into other Xtones blocks. I recommend 1 EMC for both.  Block IDs listed below.
chisel:basalt
chisel:basalt1
chisel:basaltextra
quark:basalt
quark:world_stone_bricks
xtones:base

The following emc was added for the next update
quark:biotite => 256 (same is nether quartz)
quark:basalt => 1
xtones:base => 1
EMC values from the chisel variants are also auto-calculated from the above.


Slowdown when using alloc!

Hi,
I have the following example where I observe a 2x slowdown with Bumper.alloc!.
Could you please confirm that I use the package correctly?
Do you have ideas on how to fix this?
Thank you !
using BenchmarkTools, Bumper

function work0(polys; use_custom_allocator=false)
    if use_custom_allocator
        custom_allocator = Bumper.SlabBuffer()
        @no_escape custom_allocator begin
            work1(polys, custom_allocator)
        end
    else
        work1(polys)
    end
end

# Very important work
function work1(polys, custom_allocator=nothing)
    res = 0
    for poly in polys
        new_poly = work2(poly, custom_allocator)
        res += sum(new_poly)
    end
    res
end

function work2(poly::Vector{T}, ::Nothing) where {T}
    new_poly = Vector{T}(undef, length(poly))
    work3!(new_poly)
end

function work2(poly::Vector{T}, custom_allocator) where {T}
    new_poly = Bumper.alloc!(custom_allocator, T, length(poly))
    work3!(new_poly)
end

function work3!(poly::AbstractVector{T}) where {T}
    poly[1] = one(T)
    for i in 2:length(poly)
        poly[i] = convert(T, i)^3 - poly[i - 1]
    end
    poly
end

###

m, n = 1_000, 10_000
polys = [rand(UInt32, rand(1:m)) for _ in 1:n];

@btime work0(polys, use_custom_allocator=false)
#   6.461 ms (10001 allocations: 20.26 MiB)
# 0x0000e2e1c67cdb19

@btime work0(polys, use_custom_allocator=true)
#   14.154 ms (6 allocations: 608 bytes)
# 0x0000e2e1c67cdb19

Running on
julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39 (2023-12-25 18:01 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 8 × Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

Your problem is that you don't release the allocated arrays, and thus end up allocating a quite bit out of the buffer. If you correct this, you will get a little performance increase over the "normal" version
using BenchmarkTools, Bumper

function work0(polys; use_custom_allocator=false)
    if use_custom_allocator
        custom_allocator = Bumper.SlabBuffer()
        work1(polys, custom_allocator)
    else
        work1(polys)
    end
end

# Very important work
function work1(polys, custom_allocator)
    res = 0
    for poly in polys
        @no_escape custom_allocator begin
            new_poly = work2(poly, custom_allocator)
            res += sum(new_poly)
        end
    end
    res
end

function work1(polys)
    res = 0
    for poly in polys
        new_poly = work2(poly)
        res += sum(new_poly)
    end
    res
end

function work2(poly::Vector{T}) where {T}
    new_poly = Vector{T}(undef, length(poly))
    work3!(new_poly)
end

function work2(poly::Vector{T}, custom_allocator) where {T}
    new_poly = Bumper.alloc!(custom_allocator, T, length(poly))
    work3!(new_poly)
end

function work3!(poly::AbstractVector{T}) where {T}
    poly[1] = one(T)
    for i in 2:length(poly)
        poly[i] = convert(T, i)^3 - poly[i - 1]
    end
    poly
end

## #

m, n = 1_000, 10_000
polys = [rand(UInt32, rand(1:m)) for _ in 1:n];

@btime work0(polys, use_custom_allocator=false)
#   3.430 ms (10001 allocations: 20.20 MiB)
#0x0000e20ca5f991fb

@btime work0(polys, use_custom_allocator=true)
#  3.032 ms (4 allocations: 192 bytes)
#0x0000e20ca5f991fb

The code is not that well optimized, so it will give you only a very small performance improvement.

Your problem is that you don't release the allocated arrays, and thus end up allocating a quite bit out of the buffer.

But that is precisely the point !

The problem is that, when you don't release arrays that are out of scope, you end up using much more memory than you actually need. In this example your data should fit into L1 cache (little over 8kB). But, if you don't release the temporary data, you end up with a buffer that does not fit into cache at all (about 20MB).

Thanks for your answer, sorry for being not clear, let me explain.
My example is purposefully inefficient.
In my original problem, releasing intermediate arrays cannot be done easily.
The allocation pattern in my example here roughly resembles this.


Make horizontal vector display in Inspector more compact

Make the horizontal Vector display (controlled by interface/inspector/horizontal_vector2_editing) take up 1 line instead of 2.
Current (vertical) Vector display:

Current (horizontal) Vector display:

New (horizontal) Vector display:

I can't really see much benefit to the current behavior; it seems like the intent is presumably to use more horizontal space in exchange for less vertical space...

Can you add a bit of margin between x and y?

This makes all vectors display in single line:
extends Node
@export var v2: Vector2
@export var v3: Vector3
@export var v4: Vector4

(unfortunately couldn't shrink the width more :disappointed:)
And it makes Vector3/Vector4 single-line even with interface/inspector/horizontal_vector2_editing disabled:

Needs to be this wide to show a single digit per Vector4 component (with interface/theme/spacing_preset set to Compact):

I can't really see much benefit to the current behavior; it seems like the intent is presumably to use more horizontal space in exchange for less vertical space...

Note there's a vertical scrollbar available in the inspector, meaning even on small screens the vertical space is virtually infinite. Horizontal space is bigger problem as horizontal scrolling is not supported in the inspector.

Nothing against allowing to customize it like that but I'd say the current layout should be obtainable as well.

I changed the bool setting to an enum so the user can choose from any of the existing three options.

Ideally the spin slider component would show the value rather than the label at low width, but it seems complicated and I don't want to touch it in this commit.

it makes Vector3/Vector4 single-line even with interface/inspector/horizontal_vector2_editing disabled
I wasn't able to reproduce it -- note that Vector3 and Vector4 are controlled by the 'Vector Types' editor setting.

Passing on adding margin for the compact setting, because it's not a problem for the default theme, and I want to save the space.

I like this a lot as an option, but it requires the Inspector to be decently expanded to be used meaningfully, and the editor itself struggles a bunch with horizontal space as is.

Yeah, it requires a bit more expansion if you have a Vector4 -- but given how uncommon it is, and also because you can compactify Vector2 independently from Vector3 and Vector4 -- I'm hopeful it wouldn't be a big issue.
(Besides, the biggest culprit here by far is the label, taking up gobs of space it doesn't really need...)

Vector2 and Vector3/4 are controlled by different flags, so the user can choose what to do for each independently

For Vector4, #80893 provides an alternative layout. Maybe it's worth it to provide option for every vector type separately? It's weird to have vector_types_display_style, and a separate option for Vector2. Also the compact mode for Vector4 is unusable, unless your inspector is super wide.

the compact mode for Vector4 is unusable
I agree, though it's an opt-in option so doesn't make things any worse (and FWIW it's fine in my theme). Part of this is that in my experience Vector4 fields are vanishingly rare compared to Vector3.

My personal preference would be how Unity does it (Vector2 and Vector3 are COMPACT and Vector4 is HORIZONTAL) -- happy to provide an option for each independently, but I'm not familiar with the codebase and was a bit wary of adding a setting for such niche / fine-grained control. Could be better for a future commit?

Your PR already renames the existing settings, adding a new one wouldn't be much different.

SG, will do in a follow-up commit

For what it's worth, and not necessarily in the scope of this PR, but on this topic, it'd be great that there was some automatic layout logic so the wrapping happens or not according to the available width. Also, the distribution of width between the left column (property names) and the right one would benefit a lot from a more clever logic than we currently have.

That's def out of scope for this


Fix for Closure being namespaced to \Roots\Closure

The value method is namespaced to Roots\value. The method is using the Closure class from the global namespace.
However, as it's within the Roots namespace, it is incorrectly using \Roots\Closure.
This PR adds a backtick to the Closure class to ensure it is Closure from the global namespace.

It seems the function came from https://github.com/laravel/framework/blob/v5.6.25/src/Illuminate/Support/helpers.php#L1143-L1152 - this file is not namespaced.

Thanks for the PR!


Adds ANY, SOME, and EVERY SQL aggregation function tests

Addresses https://github.com/partiql/partiql-tests/issues/79.
Adds the ANY, SOME, and EVERY SQL aggregation function tests from partiql-lang-kotlin.
Also does a slight refactor (captured in the first commit https://github.com/partiql/partiql-tests/commit/2b6f95b3f60a55bd9a963aad14d78a75d2415624) of the syntax COLL_ aggregation tests. The second commit adds the new aggregation function tests.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Following up, maybe this is a good opportunity to also include tests for COLL_X(NULL), COLL_X(MISSING), COLL_X([MISSING]), and perhaps, COLL_X(<<MISSING, MISSING>>) for a good measure?  Not sure whether this needs to be done for all X.


OCPBUGS-5790: Allow cluster to be upgraded even if using multiple datacenters

Fixes https://issues.redhat.com/browse/OCPBUGS-5790
This is basically a manual cherry-pick of https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/123 because automatic cherry-pick was throwing tons of conflicts.

/cherry-pick release-4.10

/retest

/lgtm
/label backport-risk-assessed

/jira refresh

/label cherry-pick-approved

/jira refresh


readme: add missing lang to code block

for syntax highlighting

You need to rebase onto the latest changes in master, then I can approve it.

@Stargator did easy, used UI button - good enough ? otherwise, could rebase too ?


Par Accidentally I deleted my Xampp and I just installed it again but ,,, it appear like this

par I accidentally deleted my xampp ... :(

and I reinstalled it again but it goes like what u see on the image above :(

Ok na toh?

okay na par :D
On Tue, Aug 8, 2017 at 8:57 PM, Lloric Mayuga Garcia <
<EMAIL_ADDRESS>wrote:

Ok na toh?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/lloricode/e-ttendance-laravel/issues/3#issuecomment-320947974,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AdKpEs9Dlp811QrHHKALqnIsjWHQanfrks5sWFtXgaJpZM4OweqS
.


Feature request: Filter out specific "bad actors"

I would like to filter out messages that match multiple conditions. This would allow most logs to flow normally, while filtering out specific "bad actors" producing too many logs.
While this does sound similar to what filter_grep provides, it does not support this use-case.
For example, I have logs like the following:
{"level":"debug", "log":"hello!",                 "container_name":"foo"}
{"level":"error", "log":"something bad happened", "container_name":"foo"}
{"level":"debug", "log":"hello!",                 "container_name":"bar"}

My goal is to filter out debug logs for container_name foo.
So output would be:
{"level":"error", "log":"something bad happened", "container_name":"foo"}
{"level":"debug", "log":"hello!",                 "container_name":"bar"}

It is important that only container_name foo is affected, so if I add other containers with debug messages later on then they will not be filtered out with this config.
In other words, the same config should allow new logs from container_name thing, even if they are debug level.
So with the same config, these logs should make it through:
{"level":"debug", "log":"hello!", "container_name":"thing"}
{"level":"debug", "log":"hello!", "container_name":"some_app"}

Problem
The current standard grep filter only supports AND matches between <regexp> blocks and OR matches between <exclude> blocks.
I can create a fluentd config for the above example that almost works:
<filter **>
    @type grep
    <exclude>
        key container_name
        pattern foo
    </exclude>

    <exclude>
        key level
        pattern debug
    </exclude>
</filter>

The problem with this config is that it excludes all logs from foo containers, including error logs. This also excludes debug logs from the bar containers.
Ok, so maybe we can flip it a little? Unfortunately, we now hit the complement of the same problem:
<filter **>
    @type grep
    <regexp>
        key container_name
        pattern (?!.*foo).*    # not foo
    </regexp>

    <regexp>
        key level
        pattern (?!.*debug).*  # not debug
    </regexp>
</filter>

This config still throws away all of foo's logs and any debug logs.
Possible solution
One possible solution is to modify the grep filter to support a new <exclude-all> block that ANDs together each block before throwing out a log.
Please let me know how I can help! 😄

<exclude-all> idea looks good.
Adding this condition to https://github.com/fluent/fluentd/blob/94287666b655edd9a6b4a142e6a5aa03fa0681ca/lib/fluent/plugin/filter_grep.rb#L108 is better.
if @_exclude_all
  # check key and condition pair
end

@JohnStarich I wrote some code for your feature request #1897
Could you try it?

Oh, nice. I like that idea. I have a feeling @repeatedly will want backwards compatibility so that standalone blocks of regexp and exclude will work. Perhaps these <and> and <or> blocks could be used in addition to the current behavior?

Perhaps these  and  blocks could be used in addition to the current behavior?

Yes, you can use <and>, <or> with current behavior:
<filter>
  @type grep
  <and>
    <regexp>
      key level
      pattern ^ERROR|WARN$
    </regexp>
    <regexp>
      key method
      pattern ^GET|POST$
    </regexp>
  </and>
  <regexp>
    key path
    pattern ^/login$
  </regexp>
</filter>

For more combinations, see test code.

Ah, yes you're right! I misread the changes in the diff as removing old behavior when it had really just moved.
Looks like it does exactly what I was looking for. Thanks!

Great to see this made it to master in https://github.com/fluent/fluentd/pull/1897 !
Is there a released version available for installation?

v1.2.0.pre1 is released.

Awesome! Thank you both! 🎉


Plugins not installing "fatal: repository 'latest' does not exist"

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior.
I have tried a few different syntaxes.
mise plugin add/install/i pnpm
mise install pipenv
~ > mise install<EMAIL_ADDRESS>--verbose                                                               09:41:25 PM
[DEBUG] ARGS: /Users/zack/.home/.local/bin/mise install<EMAIL_ADDRESS>--verbose
[DEBUG] Config {
Config Files: [
"~/config/mise/config.toml",
],
}
[DEBUG] Toolset: node@latest, <EMAIL_ADDRESS>ruby@latest
[DEBUG] install pnpm "latest"
cloning latest
[DEBUG] cloning latest to /Users/zack/.home/local/mise/plugins/pnpm
[DEBUG] $ git --version
[DEBUG] $ git clone -q --depth 1 latest /Users/zack/.home/local/mise/plugins/pnpm
fatal: repository 'latest' does not exist
Error:
0: command ["git", "clone", "-q", "--depth", "1", "latest", "/Users/zack/.home/local/mise/plugins/pnpm"] exited with code 128
Location:
src/git.rs:86
Version:
2024.3.1 macos-arm64 (0d95fde 2024-03-04)
Suggestion: Run with --verbose or MISE_VERBOSE=1 for more information.
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
Expected behavior
A clear and concise description of what you expected to happen.
mise doctor output
~ > mise doctor                                                                                3s 09:42:32 PM
version: 2024.3.1 macos-arm64 (0d95fde 2024-03-04)
activated: yes
shims_on_path: no
build_info:
Target: aarch64-apple-darwin
Features: DEFAULT, NATIVE_TLS, OPENSSL
Built: Mon, 4 Mar 2024 22:02:06 +0000
Rust Version: rustc 1.76.0 (07dca489a 2024-02-04)
Profile: release
shell:
/opt/homebrew/bin/zsh
zsh 5.9 (arm-apple-darwin23.0.0)
dirs:
data: ~/local/mise
config: ~/config/mise
cache: ~/cache/mise
state: ~/.local/state/mise
shims: ~/local/mise/shims
config_files:
~/config/mise/config.toml
plugins:
bun     (core)
deno    (core)
erlang  (core)
go      (core)
java    (core)
node    (core)
pnpm    https://github.com/jonathanmorley/asdf-pnpm.git#305baff
python  (core)
ruby    (core)
toolset:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>env_vars:
MISE_SHELL=zsh
settings:
activate_aggressive = false
all_compile = false
always_keep_download = false
always_keep_install = false
asdf_compat = false
cargo_binstall = true
color = true
disable_default_shorthands = false
disable_tools = []
experimental = true
go_default_packages_file = "~/.default-go-packages"
go_download_mirror = "https://dl.google.com/go"
go_repo = "https://github.com/golang/go"
go_set_gopath = false
go_set_goroot = true
go_skip_checksum = false
jobs = 4
legacy_version_file = true
legacy_version_file_disable_tools = []
node_compile = false
not_found_auto_install = true
paranoid = false
plugin_autoupdate_last_check_duration = "2 weeks"
python_compile = true
python_default_packages_file = "/Users/zack/.home/.default-python-packages"
python_pyenv_repo = "https://github.com/pyenv/pyenv.git"
raw = false
trusted_config_paths = []
quiet = false
verbose = false
yes = false
ci = false
debug = false
trace = false
log_level = "info"
python_venv_auto_create = false
[status]
missing_tools = "if_other_versions_installed"
show_env = false
show_tools = false
No problems found
REPLACE WITH OUTPUT OF `mise doctor`

Additional context
Add any other context about the problem here.
Okay well, fixed the problem myself lol. I'm migrating from RTX and I had the config from it in my mise folder. The mise doctor report may have came from after I fixed the issue. The config named the 2 plugins I bothered trying to download (pnpm/pipenv). What's odd, though, is I had attempted to download a versioned 1 of each and it still failed. I had "latest" as the version for each in the old config.
Figured I'd still post this because it is an odd error and maybe that'd affect people outside someone migrating? Who knows!

Yeah, so I'm looking in my pipenv install folder because mise upgrade pops up with an error
Error: Command '['/Users/zack/.home/local/mise/installs/pipenv/2023.12.1/bin/python3', '-m', 'ensurepip', '--upgrade', '--default-pip']' died with <Signals.SIGABRT: 6>.
And there aren't any versions in there. I'm going to tinker with forcing compilation for pipenv because that seems to be a theme and right now when I attempt to use Python 3.11.6 as written out in my local mise settings, the virtual environment ends up running with a differing version 3.11.8. I have a feeling it is because pipenv only exists in 3.11.8 (even though I did install it via mise -- as oppposed to via pip. Odd!)


Not working, attaching an example of mocking a static method when calling from another static method

Attached is an example (full project) with the simplest possible test for a scenario when a static method is calling another static method.
Can't make it to mock. The mock is ignored and the original method is executed instead.
StaticMockExample.zip

I checked your example. You should write a test a little bit different. part that mocked should be under callback in Mock.Setup method. I rewrite you example and it works for me:
        [Test]
        public void Test()
        {
            // arrange
            Mock.Setup(typeof(Callee), nameof(Callee.Call2), () =>
            {
                // act
                var mockActualResult = Caller.Call1(0);

                // assert
                Assert.AreEqual(11, mockActualResult);
            }).Returns(11);

            var rollbackedResult = Caller.Call1(0);

            // assert rollback
            Assert.AreEqual(2, rollbackedResult);
        }

thank you for the explanation, basically for my example the original test should be rewritten in the way:
[Test]
public void TestPass()
{
    // arrange
    Mock.Setup(typeof(Callee), nameof(Callee.Call2), () =>
    {
        // act
        var mockActualResult = Caller.Call1(0);

        // assert
        Assert.AreEqual(11, mockActualResult);
    }).Returns(11);
}

where I should be doing the act and assert within the Mock.Setup body only.
And if I need to mock a few calls like Callee.Call2 here, I assume by design I should be placing the whole test method code above inside the second Mock.Setup body. Something like
[Test]
public void TestPass()
{
    // arrange
    Mock.Setup(typeof(Callee), nameof(Callee.Call2), () =>
    {
        Mock.Setup(typeof(CalleeNext), nameof(CalleeNext.Call3), () =>
        {
            // act
            var mockActualResult = Caller.Call1(0);

            // assert
            Assert.AreEqual(26, mockActualResult);
        }).Returns(15);
    }).Returns(11);
}

// while the method under test is:

public class Caller
{
public static int Call1(int x)
{
x++;
int r = Callee.Call2(x);
r += CalleeNext.Call3(x);
return r;
}
}
}

Yes, your examples look good and should work. If not, let me know. And this is a good idea to support multiple classes/methods mocking in one setup. I will try to implement this in future versions.
Thank you.

What would be good is to support sequential calls instead of hierarchical
existing implementation:
`
// arrange
Mock.Setup(typeof(Callee), nameof(Callee.Call2), () =>
{
Mock.Setup(typeof(CalleeNext), nameof(CalleeNext.Call3), () =>
{
        // act
        var mockActualResult = Caller.Call1(0);

        // assert
        Assert.AreEqual(26, mockActualResult);

    }).Returns(15);
}).Returns(11);

`
to be like:
`
// arrange
Mock.Setup(typeof(Callee), nameof(Callee.Call2)).Returns(11);
Mock.Setup(typeof(CalleeNext), nameof(CalleeNext.Call3)).Returns(15);
// act
var mockActualResult = Caller.Call1(0);

// assert
Assert.AreEqual(26, mockActualResult);

`

Thank you for suggestions. Looks like good improvement. I will implement this, but also for compatibility I will leave hierarchical calls too.

looks awesome, thanks for your work

for generic methods mock you should close generic type with SetupProperties. Here is an example:
Method for test:
public static TEntity GenericTestMethodReturnDefaultWithoutParameters<TEntity>()
{
    return default;
}

Mock:
const int expectedResult = 2;

using var _ = Mock.Setup(
    typeof(TestStaticClass),
    nameof(TestStaticClass.GenericTestMethodReturnDefaultWithoutParameters),
    new SetupProperties { GenericTypes = new[] { typeof(int) } })
.Returns(expectedResult);

var actualResult = TestStaticClass.GenericTestMethodReturnDefaultWithoutParameters<int>();
Assert.AreEqual(expectedResult, actualResult);

Here a link for example in tests
https://github.com/SvetlovA/static-mock/blob/c9f089d14e0db3b5a52f62b3a79423e7fc05a6da/src/StaticMock.Tests/Tests/Sequential/ReturnsTests/MockReturnsTests.cs#L330


Created a Medium Button Style

Summary
In this PR I have created the medium button style for the button component. I have also updated the hero component to use the large button style as it was intended.

List of notable changes:

added medium button style for the button component
updated hero component to use the large button style

What should reviewers focus on?
Making sure that the Button component's new style meets the spec provided.
Steps to test:

Open the Button component in CI-deployed preview environment
Go to Button story in Storybook
Verify that Button has two configurable states for size 'large' and 'medium' with 'medium' being the default
Check that the Hero component is using the large button style

Supporting resources (related issues, external links, etc):

Issue
Spec

Contributor checklist:

[ ] All new and existing CI checks pass
[x] Tests prove that the feature works and covers both happy and unhappy paths
[x] Any drop in coverage, breaking changes or regressions have been documented above
[x] New visual snapshots have been generated / updated for any UI changes
[x] All developer debugging and non-functional logging has been removed
[x] Related issues have been referenced in the PR description

Reviewer checklist:

[ ] Check that pull request and proposed changes adhere to our contribution guidelines and code of conduct
[ ] Check that tests prove the feature works and covers both happy and unhappy paths
[ ] Check that there aren't other open Pull Requests for the same update/change

@JoshBowdenConcepts - this is looking great. Thanks for making this change.
Just a few more things I reckon we should do before it's merged...

I think medium and large parameters qualify as 'design logic', so we would ideally put a test around it IMO. You can see some of the other tests for examples of how it's done elsewhere. This is non-blocking though, and happy to discuss when you're online.
I've suggested some changes to the changeset, given it's going to be published in the Change log and we're technically issuing a breaking change by defaulting to the smaller button
I've added the new snapshots in this commit


ADDED - Human Activity Recognition with Smartphones Dataset

FIXED ISSUE
Issue Ticket Number
Fixes #167

Checklist:

[x] I have followed the contributing guidelines of this project as mentioned in CONTRIBUTING.md
[x] I have cross-checked that the dataset I'm uploading is not present in the project
[x] I have cross-checked that the dataset is present in the correct folder

Good one @Kesavan-Raman
Congrats 🎉, your PR got merged. To be eligible for the swags from ClueLess. Follow these steps.

Star this repository. ⭐
SignIn / SignUp in [ClueLess Website](https://www.clueless.tech/) .
Go to the profile section from the top right-hand corner.

Comment your profile URL under this comment tagging the person who merged your PR.

That’s it. Follow the leaderboard!
And don’t forget to share your achievement with your friends.


Why not use VGG?

I only found a SIFT features matcher....
generally, the accuracy for SIFT about 50%, and VGG can upto 90%, why not use VGG?

@coollofty I don't fully understand your question. Do you mean using VGG for image retrieval. Then I guess what you describe here is a different method from this (vocabulary tree). Using VGG and other neural net models is the state-of-the-art for global image retrieval, but it's not the scope of this repo.


Use 2D Histogram for visualization

Ongoing implementation of providing 2D heatmaps as alternative to 2d scatterplot.
Currently, the 2D heatmap replaces the 2D scatterplot. It takes into account colors as the scatterplot does. It is much much faster than the scatterplot when having a big dataset.
Next I will provide a dropdownbox that allows to switch between scatter and 2dheatmap.
I thought I already create a draft pull request so that you @zoccoler  @lazigu can check what kind of refactoring I did.
Best,
Thorsten

Alright, I added a QCombobox for switching between scatter and 2D histogram.
Right now the default is scatter, but as 2D Histogram is much faster, I would actually recommand to make this as new default. I will create a movie and post it here.

Added some changes that allows to have bigger plots

Ok, the number of bin for 2D histogram are now changeable. Here the current status (sorry for the video bad quality):

Looking forward to your feedback!

Hi @thorstenwagner ,
Thanks a lot for this contribution! It looks fantastic! I will give it a try and provide you feedback here. :)

Thanks! Do you need sample data?
08.03.2023 16:34:17 Marcelo Zoccoler @.***>:

Hi @thorstenwagner[https://github.com/thorstenwagner] ,
Thanks a lot for this contribution! It looks fantastic! I will give it a try and provide you feedback here. :)
—
Reply to this email directly, view it on GitHub[https://github.com/BiAPoL/napari-clusters-plotter/pull/196#issuecomment-1460342718], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAIP66LA4W4TXWTX7JVKZE3W3CRHPANCNFSM6AAAAAAVTTIEOM].
You are receiving this because you were mentioned.[Verfolgungsbild][https://github.com/notifications/beacon/AAIP66OKMRMEGUPJP3ZDRTDW3CRHPA5CNFSM6AAAAAAVTTIEOOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSXBMH34.gif]

Hi,
here is the test data:
https://owncloud.gwdg.de/index.php/s/rnRHi1co1Dx4ckL
Best,
Thorsten

Thank you! In the meantime, I was testing it with a small image.
First of all, the new functionality is amazing! It is really what we were looking for this type of visualization!
Now to some details regarding usability. I list them below, but the gif shows most of them.

To reproduce, I used this code and played a bit with the options (I have devbio-napari installed in this environment):
import napari
from napari_clusters_plotter._plotter import PlotterWidget
import pyclesperanto_prototype as cle
from napari_skimage_regionprops._regionprops import regionprops_table
from napari_skimage_regionprops import add_table

viewer = napari.Viewer()

viewer.open_sample(plugin='clEsperanto', sample='Blobs (from ImageJ)')
image = viewer.layers[0].data
labels = cle.voronoi_otsu_labeling(image, outline_sigma=2, spot_sigma=4)
table = regionprops_table(image, labels)
labels_layer = viewer.add_labels(labels, features=table)
add_table(labels_layer, viewer) # avoids current bug with status bar

widget, _ = viewer.window.add_plugin_dock_widget('napari-clusters-plotter', widget_name='Plotter Widget')
napari.run()

the axis names are black
an error is thrown when some selection is made and the plotting mode is changed
an error is thrown if you draws a lasso over a region without any underlying points in histogram mode
in histogram mode, clicking on the plot does not reset the labels
the aspect ratio changes constantly and sometimes this leads to a squeezed plot
the hiding of background labels is too amazing! 🤣 thanks for that!

Let me know whether these issues are easy or tricky to fix, I will try to review the code by tomorrow.

OK, I guess fixed all problems except "the axis names are black" as they are not black on my side oO

Could you please set this PR to be merged into #182 (branch fix-axes-labels-not-displayed-bug), which will be merged soon, and should fix axes colour (not visible at all) problems?
Other stuff, I'll review in a bit :)

Could you please set this PR to be merged into #182 (branch fix-axes-labels-not-displayed-bug), which will be merged soon, and should fix axes colour (not visible at all) problems? Other stuff, I'll review in a bit :)

Done!

Merged the changes of fix-axes-labels-not-displayed-bug into my branch and fixed one bug that came up after merging.

I've added one more change. Now the number of bins are automatically estimated according Freedman–Diaconis rule.

Hi again @thorstenwagner, I see you are adding more changes to this PR, should this still be converted to draft PR? It gets a bit tricky to review code changes and test everything if new changes keep coming in :) If you are also thinking about adding something else to this PR, just wanted to let you know that it would be way more convenient for us to receive multiple smaller PRs instead of one big one with many changes, so maybe further changes could go into separate PRs, e.g. you could open PRs to your other PR branches. Thanks a lot again :)

Hi @lazigu ! No, I think its stable now. I just already interacting with this plugin by external tools and noticed a problem when making the label layer invisible. That's fixed. I've no further changes in mind.
Sorry, for this "monster" PR. Will create smaller PR from now on.
Best,
Thorsten

I recently merged the PR to implement a checkbox for hiding non-selected clusters, and it appears that branches are not agreeing with each other. @thorstenwagner, would you mind solving the merging conflicts? Or I can do that if you give me the green light :)

Hi @lazigu and @thorstenwagner ,
Do you think this PR is now ready for review? I'd like to give it a new try tomorrow, hopefully merge it soon :)
Does it depend on merging another PR before?

From my side it is ready for review :-)

It depends on merging "fix axes labels not displayed bug" PR, I was waiting for Robert's or yours, Marcelo @zoccoler, review on that one, but in a way it can simultaneously be tested when testing this PR. It should have fixed axes labels being black/not visible bug.

I think the 2D histogram won't work with automated clustering procedures. I could imagine calculating the convex hull of the clustering results to create the polygons for visualization. However, this would fail in case of nested clusters. What do you think?

I think the 2D histogram won't work with automated clustering procedures. I could imagine calculating the convex hull of the clustering results that comes from an automated procedure to create the polygons for visualization. However, this would fail in case of nested clusters. What do you think?

You mean #204 right? At first I was thinking about the label image, not sure how to handle the plot yet. I guess #205 needs to be better thought then, I am turning it to Draft for now.

I am bringing #194 changes here, so both PRs get synchronized.
Once we merge #194, this could more easily follow.

I think the 2D histogram won't work with automated clustering procedures. I could imagine calculating the convex hull of the clustering results that comes from an automated procedure to create the polygons for visualization. However, this would fail in case of nested clusters. What do you think?

You mean #204 right? At first I was thinking about the label image, not sure how to handle the plot yet. I guess #205 needs to be better thought then, I am turning it to Draft for now.

Well, #204 is definitely true for the hide clusters option and I guess your change #205 fixed it. But right now, there exist no method that converts automatically found clusters (e.g. with k-means) to polygons that can be visualized with the 2D histogram method. I could imagine, that calculating convex/concave hulls could work.

But right now, there exist no method that converts automatically found clusters (e.g. with k-means) to polygons that can be visualized with the 2D histogram method. I could imagine, that calculating convex/concave hulls could work.

I don't really have any ideas about this. Getting @haesleinhuepf and maybe @allysonryan in the loop here if they have something in mind.

Another bug that came up during testing -> it works fine to select manually clusters in the plot. However, if one wants to plot the same manual clusters again and selects it here:

After clicking run such an error is received (expand for more)
```
IndexError                                Traceback (most recent call last)
File c:\users\zigut\projects\napari-clusters-plotter\napari_clusters_plotter\_plotter.py:305, in PlotterWidget.__init__..run_clicked()
    298     warnings.warn(
    299         "No axis(-es) was/were selected! If you cannot see anything in axes selection boxes, "
    300         "but you have performed measurements/dimensionality reduction before, try clicking "
    301         "Update Axes Selection Boxes"
    302     )
    303     return
--> 305 self.run(
        self =
        self.labels_select = ComboBox(value=, annotation=, name='')
        self.plot_x_axis =
        self.plot_y_axis =
        self.plot_cluster_id =
    306     get_layer_tabular_data(self.labels_select.value),
    307     self.plot_x_axis.currentText(),
    308     self.plot_y_axis.currentText(),
    309     self.plot_cluster_id.currentText(),
    310 )
File c:\users\zigut\projects\napari-clusters-plotter\napari_clusters_plotter_plotter.py:508, in PlotterWidget.run(self=<napari_clusters_plotter._plotter.PlotterWidget object>, features=           label    umap_0     umap_1    index  ... 1                  0
[4644808 rows x 6 columns], plot_x_axis_name='umap_0', plot_y_axis_name='umap_1', plot_cluster_name='MANUAL_CLUSTER_ID', redraw_cluster_image=True)
505     else:
506         number_bins = int(self.bin_number_spinner.value())
--> 508     self.graphics_widget.make_2d_histogram(
self = <napari_clusters_plotter._plotter.PlotterWidget object at 0x0000013EE0C2DC60>
self.graphics_widget = <napari_clusters_plotter._Qt_code.MplCanvas object at 0x0000013C9EF8C820>
self.data_x = 0          5.445981
1          5.654449
2          6.194092
3          7.371770
4          7.549777
...
4644803    4.677732
4644804    2.833497
4644805    3.528196
4644806    5.376968
4644807    5.482071
Name: umap_0, Length: 4644808, dtype: float64
self.data_y = 0         -10.043943
1         -10.687407
2         -10.491974
3         -10.640293
4         -10.583367
...
4644803    -7.594534
4644804    -0.596690
4644805    -3.842270
4644806    -4.647440
4644807    -7.260288
Name: umap_1, Length: 4644808, dtype: float64
cluster_colors = ['#1f77b4']
number_bins = 371
509         self.data_x,
510         self.data_y,
511         cluster_colors,
512         bin_number=number_bins,
513     )
514 self.graphics_widget.axes.set_xlabel(plot_x_axis_name)
515 self.graphics_widget.axes.set_ylabel(plot_y_axis_name)
File c:\users\zigut\projects\napari-clusters-plotter\napari_clusters_plotter_Qt_code.py:522, in MplCanvas.make_2d_histogram(self=<napari_clusters_plotter._Qt_code.MplCanvas object>, data_x=0          5.445981
1          5.654449
2       ...071
Name: umap_0, Length: 4644808, dtype: float64, data_y=0         -10.043943
1         -10.687407
2     ...288
Name: umap_1, Length: 4644808, dtype: float64, colors=['#1f77b4'], bin_number=371)
514 def make_2d_histogram(
515     self,
516     data_x: "numpy.typing.ArrayLike",
(...)
519     bin_number: int = 400,
520 ):
521     if len(colors) == 1:
--> 522         self.polygons = [self.polygons[-1]]
self.polygons = []
self = <napari_clusters_plotter._Qt_code.MplCanvas object at 0x0000013C9EF8C820>
524     self.axes.hist2d(data_x, data_y, bins=bin_number)  # extent=extent,
526     for poly_i, poly in enumerate(self.polygons):
IndexError: list index out of range

I will have a look into this bug. Thanks for testing!

@lazigu I can't reproduce this. Can you give me more detailed instructions how to reproduce this issue?

I fixed my issue.

Yes, of course, I will try it. I was trying to make a video to show the issue, but it was taking forever on my laptop with your example data, so now I made it with blobs image, but I will pull the changes first and test it again.

While working with the current implementation, we noticed that the color of the first cluster (blue) is very hard to see on the current colormap for the 2D histogram. Its just be a very simple change to change this:
 self.axes.hist2d(data_x, data_y, bins=bin_number, cmap='magma')

Here to contrast of the cluster colors and histogram is much better.
If you want, I make separate pull request out of it once this one is merged. But I guess think it would be a straight forward change that does not require extensive testing. Let me know what you prefer :-)

Thanks @lazigu , I can reproduce this issue now! I will work on it tomorrow.
Sorry for my ignorance, can you point me to your message regarding the plot size again? I can't find it. I will of course make the changes.

Sorry for my ignorance, can you point me to your message regarding the plot size again? I can't find it. I will of course make the changes.

Nono it is my mistake! You are very fast at responding so I assumed that my review message was probably not visible, just wanted to double check :) But now I see that message contains the label "pending" so I know now that Github shows it only to me as a review in progress. Sorry for the confusion!

Thanks @lazigu , I can reproduce this issue now! I will work on it tomorrow.

Great, no rush!
I will try to brainstorm with the group tomorrow regarding the visualization of the algorithmic clustering results issue.

Could you also explain me when clusters are deleted?
Because its kind of strange that you can save clusters with "update mesaurements" if the clusters are not visually shown. So far I thought, I can save cluster with "update measurement" as long as they are shown. But in your case, you reset it visually with 'run' and then save it with "update measurements", which is counter intuitive for me.

Thanks for the explanation @lazigu !
Another limitation which came to my mind:
If you select clusters on the histogram plotting feature A and B, you can't get those clusters plotted if you change to feature C and D - because the polygons are not valid anymore and the points are not necessarily clusters in your new feature selection  (C, D). That also means, there is no general valid way to calculate a polygon that for the new feature selection based on the old feature selection.

Hey @thorstenwagner ,
we were just discussing today in the group meeting. We think it might be possible to use the hand-drawn polygons on the 2D histogram to create a column in the table which represents the manual annotation (this is how it worked before). A 2D image from this table column could be generated to, similar to how the 2D histogram is created. We just use the maximum column value in a histogram bin instead of the count in the bin. Does this make sense?
Thanks again for your contribution! It's amazing to see what cool stuff you're implementing here.
Best,
Roberrt

Addendum: Having (and potentially storing) this manual annotation column is very important for post-processing it (feature selection, quality assurance,...).

create a column in the table which represents the manual annotation (this is how it worked before)

But this is what already happens right? It creates the MANUAL_CLUSTER_ID column? This is a representation of the annotation?

Yes, correct!

Not sure if I fully understand. Lets say I have selected a single cluster in histogram A vs B. Now the user selects feature C and D and press "Run". Now you propose to go through each bin, check which is the maximum MANUAL_CLUSTER_ID of the elements within that bin and set the value of the bin to this max value?
What will happen to the bins that don't have a assigned cluster?
We would also loose all density based structure, so this generated image would not be suitable to do any further selection?
If there is a mixture of clusters within a bin we would also loose that in information, right (at least visually)?

I think the 2D histogram with the intensity remains in the background. And in the foreground you draw another histogram-like/shaped image. This second image marks a pixel in blue if the class 1 is most-popular among the data points in this bin. It's not the maximum (my wrong explanation/though above), it's the mode of the bin (most popular class). I admit, I've never done that, but pandas supports it apparently. Maybe worth a try in a notebook first before integrating it in the clusters plotter.
Btw. if things become too complicated, I'm happy to schedule a zoom meeting to discuss.

I think now I got what you mean.  That requires some changes. I'm also afraid that generating this image might be quite slow huge datasets like mine.
The way I would implemented it is to generate a second RGBA image with bins with the color of the most popular class  and the overlay it on top of the original hist2d.

I'm also afraid that generating this image might be quite slow huge datasets like mine.

I would expect this to be as fast as the generation of the 2D histogram, no?

After merging all conflicts, I run in an exception:
Traceback (most recent call last):
  File "/opt/user_software/miniconda3/envs/napari-env/lib/python3.10/site-packages/matplotlib/cbook/__init__.py", line 307, in process
    func(*args, **kwargs)
  File "/opt/user_software/miniconda3/envs/napari-env/lib/python3.10/site-packages/matplotlib/widgets.py", line 2003, in release
    self._release(event)
  File "/opt/user_software/miniconda3/envs/napari-env/lib/python3.10/site-packages/matplotlib/widgets.py", line 3578, in _release
    self.onselect(self.verts)
  File "/mnt/data/twagner/Projects/napari-clusters-plotter-tw/napari-clusters-plotter/napari_clusters_plotter/_Qt_code.py", line 439, in onselect
    self.parent.manual_clustering_method(self.ind_mask)
  File "/mnt/data/twagner/Projects/napari-clusters-plotter-tw/napari-clusters-plotter/napari_clusters_plotter/_plotter.py", line 107, in manual_clustering_method
    self.run(
  File "/mnt/data/twagner/Projects/napari-clusters-plotter-tw/napari-clusters-plotter/napari_clusters_plotter/_plotter.py", line 644, in run
    cluster_image = generate_cluster_image(
  File "/mnt/data/twagner/Projects/napari-clusters-plotter-tw/napari-clusters-plotter/napari_clusters_plotter/_utilities.py", line 238, in generate_cluster_image
    return map_array(label_image, label_list, predictionlist_new).astype("uint64")
  File "/opt/user_software/miniconda3/envs/napari-env/lib/python3.10/site-packages/skimage/util/_map_array.py", line 57, in map_array
    _map_array(input_arr, out_view, input_vals, output_vals)
  File "skimage/util/_remap.pyx", line 9, in skimage.util._remap.__pyx_fused_cpdef
TypeError: Function call with ambiguous argument types

I fixed it with #0d75ef3. the label_image was clEsperanto array, so maybe you know a better fix.

I fixed it with #0d75ef3. The label_image was of type pyclesperanto_prototype._tier0._pycl.OCLArra, so maybe you know a better fix.

Thanks for catching this! A very important bug that we missed.

Shall I make a new PR or can you reopen it?

@thorstenwagner, nevermind, we were able to open this one again, and will change the branch to merge to.

Thanks for testing it!!
Added your requested change :-) Looking forward to see it merged.

Hi @thorstenwagner , hi @lazigu ,
I tested it again with the small blobs image and everything seems to be working very nicely!
I think it can be merged now :)


I wait

Ditunggu isinya gan...

Masih tahap alpha di local.

Oke gan

Promises make me crazy, sorry for late.
But today it's ready to rock.

Bahasamu Le. Hahaahahaa


Log when a paranoid durability write succeeds.

To verify that the paranoid durability feature is working as intended this PR adds a log statement for when a request succeeds in ParanoidDurabilityOperationTracker.

Codecov Report
Attention: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.

Project coverage is 18.63%. Comparing base (52ba813) to head (bfad52e).
Report is 40 commits behind head on master.

Files
Patch %
Lines

...bry/router/ParanoidDurabilityOperationTracker.java
0.00%
5 Missing :warning:

:exclamation:  There is a different number of reports uploaded between BASE (52ba813) and HEAD (bfad52e). Click for more details.
HEAD has 2 uploads less than BASE

Flag
BASE (52ba813)
HEAD (bfad52e)

3
1

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #2822       +/-   ##
=============================================
- Coverage     64.24%   18.63%   -45.61%
+ Complexity    10398     2915     -7483
=============================================
  Files           840      842        +2
  Lines         71755    72356      +601
  Branches       8611     8707       +96
=============================================
- Hits          46099    13487    -32612
- Misses        23004    57630    +34626
+ Partials       2652     1239     -1413

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


allow batch as scanner option

The "batch" option was documented but actually never allowed as an option.

I see that the CI is failing but maybe the pipeline is broken?
Another question: should we add the lib-folder to gitignore?
Cheers

regarding the lib folder, it shoud be committed to git

I sort of remember trying to work on it but I believed its REST implementation was buggy a few years back. Could you provide a test associated to this option. Also, I just fix the travis integration and you can use the docker command inside it docker run --name stargate -d -p 60080:8080 sixeyed/hbase-stargate associated with its properties cp test/properties.docker.json test/properties.json to easily run the tests.

Thanks. I'm not sure that I will succeed this weekend but I will try.
I just have a question maybe you can help: I must be able to read all cells having the same rowkey but I'm failing even if I use the option "column". I'm using a rowFilter with a GREATER op. Do you have any idea?

Current state of the batch option: I know that it works, because I raised it to 2000 in my current project, succesfully. On the other side I'm not able to get the test green :(
I have no idea what to do, please decide for yourself: we can either keep the code or revert it.

Sorry, I wasn't available to look at your changes. If I understood correctly, you're not suggesting to incorporate those changes submitted 2 weeks ago ?

sorry, I just reverted the code, I don't like red builds ;)


Exposing TypeSpec suppressions in APIView

Turn them into system comments
Feedback from API Stewardship

This is done. THe suppression directives are displayed.


@joeybab3 @chip-grab #244 is causing tracks to be COR-ed out. https://openstreetcam.org/details/2447522

@joeybab3 @chip-grab #244 is causing tracks to be COR-ed out. https://openstreetcam.org/details/2447522
Originally posted by @panthersdude2001 in https://github.com/openstreetcam/openstreetcam.org/issues/254#issuecomment-648335143

@panthersdude2001 I'm not seeing this issue on my end. The track you sent above doesn't seem to exist or at least is similar to a track of mine that I deleted but still shows up in my feed. When clicking on this deleted track it shows the same message that I get when I open the one you sent but all of my other tracks load fine.
Please keep discussion of other errors to the associated threads though it makes getting and giving help on GitHub much easier when you don't have to scroll through a crap ton of comments.
Originally posted by @joeybab3 in #254 (comment)

Ok, Try this one, https://openstreetcam.org/details/2447310/0/track-info
It is processing right now, so if it doesnt work, check back in a few minutes.
Originally posted by @panthersdude2001 in #254 (comment)

Please keep discussion of other errors to the associated threads though it makes getting and giving help on GitHub much easier when you don't have to scroll through a crap ton of comments.

Since the #244 bug has to do with processing, and the PPR (Proccessing PipeLine Rebuild), this is related,
Originally posted by @panthersdude2001 in #254 (comment)

Closing as ppr is done and everything is faster now.


Rerooting

Adds a rerooting operation for tskit trees, and does some code reorganisation.
@hyanwong can you have a look at the rerooting tests here to make sure I've implemented the right operation? Look for TestRerooting in the test_tree_ops.py module.

The operation that I've implemented is very simple, but it's quite possible I've gotten the wrong end of the stick here and implemented the wrong thing. Some eyes on the examples would be very helpful.

Similarly, for test_ternary_example_n6_leaf, node 3 is connected to only node 7, not node 9. So, I think the output tree should look like:
# 3.00┊       3   ┊
#     ┊       ┃   ┊
#     ┊       7   ┊
#     ┊    ┏━━┻━┓ ┊
# 2.00┊    9    ┃ ┊
#     ┊  ┏━┻━┓  ┃ ┊
# 1.00┊  6   8  ┃ ┊
#     ┊ ┏┻┓ ┏┻┓ ┃ ┊
# 0.00┊ 0 1 4 5 2 ┊
#     0           1

A trivial test, but could add this.
def test_example_same_root(self):
    #  1.00┊  2  ┊
    #      ┊ ┏┻┓ ┊
    #  0.00┊ 0 1 ┊
    #      0     1
    # ->
    #  1.00┊  2  ┊
    #      ┊ ┏┻┓ ┊
    #  0.00┊ 0 1 ┊
    #      0     1
    ts1 = tskit.Tree.generate_balanced(2, arity=2).tree_sequence
    ts2 = sc2ts.reroot(ts1, new_root=2)
    self.check_properties(before=ts1, after=ts2, root=2)
    tree = ts2.first()
    nt.assert_array_equal(tree.parent_array, ts1.parent_array)
    nt.assert_array_equal(ts2.nodes_time, ts1.nodes_time)

I don't understand how your algorithm works @jeromekelleher. Could you possibly explain it? Don'r we simply want to traverse through the tree-as-a-graph from a given node? I find thinking about this as an oriented tree pretty hard compared to a simple graph.

By the way, why aren't we keeping the sample node times as they were in the original TS. This should be possible with the sc2ts samples.

There's no original tree here in the actual inference problem, it's just a star tree describing the data and times get set later on.

Thanks for the input here, let me digest

Ah yes, you're right. Thanks, will fix!

The recursive version is quite easy to code up (and IMO easier to understand), if you need to have something to test against. I'm happy to code that up, if only for the test suite.

By the way, why aren't we keeping the sample node times as they were in the original TS. This should be possible with the sc2ts samples.

There's no original tree here in the actual inference problem, it's just a star tree describing the data and times get set later on.

You mean ultrametric, not star, I think? I assume we have internal topology. We do know the time of the root node (which is older than the other samples) though.

No it's a star.

Leave it with me, thanks.

No it's a star.

Oh, you mean from the matching process (i.e. it doesn't create internal nodes), not from e.g. the NJ tree inference?

Re tests, it's probably worth trying with a very deeply branched tree, e.g.
tree = tskit.Tree.generate_balanced(15)
# re-root on node 2, setting its time to e.g. 5. This requires reversing the parent-child relationship for 4 edges, I think.

Oh, you mean from the matching process (i.e. it doesn't create internal nodes), not from e.g. the NJ tree inference?

No, this is the input to the NJ inference. We just package up the data for a sample group as a tree sequence in a star topology, with all the mutations on the sample branches. It's just a straightforward way of keeping the datamodel straight.

OK, thanks very much for the input here @szhan and @hyanwong, that was super helpful. I'm pretty set on using oriented tree algorithms here as it's super simple once you get it straight. It's just reversing the parent child relationships on all nodes on the path to the (old) root.

OK, thanks very much for the input here @szhan and @hyanwong, that was super helpful. I'm pretty set on using oriented tree algorithms here as it's super simple once you get it straight. It's just reversing the parent child relationships on all nodes on the path to the (old) root.

Yep, that should be enough. Possibly worth also testing on trees with polytomies (although I don't think NJ normally gives those). E.g. reroot tskit.Tree.generate_balanced(15, arity=4) on node 14. I think the same logic should apply.
It should also be the same for trees with unary nodes above the root, right? It could be that some implementations of NJ produce these (i.e. placing a sample as the root node, rather than placing a root in the middle of an edge, which would become unary on re-rooting):

Ok, great, this seems to be working. Including rerooting in the inference is really easy with these tools and all the existing tests pass. I'm going to merge this and run with UPGMA for a bit and see how we get on. Swapping in NJ afterwards should be simple.

Yep, that should be enough. Possibly worth also testing on trees with polytomies (although I don't think NJ normally gives those). E.g. reroot tskit.Tree.generate_balanced(15, arity=4) on node 14. I think the same logic should apply.

Ternary example in their already

This is a valid unrooted NJ tree, right (dunno if any programs output this, though). Could re-root on e.g. node 2.
ts = tskit.Tree.generate_comb(4).tree_sequence.simplify([1, 2, 3, 6])
ts.draw_svg()


Foundation 5.3.3 - Reveal modal doesn't work with jquery 2.2.0

Hi there,
As mentioned here : http://foundation.zurb.com/forum/posts/37579-reveal-stopped-working
Reveal modal doesn't work anymore with the jquery update (2.2.0)
Firebug in firefox shows this message :
TypeError: g.on(...).trigger is not a function
it works if you downgrade to jquery 2.1.4

Confirmed. Same issue for me as well.

Merged #7859 which resolves this.

Why this issue is closed? The issue is not fixed!
There is no foundation 5.3.4 or something else, installing version 5.* via bower is still broken!

Agreed. Though, you can fix it by manually applying the patch as described in https://github.com/sjhewitt/foundation/commit/2e33179ad78b7d677610c50656f51f4b703034d5

@timaschew The fix is in the codebase, so we close the issue. We'll try to get a patch out for Foundation 5 soon.

This is NOT fixed in 5.5.3,
please release 5.5.4 legacy version that fixes that,
5.5.3 still needs downgrade of jQuery to 2.1.4


Component documentation with examples

I've added one main file in markdown for each group, some examples (code showcase and live example) for some of theme's components.
Some live examples doesn't work yet, for some obscure reason related to reduxAsyncConnect.
The file docs/create_example.txt is kind of a how to file about example.

And normally it should compile this time, I hope :-D

Coverage remained the same at 58.145% when pulling 94fdf688a95c044f449d371689042a33363a27e6 on VaysseB:component-documentation into 751a9940a5ac61c4cb175ead666c38a567d578be on plone:component-documentation.


failed: StreamRefsSpec

[09-01 15:20:20.134] [info] - must pass cancellation upstream across remoting before elements has been emitted *** FAILED *** (26 milliseconds)
[09-01 15:20:20.135] [info]   java.lang.AssertionError: assertion failed: expected Done, found Failure(org.apache.pekko.stream.RemoteStreamRefActorTerminatedException: Remote target receiver of data Some(Actor[pekko://StreamRefsSpec/system/Materializers/StreamSupervisor-2568/$$m-SourceRef-8#-1525367045]) terminated. Local stream terminating, message loss (on remote side) may have happened.)
[09-01 15:20:20.135] [info]   at scala.Predef$.assert(Predef.scala:279)

https://github.com/apache/incubator-pekko/actions/runs/6050882412/job/16421131496?pr=609

https://github.com/apache/incubator-pekko/actions/runs/6229433326/job/16907923622#step:6:21256

https://github.com/apache/incubator-pekko/actions/runs/6293248461/job/17083517089#step:6:22009

https://github.com/apache/incubator-pekko/actions/runs/6312975683/job/17140088676#step:7:13652


Support dry-run for create subnet

Closes ENG-780
This PR supports dry-run that, at the current stage, prints the calldata/params that sends to the blockchain. When the dry-run is set, a DryRunProvider is used instead to provide more behaviours.
While implementing dry-run, existing default parameters and hardcoded parameters are scattered, there is no one place that handles all the default parameters. This makes consistent implementation difficult. This PR also makes some code changes so that one can update or check the default parameters in one place.
When this PR is merged, other methods will be supported.

@aakoshh Initially I thought of another implementation of EvmManager, but then a lot of the methods do not really need a dry-run, such as a getter methods or like get_current_chain_head. Also their return response will be different, so at a provider level might be more appropriate.

Probably a good way is to create a DrynRun trait, then the connection() method returns the DryRun trait instance if dry-run is set.

@aakoshh I think maybe we can try implementing a DryRun version of SubnetManager, like you mentioned above in connection we can return the DryRun version based on the DryRun targeting network(evm/fvm).
The only thing we have to accept is the DryRun versions should return just defaults, as the SubetManager interfaces are set. Say the create_subnet method returns the address of the subnet created, then for the DryRun version, we just return address(0). For join_subnet method, it returns the epoch that the sender joins the subnet, then for DryRun version, we jut return epoch 0.
In this way, the interfaces are satisfied and the abstractions will not break.

@aakoshh I think your suggestion at the middleware level is very cool. But I need to do some research here because after the send_transaction is called, the caller is expecting some kind response. I'm not sure how that plays out with dry-run. We have to do some mocking of the response based on the txn sent in. I need to some checks first.

@aakoshh Interesting one! I think it should work, just dry-run will no longer work in non-cli environments.

just dry-run will no longer work in non-cli environments.

Yes, absolutely; I'm not sure that was a requirement though. My "dry run" in the materializer made every return value and input a generic parameter, so instead of returning e.g. a real subnet with fake values, it returned a "thing" that you had to pass back next time you wanted to use it. It's quite complicated. I'm clearly just suggesting a quick-and-dirty trick here, if that is compatible with the vision.

@raulk might need your input here, are we going to use dry-run outside of CLI?


Create workloads that takes into account of server/resource load

The time to finish the workload is impacted by the server/resources concurrently on the node/resource.

Co-scheduling overhead. Consider modeling that with containers:  https://simpy.readthedocs.io/en/latest/topical_guides/resources.html#containers


[ESP-WIFI-MESH] Mesh devices stop working when Root Node is in mesh sleep mode. (IDFGH-13271)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report
The problem seems to be with the Root Node's sleep mode. If I Set SLEEP MODE on ROOT Node, then this issue occurs, but if Remove SLEEP MODE on ROOT Node, then devices work properly. Can anyone please help me understand what might be happening?
Note:- Only added sleep mode in Root Node.
Old code
ESP_ERROR_CHECK(esp_mesh_disable_ps());
New code
ESP_ERROR_CHECK(esp_mesh_enable_ps());
/**  After Mesh esp_mesh_start() **/
ESP_ERROR_CHECK(esp_mesh_set_active_duty_cycle(50, MESH_PS_DEVICE_DUTY_REQUEST));
ESP_ERROR_CHECK(esp_mesh_set_network_duty_cycle(50, -1, MESH_PS_NETWORK_DUTY_APPLIED_ENTIRE));

@andy-danieal
Do you enable PS on root device only? Or both root and node devices?
what do you mean by saying stop working ? Can you provide a log to illustrate the problem ?

@zhangyanjiaoesp

Do you enable PS on root device only? Or both root and node devices?

Yes, Enable PS ONLY on Root device.

what do you mean by saying stop working ? Can you provide a log to illustrate the problem ?

Not connected to Mesh network.
We have attached child node logs with Enable PS mode and Disable PS Mode
Disable-PS-Child-Mode.log
Enable-PS-Child-Mode.log

@andy-danieal When you enable the mesh PS on the root device, you should also enable the mesh PS on the node device, otherwise the node device can't find the root.

@zhangyanjiaoesp mentioned in the comments that enabling mesh PS on the root device also requires enabling mesh PS on the child node device.
This is not mentioned in the ESP-WIFI-MESH documentation.

Could you please explain Why is this happening?

@andy-danieal When the node device receive the mesh AP's beacon, it will check the mesh PS state, only when both the mesh AP and myself support mesh PS, or neither supports mesh PS, a mesh network can be formed.
we will update the mesh documentation.


4783 poet client logging

Motivation
Part of  #4751
Closes #4783
Changes

use local logger in poet clients
provide the logger to retryablehttp to gain visibility on performed requests, retries, responses etc.
cleanup poet test harness

Test Plan
Existing tests pass

Failed TestCheckpoint which seems unrelated.
Bors merge


Support archiving/printing entire websites to Arweave

This PR adds a virtual printer to the existing printers' list in Chrome. Once selected, the printer archives the PDF version of the website/content on Arweave using Turbo.
Example website uploaded to Arweave:
https://arweave.net/1WeS6vpKYid0jizwsGBCjWUEcTVEVXoj6xNd4UpZIEw
Preview:

This idea is insane. Love it.

Merging to arconnectio:feat/print-to-arweave to push some new changes.


Call getName method in StandardReflector

It was only a remedy form my PHP 7 fork's test on MacOS.
https://www.php.net/manual/en/class.reflectiontype.php#125396
:(


Stop using to: and use bcc:

Revival of https://github.com/defund12/defund12.org/pull/832
Closes https://github.com/defund12/defund12.org/issues/640

@dionlarson @bingbongle made some updates here to support bcc only - re-requesting review - thanks!


Time trend BoxCox transform and weight trending

Resolves #499
Resolves #501
Resolves #477
The main changes here are:

Implement BoxCox transformation on time trend factors
Implement recursive get_expected in KevinHallModel, to compute expected weight from nutrients
Remove default arguments to RiskFactorAdjustableModel methods to pre-empt bugs
Added a static gender to value method to person.cpp

Happy to walk through over a chat.

It looks good to me.

I've made a few small comments... I'd be interested to have a walkthrough though if you've got time!

Sure! Today is good.

Sure! Today is good.

Sorry, only just saw this... Could do tomorrow?


Ember stream forking

Follow up to #4179. I introduced a prefetch call which is a hack to get the outer stream to terminate and finalize before inner streams do, by buffering streams to a new stream. parJoin also shuffles the results of the inner stream back out to the new stream which isn't strictly necessary.
This PR introduces a forking combinator which behaves identically to parJoin except the outer stream can terminate and finalize before the inner stream, and there is no queueing involved.
TODO:

[x] Concurrency control
[ ] Possibly upstream this back into fs2, since it's useful for network servers in general

@rossabaker if you encounter any issues merging downstream, definitely ping me to help, but I don't anticipate there being any trouble

@rossabaker if you encounter any issues merging downstream, definitely ping me to help, but I don't anticipate there being any trouble

We can contribute this to fs2?

it's extremely dangerous in the general case :)

We can contribute this to fs2?

it's extremely dangerous in the general case :)


add seu

add southeast university

@WENWU-NXP I am sorry, I have to close this request without merging, because you requested to add a non-existent domain to this repository:
$ host seu.seu
Host seu.seu not found: 3(NXDOMAIN)

Please ensure you are familiar with this repository rules located at the bottom of this page and submit a new request with correct information. Thank you for the understanding!


MDL - Green color Custom CSS theme

Hello,
I'm trying to use color green generate by "Custom CSS theme", but the result is not same that is in pallette.

There are way more than 2 colors in each theme; all of them are derived from the two main colors, with added/reduced brightness/saturation. The FAB in the preview is not using the exact accent color you are picking. But components using the exact accent color, buttons with mdl-button--accent for example, should match the accent color you picked, I believe.

We are currently working on a complete rewrite for V2. The team won't be taking any action here for V1. If someone wants to track this down and submit a patch to mdl-1.x then it is open for PRs.
Thank you.


How to get states by CC/EOM?

I want to know how to get result state obtained by CC and EOM method.
I know PySCF will return the coefficients of excitation operators, by solving the CC/EOM equation, but I don't know exact the form of these excitation operators.
I will appreciate if there are some examples.
Thank you.

The EOM calculation will return a vector v and you can do r1, r2 = eom.vector_to_amplitudes(v). For EE (neutral excitations), the elements are indexed as r1[i,a] and r2[i,j,a,b] where i,j are occupied orbitals and a,b are unoccupied orbitals, with the usual ordering. For IP they are r1[i] and r2[i,j,a] and so on.
I think that these are safe to use for qualitative analysis about the character of different transitions, but I'm not sure how carefully they've been tested for their quantitative agreement with other codes (perhaps due to spin symmetry or normalization conventions, not sure). See for example issue #1136.

So the index of r1 r2 are for MOs, but how about the spin?
Is it means, I do excitation for HF state according to the r1 r2 amplitude, I will get right eigenvector of similarity transformed Hamiltonian?
And how about the case of Coupled-Cluster? As I performed mycc = pyscf.cc.GCCSD(mf).run(), is it the same for 'mycc.t1 mycc.t2'?
And I perform the exponential of combination of these excitations to the HF state will give CC wavefunction?

I came here from #1136 as it seems related. It would be good to get a proper analyze for EOM methods. The ADC module has the sort of thing that would be good.
Anyway for the meantime this is what I understand about the analysis of the eigenvectors using say $H_2 O$ in STO-3G with a full spin treatment. Looking at, for example, IP we would have a eigenvector (v) of length (nocc) + (nocc, nocc, nvir) which would be of length 10+100.4=410. To get the quasi-particle weights you would compute np.linalg.norm[v[:10])**2. The norms as printed by ADC module are then np.einsum('p,p->', v[:10], v[:10]) and np.einsum('p,p->', v[10:], v[10:]) with the small space norm being the same as the quasi-particle weight.
To get the dominant excitations is more complex. For the small space you can simply do
    #dominant excitation in h-block
    h = v[:nocc]**2
    h = np.sqrt(h[::2] + h[1::2])               #just showing general case
    n = 1                                       #number of dominant excitations to consider
    idh = np.argsort(h)[::-1][n-1]
    x = h[idh]
    ob = 1                                      #orbital base ie count from 1
    print('Dominant excitation for h-space is {:>6.4f} from occupied orbital {:2} '.format(x, idh+ob))

and for the large space something like
    #dominant excitation in  2h-1p block
    hhp = (v[nocc:]**2).reshape(nocc, nocc, nvir)
    hhp_s = np.sqrt(hhp[::2,::2,::2].ravel() + hhp[1::2,1::2,1::2].ravel())    #same spin excitation
    hhp_d = np.sqrt(hhp[::2,1::2,::2].ravel() + hhp[1::2,::2,1::2].ravel())    #different spin excitation

    #alpha->alpha or beta->beta
    idh_s = np.argsort(hhp_s)[::-1][n-1]
    x = hhp_s[idh_s]

    spatial = (nocc//2, nocc//2, nvir//2)        #numbering corrections for spatial orbital
    ix = np.unravel_index(idh_s, tuple(spatial))
    print('Same spin dominant excitation for 2h-1p-space is {:>6.4f} from occupied orbitals {:2} {:2} to {:2} '.
           format(x, ix[0]+1, ix[1]+1, ix[2]+1+nocc//2))

    #alpha->beta, beta->alpha
    idh_d = np.argsort(hhp_d)[::-1][n-1]
    x = hhp_d[idh_d]

    ix = np.unravel_index(idh_d, tuple(spatial))
    print('different spin dominant excitation for 2h-1p-space is {:>6.4f} from occupied orbitals {:2} {:2} to {:2} '.
           format(x, ix[0]+1, ix[1]+1, ix[2]+1+nocc//2))

Something like this should give you the dominant excitation for IP. For the first root it gives
For root 0.287506 qpwt=0.8606  norm [h] 0.8606 [2h-1p] 0.1394
Dominant excitation for h-space is 0.9277 from occupied orbital  5
Same spin dominant excitation for 2h-1p-space is 0.1065 from occupied orbitals  5  3 to  7
different spin dominant excitation for 2h-1p-space is 0.1475 from occupied orbitals  3  5 to  7

which qualitatively agrees with ADC(2) and is quantitatively similar.
The problem is PySCF works with a reduced large space. For example the (nocc, nocc, nvir) part of the small space is anti-symmetric (zero diagonal) so only the lower diagonal $\frac{nocc(nocc-1)}{2}$ elements are used giving a total eigenvector length of 10 + 45.4=190 - this eigenvector is normalized. As already pointed out vectors_to_amplitudes_ip(v, nmo, nocc) reconstructs amplitudes with full dimensions ie (nocc) and (nocc, nocc, nvir) and you could reconstruct a full eigenvectors as np.concatenate((r1.ravel(), r2.ravel())) - this is now a full 410 length eigenvector which is not normalized. I think before you use these eigenvectors derived from amplitudes you must re-normalize. As a quick test I calculated the small space norm for EE by a dense matrix solver with full dimensions and got a value of 0.8066, using PySCF vectorized amplitudes without normalization I get  0.9435 and with normalization 0.8066. Maybe someone who knows more about the internal workings of PySCF can confirm or correct how I think one could calculate the character of the transitions.

@pwborthwick I think the most obvious issue with analysing the character of an EOM-CC eigenvector is that these eigenvectors are not actually proper Dyson orbitals, as \bar{H} is not a proper self-energy. The Dyson amplitudes in the true CC Green's function are something like \langle (1+\Lambda) \bar{a}^{\dagger} \bar{a} \rangle, so to properly account for the character of the CC eigenvectors you'd need to take the left- and right-hand eigenvectors and then scale by the lambda amplitudes - simply put, \bar{H} in EOM-CC doesn't depend on lambda but the Green's function should. In ADC, however, the eigenvectors directly give you the Dyson orbitals.

@pwborthwick , for a closed-shell molecule, the EOM-CCSD calculation will use RHF orbitals. What this means is that the hhp sector of the eigenvector corresponds to a low-spin (alpha-beta-alpha) excitation. In your example, you are obtaining the same spin (e.g., alpha-alpha-alpha) contribution to the eigenvector (hhp_s) by slicing hhp, which is incorrect in my opinion. The same spin contribution can be obtained as antisymmetric combination of low-spin amplitudes (see Eq. 36 in [J. Chem. Phys. 102, 3629–3647 (1995)] for EA-EOM-CCSD] for a closed-shell reference.
What you need is already implemented in the ADC code (https://github.com/pyscf/pyscf/blob/8b3fef8cf18f10d430261d4a8bea21fadf19bb1f/pyscf/adc/radc_ip.py#L603). The only thing one needs to make sure is that the EOM-CCSD eigenvector elements should be stored the same way as in IP-ADC (the doubles part is (nvir,nocc,nocc)). Other than that you should be able to use this code for IP-EOM-CCSD eigenvector analysis with RHF reference. Note that this code analyzes the eigenvector in terms of spatial orbitals, i.e. for the doubles sector only the low-spin (alpha-beta-alpha) excitations are analyzed. If you would like to get access to same spin (e.g., alpha-alpha-alpha), you can use the code from the UADC module (https://github.com/pyscf/pyscf/blob/8b3fef8cf18f10d430261d4a8bea21fadf19bb1f/pyscf/adc/uadc_ip.py#L1261). Again, one will need to make sure that the EOM-CCSD eigenvector elements are stored in the same order as in UADC. Note that, due to non-Hermitian nature of EOM-CCSD, the right and left eigenvector analyses will give different results, although the differences are not expected to be huge.

@obackhouse might not a (naive) analysis based on just the right eigenvectors be of some use for transitions where the single excitations dominate strongly? The results I get for EOM-CCSD-IP mirror the ADC(2)-IP results and certainly any qualitative inferences made with either method would concur (admittedly with a minimal test-set!). The correct way to proceed is as you point out by incorporating the $\Lambda$ amplitudes as explained in Stanton & Bartlett and section 2.7.1 of your thesis. PySCF has left-hand eigenvectors so I suppose the only way to see the difference with a proper treatment is to do it. Might then also the slight difference in quasi-particle weights found in #1136 be because Molpro calculates them from eigenvectors constructed as you suggest rather than PySCF right-hand only way?

I agree that the EOM-CCSD eigenvectors should be returned in as useful a
form as possible, or at least an analyze() method should be written that is
consistent with other codes and standard conventions. To be honest
(speaking as one of its authors), the EOM-CC code is a bit of a mess and
it's been a long-time plan to refactor, but I can't guess when that will
actually happen ...
On Tue, Feb 21, 2023 at 9:07 AM Peter Borthwick @.***>
wrote:

@obackhouse https://github.com/obackhouse might not a (naive) analysis
based on just the right eigenvectors be of some use for transitions where
the single excitations dominate strongly? The results I get for EOM-CCSD-IP
mirror the ADC(2)-IP results and certainly any qualitative inferences made
with either method would concur (admittedly with a minimal test-set!). The
correct way to proceed is as you point out by incorporating the $\Lambda$
amplitudes as explained in Stanton & Bartlett
https://www.theochem.ru.nl/files/local/jcp-98-7029-1993.pdf and section
2.7.1 of your thesis. PySCF has left-hand eigenvectors so I suppose the
only way to see the difference with a proper treatment is to do it. Might
then also the slight difference in quasi-particle weights found in #1136
https://github.com/pyscf/pyscf/issues/1136 be because Molpro calculates
them from eigenvectors constructed as you suggest rather than PySCF
right-hand only way?
—
Reply to this email directly, view it on GitHub
https://github.com/pyscf/pyscf/issues/1427#issuecomment-1438556342, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADGAJCYLYMSIFFTB4QI535DWYTD3XANCNFSM6AAAAAAQNF6LSI
.
You are receiving this because you commented.Message ID:
@.***>

@obackhouse might not a (naive) analysis based on just the right eigenvectors be of some use for transitions where the single excitations dominate strongly? The results I get for EOM-CCSD-IP mirror the ADC(2)-IP results and certainly any qualitative inferences made with either method would concur (admittedly with a minimal test-set!). The correct way to proceed is as you point out by incorporating the Λ amplitudes as explained in Stanton & Bartlett and section 2.7.1 of your thesis. PySCF has left-hand eigenvectors so I suppose the only way to see the difference with a proper treatment is to do it. Might then also the slight difference in quasi-particle weights found in #1136 be because ORCA calculates them from eigenvectors constructed as you suggest rather than PySCF right-hand only way?

When you talk about incorporating the lambda amplitudes to get the actual CC Dyson amplitudes or quasiparticle weights, that's pretty much what the area of coupled cluster Green's functions does.
I would be surprised if Molpro is calculating the lambda amplitudes by default for EOM-CC, because the lambda equations (I think more than) double the prefactor of the CC calculation. Have you compared against the projection of the left-hand vectors too, and maybe even |LR*| (though I have no idea if that's meaningful)? You can get these pretty easily from a dense form of the Hamiltonian as well as the PySCF left keyword, see script below.
import numpy as np
import scipy.linalg
from pyscf import gto, scf, cc, lib

mol = gto.Mole()
mol.atom = "O 0 0 0; O 0 0 1"
mol.basis = "sto3g"
mol.verbose = 0
mol.build()

mf = scf.RHF(mol)
mf.kernel()

ccsd = cc.CCSD(mf)
ccsd.kernel()

eom = ccsd.eomip_method()
matvec, diag = eom.gen_matvec()
hbar = np.column_stack(matvec(np.eye(diag.size)))

w, vl, vr = scipy.linalg.eig(hbar, left=True, right=True)
mask = np.argsort(w.real)
w, vl, vr = w[mask], vl[:, mask], vr[:, mask]

print("dense:")
print("left: ", np.linalg.norm(vl[:np.sum(mf.mo_occ > 0), 0])**2)
print("right:", np.linalg.norm(vr[:np.sum(mf.mo_occ > 0), 0])**2)

print("davidson:")
w, v = eom.kernel(nroots=5, left=True)
print("left: ", np.linalg.norm(v[0][:np.sum(mf.mo_occ > 0)])**2)

w, v = eom.kernel(nroots=5, left=False)
print("right:", np.linalg.norm(v[0][:np.sum(mf.mo_occ > 0)])**2)

@obackhouse Thanks Ollie. I don't use ORCA (not Molpro as I originally thought) so am only going by what was written in #1136. But I suspect ORCA just looks at the singles space on the basis of right-hand vectors and the small differences with PySCF are just differences in methodology here and there. I have my own code for CCSD, $\Lambda$-CCSD and EOM-CCSD so will play around with those (and your script).
@ssh2 By 'full spin' in my post I mean't to use GCCSD so I think the same-spin aaa and bbb are right. Sorry for not being clear.
@tberkel Yes I think an analyze() function would be really useful and sort out some of the issues brought up here in the process. Thanks.


Random insignificant health bars occasionally pop up for a split second on the screen sometimes

Update: It happens (on average) about a second after structures and enemies die, and at the same location where they died.


Research: Basic features of the wallet

https://www.figma.com/design/cvV48t0f7O2znT6QBxK0Zj/Tonomy-ID?node-id=8906-5687&t=1VANH5JpUfLpM6YY-1
All the features that conisdered to be basic and should be added have been marked with green


README example doesn't work

The following example code is taken from the determinant section of README.md:
julia> using LinearAlgebraX
julia> A = rand(Mod{10},5,5)
ERROR: UndefVarError: Mod not defined

Sorry 'bout that. Need using Mods as well to make this work.