[BUG] Volumens can't be attached anymore

Describe the bug
Longhorn was running fine the last weeks. Since two days all the volumes can't be attached anymore. The volumes are attaching and deattaching in a loop. -> Rancher can't use the volumes anymore for the pods.
To Reproduce
Steps to reproduce the behavior:

No direct steps ... unfortunately.
Using Rancher (v2.6.3) with k3s with Longhorn (v1.2.3)

Expected behavior
Volumens can be attached to the Rancher pods.
Log or Support bundle
longhorn-support-bundle_598b3b59-09b8-4494-b9d1-1d3d4f353462_2022-03-29T08-58-49Z.zip
Environment

Longhorn version: v1.2.3
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Rancher
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: K3s

Number of management node in the cluster: 1
Number of worker node in the cluster: 1

Node config

OS type and version: Ubuntu 20.04 LTS
CPU per node: 16
Memory per node: 64GB
Disk type(e.g. SSD/NVMe): SSD
Network bandwidth between the nodes:

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): KVM
Number of Longhorn volumes in the cluster: 23

Additional context
I'm not sure what happened to be honest. Everything was working perfectly. Now the volumes are not attaching.

Can you provide the volume name which is stuck in detaching/attaching?

@PhanLe1010 yes, all volumes are affected. I though it easier to find the issue if not so much deployments are running. Therefore I scaled down the deployments in Rancher which are using 4 volumes. Now only these 4 volumes are in the loop of attaching and de-attaching. The rest is idle because the deployments are scaled to 0.

Okay, this cluster only has 1 node. But the instance manager r is in not ready state:
  status:
    phase: Failed
    conditions:
    - type: Initialized
      status: "True"
      lastprobetime: "0001-01-01T00:00:00Z"
      lasttransitiontime: "2022-03-13T13:05:49Z"
      reason: ""
      message: ""
    - type: Ready
      status: "False"
      lastprobetime: "0001-01-01T00:00:00Z"
      lasttransitiontime: "2022-03-28T04:32:12Z"
      reason: ContainersNotReady
      message: 'containers with unready status: [replica-manager]'
    - type: ContainersReady
      status: "False"
      lastprobetime: "0001-01-01T00:00:00Z"
      lasttransitiontime: "2022-03-28T04:32:12Z"
      reason: ContainersNotReady
      message: 'containers with unready status: [replica-manager]'

It means Kubelet might have killed the instance manager r because of resources pressure. I can see the ready condition was fliped around 2022-03-29T08:25:06:
      Ready:
        type: Ready
        status: "True"
        lastprobetime: ""
        lasttransitiontime: "2022-03-29T08:25:06Z"
        reason: ""
        message: Node v2202112162120172526 is ready

Actions:

Get the instance-manager-r-xxx pod to running state (by cleanup some resources?)
Since you only have 1 node, set the csi deployment scale to 1: https://github.com/longhorn/longhorn/blob/bd3655b122e72bad0653783d8edfdada9a50cdab/chart/values.yaml#L74-L77
Set the volume replica count to 1: https://longhorn.io/docs/0.8.0/users-guide/volume/#changing-replica-count-of-the-volumes

@PhanLe1010 thank you for the quick help. Do you have any idea how I can get the instance-manager-r running? Resources should not be the issue. With kubectl get pods -n longhorn-system I see that the instance-manager-r is in terminating state.
NAME READY STATUS RESTARTS AGE
instance-manager-r-3fad300b 0/1 Terminating 0 16d
I tried to delete it with kubectl delete pod instance-manager-r-3fad300b -n longhorn-system but after the deletion confirmation I get a time-out and nothing happens.

Unfortunately the logs of the pod are not so helpful with kubectl logs instance-manager-r-3fad300b -n longhorn-system

Error from server (BadRequest): container "replica-manager" in pod "instance-manager-r-3fad300b" is terminated

Is there any finalizer on the instance manager pod? Can you do kubectl get pod instance-manager-r-3fad300b -n longhorn-system -o yaml and paste the output here?
How about force delete the pod?

You can also check the kubelet logs for hints about why pods cannot be deleted. Instruction about getting the logs: https://rancher.com/docs/k3s/latest/en/faq/
Also, You might want to upgrade Longhorn v1.2.4. We have made improvement here regarding to how we manage the life cycle of the instance manager pods


[SPARK-10735] [SQL] Generate aggregation w/o grouping keys [WIP]

PR is based on #10735
The benchmark show that generated aggregation could be 6X time faster than non-generated (with generated filter/range).
It also showed that declarative function is much faster than imperative one (5X faster for stddev), we may need to switch to a declarative version. (the difference need to be measured when there are grouping keys).

Test build #49526 has started for PR 10786 at commit 640a4b5.

Test build #49526 has finished for PR 10786 at commit 640a4b5.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class StddevAgg(child: Expression) extends DeclarativeAggregate
case class StddevPop1(child: Expression) extends StddevAgg(child)
case class StddevSamp1(child: Expression) extends StddevAgg(child)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49526/
Test FAILed.

Test build #49537 has started for PR 10786 at commit 6911a05.

Test build #49537 has finished for PR 10786 at commit 6911a05.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49537/
Test FAILed.

Test build #49540 has started for PR 10786 at commit 10f6bb9.

Test build #49540 has finished for PR 10786 at commit 10f6bb9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49540/
Test FAILed.

Merged build finished. Test FAILed.

Test build #49546 has started for PR 10786 at commit 4c655dd.

Test build #49546 has finished for PR 10786 at commit 4c655dd.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49546/
Test FAILed.

Merged build finished. Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49547/
Test FAILed.

Test build #49548 has started for PR 10786 at commit c8ed5d9.

Test build #49548 has finished for PR 10786 at commit c8ed5d9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49548/
Test FAILed.

Merged build finished. Test FAILed.

This PR will be replaced by multiple small PRs.


Add copy of BootCamp certificate

Make a certifications component that can display multiple certifications
Add image object
Add descriptions?
Completion dates
zoom in when clicked (modal)

Didn't use modal. Added cert image to each Certification card with bootstrap card-img-bottom. Each certification image is showing. No descriptions added. Completion dates are listed.


Add support for AWS ec2 r5a and r5n instance types for Rancher 2.x

What kind of request is this (question/bug/enhancement/feature request):
Feature Request:
Please add support for AWS r5a and r5n instance types (same performance as R5 but with 10% cost savings (for r5a) and with better network throughput (for r5n)).
Other details that may be helpful:
This is the same request as this but for r5a and r5n instance types
gz#15440

Please consider this as well https://github.com/rancher/rancher/issues/31475

Is there any reason as to why these instance types are not in the instance selection for Rancher?


How to implement a callback after execution of a widget

Hi,
I'm using a rangeSlider to filter the hits and I would like to display the interval price chosen by the user

To achieved that, I think I need a callback on the currentRefinedValues widget. Do you know How I could do that and if it is my best option?

Hi @beHappyBeLindy thanks for contributing to the project with this issue. To customise the way the values are displayed you should use the CurrrentRefinenements connector: connectCurrentRefinements
It is an higher order component that let you completely customize the view of this widget based on your understanding of your search interface.

Thanks a lot @bobylito for this quick answer. But unfortunately, I'm not using react in my project.
Is there another existing solution with basic Js?

@Jerskouille I believe you solved a similar problem inside shopify? I do not remember the right way to do this (merging numeric refinements in this widget)

Oh I didn't with Shopify, I just actually indexed price ranges as strings.
But I'm pretty sure this can be failry easily done with a custom widget that would call getNumericRefinements('price').

Thanks @Jerskouille, Helper is really powerfull, it helped a lot !

Happy to have helped you! @bobylito is the one who did an awesome job with the helper.
I'll close the issue, feel free to reopen it any time. :)


Rework styles of the homapage audio/image media type switcher

The homepage media switcher implemented in #271 should be updated to the following design from the transitional styles figma file:

Closing, this design will no longer be used and a new one has been proposed in the redesigned homepage.


Firestore adapter lifecycle lint checks

This PR adds the following Lint checks:

Missing call on startListening() when using a FirestoreRecyclerAdapter.

Missing call on setLifecycleOwner() in case FirestoreRecyclerOptions are being used.

Relevant issue: https://github.com/firebase/FirebaseUI-Android/issues/1339

@pavlospt thanks for making the changes!  This looks good for merging into internal and then we'll have to schedule the work of making a public lint module.

Glad to see so :D Thanks @samtstern !


Fixes available? 1.10.2

This is not really an issue but more of curiosity. I am glad to see that fixes and patches are being done while Aiden is absent but how are these being distributed? Is there a new download repository?

Not yet, working on setting up a CI system to build off the 1.10 branch to my download server.

excellent thanks for quick response.


How to compile rust-rdkafka with cargo lambda?

I want to produce some Kafka messages from a Rust Lambda function.  rust-rdkafka seems to be the recommended client library. But I have some trouble getting rust-rdkafka to work with cargo lambda.
My Cargo.toml contains the following dependency config:
rdkafka = { version = "0.36", default-features = false, features = ["tokio", "ssl-vendored", "sasl", "gssapi-vendored"] }

I try to run the following command: cargo lambda build --release --arm64 --output-format zip but encounter a cross-compile issue, see attached output. Do someone know what must be done in order to get this working, and could it be added to the Known cross compilation issues section of the docs?
 -Wl,-rpath,
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /opt/homebrew/bin/ggrep
checking for egrep... /opt/homebrew/bin/ggrep -E
checking for fgrep... /opt/homebrew/bin/ggrep -F
checking for ld used by aarch64-unknown-linux-gnu-gcc... /opt/homebrew/Cellar/aarch64-unknown-linux-gnu/13.2.0/toolchain/aarch64-unknown-linux-gnu/bin/ld
checking if the linker (/opt/homebrew/Cellar/aarch64-unknown-linux-gnu/13.2.0/toolchain/aarch64-unknown-linux-gnu/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /opt/homebrew/bin/aarch64-unknown-linux-gnu-nm -B
checking the name lister (/opt/homebrew/bin/aarch64-unknown-linux-gnu-nm -B) interface... BSD nm
checking the maximum length of command line arguments... 786432
checking how to convert aarch64-apple-darwin23.5.0 file names to aarch64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert aarch64-apple-darwin23.5.0 file names to toolchain format... func_convert_file_noop
checking for /opt/homebrew/Cellar/aarch64-unknown-linux-gnu/13.2.0/toolchain/aarch64-unknown-linux-gnu/bin/ld option to reload object files... -r
checking for aarch64-unknown-linux-gnu-file... no
checking for file... file
checking for aarch64-unknown-linux-gnu-objdump... aarch64-unknown-linux-gnu-objdump
checking how to recognize dependent libraries... pass_all
checking for aarch64-unknown-linux-gnu-dlltool... no
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for aarch64-unknown-linux-gnu-ar... aarch64-unknown-linux-gnu-ar
checking for archiver @FILE support... @
checking for aarch64-unknown-linux-gnu-strip... (cached) aarch64-unknown-linux-gnu-strip
checking for aarch64-unknown-linux-gnu-ranlib... aarch64-unknown-linux-gnu-ranlib
checking command to parse /opt/homebrew/bin/aarch64-unknown-linux-gnu-nm -B output from aarch64-unknown-linux-gnu-gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for aarch64-unknown-linux-gnu-mt... no
checking for mt... no
checking if : is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if aarch64-unknown-linux-gnu-gcc supports -fno-rtti -fno-exceptions... no
checking for aarch64-unknown-linux-gnu-gcc option to produce PIC... -fPIC -DPIC
checking if aarch64-unknown-linux-gnu-gcc PIC flag -fPIC -DPIC works... yes
checking if aarch64-unknown-linux-gnu-gcc static flag -static works... yes
checking if aarch64-unknown-linux-gnu-gcc supports -c -o file.o... yes
checking if aarch64-unknown-linux-gnu-gcc supports -c -o file.o... (cached) yes
checking whether the aarch64-unknown-linux-gnu-gcc linker (/opt/homebrew/Cellar/aarch64-unknown-linux-gnu/13.2.0/toolchain/aarch64-unknown-linux-gnu/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for sys/socket.h... yes
checking for ws2tcpip.h... no
checking for socket... yes
checking for connect... yes
checking for res_search... no
checking for dn_expand... no
checking for dns_lookup... no
checking DB path to use... /etc/sasldb2
checking DB library to use... none
checking if DB handle is kept open in SASLDB... no
checking for dlopen in -ldl... yes
checking PRNG to use... /dev/urandom
checking for nm... /opt/homebrew/bin/aarch64-unknown-linux-gnu-nm -B
checking for underscore before symbols... no
checking for syslog... yes
checking if I should include saslauthd... /var/state/saslauthd
checking to include Courier authdaemond support... /dev/null
checking if I should include pwcheck... no
checking if I should include the alwaystrue verifier... no
checking if we should enable sasl_checkapop... disabled
checking CRAM-MD5... disabled
checking size of long... 8
checking what directory libraries are found in... lib
checking for RSAPublicEncrypt in -lrsaref... no
checking for openssl/evp.h... no
checking for OpenSSL... no
checking for des_cbc_encrypt in -ldes... no
checking for des_cbc_encrypt in -ldes425... no
checking for des_cbc_encrypt in -ldes524... no
checking for RSAPublicEncrypt in -lrsaref... (cached) no
checking for des_cbc_encrypt in -lcrypto... no
checking DIGEST-MD5... disabled
checking SCRAM... disabled
checking OTP... disabled
checking SRP... disabled
checking KERBEROS_V4... disabled
checking for crypt... no
checking for crypt in -lcrypt... yes
checking for aarch64-unknown-linux-gnu-gcc options needed to detect all undeclared functions... none needed
checking for gssapi.h... yes
checking for gssapi/gssapi_ext.h... yes
checking for res_search in -lresolv... no
checking for gss_unwrap in -lgssapi... no
checking for krb5int_getspecific in -lkrb5support... yes
checking for gss_unwrap in -lgssapi_krb5... yes
checking GSSAPI... with implementation mit
checking for res_search in -lresolv... (cached) no
checking for gsskrb5_register_acceptor_identity... no
checking for gssapi/gssapi_krb5.h... yes
checking whether gsskrb5_register_acceptor_identity is declared... yes
checking for gss_decapsulate_token... yes
checking for gss_encapsulate_token... yes
checking for gss_oid_equal... yes
checking for gss_get_name_attribute... no
checking for gss_inquire_sec_context_by_oid... no
checking whether gss_inquire_sec_context_by_oid is declared... yes
checking for SPNEGO support in GSSAPI libraries...
--- stderr
configure: WARNING: using cross tools not prefixed with host triplet
configure: WARNING: Disabling SASL authentication database support
configure: WARNING: OpenSSL not found -- SCRAM will be disabled
configure: WARNING: OpenSSL not found -- OTP will be disabled
configure: WARNING: OpenSSL not found -- SRP will be disabled
configure: error: in /<private-path>/target/aarch64-unknown-linux-gnu/release/build/sasl2-sys-63d453177ca6463f/out/sasl2':   configure: error: cannot run test program while cross compiling   See config.log' for more details
thread 'main' panicked at //.cargo/registry/src/index.crates.io-6f17d22bba15001f/sasl2-sys-0.1.22+2.1.28/build.rs:127:10:
configure failed: Custom { kind: Other, error: "command ["//target/aarch64-unknown-linux-gnu/release/build/sasl2-sys-63d453177ca6463f/out/sasl2/configure", "--prefix=//target/aarch64-unknown-linux-gnu/release/build/sasl2-sys-63d453177ca6463f/out/install", "--enable-static", "--disable-shared", "--disable-sample", "--disable-checkapop", "--disable-cram", "--disable-scram", "--disable-digest", "--disable-otp", "--enable-gssapi=//target/aarch64-unknown-linux-gnu/release/build/krb5-src-88d76e4bc128865b/out/install", "--disable-plain", "--disable-scram", "--disable-anon", "--with-dblib=none", "--with-pic", "CPPFLAGS=", "CFLAGS= -fPIC", "--host=aarch64-unknown-linux-gnu"] exited with code 1" }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

I got it to work by changing my dependency to this:
rdkafka = { version = "0.36", features = ["cmake-build", "ssl-vendored"] }


update_message: Use rendering_only flag for notifications logic.

When update_message events were updated to have a consistent format for both normal message updates/edits and special
rendering preview updates (#20803), the logic used in the tornado event queue processor to identify the special events for sending notifications no longer applied.
Updates that logic to use the rendering_only flag that was added to the update_message event format to identify if the event processor should potentially send notifications to users.
Fixes #16022.
Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[x] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

I've added a TODO/compatibility block to where rendering_only_update is set.
First, it checks for the new event format that contains rendering_only. If that key is not in the event_template, then it sets the value for rendering_only_update based on whether the user_id key is absent from the event_template. Previously, events for rendering preview updates omitted this key/value, unlike normal message update/edit events.

Merged, after a bit of a tweak to the TODO/compatibility comment. Thanks for clearing this out @laurynmm!


Domain Expiry Monitor

⚠ Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find similar feature request

🏷 Feature Request Type
New Monitor
🔖 Feature description
It would be awesome if we can add domains, and monitor their expiry status.
✔ Solution
We should be notified when there is a change to the DNS or expiry date, and be notified x days before expiry.
❓ Alternatives
No response
📝 Additional Context
11Uptime has a similar feature, as well as Better Uptime

That would be awesome!

I think it is hard to retrieve domain information from tld companies/organization. For example there is no public api for hungarian domains.
If you want to detect dns (host->ip) resolve changes that is other issue.

it is hard to retrieve domain information from tld companies/organization

It is hard and not reliable without a standardized API, but for many domains WHOIS scraping would go a long way. Still not too fun to implement...

Seems like there are quite a few APIs, some with big limits, and some completely free:
https://blog.api.rakuten.net/top-domain-whois-api/

This would be awesome +1 🎉

Just found a library. I think it is quite promising, because it directly queries the whois 43 port server.
As long as Uptime Kuma caches the expiration date, it should not hit the rate limit.
https://github.com/eithan1231/whois-light

+1 for this. I was looking for something self-hosted for this when I came across domain-monitor.io, and that looked suspiciously like Uptime Kuma so I came over here to search. This and SSL expiry notifications would be super cool.

Just found a library. I think it is quite promising, because it directly queries the whois 43 port server. As long as Uptime Kuma caches the expiration date, it should not hit the rate limit. https://github.com/eithan1231/whois-light

According to open issues, it seems that the library is not completely consistent for all TLDs.
This could have a negative effect on the implementation into Uptime Kuma, as it would not be possible to have one code for all TLDs.
If you are interested, I could try to look at it and try to make a universal WHOIS library for most domains. But I can't promise anything.
Would that help?

@louislam As I wrote in the comment above, I started working on a library to retrieve domain data from WHOIS servers.
I'm posting a link to my GitHub repository.

You're overcomplicating it, just let users enter an expiration date for the domain name when adding a new monitor. And then send notifications when the specified date approaches.

+1 for this. I'm commenting in support because I can't tell where things landed with the request, though I did see the Github repo and closed, duplicate request.
Regarding the comment before mine, I have to say it completely misses the point: the reason for the feature request is to account for absentmindedness/human error and if we all took the time to track when domains were due for renewal then there would be no need for such a thing... but people are going to people. I love the idea of Uptime Kuma offering an additional check, and I already have every domain set to autorenew. Sometimes things happen and people forget to update their credit card's expiration date for instance.

@anseltaft In fact, a big problem is the inconsistent WHOIS servers of different companies. As I wrote above, I started working on the Universal WHOIS library, but basically most registrars had different results when requesting information about the domain. Personally, I think that we are currently unable to provide full support for most domain extensions, and therefore this requirement is in a frozen state.
I'm not sure if I once posted recent changes to GH, but if you look at my repository, you'll probably understand what I'm talking about.

Go with the manual approach first, then work on the automation later.

@alexandernst In fact, I thought about it too. I planned to eventually use it in my library, and it could then be used as the only dependency. But I haven't gotten to it yet... maybe on the weekend.
However, it is true that at the moment this is probably the only solution.

Would love to have this feature, even if it's something manually entered somewhere for now (Having "Check domain expiry for xxx.tld" show up in slack at 30, 14, and 7 days before would be just fine to let the team know to check in this, and update the date in Uptime Kuma if needed).
On a side note, I did discover this check domain expiry plugin for Nagios, and it's not trivial (it's written in SH, and uses the whois tool directly). Search for the get_expiration() function.
https://github.com/glensc/monitoring-plugin-check_domain/blob/master/check_domain.sh

RDAP is the replacement for WHOIS, see some more details on Wikipedia. It's very useful as it implements a structured JSON format for representing domain metadata, including the expiration.
The way it'd work is that Uptime-Kuma would have to cache the RDAP Bootstrap list (https://data.iana.org/rdap/dns.json) to get a list of TLD -> RDAP servers and then query the RDAP server on a regular basis for the expiry date.

RDAP is the replacement for WHOIS, see some more details on Wikipedia. It's very useful as it implements a structured JSON format for representing domain metadata, including the expiration.
The way it'd work is that Uptime-Kuma would have to cache the RDAP Bootstrap list (https://data.iana.org/rdap/dns.json) to get a list of TLD -> RDAP servers and then query the RDAP server on a regular basis for the expiry date.

Wow, it could be so helpful.
Surprisingly, I don't know why I didn't find anything about RDAP.
Haven't tested it yet, but I just found a nodejs package for rdap:
https://github.com/dcoffey-zengenti/node-rdap

Looks like not all TLDs have working RDAP yet (.io seems to be one, see: https://openrdap.org/demo?cmd=rdap+-v+codepen.io vs https://www.whois.com/whois/codepen.io ), but this would be better than nothing and RDAP should be the path for this check going forward.

On May 25, 2023 5:28:53 PM UTC, Nicholas Istre @.***> wrote:

Looks like not all TLDs have working RDAP yet (.io seems to be one, see: https://openrdap.org/demo?cmd=rdap+-v+codepen.io vs https://www.whois.com/whois/codepen.io ), but this would be better than nothing and RDAP should be the path for this check going forward.
--
Reply to this email directly or view it on GitHub:
https://github.com/louislam/uptime-kuma/issues/944#issuecomment-1563263456
You are receiving this because you commented.
Message ID: @.***>
Yeah, all of the generic TLDs must implement RDAP per ICANN policies, but the country codes are free to do whatever their government wants.

I hope this gains some tractions and seems like it would be trivial to add for the most part. Yes some of the TLDs aren't doing RDAP, in those cases, a simple whois seems to work just fine. I get it, there is no standardization around whois and what is returned....But I think RDAP would cover a large chunk of domains out there.

You are right, the output of WHOIS is always different, except for the line:
Registry Expiry Date:

Although, just as mentioned above, I also did not find these details in the RFC
I haven't seen output for any existing domain name that doesn't have an expiration line, please correct me if the expiration output is different somewhere.

I have a Remote Probe implementation to work with Push monitor type that has a function to check for Domain Expiration. May help somebody.
Branch

Looking at some stats about RDAP support (source https://deployment.rdap.org/) it looks like 72.1% of domains support it, with all gTLDs supporting it but only 22.7% of ccTLDs supporting. Given that 72.1% of domains support it, I would be inclined to suggest that we start by implementing a domain expiry using RDAP only. That should simplify implementation a bit hopefully and still cover the majority of use cases. If in future it is determined that accessing those domains that don't currently support RDAP, we could perhaps think about how we want to deal with that problem (maybe if enough people ask ccTLDs for RDAP, they will provide? :wink:).
I think that I might have a bit of a play with this over the coming days and see if I can get anything working. As far as implementation, would we want to create an entirely new monitor type (something with a long interval), or something more like what we have with the certificate expiry, with the monitors URL / hostname being used as the domain to check?

I would argue that this should be a dedicated monitor, but this is just

my feeling of how generally useful (as in useful for everybody),
technical debt (how complex is the monitor?) and
UX-friedly (there are plenty of options for monitors already ⇒ adding more (without them being required) might be overwhelming)

If you decide that this is a good decision, please have a look at the new way of writing monitors (#3178) instead of the one used in older PRs ^^

whoiser package works great.
The initial check should occur when adding a site and subsequent checks should be carried out periodically, similar to how SSL certificate checks are done.

This might be used as an alternative: #3633 .

+1
I have several domain names where I would want to get a 90 day advance expiration (or WHOIS data change notification). This to allow me to find the cheapest domain renewal rates and so on.

@TheLinuxGuy
Please refrain from posting +1 / requests for updates things on issues, as this makes issue-management harder.
Issues are for discussing what needs to be done how by whom.
We use 👍🏻 on issues to prioritise work, as always: Pull Requests welcome.
An example-pr that implements a monitor according to our new syntax (not just piling anything into monitor.js) can be found here: https://github.com/louislam/uptime-kuma/pull/3178 or https://github.com/louislam/uptime-kuma/pull/3308

Wouldn't it possible to use the Linux whois command somehow?
Some output examples
whois google.com
...
Registry Expiry Date: 2028-09-14T04:00:00Z

whois github.com
...
Registrar Registration Expiration Date: 2024-10-09T00:00:00+0000

Note that this is not blocked on a technical level, but on the implementation side.
If you'd like to propose a monitor here is our contribution guide:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md
As noted above, whois does not have a stable interface.
⇒ RDAP would be preferable for the TLDs which support it

fetching domain expiration has been implemented for whois and RDAP in the "domain_exporter" for prometheus by caarlos0!
It's written in go, but perhaps someone could use it as inspiration for implementing it here in js? I lack the nodejs competence to do it myself, but hoping this helps :)

@tahaipek
No change in comparison to the previous state.
Please refer to my previous comment https://github.com/louislam/uptime-kuma/issues/944#issuecomment-1756306092:

Note that this is not blocked on a technical level, but on the implementation side.
If you'd like to propose a monitor, here is our contribution guide:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md
An example-pr that implements a monitor according to our new syntax (not just piling anything into monitor.js) can be found here: https://github.com/louislam/uptime-kuma/pull/3178 or https://github.com/louislam/uptime-kuma/pull/3308

As noted above, whois does not have a stable interface.
⇒ RDAP would be preferable for the TLDs which support it. node-rdap seems like a solid choice, but we would be open to alternatives if somebody knows a better one ^^

Hi @louislam
Do we have any progress in the plan to implement this functionality ?

Hi @louislam Do we have any progress in the plan to implement this functionality?

I made this using n8n. It's particularly challenging because each whois provides the date/time expiration in a different format, I don't think that will be implemented soon here :/

Hello!
I think RDAP could be used to monitor this.
Some API examples:
OpenRDAP
RDAP.org

Hi @louislam Do we have any progress in the plan to implement this functionality ?

I made this using n8n. It's particularly challenging because each whois provides the date/time expiration in a different format, I don't think that will be implemented soon here :/

Looking at some stats about RDAP support (source https://deployment.rdap.org/) it looks like 72.1% of domains support it, with all gTLDs supporting it but only 22.7% of ccTLDs supporting. Given that 72.1% of domains support it, I would be inclined to suggest that we start by implementing a domain expiry using RDAP only. That should simplify implementation a bit hopefully and still cover the majority of use cases. If in future it is determined that accessing those domains that don't currently support RDAP, we could perhaps think about how we want to deal with that problem (maybe if enough people ask ccTLDs for RDAP, they will provide? 😉).
I think that I might have a bit of a play with this over the coming days and see if I can get anything working. As far as implementation, would we want to create an entirely new monitor type (something with a long interval), or something more like what we have with the certificate expiry, with the monitors URL / hostname being used as the domain to check?
I know about #1775 but that make use of whois which we know to be a bit problematic and it hasn't been updated for over a year.

It's not true that all gTLD support it, .ES of Spain for example do not have it.


Protection from copying a directory into itself

Please describe the problem you're trying to solve

cp has protection against copying a directory into itself.

This is not present in yazi resulting in an operation running forever.
Of course you could just quit yazi, so it is not an urgent feature, just a nice addition to yazi.
Would you be willing to contribute this feature?

[ ] Yes, I'll give it a shot

Describe the solution you'd like
A warning message when you are about to copy a directory into itself.
Additional context
No response

Hey, you can open the task manager by pressing the w key, then press x to cancel the failed task, keep the failed tasks in the task manager and display a red progress bar at the bottom to indicate to the user that it's an intentional design.
I don't think notifications are suitable for this scenario. For example, if a user copies 100 files and 30 of them fail, constantly popping up 30 notifications would be annoying. So I decided to just keep them in the task manager and use the red progress bar to quietly inform the user.

Regarding adding protection checks, it would be nice to have and this would make nested copying/moving more secure.
Anyone want to give it a try? I think file_cut() and file_copy() are the only two places that need to be changed:
https://github.com/sxyazi/yazi/blob/e7dc971416af2c59b0d1498579102318885642ca/yazi-scheduler/src/scheduler.rs#L70
https://github.com/sxyazi/yazi/blob/e7dc971416af2c59b0d1498579102318885642ca/yazi-scheduler/src/scheduler.rs#L102

@sxyazi There is already some similar code preventing the file to be copied/cut onto itself: https://github.com/sxyazi/yazi/blob/e7dc971416af2c59b0d1498579102318885642ca/yazi-core/src/tasks/file.rs#L24
Shouldn't the protection check be added there as well, so in that case it doesn't even reach the scheduler?
Assuming you still want it to be in the scheduler, do you think that just checking and exiting earlier is enough?

Hey @alexandruoprinca, this is different. When the user moves/copies A to A, i.e., A -> A, Yazi assumes that the operation has been successful because nothing (and should not) change, so there is no need to give the user an error message.
However, the second image provided by OP is a process of A -> A/A, which leads to an infinite loop of A -> A/A/A/....

Hey @alexandruoprinca, this is different. When the user moves/copies A to A, i.e., A -> A, Yazi assumes that the operation has been successful because nothing (and should not) change, so there is no need to give the user an error message.
However, the second image provided by OP is a process of A -> A/A, which leads to an infinite loop of A -> A/A/A/....

Sorry for not being more explicit, i meant that there is already a particular place in the code where there is a check for 'A -> A', shouldn't the check for 'A -> A/A' be put there as well? So in the end in this if:
https://github.com/sxyazi/yazi/blob/e7dc971416af2c59b0d1498579102318885642ca/yazi-core/src/tasks/file.rs#L23
Another check can be added, something like:
if force && *u == to && to.startsWith(*u) { ....


fix pnpm run create directions

per https://github.com/getferdi/recipes/blob/master/docs/integration.md and testing, I think it should be pnpm run create <recipe>. However, I don't know pnpm particularly well.

@all-contributors pls add @langdon for docs


unrecognized option: S for BusyBox's env in index.ts shabang

As of right now index.ts has the following shabang:
https://github.com/dstoyanoff/directus-auto-migrate/blob/db50e821464d9eef4398b02cfd5afef7d242bd49/src/index.ts#L1
It was added in c4bbbf32a2e2d61235fd4f91c8fa8cb105e19b07.
But unfortunately BusyBox's env does not have -S option, furthermore directus official docker image is based on BusyBox (as it is based on node:16-alpine).
# env --help
BusyBox v1.34.1 (2022-04-04 10:19:27 UTC) multi-call binary.

Usage: env [-i0] [-u NAME]... [-] [NAME=VALUE]... [PROG ARGS]

Print current environment or run PROG after setting up environment

        -, -i   Start with empty environment
        -0      NUL terminated output
        -u NAME Remove variable from environment

How to reproduce:

Run directus/directus:latest docker image.

Exec npx directus-auto-migrate my-first-migration inside the container.
# npx directus-auto-migrate my-first-migration
Need to install the following packages:
  directus-auto-migrate
Ok to proceed? (y) y
/usr/bin/env: unrecognized option: S
BusyBox v1.34.1 (2022-04-04 10:19:27 UTC) multi-call binary.

Usage: env [-i0] [-u NAME]... [-] [NAME=VALUE]... [PROG ARGS]

Print current environment or run PROG after setting up environment

        -, -i   Start with empty environment
        -0      NUL terminated output
        -u NAME Remove variable from environment

Why do we need this shabang? Why do we need typescript (ts-node) in the final build (index.js)?
Why doesn't directus-auto-migrate read from env vars (only .env file)? Should I create a different issue for this one?

Hello,
Is there a reason to still use this package? I have stopped maintaining it since directus released their built-in schema snapshots functionality, which is influenced by this project, but adapted more on their own preferences. I would suggest going there first. If you insist to use this package, PR's are welcome, but I am not planning to invest time in testing the changes myself.

Might be related: https://github.com/dstoyanoff/directus-auto-migrate/issues/7
@dstoyanoff
If this package is not actively maintained because similar functionality is available with Directus CLI, please update the README.md.
I just started working with Directus and I'm not aware yet of what's included and what is not.


Code uses WebClient but blocks

The code uses the reactive WebClient but ends up calling #block() which, as the name suggests, blocks and therefore defeats the whole purpose of using WebClient in the first place. Examples:

https://github.com/admin-ch/CovidCertificate-Api-Gateway-Service/blob/4633b186a9cb102c33094b890b39561758000fb0/src/main/java/ch/admin/bag/covidcertificate/gateway/service/ValueSetsService.java#L51
https://github.com/admin-ch/CovidCertificate-Api-Gateway-Service/blob/4633b186a9cb102c33094b890b39561758000fb0/src/main/java/ch/admin/bag/covidcertificate/gateway/service/CovidCertificateRevocationService.java#L39
https://github.com/admin-ch/CovidCertificate-Api-Gateway-Service/blob/4633b186a9cb102c33094b890b39561758000fb0/src/main/java/ch/admin/bag/covidcertificate/gateway/service/CovidCertificateGenerationService.java#L52

The code should instead do one of the following:

Avoid calling #block() and instead call #subscribe()
Switch from the reactive WebClient to the blocking RestTemplate

Considering all the rest of the code is blocking (WebServiceTemplate, JpaRepository, …) WebClient seems like the better match.

@marschall Thanks for your feedback.
According to https://stackoverflow.com/questions/58513071/spring-mvc-to-spring-webflux-migration-block-vs-subscribe, using block seems to be the best way in our situation.

The thread is explicit that going Mono / Flux and #subscribe is the way to go if you want to be reactive. #block is a good choice for cases when you want to migrate an existing servlet based application to reactive in a step by step fashion. If you decide to go down that route you still needs solutions for JPA and SOAP which to my knowledge have no reactive options.


Broker can be hacked through HAService Port

If someone knows the master's ha service ip and port , he can disguise himself as a slave broker, then he can sync the msgs from master even through there is acl exists in master.
One simple way to fix it:  add iptables limit the ha port connections.
I didn't see anyword about it, So if it is the goal of rocketmq, then i think it is better to write on best_practices or somewhere else. Or it can use the acl  together.

@zongtanghu Could you please help to check the ACL about HA service?

link #1842

link #1842

I think it is a independent issue beyond #1842 , i can use some tool to find out master's ip and port.

@ShannonDing Okay I will review this pr next week.

@ShannonDing @RongtongJin @zongtanghu Hi，I think this proble is very important, could you check this asap?


sorting the names

the names should be sorted alphabetically when the refresh button is clicked

Good suggestion. What are your ideas around achieving this?

using node.js to read and edit the text.txt file should be able to achieve it

You can check the sorted.js file. I have an implementation for that and it includes read and writing with nodejs. Let me know what should be done. Next. Thanks for jumping on this issue.

I will close this issue @Darrkzero. Waiting for your response.

the sorted.txt file is perfect.  You can create a different button to switch from text.txt file to sorted.txt file to make the names appear in alphabetical order.

Yes that will be a great one but we could use JavaScript to sort it on the fly and replace instantly.

That will also work.

Attempted this in #743. Kindly review and give feedback. Also, i don't know if the ordered.js and sorted.txt files are needed anymore.


React routing

I'm coming from using Angular every day, and was just experimenting with React, and I used the react sub-generator to create a new component (as outlined in the docs). However, it's not entirely clear how creating a new component can lead to making a new route. I tried adding the route in routes.js, and the method that it calls is virtually identical to the one that renders the index view
var TestComponent = React.createFactory(require('./components/testcomponent.jsx'));

then
var test = function(){ render(TestComponent); };

and in the routes object I added
'/test': test

but it doesn't work. Besides adding the component and adding the route, is there some third step I'm unaware of? If I add my test component to an existing route, it works fine, it's just any new route I create isn't recognized.

Hi @tjslater,
Can you post the contents of your .yo-rc.json file? (Remove any private info first)
It is located in the root folder of your generated project. It is also a hidden file, so you might need to enable hidden files for your file manager to view it.
This will help me debug and hopefully answer the question you've posed. Thanks!

Sure enough!
{
  "generator-yeogurt": {
    "config": {
      "projectName": "office-365-sample",
      "versionControl": "git",
      "useServer": true,
      "dbOption": "mongodb",
      "dbHost": "localhost",
      "dbPort": "27017",
      "dbName": "yeogurt_db",
      "dbUser": "",
      "dbPass": "",
      "useAuth": true,
      "singlePageApplication": true,
      "jsFramework": "react",
      "useJsx": true,
      "cssOption": "less",
      "useJsdoc": true,
      "useKss": true,
      "useTesting": true,
      "testFramework": "jasmine",
      "jsOption": "browserify"
    },
    "version": "0.14.3"
  }
}

Ok, perhaps I'm not very familiar with React or Flux semantics.
I can get to the route if I:

Go to it by link from another page
Put a # before the url if I try to go to it directly: e.g. (/#test)

But going directly to '/test' always results in a 404, in Chrome, FF, and Safari. Is this normal behavior for React?

Hey @tjslater
So the issue you are running into is the fact that HTML5 pushState is on by default. This means that when you go to a normal route (i.e. /test), the server is expecting to render a template. Because you do not have a server template rendering for that route, you will get a 404 not found page.
To remedy the issue, you need to create a /test route for your express server. So create a test.js file within the server/routes/ folder and copy in the following contents:
/**
 * Test Routes
 */

'use strict';

var testController = require('../controllers/test');

var routes = function(app) {

  // Test
  app.get('/test', testController.index);

};

module.exports = routes;

Then create a new test.js file within server/controllers/ folder with the following contents:
/**
 * Test Controller
 */

'use strict';

var settings = require('../config/env/default');
var path = require('path');

var testController = function(req, res) {
  // Render index.html to allow application to handle routing
  res.sendFile(path.join(settings.staticAssets, '/index.html'), { root: settings.root });
};

module.exports = {
  index: testController
};

This will now allow you to render your /test route without the 404 error.
If you don't want to use HTML5 pushState (i.e. use /#test hash routing) and/or avoid creating server routes for every page, then you can go into the client/scripts/main.js file and change the html5history: true to html5history: false

Thanks! That did the trick. I suppose I didn't realize what was involved in creating routes in this stack, a bit different than what I'm accustomed to! Cheers.


A little question about Cluprocessor

A little question.
If I use Cluprocessor  to make my index, I can't  use the rule like:

name: pets_adoption
type: event
label: Adoption
priority: 2  # will run in the second round of extraction, can reference priority 1 rules
pattern: |
trigger = [lemma=adopt]
adopter = >nsubj []   # note: we didn't specify the label, so any token will work
pet: Pet = >dobj []

to extract information?
what's more, I try the  rule of document.event_queries.priorities, and the next.
http://gh.lum.ai/odinson/event_queries.html#priorities
all of them can't work if I use Cluprocessor  to make my index.
If I use fastnlp, it takes really long time to annotate the text.
so, maybe, we can do some work to make it fit the syntactic parse of Cluprocessor .

New changes to cluprocessor may give us what we we lacking.

[ ] profile the annotation/indexing on a given corpus
[ ] switch to coreclu as default for extra/AnnotateText
[ ] add a mkPartialAnnotation method as default (configurable?), particularly since we can't currently use the expensive cross-sentence annotations anyway
[ ] add cool new fields to the defaults (e.g., enhanced SRLs)
[ ] profile again, celebrate or cry

This is the plan, I'll try to work on it as I can.  PLEASE all (@myedibleenso , @marcovzla , @dvzhang ) feel free to comment/complain/add/subtract

enhanced SRLs

@BeckySharp , are you talking about indexing multiple graphs or just some new token attributes?  @marcovzla and I discussed indexing multiple graphs per sentence before, but opted not to do it at that time.  Eventually, though, I think we'll want to do this and use a field in the rule to tell us which graph to match against. One proposal was to index one merged graph.  I think both approaches (one vs many) have advantages and disadvantages.

are you talking about indexing multiple graphs or just some new token attributes?

Great point, I wasn't fully thinking it through, but now that you mention it I remember this and several discussions about how to approach it.  I think the lowest hanging fruit is to merge (e.g., deps and SRLs), configurable of course, as it would also give access to mixed rules without needing to make a change to the rule syntax.
That said -- looking longer term, IMO we should be able to write rules against multiple graphs, to be able to add more complex constraints.  However, IDK when that will be needed vs simple anticipated as useful...
How do you feel about the merged plan for now?
And really thanks @myedibleenso  for thinking it through for/with me!

An option under the conf for extra related to graph merging is definitely the path of least resistance.  Seems like it would be useful.  It's straightforward to just combine and distinct edges: List[Edge] and roots: Set[Int] from each processors doc's Sentence to construct a single DirectedGraph and then replace the GraphMap for each processor Sentence before creating the Odinson doc.
Looks like there's already an entry in the processors GraphMap that merges a couple of the graphs (it looks as if it excludes basic):

https://github.com/clulab/processors/blob/b8fb35a38d146a67817b6fad00031ab6732ee53a/main/src/main/scala/org/clulab/struct/GraphMap.scala#L17

I sort of think that we might as well merge them all, though...

I sort of think that we might as well merge them all, though...

i'd be ok merging them all unless there are overlapping labels between the basic and enhanced that point to diff things, any easy way to check that you think?

My guess is that it's most likely to happen with adpositions/case relations.  Here is where the conversion from basic -> enhanced takes place in processors:

https://github.com/clulab/processors/blob/master/main/src/main/scala/org/clulab/utils/ToEnhancedDependencies.scala

I didn't have a hand in that code, but I know it is a deterministic process.
I guess one strategy that still keeps things fairly simple is to just merge in a way that prefers enhanced (I assume that's what you want?) where any edge in basic that has the same source and relation label should be filtered out of the merged.

that strategy would account for one issue (that TBH I didn't think about), but I was thinking more that idk, something like >dep in basic would go to word_3 and in enhanced it would go to word_4, and when we wrote the rules we were thinking about enhanced (which we're more familiar with), and so we would expect/want word_4 not word_3....
maybe i'm worried about nothing tho

If we want to prefer X over Y, I think we can achieve that using the filtering step I described.  The downside is we lose some info.
I think we're really just trying to come up with reasonable default behavior.
If a user wants something more customized, that person can should create custom Odinson doc JSON and simply index. This annotation runnable is really just to get started quickly with something that is (hopefully) useful to many.
At this point, anyone who wants to index non-English docs or use custom components (ex. sequence taggers for NER) must roll their own Odinson doc creation pipeline.
One of these days I'm going to provide an example that shows how to use spaCy to create Odinson doc JSON.

on it :)


Update textobject.py

Documentation spelling and grammar fixes

Sorry if this was the wrong repository. I'll try to put this pr in the other repo too.


Update o365-data-locations.md

added covid 19 statement.

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks.

Do I need to do something to approve his changes?  I do not see it live yet.
From: Joyce Y<EMAIL_ADDRESS>Sent: Monday, March 30, 2020 3:06 PM
To: MicrosoftDocs/OfficeDocs-Enterprise<EMAIL_ADDRESS>Cc: Dean Wierman<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/OfficeDocs-Enterprise] Update o365-data-locations.md (#771)
@JohanFreelancer9https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJohanFreelancer9&data=02|01|dean.wierman%40microsoft.com|ca577595f089417d715508d7d4f677b7|72f988bf86f141af91ab2d7cd011db47|1|0|637212027364792553&sdata=NnCYlhoOVe4wZU24t47PCe1%2BnOir3pHSrICFByAD35U%3D&reserved=0 - kindly do the copy editor review for this PR. Thanks.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2FOfficeDocs-Enterprise%2Fpull%2F771%23issuecomment-606275884&data=02|01|dean.wierman%40microsoft.com|ca577595f089417d715508d7d4f677b7|72f988bf86f141af91ab2d7cd011db47|1|0|637212027364792553&sdata=POvXkHFNBVT1r4mTAVL5hbbu92OXWH1JRs5LH4Mgngg%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOPQZVCXU2WITEPTRJGMHQLRKEJSZANCNFSM4LW2RRNQ&data=02|01|dean.wierman%40microsoft.com|ca577595f089417d715508d7d4f677b7|72f988bf86f141af91ab2d7cd011db47|1|0|637212027364802548&sdata=6wEcgEF94N0s0oes5ni%2B%2BDdJdGtrrXh22xkcVmvMabw%3D&reserved=0.

@dmwmsft - please merge the commit on the copy/edit review done by JohanFreelancer9 or make the necessary changes on the PR following the suggestions made. Thank you.

@dmwmsft - please merge the commit on the copy/edit review done by JohanFreelancer9 or make the necessary changes on the PR following the suggestions made. Thank you.

let me know if I did it correctly.  I did not see a "merge" option.


Docs: Which Package Manager should I use?

Add page in documentation about yarn vs. yarn 2+ vs. pnpm vs. npm. Compare features:
Dimensions

Support for resolutions/overrides, hoisting
Workspace-aware installs
Support for React Native
Install speed comparison
Support for turbo

Something interesting I found out about yarn 3 (2+ too I guess) is the local cache feature. It allows to keep the cache across yarn.lock changes, by pruning old packages and installing the new ones.
In github action with a restore_key
      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn config get cacheFolder)"

      - name: Restore yarn cache
        uses: actions/cache@v2
        id: yarn-cache
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: yarn-cache-folder-${{ hashFiles('**/yarn.lock', '.yarnrc.yml') }}
          restore-keys: |
            yarn-cache-folder-

When installing a new dependency, ie: superjson, there's no need to invalidate the whole thing.
yarn install --immutable
➤ YN0000: ┌ Fetch step
  ➤ YN0013: │ 1719 packages were already cached, one had to be fetched (superjson@npm:1.7.5)
➤ YN0000: └ Completed in 1s 124ms

Cache Size: ~127 MB (133261279 B)
Cache saved successfully

See also https://github.com/actions/setup-node/issues/325
Worthy to share.

I tried pnpm before transitioning to npm workspaces, as prisma and next misbehaved with package hoisting


Socket options wrapper: Expose configuring socket options through envoy at Filters

Description:

Several customized tuning can be performed on the socket file descriptor created during a connection for optimizing performance and building measurements. Here are the several options that can be tuned on the socket

https://linux.die.net/man/7/socket
(eg: SO_RECVBUF, SO_LINGER, etc. ..).
Would you folks think it is useful for envoy to expose an API (wrapper on setsockopt and getsockopt)available at a filter (l4/l7) abstraction to configure these settings?
I have a variation of this implementation used for an internal project. Let me know if this is something that is interesting for upstream?
Thx!
Rama
[optional Relevant Links:]

https://linux.die.net/man/2/setsockopt

For the L4 filter stack, yes this sounds like a worthwhile proposal. For the L7 filter stack, I don't think so. Can you potentially provide a complete proposal? Thank you.

Hello, thanks for the comments.
There are certain uses cases that we have seen where we would like to tweak the connection based on the http requests that are delivered through it. Specifically, we want to apply some intelligence based on the http headers to conserve the compute resources, memory utilization on server and preserve network bandwidth utilization.
If you would like to understand one particular motivation, please read ahead...

Consider a scenario where we have a single connection per user; though usually it is not the case, for the sake of identifying the motivation; I would like to propose this case.

•    With the said setup, say there are about 20,000 connections and 20,000 users accessing our application servers through envoy.
•    Each of these users has an unique IPv4 address used to establish connection and are sending requests at around 5K RPS to envoy; which envoy forwards upstream to the application server pool.
•    Say, a malicious user has about 1,000 of these connections with unique IPv4 addresses and wants to do a distributed denial of service attack through flooding http/https `get` requests to the upstream application server pool. The person would increase the requests per second on these connections from 5K RPS to 30K RPS. This would also mean; these requests would consume the network bandwidth in the multi-tenant back bone and our network infrastructure.
•    This would effect the requests per second for the good users who are genuinely interested in getting their requests served. This service to them is now degraded.

In the case, where we wanted to act purely based on IPv4 address, tuning socket options to reduce the attack footprint; at L4 layer of envoy would help. On the other hand if we would want to take action based on any of the http headers; say `user-agent` this can not be done at the L4 layer.

For this very reason, this is a proposal to expose the socket options at the L7 layer.

Please let me know your opinions.
Thanks!
Rama

@ramachaitanyak to date we don't allow L7 filters to directly modify the connection as there can be multiple streams per-connection, and the abstraction breaks down pretty badly if you allow connection modification.
My suggestion for this would be to create a custom L4/L7 filter pair and have the L7 filter communicate to the L4 filter that it should set socket options.
@alyssawilk any thoughts on this?

I have lots of thoughts on this - we've got a couple of options we set in-house from L7-L4 and they have fairly individual behavioral characteristics.  Some we only set early on a connection, some are per-stream and we have a manager which coordinates per-stream settings with live-streams to calculate what the per-connection value ought to be (and tweaks it as streams are created and destroyed).   I'd love some utility classes with generalizable rules which could be used for all of them!
We also might need a similar option for QUIC, where most things would be set per logical QUIC connection (you don't want to accidentally change the size of the send buffer for a globally shared UDP socket, but you might want it to convert into how much you'd be willing to buffer in user-space)

Per @alyssawilk, to be clear, the utility of this is obvious to me, but the APIs are tricky since we have to have a sane way for streams to interact with the connection. I will meet with @alyssawilk and discuss some use cases and then let's see if we can come up with some common utilities?
I can imagine generalizing the "L4 filter" I propose above as a generalized connection settings manager that is exposed to filters that provides some safeguards and sanity around what can be done.

Thanks @mattklein123 & @alyssawilk for the feedback. We currently have an internal implementation similar to RBAC filter(L7 filter) to plug into this API. We can improve it based on the feedback. OTOH if it is convenient for you folks; I would enjoy to be a part of the aforementioned discussion if it has not already happened. I can join over a voice call. Please let me know.
Rama

Hello @mattklein123 and @alyssawilk; thanks for the discussion. I am following up on this thread to gather a summary of design proposals made during the meeting with respect to having a singleton in HCM for exposing this socket options per connection.
In addition, it would be great to get opinion on the common utilities for reading the socket file descriptor.
It would be great if the design is elaborated here. I will work on the implementation once there is a consensus on it.
Rama

This is roughly what I had in mind:
mklein@localhost:~/Source/envoy$ git diff
diff --git a/include/envoy/http/filter.h b/include/envoy/http/filter.h
index 116506541..ec0f60f6b 100644
--- a/include/envoy/http/filter.h
+++ b/include/envoy/http/filter.h
@@ -119,6 +119,29 @@ enum class FilterMetadataStatus {
   Continue,
 };

+/**
+ * An interface for a per-connection object that is created by a filter and then managed by the
+ * HTTP connection manager. This pushes singleton handling into the HTTP connection manager.
+ */
+class PerConnectionObject {
+public:
+  virtual ~PerConnectionObject() = default;
+};
+using PerConnectionObjectSharedPtr = std::shared_ptr<PerConnectionObject>;
+
+/**
+ * An interface for connection attributes that are modifiable by HTTP filters. Generally, these
+ * should be modified only via a PerConnectionObject.
+ */
+class MutableHttpConnection {
+public:
+  virtual ~MutableHttpConnection() = default;
+
+  // Fill with the things that we want HTTP filters to be able to do to the connection such as
+  // setting certain socket options, etc. Everything needs to be abstracted and then flowed through
+  // to the underlying writable Network::Connection.
+};
+
 /**
  * The stream filter callbacks are passed to all filters to use for writing response data and
  * interacting with the underlying stream in general.
@@ -132,6 +155,21 @@ public:
    */
   virtual const Network::Connection* connection() PURE;

+  /**
+   * Create a per-connection object. The connection manager will track whether the object exists
+   * already and if so, return it. Otherwise it will be created via the creation function. Objects
+   * are keyed by name and thus can be shared between filters if needed.
+   * @param object_name supplies the name of the object to create or fetch.
+   * @param creation_function supplies function used to create the object if it does not already
+   *        exist. The creation function will be passed a mutable HTTP connection object that it
+   *        can use for singleton operations against the connection.
+   */
+  using PerConnectionObjectCreator =
+      std::function<PerConnectionObjectSharedPtr(MutableHttpConnection& connection)>;
+  virtual PerConnectionObjectSharedPtr
+  createPerConnectionObject(const std::string& object_name,
+                            const PerConnectionObjectCreator& creation_function) PURE;
+
   /**
    * @return Event::Dispatcher& the thread local dispatcher for allocating timers, etc.
    */

@alyssawilk WDYT about ^?

@mattklein123 & @alyssawilk this level of abstraction works for the implementation. I will wait for Alyssa to give the go-ahead before I take a stab on the implementation.
Another thought that occurred to me is whether or not this API should be OS dependent. I understand that macOS supports setsockopt and getsockopt. I am not sure about windows. Any thoughts on this ?

I think this is fine at a high level - I'm more interested in what we do with MutableHttpConnection once we have it, so feel free to start in on this and send a draft PR my way, and you or I can tag @danzh2010 as well to make sure it'll play well with non-TCP "connections"

@alyssawilk thanks for the comments. I will work on it. @danzh2010 if you have comments please let me know.
My impression for this patch; is to only include a wrapper for getsockopt and setsockopt (specifically a wrapper on http://man7.org/linux/man-pages/man2/setsockopt.2.html)
What we could do with this information; is something I have addressed as other issues like tarpit and gather metrics.
I will send a draft PR as soon as possible.
Rama

I'm more inclined to start with explicit options we want to set and set them via methods, so we can make clear what is supported by QUIC and across various platforms but I'll go with whatever Matt suggests makes sense

+1 to explicit for now until we prove that won't work.

Hello, I have a draft patch for this. I will open a PR with draft tag soon. Working between schedules, I am trying to resolve abseil compilation issues. Should be soon to get a patch out.
Apologize for the delay.
Rama

Hello, I have created a PR here https://github.com/envoyproxy/envoy/pull/8851/
I am running into these errors while building:
source/common/http/conn_manager_impl.cc:1993: error: undefined reference to 'Envoy::Http::ConnectionManagerImpl::per_connection_object_map_[abi:cxx11]'
source/common/http/conn_manager_impl.cc:1994: error: undefined reference to 'Envoy::Http::ConnectionManagerImpl::per_connection_object_map_[abi:cxx11]'
source/common/http/conn_manager_impl.cc:2004: error: undefined reference to 'Envoy::Http::ConnectionManagerImpl::per_connection_object_map_[abi:cxx11]'

I will work on resolving those; however if any of you have suggestions to fix this, please let me know.
Take a look at the PR and let me know your thoughts and hope it aligns with the initial thinking of design.
Thanks!
Rama

the linker errors is because if you have a declaration of a static member static std::map<const std::string, PerConnectionObjectSharedPtr>  per_connection_object_map_;   without a definition.
you need to defined it in the .cc file like so:
   std::map<const std::string, PerConnectionObjectSharedPtr>  ConnectionManagerImpl ::per_connection_object_map_;

btw - not sure if a static variable will work here as IIUC this code can be called concurrently so it is not thread safe.

Thanks @yuval-k for pointing this out. I am able to get past this error; following your suggestion.
Based on my understanding we are required to maintain a single instance of the ConnectionManagerImpl::per_connection_object_map_ maintained per ConnectionManagerImpl instance.
This allows multiple filters running on the connection to manipulate and reuse connection states.
I recognize having a static map will allow access to multiple ConnectionManagerImpl instances to manipulate the map. I have fixed this by accessing the parent_.connection_manager_ instance in
ConnectionManagerImpl::ActiveStreamFilterBase::createPerConnectionObject and removed it from being defined as static.
Furthermore; if my interpretation is correct we still will need some synchronization primitives around the access to the map; since there could be two or more filters simultaneosly trying to access/modify the contents inside the map.
Please let me know if you have other thoughts and correct me if my interpretation is inaccurate.
Thanks!
Rama


JToken.ToObject does not use Converters specified on enum type

Here's the scenario:
Enum with a converter specified and EnumMember:
[JsonConverter(typeof(StringEnumConverter))]
public enum FooBar
{
[EnumMember(Value = "SOME_VALUE")]
SomeValue,

[EnumMember(Value = "SOME_OTHER_VALUE")]
SomeOtherValue
}

During serialization/deseriazation, this works as expected. I can have a type like so and the values are read and generated correctly:
public class MyObject
{
    public FooBar FooBar { get; set;}
}

However, if I want to get my value using a JObject, as I do in the case of implementing a custom converter to deal with abstract base classes and new'ing up the right type for deserialization, I need to get my enum value first.
So I do var jo = JObject.Load(reader) and then I want to do var f = jo["FooBar"].ToObject<FooBar>(). That does not work though. The code in ToObject does an Enum.Parse() and that fails as the value is SOME_VALUE. The declared JsonConverter does know how to handle it though, as does EnumUtils.ParseEnumName
Can ToObject be "fixed" to check for and use any JsonConverter that might be specified on the type? Also, not sure if there are other places that do this as well, but I'd expect consistent behavior anyplace I ask for a type.
Thanks

Done https://github.com/JamesNK/Newtonsoft.Json/commit/2c7a51c50a2dfe4ca37871ac27488abdbc924669


Mailpile as webmail

Hello !
I didn't found any history about this, but what do you think to use mailpile as default webmail in Modoboa?
Thanks for your answer.

Roundcube Next is more likely as it has groupware features by default. The project started and looks promising.
https://github.com/roundcube-next

@titilambert I think your question is more related to the installer.
It would be pretty easy to add the possibility to deploy a custom webmail during the installation process.

just saw Mailpile. looks great! I would also like to know how to add it post installation

@tonioo any plan to allow us add custom webmail during installation process?

@hadifarnoud Feel free to modify the installer, add this feature and propose a pull request ;)

@tonioo if you can help me get started, I would give this a try. I would've done it if I knew how :)

@hadifarnoud If you need to support multiple users through webmail, I'm not sure that Mailpile is quite capable enough yet to do this. They won't have full/stable multi-account support until their 1.0 milestone. If single user works in your situation, then it's perfect, especially considering the baked in PGP/GPG features. From what I can tell, it's not too bad to get running -- just clone the repo, do the pip setup, and run the setup from the Mailpile CLI. Then set the web address and reverse proxy!

my setups is multi-user. I guess I need to add RTL to existing webmail that comes with Modoboa. Will submit a PR soon.
I think others might like a tutorial though if you know how to do this.

@hadifarnoud I'll look in to doing that after Mailpile has proper multi-user support :)

I close this issue since it is more an installer request. Feel free to reopen it in the appropriate repository.

multi user support is added @pirogoeth https://github.com/mailpile/Mailpile/issues/820


Bring Phase 2 live

[ ] Check master cluster deployment stability @aismail
[ ] Restore login requirement for ETL if easily possible in routes @mattfullerton

@aismail please make sure hat our master branch is also stable not only the staging one.

@davidmihalyi ack, will be checking it today

@KseniyaKuprina : when are you back from leave?

@davidmihalyi getting back on Monday

@davidmihalyi I will double check the servers, but I'm afraid the problem was specific to staging.
In-depth explanation: we have 2 recurring failures that happen to affect the 3 nodes that host the deployments:

number of inodes gets filled up
nodes are dropped from the cluster (I think it's a Zookeeper problem, still investigating)

Since staging was previously deployed only on one node, if that particular node had problems, it went down. The production deploy is on 3 nodes, so it was never directly affected.
The other staging-specific problem that plagued us was that the MongoDB password was changed to something without updating the config, so that doesn't affect staging either.
I will double check the servers, but what we agreed on Slack is a bit better: offer ops assistance coupled with a manual deploy this time. I will need a pull request to review with the changes that are going in production, and we may have to do a quick plan of how to roll things back if anything goes wrong. Do you think that's a good idea? Thank you!


SqlServerDsc: Stop instances in integration tests

Pull Request (PR) description

Changes to SqlAlwaysOnService

Minor code cleanup in integration test, fixed the scope on variable.

Changes to SqlSetup

Updated the integration tests to stop some services after each integration test.
This is to save memory on the AppVeyor build worker.
Updated the integration tests to use a SQL Server 2016 Service Pack 1.

Changes to SqlRS

Updated the integration tests to stop the Reporting Services service after
the integration test. This is to save memory on the AppVeyor build worker.
The helper function Restart-ReportingServicesService should no longer timeout
when restarting the service (issue #1114).

Changes to SqlServiceAccount

Updated the integration tests to stop some services after each integration test.
This is to save memory on the AppVeyor build worker.

This Pull Request (PR) fixes the following issues
Fixes #1114
Task list

[x] Added an entry under the Unreleased section in the CHANGELOG.md? Entry
should say what was changed, and how that affects users (if applicable).
[x] Resource documentation added/updated in README.md?
[x] Resource parameter descriptions added/updated in README.md, schema.mof
and comment-based help?
[x] Comment-based help added/updated?
[x] Localization strings added/updated in all localization files as appropriate?
[ ] Examples appropriately added/updated?
[x] Unit tests added/updated?
See DSC Resource Testing Guidelines.
[x] Integration tests added/updated (where possible)?
See DSC Resource Testing Guidelines.
[x] New/changed code adheres to
DSC Resource Style Guidelines
and Best Practices?

This change is

This will hopefully resolve all PR's being blocked by failing tests. I will merge this one as soon as it passes all checks.

Codecov Report

Merging #1115 into dev will increase coverage by <1%.
The diff coverage is 100%.

@@          Coverage Diff          @@
##            dev   #1115    +/-   ##
=====================================
+ Coverage    97%     97%   +<1%
=====================================
  Files        32      32
  Lines      3930    3936     +6
=====================================
+ Hits       3844    3850     +6
  Misses       86      86

Reviewed 1 of 8 files at r1, 1 of 1 files at r4.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Feature request: Calculate Log Sampling Per Request

Use case
The log sampling concept is great but not very useful in practice if you have provisioned concurrency turned on due to the decision happening at initialization.
Solution/User Experience
Per request sampling would be ideal, as opposed to per log sampling. That would give the user the ability to see the entirety of a request that was sampled rather than a mix of logs for different requests.
Alternative solutions
No response
Acknowledgment

[x] This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
[x] Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers
Please react with 👍 and your use case to help us understand customer demand.

Hi @jasonwadsworth, thanks for opening this issue.
This is an intended limitation that is documented in the docs, and we designed this feature this way to avoid adding overhead for those customers who don't use the feature.
Up until now we have never gotten any demand of changing this behavior, but it's something we could consider provided that there's enough demand from multiple customers.
In the meantime, you can call the logger. refreshSampleRateCalculation() method manually at the beginning or end of your function handler to recalculate the sampling.

I'm not, but I was going to try your first suggestion for now.

Thank you for clarifying.
As of now, unless you're using one of these two methods then the Logger has no way of knowing when a request starts/ends, so it'd be tricky to implement.


Gain access to new WOS API and associated documentation

Swagger API minimal documentation: https://developer.clarivate.com/swagger-ui/?url=/apis/wos/swagger

Will ping Clarivate about access keys

The current REST API Documentation is linked from here and I suspect the fields available and queryable parameters will be the same or identical: https://github.com/sul-dlss/sul_pub/wiki/Clarivate-APIs

@peetucket will reach out to Clarivate again per today's storytime discussion.

Key now in rialto-etl-prod shared configs branch.
Documentation here: https://developer.clarivate.com/apis?filter=on&Web+of+Science=on (but may need to register for a free account to see it)


StatusBar is not defined error in Tabs example

Just created a new project using:
"ionic start myApp tabs"
When it's running in Chrome (on OSX) and the console shows the following error:
Uncaught ReferenceError: StatusBar is not defined          app.js:12
(anonymous function)                                                        app.js:12
(anonymous function)                                                        ionic.bundle.js:33051
onPlatformReady                                                               ionic.bundle.js:2261
onWindowLoad

I got a same error with @ace-han

Getting same error with Cordova 4.3.0 and Ionic 1.0.0-rc.2.
I'd really like to use this, hopefully there's a fix in the pipeline.

The same problem here with Cordova 4.3.0

@ace-han Yeah, the StatusBar plugin is not currently supporting the Android platform, mostly since Android, if you take a look, has the status bar blacked out and pinned to the top, like iOS 6 and prior did. The reason it only works with iOS is so that developers can give the app that iOS 6-y look, or change the colors and style it independently.
Kinda funny if you ask me.

Getting this same error.
@wilsonhobbs Android is capable of changing the Status Bar since 5.0.
Here's a thread of people who got it to work, though not with this error. http://forum.ionicframework.com/t/problem-with-status-bar-color/19375/4

Put them under the $ionicPlatform.ready(function() and not in the root of  .run() method

$ionicPlatform.ready( [ "$cordovaPluginA", "$cordovaPluginB", ( $cordovaPluginA, $cordovaPluginB )=> $cordovaPluginA.method() ]);

If you're using Angular you have to load the plugins independently and use ngCordova
angular.module( "module",
[ "$cordovaPluginA",
  "$cordovaPluginB",
  ( $cordovaPluginA, $cordovaPluginB )=> {
    $cordovaPluginA.method() ;
    $cordovaPluginB.methodC();
}]);

I had missed installing this plugin:
https://www.npmjs.com/package/cordova-plugin-statusbar


Azure Cloud Checks - Make Optional

I note that there is a check whether the cluster is deployed into Azure Cloud .. see below.
I don't have a problem with that per se except that in a corporate environment I don't want to request a proxy whitelist exception with my CISO for an endpoint that we have no use for (http://www.azurespeed.com). Moreover, whilst this test will eventually timeout and the checks resume, it delays the process by whatever our default timeout is (120 secs I think) which is not ideal.
Could you make this check optional using something like this ...
parser.add_argument('--azurechk', action="store_true", help="whether to check if the cluster is deployed on azure cloud - defaults to true")
...
class HostDiscoveryHelpers:
    @staticmethod
    def get_cloud(host):
        if config.azurechk:
            ...

Kind Regards
Fraser.
class HostDiscoveryHelpers:
    @staticmethod
    def get_cloud(host):
        if config.azurechk:
            try:
                logging.debug("Checking whether the cluster is deployed on azure's cloud")
                metadata = requests.get("http://www.azurespeed.com/api/region?ipOrUrl={ip}".format(ip=host)).text
            except requests.ConnectionError as e:
                logging.info("- unable to check cloud: {0}".format(e))
                return
        if "cloud" in metadata:
            return json.loads(metadata)["cloud"]

I would like to see this too.  I can't have a security tool making calls to the outside.


[Core] Failed to build <x64>-windows-static with vcpkg

Only happen when powershell is installed on Windows from the Windows Store.

Workaround:
Uninstall PowerShell and re-install from the msi executable and not from the Windows Store.


Suggestion for README.md

The updated sentence might shorten the length of the description which is more concise.

Thanks Kelvin, your PR is merged, be free to send other README changes if you see that something may be improved


Customizable graph text #8

Made it possible to customize the title and the axis labels depending on
the tool implementation.

Good suggestions, I just pushed the changes.

🎉 thanks again!

🎉  thanks again!


HDDS-8662. Improving S3G-related metrics

What changes were proposed in this pull request?

Copy/Put/Get Add Metadata Latency Metrics.
Adjust the original createKeySuccessLatencyNs/copyObjectSuccessLatencyNs/getKeySuccessLatencyNs latency metric to the time taken for a complete operation, including metadata time and data time. In detail these metrics are adjusted to be calculated after stream close(), because some data is written to the DN during the close() method.
Add Copy/Put/Get IO byte count metric, which can be used to calculate read/write traffic (byte/s)
Add list_key_count used to measuring the load of user list requests.

Example
Copy/Put/Get Metadata Latency Metrics
s3_gateway_metrics_copy_key_metadata_latency_ns_avg_time{hostname="VM-8-3-centos"} 0.0
s3_gateway_metrics_copy_key_metadata_latency_ns_num_ops{hostname="VM-8-3-centos"} 0
s3_gateway_metrics_get_key_metadata_latency_ns_avg_time{hostname="VM-8-3-centos"} 0.0
s3_gateway_metrics_get_key_metadata_latency_ns_num_ops{hostname="VM-8-3-centos"} 0
s3_gateway_metrics_put_key_metadata_latency_ns_avg_time{hostname="VM-8-3-centos"} 0.0
s3_gateway_metrics_put_key_metadata_latency_ns_num_ops{hostname="VM-8-3-centos"} 0

Copy/Put/Get IO byte count metric,
s3_gateway_metrics_copy_object_success_length{hostname="VM-8-3-centos"} 0
s3_gateway_metrics_get_key_success_length{hostname="VM-8-3-centos"} 0
s3_gateway_metrics_put_key_success_length{hostname="VM-8-3-centos"} 0

list_key_count
s3_gateway_metrics_list_key_count{hostname="VM-8-3-centos"} 0

What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8662
How was this patch tested?
manual tests

@tanvipenumudy would you like to take a look?


Use the Procedure.code and Procedure.category instead of profile tag

https://github.com/mcode/rttd-summary-webapp/blob/93d0496656cb9c98edbd324aba93c563b7d25a7c/src/mappingUtils.js#L84
Resources are not guaranteed to have profile tags so we should be using the code and category codes to determine when a procedure is of interest.  Or at the very least not looking for whether it is the first profile in the list but one of the profiles on the resource.

This is addressed for specifically TreatmentPhase, but not more holistically. We have another task for addressing this across the board.


Missing Link for Chainsformer Spark Connector in Documentation

I was going through the Chainsformer documentation and noticed that there's a mention of the "Chainsformer Spark Connector" for structured data streaming. However, the documentation also mentions that a link to the Spark Connector is to be added later  (LINK TO BE ADDED LATTER).
Could you please provide the missing link for the Chainsformer Spark Connector or update the documentation with the necessary information?
Thank you!

We are working on it - ETA is end of FEB


chore(GroupTheory/GroupAction/Blocks): correct doc and change one name

Correct two docstrings that were incorrect after refactor
Change one name (by adding of_…) for consistency
Remove one assumption from code (now useless with the new handling of variables)

Add the deprecated alias please

Thanks!
bors merge


High rate of dropped events

I'm running Falco (latest from 3 weeks ago) as a daemonset in one of our staging environments in GKE (1.11.6-gke.6). It's a pool of around 20 nodes using Ubuntu. I'm using BPF probe.
I'm experiencing a very high drop rate of events, at around 72%, on nodes with around 8 of CPU load. On nodes where the CPU load is less (below 1), the drop rate is almost inexistent or even zero.
Also tested in a less busy GKE+COS environment, with BPF enabled, and still got around 12% drop rate for a node with 1.2 of CPU load.
I'm attaching two PDFs with the detailed data that I measured, showing CPU load, memory, disk, pod restarts, drop rate, etc.
I also tested using the kernel probe on the Ubuntu node pool, and the drop rate was actually higher (around 74% on the busier nodes), so not sure if it's totally related to BPF.
Falco Dropped syscalls - GKE + Ubuntu + BPF.pdf
Falco Dropped syscalls - GKE + COS + BPF.pdf

Do you have any data on how many containers per node you run on average and also info on how quickly containers churn? We have a fix going in for how we pull container metadata which we believe is causing the dropped events.

It's around 16 to 20 containers (including kube-system ones) on each node and most of them seem to be long lived (most of them have the same age as the node). There's one pod that have some restarts. I'm attaching a file with more detailed data from kubectl:
data.txt

Just an update on this,
After fixing the k8s metadata issue (https://github.com/falcosecurity/falco/pull/562), I was able to reduce the drop rate to almost zero on the heavy CPU load nodes (0.0001% drop rate).
Was the inability of Falco to reach https://kubernetes.default, the root cause of all these drops?

Have you seen any improvement further with the newer falco builds? We've further fixed pulling container info as well as alerting on drops.
And yes, I am sure the DNS was blocking somewhere which would have caused this.

I haven't noticed much difference. The DNS fix was what really pulled down the number of drops.


targets: Restore example targets

Those were accidentally moved to CI targets but are expected to be used by users.

@mkasenberg FYI


 =>
 transformation

Describe the solution you'd like
It would be great to have
 =>
 transformation - wrapping text blocks surrounded by
 with

Example:
This is the greatest game of all times.

The game provides such features like...

 =>

 This is the greatest game of all times.

 The game provides such features like...

Thank you!


ci: add gh workflow to validate all workflows

This change adds a json schema for github action workflow files and runs validation against this schema.
The schema is taken from https://github.com/cschleiden/github-actions-parser/blob/main/schemas/schema.json and was updated by me to support some newer features (pull_request_target and environment).
This change will help us partially validate workflow files, especially ones that run on pull_request_target event before merging.

Seems like we need to resolve the most recent comment from @aanm before continuing? Will mark as draft.


Bump packages to v3.X.X

Android package updated

iOS package updated
https://github.com/Kebechet/Maui.MicrosoftClarity/pull/8


A problem occurred configuring root project 'android'.

My mistake：
* Error running Gradle:
ProcessException: Process "C:\Users\shunyue\mydemo\flutter_app\android\gradlew.bat" exited abnormally:

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'android'.
> Could not resolve all artifacts for configuration ':classpath'.

My solution：
modification android\build.gradle
buildscript {
    repositories {
        google()
        mavenCentral()   //add
        jcenter()
    }

    dependencies {
        classpath 'com.android.tools.build:gradle:3.2.1'
    }
}

allprojects {
    repositories {
        google()
        mavenCentral()   //add
        jcenter()
    }
}

This is a problem that could be caused by lacking access to the google or jcenter mirrors. It sounds like you solved the issue though?

This is a problem that could be caused by lacking access to the google or jcenter mirrors. It sounds like you solved the issue though?

yes,  That's how I solved it

Coolio. In general there isn't really much we can do about this. If the issue is solved I'm going to close the bug.

I have met the same problem, and my gradle script is:
repositories {
google()
jcenter()
mavenLocal()
}
and I change it to:
repositories {
google()
jcenter()
mavenLocal()
mavenCentral()
}
but the errors still exist.
唉

Hello Shunyue1230, please how did you modify your android/build.gradle as you stated up there. I'm new to coding and even flutter, I'm experiencing the same problem but I don't know how to go about your solution.

@Adamsay93a  answer is above ↑
Between google() and   jcenter()  add to  mavenCentral()
I don’t know if it’s useful now, and it’s been a long time！

@Adamsay93a
If it is because gradle-x.x-all.zip download failed
Go to the gradle-x.x-all.zip to download
Copy to your folder after downloading: C:\Users\yourUserName\.gradle\wrapper\dists\gradle-x.x-all\xxxxxx\ in window
Download the version here：

Hello Shunyue1230, please how did you modify your android/build.gradle as you stated up there. I'm new to coding and even flutter, I'm experiencing the same problem but I don't know how to go about your solution.

For me I solved that issue by downloading gradle and extracting it in the root folder of android studio. and add it to the system variables path and go in android studio > settings > Build, Exception, Deployment. and point it to the folder where you extracted you gradle.  Download gradle from here https://services.gradle.org/distributions/


Adding missing ReadMe.md file for few samples.

Summary
Found few samples without README.md files. So, added README.md files with links to the respective topics for an easy and quick reference of the article.
Following are the list of articles added as references in the README.md files.

https://docs.microsoft.com/en-in/dotnet/csharp/tutorials/intro-to-csharp/introduction-to-classes
https://docs.microsoft.com/en-in/dotnet/csharp/tutorials/intro-to-csharp/branches-and-loops-local
https://docs.microsoft.com/en-in/dotnet/csharp/tutorials/intro-to-csharp/numbers-in-csharp-local
https://docs.microsoft.com/en-in/dotnet/csharp/tutorials/intro-to-csharp/arrays-and-collections

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: balaji-dabbara sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @BillWagner for your valuable suggestions. I have committed your suggestions to remove locale from the URL links.


Problem with 20240422 Release of SDR++

The alternative set of dlls provided by Sultan-papgani that allow for additional AGC and gain settings for the RTL-SDR Blog V4 have worked with past versions of SDR++, but with the latest release of SDR++ on 20240422, the modified dll (new_rtl_sdr_source.dll) causes the program to hang when launching the program in a folder from a prior installation.  If one deletes the json files, the program will start, but the new_rtl_sdr module isn't recognized.  If one tries to create the new module, the program crashes.  I just wanted to make you aware of this issue.  Thank you for such a wonderful enhanced set of dlls.
Ken -- WBØOCV

yeah this repo is old 🤷♂ use my newer  source and delete the old one (for compatibility)

Greetings,
Im still seeing the same behavior with the two most recent updates to SDR++, and both the old and new versions of the modified dlls. Both cause SDR++ to crash when trying ot install the new module.
One other thing I noticed is that the Zip-file for the new version does not contain the rtlsdr.dll file that the old Zip-file did. The instructions state to: "...Replace the rtlsdr.dll with the "librtlsdr.dll".  Do you mean to just copy the librtlsdr.dll into the root folder and delete the rtlsdr.dll file, or to copy the librtlsdr.dll into the root folder and remane it to rtlsdr.dll?  I'm a bit confused on what exactly you mean by "Replace the rtlsdr.dll with the "librtlsdr.dll"
Thank you and 73
Ken -- WBØOCV

i fixed the new(er) source, and no there is no renaming, just put the librtlsdr.dll to the same folder (you can delete the old rtlsdr.dll)


Pin cysignals on macOS

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


2.2.2

Convert SLP address to legacy and cash address

Pull Request Test Coverage Report for Build 1618

0 of 0   changed or added relevant lines in 0 files are covered.
12 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.07%) to 76.986%

Files with Coverage Reduction
New Missed Lines
%

dist/routes/v2/slp.js
12
59.73%

Totals

Change from base Build 1612:
-0.07%

Covered Lines:
1758

Relevant Lines:
2146

💛  - Coveralls

:tada: This PR is included in version 1.14.7 :tada:
Your semantic-release bot :package::rocket:


ValueDecl: endInit restore context use defer

fix https://github.com/goplus/gop/issues/1330

Codecov Report

Merging #207 (5cf5ad9) into main (03f8be9) will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #207      +/-   ##
==========================================
+ Coverage   94.77%   94.81%   +0.04%
==========================================
  Files          12       12
  Lines        5777     5825      +48
==========================================
+ Hits         5475     5523      +48
  Misses        234      234
  Partials       68       68

Impacted Files
Coverage Δ

type_var_and_const.go
98.63% <100.00%> (+<0.01%)
:arrow_up:

package.go
100.00% <0.00%> (ø)

codebuild.go
95.83% <0.00%> (+<0.01%)
:arrow_up:

stmt.go
94.58% <0.00%> (+0.08%)
:arrow_up:

builtin.go
94.43% <0.00%> (+0.22%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 03f8be9...5cf5ad9. Read the comment docs.


Is there a way to dynamically update the options of a line chart

I have a chart in which I would like to dynamically update the scale. Is that possible?
my data is dynamically changed as time goes by and I would like to do the same for the options.
For example, each 500 milliseconds, I would like to change these values:
scaleOverride: true,
scaleSteps: Math.ceil((max-min)/step),
scaleStepWidth: 1
scaleStartValue: a min value dynamically calculated
I would like to avoid creating a new chart each time.
My code is very similar to this: http://jsbin.com/yaxafehixe/1/edit?html,js,output
How should i go about this? Re-create the chart everytime? Or is there a way to change dynamically the options?
Also, on a side note, is there a way to change scaleStepWidth (in order to get only integer values on y axis), without having to change other parameters such as scaleStartValue or scaleSteps. I would just like to have integers on the y axis and don't mind if the starting point and the number of steps are calculated for me.
Thanks very much for your help.

@c-desmarais I think you would need to find the options that were merged with the global options. They should be stored on the chart instance but I'm not sure of the variable name

i am trying to do something like myChart.scaleStartValue = some value;
but it doesn't seem to work, am i doing something wrong?

its probably something like: I'm not sure
myChart.options.scaleStartValue = some value;

no it does not work either :(

@c-desmarais here's a value that seems to work, but it's not ideal. http://jsbin.com/rabado/1/edit?html,js,output

oh great! well. that is indeed better than nothing :) thanks so much! However, where can i look for these values? For instance, if i want to change scaleStartValue, how do i go about this... ? where do i have to look in the chartjs code or the docs to see what is available under myChart.scale.someOptions = ...

@c-desmarais I found it by inspection in the debugger. Unfortunately, V1 is lacking in internal docs for this kind of stuff :(

@etimberg Oh i see. thanks very much!  do you think i should change for version 2 then ? Would it be possible to change the chart options more easily? (Because i thought the docs for version 1 are more complete than for version 2.. (I mean the whole chartjs site with the docs on it) but please correct me If i am wrong Here .

@c-desmarais if you're just starting out using Chart.js I would recommend v2. If you already have a lot of stuff using v1, it may be best to stick with it as the upgrade is a bit messy. The docs on the site are for v2.

@etimberg :+1:  Thank you very much for your help ! I really appreciate it, i've been stuck trying to figure this out all day believe me or no lol . But I am quite a newbie with chartjs here I must admit.  Anyways, I've rearranged my chart with v2 as you recommended and here is a jsbin in case it can help anybody else. It now works as intended.
Jsbin :
http://jsbin.com/bamemuliyu/3/edit?html,js,output

Closing as duplicate of #2060 which asks for the same request


playbook explanation and generation UI

This PR introduces the early UI for two new features for Lightspeed. The code is not fully operational yet, but we would like to start to merge the elements that we feel are ready.
These new UI are not exposed to the users.
This PR also introduces a new LSP server that is dedicated to Lightspeed. Our goal is to use this server as an interface between the editor and the AI/model interaction.
In the future, we would like to:

also move into this server the functions that are used for the task and multiline predictions.
consolidate everything in the existing LSP server (ansible-language-server)

Co-Author: Tami Takamiya<EMAIL_ADDRESS>

@ganeshrn @ssbarnea this PR add our new UI, there is already quiet a lot of changes, but nothing is enabled by default. We would like to get this PR land to avoid building a gigantic PR that we will have to rebase again and again. The PR slightly reduces the coverage, we are aware of that and already working on new vscode-extension-tester tests.


fix: Fix getting branch name function

Pull Request Test Coverage Report for Build #168
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 80.568%

Totals

Change from base Build #167:
0.0%

Covered Lines:
369

Relevant Lines:
458

💛  - Coveralls


Fixes #2077

Workaround IE text node splitting issue that can make text bindings fail. The behavior is such that node.normalize() fails and so we address adjacent text nodes manually.

LGTM