Fix static provisioning edge cases

Description of your changes

Fixes #28
Quoting the above issue:

There are three scenarios we need to make sure the claim reconciler handles:

Dynamic provisioning. No managed resource exists. A resource claim is created with a portable class reference and without a managed resource reference. In this case the claim reconciler should dynamically provision and then bind to a managed resource.
Resource-first static provisioning. A managed resource is created explicitly, without using a resource class. Later (or simultaneously), a resource claim that references (via its resource reference) that managed resource is created. We should skip dynamic provisioning and go straight to binding.
Claim-first static provisioning. A resource claim is created that references (via its resource reference) a managed resource that does not yet exist. We should fail to reconcile that resource claim until such time as the referenced managed resource is created (out of band) and becomes bindable, at which point we should bind to it.

While investigating I found that scenario 3 lead to a nil pointer exception, and scenario 2 was partially broken; static provisioning would only work when the statically provisioned managed resource was available for binding at the time the claim was submitted. If the resource was still being created when the claim was submitted it would never bind. I also realised we were susceptible to a nil pointer in a pathological case in which a managed resource referenced a claim that had neither a resource reference or a class reference. I believe this PR fixes all of these issues.
Checklist

I have:

[x] Run make reviewable to ensure this PR is ready for review.
[x] Ensured this PR contains a neat, self documenting set of commits.
[ ] Updated any relevant documentation, examples, or release notes.
[ ] Updated the RBAC permissions in clusterrole.yaml to include any new types.

$ make reviewable test build
14:52:24 [ .. ] go mod vendor
14:52:25 [ OK ] go mod vendor
14:52:25 [ .. ] go generate darwin_amd64
apis/core/v1alpha1/zz_generated.deepcopy.go
14:52:26 [ OK ] go generate darwin_amd64
14:52:26 [ .. ] verify dependencies have expected content
all modules verified
14:52:30 [ OK ] go modules dependencies verified
14:52:30 [ .. ] golangci-lint
14:52:38 [ OK ] golangci-lint
14:52:38 [ .. ] verify dependencies have expected content
all modules verified
14:52:40 [ OK ] go modules dependencies verified
14:52:40 [ .. ] go test unit-tests
?       github.com/crossplaneio/crossplane-runtime/pkg/logging  [no test files]
ok      github.com/crossplaneio/crossplane-runtime/pkg/meta     0.257s  coverage: 100.0% of statements
ok      github.com/crossplaneio/crossplane-runtime/pkg/resource 0.287s  coverage: 93.9% of statements
?       github.com/crossplaneio/crossplane-runtime/pkg/test     [no test files]
ok      github.com/crossplaneio/crossplane-runtime/pkg/util     0.370s  coverage: 78.6% of statements
?       github.com/crossplaneio/crossplane-runtime/apis [no test files]
ok      github.com/crossplaneio/crossplane-runtime/apis/core/v1alpha1   0.302s  coverage: 27.3% of statements
14:52:45 [ OK ] go test unit-tests
14:52:46 [ .. ] verify dependencies have expected content
all modules verified
14:52:47 [ OK ] go modules dependencies verified
14:52:47 [ .. ] golangci-lint
14:52:54 [ OK ] golangci-lint
14:52:54 [ .. ] go build linux_amd64
14:52:54 [ OK ] go build linux_amd64

Have you had the chance to test it out with a real scenario?

I have. I built stack-gcp against this branch and tested that all scenarios discussed in this issue work as expected.

@negz awesome! I am comfortable with this being merged :) also, we will need to bump dependencies in all stacks


Able to set Debug option for each HelmChartProxy individually

User Story
Right now, debug option can be enabling for whole controller and components centrally as running flag. And it's not possible to setup it for each HelmChartProxy, but in some cases, whole controller reconfiguration are not allowing.
As a user, I would like to enable debug option for each HelmChartProxy individually like a helm cli command, i.e.
helm install -f myvalues.yaml myredis ./redis --debug

Detailed Description
For this case, I suggesting to create an optional Debug (bool) field inside the HelmOptions API struct, propagate it into the HelmInit method and set log level to 2 (default) if this option is enabled.
/kind feature

/triage-accepted
Will take a look on Thurs.

Thanks for accepting, @Jont828
Was planning to fix it soon

/assign

I took a closer look at your issue, and I think passing the --debug flag in to helm install or helm upgrade for an individual HelmChartProxy instance makes sense. Only thing is that I'm not sure if we can set the log level on the CRD level, since that gets set by the controller args for the whole container when the caaph-controller-manager pod gets created, not on each reconciliation loop.


Minor improvements (mainly to the UI)

Hey Azagaya! Thanks for releasing the project! It looks quite interesting and I will need to take a closer look at how it all works soon, especially the shader for making the normal map.
I noticed the plugin doesn't resize nicely with different windows, so I made some changes to the UI so it scales better with different sized windows. It is not perfect, but it resizes a little bit better, in my opinion. While I was making the UI changes, I kinda got carried away and added some additional functionality.
Changes are as follows:

Removed the VBoxContainer and replaced it with a normal Control node called GUI.

This allowed for better control over how the plugin changes with the window size.

Both the Normal Map Generate node and the GUI node have their layouts set to Full Rect so they will resize to take the entire available plugin window space.
All the children of the GUI node now have anchors set to either to Top Wide or Bottom Wide so they resize correctly.

One exception is the ViewportContainer node, as I couldn't get it resizing correctly. It's anchor is set to  Center Top

Added a new HBoxContainer for setting the color of the Light2D node.

The color_changed signal is connected from the ColorPickerButton to a function that sets the color of the Light2D Node.
Changed _ready so that it sets the color of the button to the default Light2D node color.

Added the _input function to Normal Map Generator.gd.

Now when a click is detected within the ViewportContainer node, the Light2D node will be moved to the cursor position.
Changed the cursor icon showed when over the ViewportContainer node to a Cross (was an arrow)
This should work with any sized Viewport/Texture, but I didn't do extensive testing.

Added a couple new class variables to Normal Map Generator.gd
Other small improvements/changes I forgot... I think that is all of them, but I wasn't writing this done as I went, so there might be a few I missed.

I can separate these additions into different pull requests if you want, to make it easier to manage and so changes can be added/removed.
One minor thing I noticed is that the plugin really slows down when a large texture is loaded. I loaded a 2048x2048 texture into the plugin as a test and it made the editor run really slow. I think it probably has to do with the pixel density of the texture, but I don't know for sure.
This probably isn't a huge problem, I doubt many people are inputting 2048x2048 textures into the plugin, but I thought I'd mention it before I forgot.
Regardless, hopefully these changes are helpful. 🙂

Hi! thanks a lot!! Indeed, i have little to zero experience with gui in godot, so i needed someone to help me. I'll try the changes now.

Added the _input function to Normal Map Generator.gd.
Now when a click is detected within the ViewportContainer node, the Light2D node will be moved to the cursor position.
Changed the cursor icon showed when over the ViewportContainer node to a Cross (was an arrow)
This should work with any sized Viewport/Texture, but I didn't do extensive testing.

Well, i tried to do the same thing, but i could not manage to get it working in the dock.

One minor thing I noticed is that the plugin really slows down when a large texture is loaded. I loaded a 2048x2048 texture into the plugin as a test and it made the editor run really slow.

Yes, this is why i mainly said in my tweets that it was just for testing.. The shader is calculating everything (distance, blur, normals) every frame, and this makes no sense, cause once the normal is calculated, the next frame should be the same unless some controls change.. i couldn't find any way to only render once (tried RENDER_ONCE with no luck).


Encoder-pio not working with more than one encoder

Hello, I have been testing the code available in the branch encoder-pio and I have run into a few issues. I am not sure it is still under development, since the branch has been untouched for a while, but I'd like to address them anyway.
It seems to me that the code was written to allow more than one encoder to be defined (for example in the case of a robot with two wheels), but I believe there are some issues in that case. In my case I am specifically working with two encoders.
In the following line there is an if condition when the first encoder is defined:
https://github.com/pimoroni/pimoroni-pico/blob/a15646d922b185622036ce81423f30c256e5645d/drivers/encoder-pio/encoder.cpp#L87
However, I believe the two following lines inside this if should be called for each encoder, which is taken care by a single state machine:
https://github.com/pimoroni/pimoroni-pico/blob/a15646d922b185622036ce81423f30c256e5645d/drivers/encoder-pio/encoder.cpp#L90
https://github.com/pimoroni/pimoroni-pico/blob/a15646d922b185622036ce81423f30c256e5645d/drivers/encoder-pio/encoder.cpp#L91
To solve that, I put those two lines outside of the if condition to force them to be executed for each encoder's initialization.
But it still did not solve my problem...
I tried to check if the calls for the hw_set_bits(&enc_pio->inte0, 1u << enc_sm) function where ok and it seems to work fine, which means both state machines should generate an interrupt that would call the interrupt handler, but that is not the case. For some reason, it seems that setting a second state machine sharing the same code pio code defined in encoder.pio as the first state machine will cause the first one to not generate interruptions, or maybe not work at all.. I still haven't been able to  figure out which is the case.
Has anyone ever tried this implementation with more than one encoder?

Hi, thanks for raising this! You are correct that this branch hasn't received attention in a long while, and I honestly cannot remember if I ever tested it with multiple encoders.
I am planning to revisit it now we're in the new year, as we may have a product on the horizon that requires it, so knowing that this is broken is a good thing for me to look at first.

Hey @guilhermebene. Sorry it has taken so long, but I have finally had chance to look into this.
There were a bunch of issues with the code, but the main reason why only a single encoder would run is because hw_set_bits(&enc_pio->inte0, 1u << enc_sm) was being called within the if(pio_claimed_sms[pio_idx] == 0) {  statement, meaning it only got registered for the first SM (whichever that may have been). There were also thing wrong with setting up both PIOs as well as tearing them down at the end. These "should" be fixed now.
I am giving the rest of the encoder code a thorough look over as I polish it up for release, so class and function names and behaviour may change going forward, but if you want to have a play around over the weekend, I would welcome your feedback.
The latest code is within the encoder-pio branch. I also rebased it from main so it has all our other libraries and improvements, if that's useful to you.

Hi @guilhermebene. I forgot to reply to this, but the encoder support was rolled into our libraries back in the summer to coincide with the launch of Motor 2040. This fixed the issues with running multiple encoder instances at once, so worth you having a look at if it's still relevant to your project.


Suggest a feature: Option to click on the data point

I would like to be able to click on a data point and the details just pops up similar to the hovering except the pop-up just stays on the screen until you close it. Sometimes hovering alone isn't enough. If this happens, it will be nice to be able to move that pop-up box.
If we already have that feature, I apologize, iono how to use it. But if we don't, it would be nice to add it.

I'm assuming you're using the Bokeh backend?
Just leaving this here so we can track the requests for this across multiple libraries.

It's not exactly the same thing, but soon you will be able to define a custom persistent popup in HoloViews.
https://github.com/holoviz/holoviews/pull/6168


SQS Transport Capability

Hello:
This PR adds a SQSTransport class to the Transports.
What does this do?
Instead of forwarding messages directly to Sentry, it forwards them over to an SQS queue. Then, an SQS poller pops off the messages before sending them onto Sentry.
I just created some simple code on GitHub here that does this: https://github.com/Netflix-Skunkworks/raven-sqs-proxy
Why is this useful?
The primary use case of this is with AWS Lambda functions. For lambda functions to write to an AWS Sentry instance, it would need to reside within a VPC. Running lambdas inside of a VPC requires that the lambda be assigned an ENI.
There are many use-cases where running a lambda within a VPC is not feasible, such as:

An AWS account doesn't have a useful VPC (special purpose accounts)
An AWS account doesn't have a VPC that is peered to a VPC where Sentry is running
Cross-region use cases where Sentry lives in an internal VPC without external connectivity
ENI Exhaustion Concerns: It is possible to exhaust the ENIs within a VPC if you have many, many lambdas running. This can break new deployments within a VPC.

For any of the above use cases, SQS alleviates the drawbacks. In this case, the lambda is still able to take advantage of Sentry by posting to an SQS queue. A VPC ENI need not be required. Security to the SQS queue is handled via IAM Roles.
Potential Concern
This PR simply passes the headers with the DSN credentials onto SQS. If this is a concern, I can add a KMS option to encrypt credentials before passing onto SQS.
Please let me know if there are additional changes that can be made or if you have any questions about my approach.
Thank You

I'd personally rather this be a separate library that just depended on the raven package. It's a pattern we're taking in a lot of situations to avoid too tightly coupling things into the core lib that aren't heavily used.
Aside, we intend to solve the 'async' issue natively at some point with a uniform binary you can install on a system.

(also if this was talked about elsewhere, ignore me, i just happened to be paying attention to my inbox tonight)

I went ahead and moved this plugin over to our raven-python-lambda project here in this PR: https://github.com/Netflix-Skunkworks/raven-python-lambda/pull/14
There was an initial discussion taking place here: https://github.com/Netflix-Skunkworks/raven-python-lambda/issues/7 with regards to raven incorporating much of that logic.
Not sure what additional complexity is added, but if raven were to take in Lambda support, it should also take in SQS support for the reasons mentioned above.

@mikegrima its more around us having to maintain it. lambda I think makes a bit more sense given the requirement of it, but if e.g. sqs is required for lambda, thats a different story. Just my two cents, as I'm not actively involved in these projects

Closing this as we have this implemented in our raven-python-lambda project.


Rotation scan ispyb comment should contain metadata

Currently the comment in ispyb for a rotation scan is fixed:

We should add the same metadata that's in teh comment for GDA:

That is:

Motor position
Some user defined comment - which comes in through UDC
Xray centre box selected
Aperture

Requires https://jira.diamond.ac.uk/browse/I03-922

Will be easier if we do https://github.com/DiamondLightSource/hyperion/issues/1032 first

I don't think this has been started yet so I'm going to put it back on the backlog

To do this:

Confirm that we get comment in from GDA
Confirm that we're using the comment param properly
Get the position as we have in https://github.com/DiamondLightSource/hyperion/pull/1411
Get the aperture name as read from the device


TestYaegiCmdCancel is failing on non 48-bit VMA

Hello there, I'm the Debian maintainer for the yaegi package :)
While building the package, our builders have reported failure for the test TestYaegiCmdCancel on some archs  (see the package build status).
This is because of the -race flag being used on non 48-bit VMA.
See: https://github.com/traefik/yaegi/blob/451c75406813ea990296abb9f423ae87390f91d6/cmd/yaegi/yaegi_test.go#L52
I think the -race flag should only be append on 48-bit VMA (linux/amd64, freebsd/amd64, darwin/amd64, windows/amd64, linux/ppc64le and linux/arm64)
Bug has been reported in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=992941

I'll submit a PR asap.
Cheers!

Here's the patch. I've tried building yaegi again on armhf and it looks like the build is passing :)
I'll upload a new version of yaegi on the Debian archive to see if it solves the issue on all arches.
Cheers,


Documentation updates to recognise class renaming

Acceptance criteria

Classes in Markdown and PHPDoc documentation are using the correct names and namespaces

From Ingo's comments on #5734:
We need to adjust heaps of markdown docs - is there a follow up task for this? It's going to be at least 50% manual I'd say.
PHPDoc follows the same namespace rules as PHP which means we don't need to fix @param, @return and such. But: {@link MyClassName} can link to any class (not required to be listed in use statements), so we need to go through those. Could be deferred, as long as its tracked in a ticket?

In terms of the markdown docs that use the API shortcode, there could be some that could be updated in bulk search and replace operations e.g. [api:DataExtension] -> [api:SilverStripe\ORM\DataExtension]. I've raised an issue to track the required changes to handle this as well: https://github.com/silverstripe/silverstripe-docsviewer/issues/132

@robbieaverill has written a fix to the api.ss.org lookup: https://github.com/silverstripe/api.silverstripe.org/pull/38. Consider how this works before renaming [api:MyClass] notations

@chillu so the way that is designed is that existing [api:MyClass] notations will be run through the combined core module upgrader mappings and potentially be namespaced before being used, but you could still write [api:My\FCQN\ClassName] if you wanted to - this would allow a more gradual transition for changing these in the docs

Found in https://github.com/silverstripe/silverstripe-framework/issues/7068; Don't forget to include changelog (4.0.0.md) in your search. :P


Auto-and Predicates

See section 6.1.3 "Auto-AND predicates" in the "Qualcomm Hexagon V66 Programmer’s Reference Manual"

If multiple compare instructions in a packet write to the same predicate register, the result is the logical AND of the individual compare results.

This is not handled in HexagonPcodeEmitPacked, and will silently result in only the last assignment to P3:0 being used in a subsequent compare

If only instructions of the form PdV = cmp.XX(...) are allowed, then these are all the relevant opcodes:
C2_cmpeq
C2_cmpgt
C2_cmpgtu
C2_cmpeqp
C2_cmpgtp
C2_cmpgtup
C2_cmpeqi
C2_cmpgti
C2_cmpgtui
A4_cmpbeq
A4_cmpbeqi
A4_cmpbgtu
A4_cmpbgtui
A4_cmpbgt
A4_cmpbgti
A4_cmpheq
A4_cmphgt
A4_cmphgtu
A4_cmpheqi
A4_cmphgti
A4_cmphgtui

https://github.com/qemu/qemu/blob/d940d468e29bff5eb5669c0dd8f3de0c3de17bfb/tests/tcg/hexagon/misc.c has some good examples of auto-and predicates

Qemu handles auto-and predicates like so https://github.com/qemu/qemu/blob/aeb0ae95b7f18c66158792641cb6ba0cde5789ab/target/hexagon/translate.c#L356


Please add support for dns names in bootnodes records

Describe the feature
For the --bootnodes flag, it would be really nice to support dns names instead of just ip addresses.
For the berachain bArtio testnet, an example EL bootnode:
enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303

To use this with reth, we have to convert all of the hostnames to ip addresses. Note that we are using v0.2.0-beta.4 and have not had a chance to upgrade yet.
Additional context
No response

We recently added this support in beta.9:
https://github.com/paradigmxyz/reth/pull/8188
Highly recommend upgrading to the latest version!

@Rjected seems to not work when specified in the configuration file. One of my coworkers can give you more details.

Hi! We've upgrade to reth v1.0.0 (🎉 congrats!) and can now use this feature! But I am still having some problems with it still.
Here is it working:
start.sh
#!/bin/sh

set -e

curl -L https://raw.githubusercontent.com/berachain/beacon-kit/988cfc795108c81e30e4c98e18da36cdcfff746f/testing/networks/80084/eth-genesis.json -o ~/genesis.json
openssl rand -hex 32 | tr -d '\n' > /tmp/secret.hex

cat > ~/reth.toml <<EOF
EOF

/usr/local/bin/reth node \
    --full \
    --chain ~/genesis.json \
    --datadir ~ \
    --config ~/reth.toml \
    --addr <IP_ADDRESS> \
    --port 30303 \
    --http \
    --http.addr <IP_ADDRESS> \
    --http.port 8545 \
    --http.api all \
    --authrpc.addr <IP_ADDRESS> \
    --authrpc.port 8552 \
    --authrpc.jwtsecret /tmp/secret.hex \
    --metrics "<IP_ADDRESS>:9001" \
    --trusted-peers "enode://0401e494dbd0c84c5c0f72adac5985d2f2525e08b68d448958aae218f5ac8198a80d1498e0ebec2ce38b1b18d6750f6e61a56b4614c5a6c6cf0981c39aed47dc@<IP_ADDRESS>:30303,enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303" \

This runs and peers successfully:
logs
$ ./start.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7232  100  7232    0     0  51410      0 --:--:-- --:--:-- --:--:-- 51657
2024-07-03T22:00:15.762067Z  INFO Initialized tracing, debug log directory: /data/reth/.cache/reth/logs/80084
2024-07-03T22:00:15.763933Z  INFO Starting reth version="1.0.0 (83d412d)"
2024-07-03T22:00:15.764569Z  INFO Opening database path="/data/reth/db"
2024-07-03T22:00:15.907568Z  INFO Saving prune config to toml file
2024-07-03T22:00:15.907756Z  INFO Configuration loaded path="/data/reth/reth.toml"
2024-07-03T22:00:15.907780Z  INFO Adding trusted nodes
2024-07-03T22:00:15.935638Z  INFO Verifying storage consistency.
2024-07-03T22:00:15.939323Z  INFO Database opened
2024-07-03T22:00:15.939335Z  INFO Starting metrics endpoint addr=<IP_ADDRESS>:9001
2024-07-03T22:00:15.939552Z  INFO
Pre-merge hard forks (block based):
- Homestead                        @0
- Dao                              @0
- Tangerine                        @0
- SpuriousDragon                   @0
- Byzantium                        @0
- Constantinople                   @0
- Petersburg                       @0
- Istanbul                         @0
- MuirGlacier                      @0
- Berlin                           @0
- London                           @0
- ArrowGlacier                     @0
- GrayGlacier                      @0
Merge hard forks:
- Paris                            @0 (network is known to be merged)
Post-merge hard forks (timestamp based):
- Shanghai                         @0
- Cancun                           @0
2024-07-03T22:00:15.939935Z  INFO Transaction pool initialized
2024-07-03T22:00:15.940060Z  INFO Loading saved peers file=/data/reth/known-peers.json
2024-07-03T22:00:15.941568Z  INFO StaticFileProducer initialized
2024-07-03T22:00:15.941907Z  INFO Pruner initialized prune_config=PruneConfig { block_interval: 5, segments: PruneModes { sender_recovery: Some(Full), transaction_lookup: None, receipts: None, account_history: Some(Distance(10064)), storage_history: Some(Distance(10064)), receipts_log_filter: ReceiptsLogPruneConfig({}) } }
2024-07-03T22:00:15.942088Z  INFO Consensus engine initialized
2024-07-03T22:00:15.942132Z  INFO Engine API handler initialized
2024-07-03T22:00:15.944685Z  INFO RPC auth server started url=<IP_ADDRESS>:8552
2024-07-03T22:00:15.944733Z  INFO RPC IPC server started path=/tmp/reth.ipc
2024-07-03T22:00:15.944740Z  INFO RPC HTTP server started url=<IP_ADDRESS>:8545
2024-07-03T22:00:15.944750Z  INFO Starting consensus engine
2024-07-03T22:00:18.948572Z  INFO Status connected_peers=2 freelist=1806678 latest_block=980558

If I put the IPv4 address in to reth.toml rather than on the command line, it also peers:
--- start.sh	2024-07-03 22:01:27.727068910 +0000
+++ start.sh	2024-07-03 22:02:27.149421127 +0000
@@ -6,6 +6,11 @@
 openssl rand -hex 32 | tr -d '\n' > /tmp/secret.hex

 cat > ~/reth.toml <<EOF
+[peers]
+# A list of ENRs for trusted peers, which are peers reth will always try to connect to.
+trusted_nodes = [
+    "enode://0401e494dbd0c84c5c0f72adac5985d2f2525e08b68d448958aae218f5ac8198a80d1498e0ebec2ce38b1b18d6750f6e61a56b4614c5a6c6cf0981c39aed47dc@<IP_ADDRESS>:30303",
+]
 EOF

 /usr/local/bin/reth node \
@@ -23,18 +28,10 @@
     --authrpc.port 8552 \
     --authrpc.jwtsecret /tmp/secret.hex \
     --metrics "<IP_ADDRESS>:9001" \
-    --trusted-peers "enode://0401e494dbd0c84c5c0f72adac5985d2f2525e08b68d448958aae218f5ac8198a80d1498e0ebec2ce38b1b18d6750f6e61a56b4614c5a6c6cf0981c39aed47dc@<IP_ADDRESS>:30303,enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303" \
+    --trusted-peers "enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303" \

start.sh
#!/bin/sh

set -e

curl -L https://raw.githubusercontent.com/berachain/beacon-kit/988cfc795108c81e30e4c98e18da36cdcfff746f/testing/networks/80084/eth-genesis.json -o ~/genesis.json
openssl rand -hex 32 | tr -d '\n' > /tmp/secret.hex

cat > ~/reth.toml <<EOF
[peers]
# A list of ENRs for trusted peers, which are peers reth will always try to connect to.
trusted_nodes = [
    "enode://0401e494dbd0c84c5c0f72adac5985d2f2525e08b68d448958aae218f5ac8198a80d1498e0ebec2ce38b1b18d6750f6e61a56b4614c5a6c6cf0981c39aed47dc@<IP_ADDRESS>:30303",
]
EOF

/usr/local/bin/reth node \
    --full \
    --chain ~/genesis.json \
    --datadir ~ \
    --config ~/reth.toml \
    --addr <IP_ADDRESS> \
    --port 30303 \
    --http \
    --http.addr <IP_ADDRESS> \
    --http.port 8545 \
    --http.api all \
    --authrpc.addr <IP_ADDRESS> \
    --authrpc.port 8552 \
    --authrpc.jwtsecret /tmp/secret.hex \
    --metrics "<IP_ADDRESS>:9001" \
    --trusted-peers "enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303" \

logs
$ ./start.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7232  100  7232    0     0   146k      0 --:--:-- --:--:-- --:--:--  147k
2024-07-03T22:03:12.205065Z  INFO Initialized tracing, debug log directory: /data/reth/.cache/reth/logs/80084
2024-07-03T22:03:12.206754Z  INFO Starting reth version="1.0.0 (83d412d)"
2024-07-03T22:03:12.207376Z  INFO Opening database path="/data/reth/db"
2024-07-03T22:03:12.311714Z  INFO Saving prune config to toml file
2024-07-03T22:03:12.311914Z  INFO Configuration loaded path="/data/reth/reth.toml"
2024-07-03T22:03:12.311937Z  INFO Adding trusted nodes
2024-07-03T22:03:12.341205Z  INFO Verifying storage consistency.
2024-07-03T22:03:12.344997Z  INFO Database opened
2024-07-03T22:03:12.345009Z  INFO Starting metrics endpoint addr=<IP_ADDRESS>:9001
2024-07-03T22:03:12.345237Z  INFO
Pre-merge hard forks (block based):
- Homestead                        @0
- Dao                              @0
- Tangerine                        @0
- SpuriousDragon                   @0
- Byzantium                        @0
- Constantinople                   @0
- Petersburg                       @0
- Istanbul                         @0
- MuirGlacier                      @0
- Berlin                           @0
- London                           @0
- ArrowGlacier                     @0
- GrayGlacier                      @0
Merge hard forks:
- Paris                            @0 (network is known to be merged)
Post-merge hard forks (timestamp based):
- Shanghai                         @0
- Cancun                           @0
2024-07-03T22:03:12.345611Z  INFO Transaction pool initialized
2024-07-03T22:03:12.345708Z  INFO Loading saved peers file=/data/reth/known-peers.json
2024-07-03T22:03:12.347179Z  INFO StaticFileProducer initialized
2024-07-03T22:03:12.347437Z  INFO Pruner initialized prune_config=PruneConfig { block_interval: 5, segments: PruneModes { sender_recovery: Some(Full), transaction_lookup: None, receipts: None, account_history: Some(Distance(10064)), storage_history: Some(Distance(10064)), receipts_log_filter: ReceiptsLogPruneConfig({}) } }
2024-07-03T22:03:12.347608Z  INFO Consensus engine initialized
2024-07-03T22:03:12.347645Z  INFO Engine API handler initialized
2024-07-03T22:03:12.349878Z  INFO RPC auth server started url=<IP_ADDRESS>:8552
2024-07-03T22:03:12.349925Z  INFO RPC IPC server started path=/tmp/reth.ipc
2024-07-03T22:03:12.349934Z  INFO RPC HTTP server started url=<IP_ADDRESS>:8545
2024-07-03T22:03:12.349942Z  INFO Starting consensus engine
2024-07-03T22:03:15.354643Z  INFO Status connected_peers=2 freelist=1806678 latest_block=980558

But if I put the DNS name into the config file, it fails with

Error: Could not load config file "/data/reth/reth.toml"
1: Failed to parse url: invalid IPv4 address syntax

--- start.sh	2024-07-03 22:04:58.031393273 +0000
+++ start.sh	2024-07-03 22:04:27.246174805 +0000
@@ -10,6 +10,7 @@
 # A list of ENRs for trusted peers, which are peers reth will always try to connect to.
 trusted_nodes = [
     "enode://0401e494dbd0c84c5c0f72adac5985d2f2525e08b68d448958aae218f5ac8198a80d1498e0ebec2ce38b1b18d6750f6e61a56b4614c5a6c6cf0981c39aed47dc@<IP_ADDRESS>:30303",
+    "enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303",
 ]
 EOF

@@ -28,5 +29,8 @@
     --authrpc.port 8552 \
     --authrpc.jwtsecret /tmp/secret.hex \
     --metrics "<IP_ADDRESS>:9001" \
-    --trusted-peers "enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303" \

start.sh
#!/bin/sh

set -e

curl -L https://raw.githubusercontent.com/berachain/beacon-kit/988cfc795108c81e30e4c98e18da36cdcfff746f/testing/networks/80084/eth-genesis.json -o ~/genesis.json
openssl rand -hex 32 | tr -d '\n' > /tmp/secret.hex

cat > ~/reth.toml <<EOF
[peers]
# A list of ENRs for trusted peers, which are peers reth will always try to connect to.
trusted_nodes = [
    "enode://0401e494dbd0c84c5c0f72adac5985d2f2525e08b68d448958aae218f5ac8198a80d1498e0ebec2ce38b1b18d6750f6e61a56b4614c5a6c6cf0981c39aed47dc@<IP_ADDRESS>:30303",
    "enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303",
]
EOF

/usr/local/bin/reth node \
    --full \
    --chain ~/genesis.json \
    --datadir ~ \
    --config ~/reth.toml \
    --addr <IP_ADDRESS> \
    --port 30303 \
    --http \
    --http.addr <IP_ADDRESS> \
    --http.port 8545 \
    --http.api all \
    --authrpc.addr <IP_ADDRESS> \
    --authrpc.port 8552 \
    --authrpc.jwtsecret /tmp/secret.hex \
    --metrics "<IP_ADDRESS>:9001" \

logs
$ ./start.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7232  100  7232    0     0  53562      0 --:--:-- --:--:-- --:--:-- 53970
2024-07-03T22:06:02.214560Z  INFO Initialized tracing, debug log directory: /data/reth/.cache/reth/logs/80084
2024-07-03T22:06:02.216271Z  INFO Starting reth version="1.0.0 (83d412d)"
2024-07-03T22:06:02.216889Z  INFO Opening database path="/data/reth/db"
2024-07-03T22:06:02.304013Z ERROR shutting down due to error
Error: Could not load config file "/data/reth/reth.toml"

Caused by:
   0: Bad TOML data
   1: TOML parse error at line 5, column 5
   1:   |
   1: 5 |     "enode://e9675164b5e17b9d9edf0cc2bd79e6b6f487200c74d1331c220abb5b8ee80c2eefbf18213989585e9d0960683e819542e11d4eefb5f2b4019e1e49f9fd8fff18@berav2-bootnode.staketab.org:30303",
   1:   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   1: Failed to parse url: invalid IPv4 address syntax

Location:
    /reth-v1.0.0/crates/node/builder/src/launch/common.rs:90:14

Version:
$ reth --version
reth Version: 1.0.0
Commit SHA: 83d412d
Build Timestamp: 2024-07-02T15:19:26.620030609Z
Build Features: jemalloc
Build Profile: release

I realize I'm using --trusted-peers instead of --bootnodes; I'm not entirely clear what the difference is, if I'm honest; also there doesn't seem to be an equivalent to --bootnodes in the config file. But regardless, please may I request DNS names in both trusted-peers and bootnodes, no matter how they are passed in.
This isn't urgent -- we can live with using the CLI -- but it would help tidy up our configs!
Thanks for all the work you do.  You're pushing chain performance far!

So, it turns out this is against spec

The hostname can only be given as an IP address; DNS names are not allowed.

Too bad, it's super useful.

So, it turns out this is against spec

The hostname can only be given as an IP address; DNS names are not allowed.

Too bad, it's super useful.

@kousu  I think this would be a reasonable ask for the config, despite the spec. Do you mind creating an issue and copy-paste the message above containing the diffs / scripts etc? By the way, I really appreciate the detail in your message explaining the expected behavior!


Fix minor style issues

Fix minor style issues

Coverage remained the same at 54.995% when pulling 448555e952b42ce0b7f6f43a3b4effb205160ca6 on steven-zou:dev into 8a5f024a1acea1a6cd8e58a671482fed74566a7b on vmware:dev.


How to add two floating values

{{ order.get_total | add:7.99 }}
where order.get_total is 12.99 . I am trying to add 12.99 with 7.99 but while adding getting an output of 19

What data type is order.get_total?

Oh, by the way, add is not even part of django-mathfilters, it's a django built-in and always coerces values to integers. You might want addition instead. See the README.


[AutoPR billing/resource-manager] chore: jsonfmt billing

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/5923

This PR has been merged into https://github.com/Azure/azure-sdk-for-js/pull/2888


Siproxd plugin

Siproxd plugin as requested in #246
In #246 are some screenshots how it looks like.
Current problem are the "Current registrations", the newlines are not correctly formatted. Perhaps you have some ideas? I could als try to write a script but a oneliner would be smarter I think.
setup.sh needs a chmod 755 as always when I contribute :)

you can use <textarea> or <pre> for monospace output

I removed the WIP, if you don't find anything else we could merge and let @NicholasRush test the whole thing.

Merged, have some time, will fix my comments in the tree


Handle gracefully raise X from a missing dependency

Fixes https://github.com/ManderaGeneral/generalimport/issues/4

The PR changes the behavior of the following snippet (using langdetect as my missing dependency):
from generalimport import generalimport
generalimport("langdetect")

from langdetect import LangDetectException as MyException
raise MyException

from
TypeError: exceptions must derive from BaseException

to:
Traceback (most recent call last):
  File "/path-to-project/snippet.py", line 44, in <module>
    raise MyException
generalimport.exception.missing_exception.<locals>.MissingOptionalDependencyException: Optional dependency langdetect (required by 'LangDetectException') was used but it isn't installed.

It also handles a different scenario (the one that led me to this bug to begin with) which looks like this:
from generalimport import generalimport
generalimport("langdetect")

from langdetect import LangDetectException, detect

try:
    language = detect(document)
except LangDetectException:
    logger.warning("Langdetect cannot detect the language of document: %s", document)

If langdetect was not installed, the except LangDetectException: call would raise:
TypeError: catching classes that do not inherit from BaseException is not allowed

This scenario now also raises:
generalimport.exception.missing_exception.<locals>.MissingOptionalDependencyException: Optional dependency langdetect (required by 'LangDetectException') was used but it isn't installed.

Probably not the best error name, but at least can be catched and improved upon.
I added no tests for now but I believe there should be. I'll wait for you to review the changes and maybe to give  me a few hints how you want the tests to look like.

Relying on the name is an interesting idea! If it wasn't for PEP8 I wouldn't have been too excited about it, would have preferred relying on some magical dunder method that's only called by raise, alas, such a method probably doesn't exist, so this could work!
Using item in the error message is a nice touch
However, I'm not too excited about the other implementation

The new missing_exception function doesn't seem necessary, could we go through error_func instead?

I'd really like to stay DRY, I'll try to list these principles in https://github.com/ManderaGeneral/generalpackager/issues/81

The new self.trigger also doesn't seem necessary, and I'm not sure it should even default to spec.name

It doesn't really make sense though to store the item value in FakeModule since we cannot be sure the latest yielded object is the one that will be called, see the example:

from langdetect import hello
from langdetect import hi

hello()

generalimport.exception.MissingOptionalDependency: Optional dependency 'langdetect' (required by 'hi') was used but it isn't installed.

It says "required by 'hi'". I think this has potential though! Could probably create a protected _error_func that error_func relays to and then have _error_func be directly callable with the optional item parameter

Relying on the name is an interesting idea! If it wasn't for PEP8 I wouldn't have been too excited about it, would have preferred relying on some magical dunder method that's only called by raise, alas, such a method probably doesn't exist, so this could work!

I also wasn't very happy with this, there's always going to be some library that misbehaves (even though that would be their fault :grin: ). I put those patterns in a constant so in case of need people could always modify it at runtime. Ugly, but it's an emergency solution after all. If I did the same matching inline in L28 it would have been impossible to extend.
I really looked for those dunder methods too... but no way. Exception matching is baked into CPython: i had to dig into the C source to make sure and yep, it's done in C. No way around it.

The new missing_exception function doesn't seem necessary,

I found it necessary to be generated on the fly because there's no other (evident) way to pass the dependency value to it. __getattr__ must return a class for the whole thing to work, not an instance, so this was the cleanest way to make sure the name of the dependency was carried over. I might be wrong though. Feel free to simply return the class, like:
def __getattr__(self, item):
    if any(str(item).endswith(pattern) for pattern in EXCEPTION_NAMING_PATTERNS):
        return MissingOptionalDependency  # Can't return MissingOptionalDependency("the error message"): it must be a class.
         ...

to see how the error looks like. Of course if you find a better way I'm happy with anything that works!

The new self.trigger also doesn't seem necessary

Fair, that was mostly for me to debug, I admit. I'll spend a bit more time to see if there's a neater way to make it work, otherwise it's going to disappear from this PR.

I might have found a nice way of keeping trigger while fixing the bug you highlight above. I push that in the next commit
It makes this snippet:

from generalimport import generalimport
generalimport("langdetect")

from langdetect import hello
from langdetect import hi

hello()

return this message:
generalimport.exception.MissingOptionalDependency: Optional dependency 'langdetect' (required by 'hello') was used but it isn't installed.

In the very opinionated changes commit I also added some "cosmetic" changes that I actually used for most of the development of this feature, and that I thought you might like. If you don't like them, no issue! I will absolutely revert them back.  Could I please ask you to separate these changes into one or two other PRs before I process them? My IDE is looking like a Christmas tree haha

Alright the diff should be back to a manageable size :smile:

Adding some changes to this PR and merging master onto it

I think code is OK but struggling with dev workflow. I thought that the "missing-exceptions" branch would be available on my repo but I'm guessing it has to use the current fork too

Hey @Mandera! Yes that's normal, missing-exceptions should not be available on your main repository. However, you should have full access to my fork.
If you want/need to see the branch on your repo you can either:

Merge my fork's branch onto an equally named branch of your repo before merging to master (may be automated, but be careful with this), or
Give me write access to your repo (even more dangerous than the previous one ofc).

As a third, very rough alternative you can use GitHub's own web editor to edit my PR's code without checking out my repo, but that might not be what you need.

On top of this, I have one last bit of unsolicited advice :sweat_smile: By looking at your CI it seems to me like you would like to test all your libraries for integration every time anything happens on any of them. While that is surely useful, why replicating the effort on every single project's CI? That's a lot of duplication to handle.
I'd recommend setting up an additional repo with only the integration tests, and run those tests with a repository_dispatch hook: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#repository_dispatch. In this way, every time an event of your choice occurs on any of your repos, you can trigger the tests there and make sure they pass, and you don't need to keep or clone these tests anywhere else.
In addition, as a contributor it would be really weird if I saw tests failing for unrelated projects on my PRs. I think this is information that you can use to decide whether to merge a PR or not, but you should not directly force everyone to fix failing tests on other repos in order to get their code merged in this one. Otherwise these libraries are just pretending to be self-sufficient and are instead a monolith that I have to understand from top to bottom in order to modify it.
Sorry if I've been a bit direct :sweat_smile: but I believe this is quite important for the success of this project.

Ah I understand! No that's totally fine! I really appreciate your advice 👍
I think it's a good idea that the workflow only runs unit tests for its own repo. I definitely want each repo to be self-sufficient! It's an easy change for the dev workflow.
To change it in the main workflow (push to master) I'd have to rethink how the repos are synced and published, a bit of a bigger job but definitely on the radar!
I like the idea of a separate repo with those (integration?) tests, perhaps the syncing, version bumping, and publishing could take place there 🤔

Ah I understand! No that's totally fine! I really appreciate your advice 👍
I think it's a good idea that the workflow only runs unit tests for its own repo. I definitely want each repo to be self-sufficient! It's an easy change for the dev workflow.
To change it in the main workflow (push to master) I'd have to rethink how the repos are synced and published, a bit of a bigger job but definitely on the radar!
I like the idea of a separate repo with those (integration?) tests, perhaps the syncing, version bumping, and publishing could take place there 🤔

Tests passed except for the two from

https://github.com/ManderaGeneral/generalimport/issues/20

Not sure why that's happening. I'll go ahead and merge this because I believe those tests are passing on the main workflow, let's see!


Truvari v1.2 and v1.3.2 give different results

Hi,
we upgraded Truvari from v1.2 to v1.3.2, and given the same input data and Truvari parameters, we get quite different results.
E.g. results for Manta DEL:
For variants larger that 1000bp, v1.2 reports approximately 450 true positives, while v1.3.2 reports 2
The two versions agree on variants smaller than 1000bp (only missing 5 of 1000 in 300to999 range)
Is this behaviour expected?

I can report the same problems, here is the results using the exact same commands on the same vcf's, but with the two different versions.
Version 1.2:
https://docs.google.com/spreadsheets/d/1FvZg1fTezi0OAS4ogcH3UvGtJqWOyyTfzmsEbFuRK1Q/edit?usp=sharing
Version 1.3.2:
https://docs.google.com/spreadsheets/d/1ifmh-4sSHFy4c0xZkucBJcmC0dQFIzyb8iMAxpDQx1g/edit?usp=sharing

Hello,
Could you please provide a set of comparison calls' VCF so that I can look for what might be happening.
Alternatively, if you could run the v1.2 and v1.3.2 commands again with --debug and provide the full log.txt files, I may be able to get an idea from those as well.
Have a great day,
~/Adam

Thanks for quick response Adam, I've created a git-lfs repo at this location:
https://github.com/scalavision/truvari_troubleshoot
You will find both logs and vcfs there. The log for 1.3.2 is clipped at 5000 lines, because it was more then 70mb big. Please let me know if you need anything else.

Hello,
I just put in a commit that may address this problem.
This issue was arising because pyvcf did not provide accurate end points for events that are not sequence-resolved (e.g. ALT:), but pysam does provide an accurate stop. Version 1.2 and prior worked because the create_haplotype method worked around the pyvcf behavior. However, that same approach did not work after the change to pysam.
I only tested far enough to get your manta_NACG-HG002-BenchM_diploidSV.vcf.gz to behave the same in the fix as v1.2. Below is the summary of the three results.
Could you please checkout 4dcac156096db30ac3f6c9c6fad6b7d1b6237993 and see if that gives results consistent with your expectations on all the other vcfs? If so, I'll push a patch bump.
Thanks,
~/Adam
$ grep -w TP-base */summary.txt
fix/summary.txt:    "TP-base": 2699,
v1.2/summary.txt:    "TP-base": 2699,
v1.3/summary.txt:    "TP-base": 2241,
$ consistency_report */tp-base.vcf
#
# Total 2699 calls across 3 VCFs
#
#File	NumCalls
fix/tp-base.vcf	2699
v1.2/tp-base.vcf	2699
v1.3/tp-base.vcf	2241
#
# Summary of consistency
#
#VCFs	Calls	Pct
3	2241	83.03%
2	458	16.97%
#
# Breakdown of VCFs' consistency
#
#Group	Total	TotalPct	PctOfFileCalls
111	2241	83.03%	83.03% 83.03% 100.00%
110	458	16.97%	16.97% 16.97% 0%

Thanks!! I'll have a loot at it :-)

@ACEnglish , I've now tested against your last release, the 1.3.4 version. As far as I can see, I now get exactly the same results between 1.2 and 1.3.4. Thank you very fixing this :-)
Wish you a nice weekend!

Note that I had a bad push to 1.3.3, so 1.3.4 will be your new desired version. Thanks for the help!


Failure to open database after Teku upgrade

Report from stefa2k#3917 on Discord:

got an issue when updating from 21.3.2 to 21.4.0.

I've requested the full version info: I believe that we have made some changes to the DB columns, but I don't think they are in a release yet. Note the RocksDBException: You have to open all column families. Column families not opened: message.
beacon_1         | 11:32:25.770 INFO  - Beacon data directory set to: /opt/teku/data/beacon
beacon_1         | 11:32:27.758 FATAL - Failed to open database at path: /opt/teku/data/beacon/archive
beacon_1         | tech.pegasys.teku.storage.server.DatabaseStorageException: Failed to open database at path: /opt/teku/data/beacon/archive
beacon_1         |      at tech.pegasys.teku.storage.server.DatabaseStorageException.unrecoverable(DatabaseStorageException.java:31) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.storage.server.rocksdb.RocksDbExceptionUtil.wrapException(RocksDbExceptionUtil.java:23) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.storage.server.rocksdb.core.RocksDbInstanceFactory.create(RocksDbInstanceFactory.java:110) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.storage.server.rocksdb.RocksDbDatabase.createV4(RocksDbDatabase.java:130) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.storage.server.VersionedDatabaseFactory.createV5Database(VersionedDatabaseFactory.java:242) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.storage.server.VersionedDatabaseFactory.createDatabase(VersionedDatabaseFactory.java:151) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.services.chainstorage.StorageService.lambda$doStart$0(StorageService.java:62) ~[teku-services-chainstorage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.infrastructure.async.SafeFuture.fromRunnable(SafeFuture.java:151) ~[teku-infrastructure-async-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.services.chainstorage.StorageService.doStart(StorageService.java:49) ~[teku-services-chainstorage-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.service.serviceutils.Service.start(Service.java:33) ~[teku-services-serviceutils-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.services.ServiceController.doStart(ServiceController.java:29) ~[teku-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.service.serviceutils.Service.start(Service.java:33) ~[teku-services-serviceutils-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.AbstractNode.start(AbstractNode.java:83) ~[teku-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.Teku.start(Teku.java:46) ~[teku-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.cli.BeaconNodeCommand.call(BeaconNodeCommand.java:282) [teku-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.cli.BeaconNodeCommand.call(BeaconNodeCommand.java:71) [teku-21.4.0.jar:21.4.0]
beacon_1         |      at picocli.CommandLine.executeUserObject(CommandLine.java:1933) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at picocli.CommandLine.access$1100(CommandLine.java:145) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at picocli.CommandLine$RunLast.handle(CommandLine.java:2326) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at picocli.CommandLine$RunLast.handle(CommandLine.java:2291) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at picocli.CommandLine.execute(CommandLine.java:2058) [picocli-4.4.0.jar:4.4.0]
beacon_1         |      at tech.pegasys.teku.cli.BeaconNodeCommand.parse(BeaconNodeCommand.java:231) [teku-21.4.0.jar:21.4.0]
beacon_1         |      at tech.pegasys.teku.Teku.main(Teku.java:32) [teku-21.4.0.jar:21.4.0]
beacon_1         | Caused by: org.rocksdb.RocksDBException: You have to open all column families. Column families not opened:
beacon_1         |      at org.rocksdb.TransactionDB.open(Native Method) ~[rocksdbjni-6.11.4.jar:?]
beacon_1         |      at org.rocksdb.TransactionDB.open(TransactionDB.java:90) ~[rocksdbjni-6.11.4.jar:?]
beacon_1         |      at tech.pegasys.teku.storage.server.rocksdb.core.RocksDbInstanceFactory.create(RocksDbInstanceFactory.java:83) ~[teku-storage-21.4.0.jar:21.4.0]
beacon_1         |      ... 22 more
beacon_1         | Failed to open database at path: /opt/teku/data/beacon/archive

after deleting the data it starts up correctly and syncs

I've tested this locally and can't reproduce it going between those two versions.  The error message indicates that a RocksDB column had previously been used on the stored database but was no longer being specified.  Adding new columns is allowed, but removing them is not backwards compatible.
There was a new column added in 21.4.0 and another new column added in 21.4.1 but none were removed.  Both those changes were in the archive database which is what's failing to open so it seems likely that somehow a newer version of Teku opened the database which created the additional columns though the user doesn't believe a develop version was used on that server.  In any case the latest teku release includes all the new columns so should work with the database if it's still available.
I'm not sure there's much more we can do to investigate unless we can get a copy of the db and its still failing to open with 21.4.1.

Unfortunately I can't provide a db, because it was a prater testnet setup I just deleted the files and carried on to test our software. I can say for sure that it were these versions, but I can't rule out that we originally already migrated multiple times from older versions to newer versions and then to 21.3.2 (never going back in time but making upgrades multiple times). What exact versions we used exactly I can't say, we usually use all the releases and update and test as soon as you guys release a new one.

Going to close this one as we don't have any further information to go on.  Thanks for taking the time to report it though - we've finished the database changes now anyway but if this does occur again being able to get a copy of the database would definitely be very useful.


Ollie: Watch convertertogenerator.com

Ollie requests the watch of the watch_keyword convertertogenerator\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

convertertogenerator\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ


Cannot interrupt or kill the program due to signal handler

When acme's environment_loop is running (with some launchpad/reverb programming running as children processes), I cannot interrupt or kill the program. I am on a standard linux environment where the main python process is spawned from shell (bash/zsh) using commands like python train.py.
When pressing CTRL+C, I expect SIGINT signal works to interrupt the program (with some graceful handling of child processes as well). But what I see in the console is a bunch of ^C^C^C -- the signal handler seems eating the signal.
I guess this is related to acme/utils/signal.py: https://github.com/deepmind/acme/blob/fbdc7b9b8d51f10b5c397fa7715b3b4f3a5d7000/acme/utils/signals.py#L28-L53. Any helps would be appreciated.

What is the version of Acme you are running? There were some changes recently for termination handling. Could you give v0.4.0 a try?

My working copy of acme was based on d425a3c, but I don't see any relevant changes in d425a3c..b81159b (0.4.0). I will give v0.4.0 a try tomorrow and let you know. FYI my agent was based on a LocalLayout (not distributed one), but I believe both will share the same problem.

LocalLayout doesn't use Launchpad and it doesn't support clean termination. Please give distributed layout a try. That should work fine.

I can confirm what @wookayin says and have encountered a similar issue with local agent. The examples bc_jax agent cannot be interrupted for some reason. I am using dm-acme 0.4.0

OK so after some bisecting, I think I have found the issue.
The problem is in this line:
https://github.com/deepmind/launchpad/blob/14bd68213af87a2989149dcecb46a056195ee092/launchpad/launch/worker_manager.py#L64-L65
Looks like the keyboard interrupt is ignored with the try catch pass statements. Not sure why this is done here, but removing that or disable registering the SIGINT handler in
https://github.com/deepmind/launchpad/blob/14bd68213af87a2989149dcecb46a056195ee092/launchpad/launch/worker_manager.py#L85
seems to solve the problem for me.

Thanks for investigating this problem. Fix is under development.

Fix is in - changes are both on Acme and Launchpad side - could you give it a try installing Acme / Launchpad from head (not PyPi)?

I don't have a local setup for building launchpad so I suppose I can try it out once the change is in nightly.

@qstanczyk Thanks a lot for the fix. This appears to be working for me.
I hope this can be made into the released version soon. Out of curiosity, how are you managing the release process of Acme?

Please reopen if there are any further questions on the topic.


Grandmaster capes

Add grandmaster skillcapes if you get 5b xp in a skill - functions same as master capes, just cosmetically different?

This sounds neat. Maybe even just like, a golden glow around it?

Add grandmaster skillcapes if you get 5b xp in a skill - functions same as master capes, just cosmetically different?

or how about a pair of golden nerd glasses 🤓

lol

BSO Suggestions now need to go in the #bso-vote channel so it can be voted on.


Spinner is blank

On an Android 6 custom ROM when I try to select a different sort method the text is invisible, though I can stil select by guessing. It must be white text on white background.

Thanks for your help. The problem is solved, there is no more Spinner !


fail to open file TmpGifs/312733417272.gif

Hi,
The standard blastn result page output fail to open file TmpGifs/312733417272.gif rather showing the alignment graphic.
What did I miss?
Thank you in advance,
Michal

Solution is  chmod o+w  /var/www/html/blast/TmpGifs/

Fixed in commit 0ef4462
thanks for the solution!


Update assign command to not allow assigning if role not given

Codecov Report

Merging #180 (a8e461e) into master (c23b469) will decrease coverage by 0.11%.
The diff coverage is 0.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff              @@
##             master     #180      +/-   ##
============================================
- Coverage     59.89%   59.79%   -0.11%
  Complexity      613      613
============================================
  Files           112      112
  Lines          2354     2358       +4
  Branches        270      271       +1
============================================
  Hits           1410     1410
- Misses          877      879       +2
- Partials         67       69       +2

Impacted Files
Coverage Δ

...main/java/seedu/address/commons/core/Messages.java
0.00% <ø> (ø)

...eedu/address/logic/commands/AssignTaskCommand.java
94.87% <0.00%> (-5.13%)
:arrow_down:

...c/main/java/seedu/address/model/person/Person.java
93.33% <0.00%> (-4.35%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add breadcrumbs with links to parent pages in the article header

Many websites put "breadcrumbs" at the top of the page so that you can quickly see what parent pages are above you, and click through to them. For example:
The RTD theme:

The MDN docs:

What do folks think about using our article-header area to include breadcrumbs for each page? If we were worried about horizontal space, then we could restrict it to 2 parent pages, or limit the number of characters per header title or something.

I'm open to supporting this as long as it's optional / can be turned off in config. Are the intermediate levels in a breadcrumb guaranteed to exist though? (will there always be an index.html in every subfolder in _build/html/?)  Off the top of my head, I'm not sure, but my guess is that it is not guaranteed by Sphinx.

I think the way you'd do it is by looping through a page's parent documents and using their titles + hrefs...those always do need to exist. So the folder structure in the filesystem wouldn't matter as much as the toctree structure of the documents

+1 on this one as this would help with navigation, especially if adding https://github.com/pydata/pydata-sphinx-theme/issues/1072
We should also consider how many levels will be displayed in the breadcrumbs or/and make this customisable. I think if there is a doc nested a few levels this might take too much space or would not fit the width at all

Fwiw, it looks very simple to implement. It is just this couple of lines in the RTD theme (assuming that the "parents" variable is built into sphinx)
https://github.com/readthedocs/sphinx_rtd_theme/blob/6444ca893043d8a0ca704474f7cf7d5a148f6767/sphinx_rtd_theme/breadcrumbs.html#L24


Added activateGuestPass

https://dev.doctormckay.com/topic/665-is-there-a-methodway-to-accept-and-activate-gifts/#entry2113

When I try to call this function, it gives me access denied


Try Laravel UI for Vue with Auth

This project webpack version is 4.
But laravel mix need webpack 5 (dependent?).

Applay Laravel UI without Vue.


feat: add codecov

Summary of changes

Changes introduced in this pull request:

add code coverage to the repository

Reference issue to close (if applicable)

Closes
Other information and links

Change checklist

[x] I have performed a self-review of my own code,
[x] I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
[x] I have added tests that prove my fix is effective or that my feature works (if possible),

@LesnyRumcajs Hmm we get codecov uploading error even with the ENV VAR

@LesnyRumcajs Hmm we get codecov uploading error even with the ENV VAR

It was an intermittent failure.


POC Logging Plugin & Replayer

Signed-off-by: Mikayla Thompson<EMAIL_ADDRESS>Signed-off-by: Greg Schohn<EMAIL_ADDRESS>Description
This PR is a follow up to #110. See that issue for more info on the plugin.
As for the replayer, it is very much a work-in-progress, but it successfully does the following:

Read in logged request/response files from a directory
Replay the requests packet-by-packet
Log the results in the form of "triple" jsons -- {"request" , "primaryResponse", "shadowResponse"} -- for the traffic comparator, with latencies for both responses.

There are definitely a lot of bugs and TODOs -- we're hoping to surface a lot of those in this PR.
Issues Resolved
MIGRATIONS-970
Testing
A lot of manual testing, but it is woefully lacking on unit and integration tests.
Check List

[ ] New functionality includes testing

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented
[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

updated code in response to comments here, but due to commit signing issues in #110 , it needed to be on a new branch. Opened here: https://github.com/opensearch-project/opensearch-migrations/pull/117


Add visual story page to pa11y, updown and check-url

Once we're happy with the Visual story content type and we have an existing one, we should add it to pa11y tests as well as the Check URL list and Updown checks.

Need a real VS

The Genetic Automata VS can be used for this


Is it safe to use xor for the inner additions in (E)NH ?

The codegen would be slightly neater for full-sized blocks if we could xor the message with our key in AVX256-sized blocks.

In my experience, you can easily mix carryless mult. with XOR, but it gets harder to interleave regular multiplications and additions.
(This does not answer your question, of course.)

Seems unlikely. (m ^ k) - k is xor-linear in k (http://www.goubin.fr/papers/arith-final.pdf), i.e., very predictable.
The core of the collision bound is kc = (m' ^ k) c' + y, or, equivalently, kc = (m' + k - f_m'(k))c' + y, where f_m'(k) = (m' + k) - (m' ^ k),
The condition becomes k(c - c') + f_m'(k)c' = y, where all but k (including f_m') is attacker-controlled. It's not hard to imagine that f_m' might be affine wrt modular arithmetic for some m'. The expression becomes k(c - c') - (dk + e)c' = k(c - c' + d) + ec' = y, and at that point, an attacker simply has to make sure c' = c - d.


Cannot run test inside IDEA

When trying to run or debug a test inside IDEA I'm getting this:
Testing started at 20:42 ...
Exception in thread "main" java.lang.ClassNotFoundException: utest.framework.TestTreeSeq
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.jetbrains.plugins.scala.testingSupport.uTest.UTestRunner.main(UTestRunner.java:316)

Process finished with exit code 1

The test works perfectly when running directly from sbt. Perhaps this is an IDEA issue, not a uTest one?

Sounds like https://youtrack.jetbrains.com/issue/SCL-12660. Could you try their latest EAP?

It definitely looks like that one, thanks for the pointer. I'll try the EAP later :)

I'm using one of the latest versions of IntelliJ, but it still doesn't seem to work. Can this issue be reopened?
Details:
IntelliJ IDEA 2017.3 EAP (Ultimate Edition)
Build #IU-173.3415.22, built on October 25, 2017
IntelliJ IDEA EAP User
Expiration date: November 24, 2017
JRE: 1.8.0_152-release-1024-b6 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.12.6

are you on the latest version of the Scala Plugin? I think it's versioned separately from IntelliJ IDEA itself

I was on version Version: 2017.3.4 and just updated to Version: 2017.3.5. Now it works. Was confused by the message in the referenced thread on the IntelliJ issue tracker which stated

Try it please on latest Scala plugin EAP #2<IP_ADDRESS>. It's expected to be fixed there.

Thought 2017.3.4 was new enough. Thanks for the quick reply though!


elektroid: 2.5.2 -> 3.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/dagargo/elektroid/releases.
meta.description for elektroid is: Sample and MIDI device manager
meta.homepage for elektroid is: https://github.com/dagargo/elektroid
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 3.0 with grep in /nix/store/3z6h0j73k463kdhg1wfwzv92pr4biibh-elektroid-3.0
found 3.0 in filename of file in /nix/store/3z6h0j73k463kdhg1wfwzv92pr4biibh-elektroid-3.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
elektroid

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/3z6h0j73k463kdhg1wfwzv92pr4biibh-elektroid-3.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A elektroid https://github.com/r-ryantm/nixpkgs/archive/984908965607c5de2f304f8c1d6119849f675df2.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/984908965607c5de2f304f8c1d6119849f675df2#elektroid

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/3z6h0j73k463kdhg1wfwzv92pr4biibh-elektroid-3.0
ls -la /nix/store/3z6h0j73k463kdhg1wfwzv92pr4biibh-elektroid-3.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

elektroid

Maintainer pings
cc @camelpunch for testing.

LGTM


HP Omen (Rival 300) not recognised

ratbagd and piper installed. Mouse does not get recognized
Kubuntu 20.04

@perigoso It's similar but not quite the same, looking https://github.com/flozz/rivalcfg, it seems like "SteelSeries Rival 300 HP OMEN Edition" has a different product_id.
That said it seems like the different product_ids are just a variation of the same Rival 300 because looking at rivalcfg code I don't see any special handling for different product_id's it supports, so it's safe to assume libratbag would support HP Omen also with provided .device file
[Device]
Name=SteelSeries Rival 300 HP Omen
DeviceMatch=usb:1038:1718
Driver=steelseries

[Driver/steelseries]
DeviceVersion=1
Buttons=6
Leds=2
DpiRange=50:6500@50
MacroLength=1

@nyhilium can you drop the above file as somename.device into /usr/share/libratbag/ and restart ratbagd`. Tell us if it worked?

Hm you're right, but surely it's for marketing reasons and the firmware is the same.
I was under the impression the driver for the 300 wasn't behaving well, so the device file might not be enough

as of #1209 the device should now be recognized, the driver still has some issues, but that is being tracked in #804


Returned None Values when computing Gradients

Describe the current behavior
I am using the Resnet50 code maintained under Eager Execution as is. When computing gradients
*grads = tape.gradient(loss, model.variables)
I get NONE values return in grads. I traced down the issue and realized that those none values correspond to the BatchNormalization layers. The layers are set to trainable=True, so I am not quite sure why they do not train or why they return None values.
**Obviously, model.variables != tape.variables
Code to reproduce the issue**
With tf.GradientTape() as tape:
prediction = model(x, training=True)
var_list = tape.watched.variables()
loss = compute_loss(prediction, label, operation = 'L2')
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(zip(grads, model.variables), global_step = step))
Could you please explain why am I getting those None values?

You should use model.trainable_variables instead of model.variables.

Thank you @ppwwyyxx , That did get rid of the NONEs. Does this mean BatchNormalization layers hold non-trainable variables?

Does this mean BatchNormalization layers hold non-trainable variables?

Yes.


Update encryption algo

Coverage decreased (-9.09%) to 52.273% when pulling c72f95f462805a614d004bf418f896b41912e4f7 on cpacia:master into e0d34f55fb65917a53d6d4dc3258852a4a5e16c0 on OpenBazaar:master.

Coverage decreased (-12.6%) to 48.796% when pulling 93fb166c90a0e4528ec005749d3683c6a9769783 on cpacia:master into e0d34f55fb65917a53d6d4dc3258852a4a5e16c0 on OpenBazaar:master.

Coverage decreased (-12.6%) to 48.796% when pulling 93fb166c90a0e4528ec005749d3683c6a9769783 on cpacia:master into e0d34f55fb65917a53d6d4dc3258852a4a5e16c0 on OpenBazaar:master.


Need help

Hey it keeps on saying this
Error: Cannot find module 'C:\Users\byoun\Downloads\DiscordJS-V14-ModMail-Bot-main\index.js'. Please verify that the package.json has a valid "main" entry
at tryPackage (node:internal/modules/cjs/loader:444:19)
at Module._findPath (node:internal/modules/cjs/loader:715:18)
at resolveMainPath (node:internal/modules/run_main:26:23)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:24)
at node:internal/main/run_main_module:28:49 {
code: 'MODULE_NOT_FOUND',
path: 'C:\Users\byoun\Downloads\DiscordJS-V14-ModMail-Bot-main\package.json',
requestPath: 'C:\Users\byoun\Downloads\DiscordJS-V14-ModMail-Bot-main'
}
i have tried everything can you help?

lol i had the same issue,  try changing "example.config.js" to "config.js" it worked for me

nvm


Support Prettitier

This issue is closed https://github.com/Chiel92/vim-autoformat/issues/191 so I open this one to make it work. It will be nice if vim-autoformat support prettitier via cli, but now I have vim-prettitier plugin and :Prettitier command works. How run it via  vim-autoformat?

What do you want to achieve? I'm not sure what you mean by "running pretier via vim-autoformat".
If you want the same key for either formatting command, just map your keys differently for the filetypes where you want to use prettifier.

I'd like to use only one key binding for all file types. If file type is  'js' or 'jsx' I wish Autoformat run :Prettier command on these file types. How to achieve this? Could you help me how to configure it?

Try something like (untested)
map <F3> :Autoformat
au filetype javascript map <F3> :Prettifier

Thanks a lot!

noremap <F3> :Autoformat<CR>
noremap! <F3> <esc> :Autoformat<CR>

au filetype javascript,javascript.jsx,typescript,less,scss,css,json,graphql noremap <F3> :Prettier<CR>
au filetype javascript,javascript.jsx,typescript,less,scss,css,json,graphql noremap! <F3> <esc> :Prettier<CR>

Hmm, this config do not work. :Prettier runs on every file type, on VIM files also.

If anyone else stumbles across this issue, you probably want
map <F3> :Autoformat
au FileType javascript map <buffer> <F3> :Prettier

with map <buffer> so that the special key binding only applies to the current buffer. Otherwise, if you open a JavaScript file and then open a Python file in a new buffer, Vim will still try to use Prettier for the Python file.

Otherwise, if you open a JavaScript file and then open a Python file in a new buffer, Vim will still try to use Prettier for the  Python file.

How is this possible if we use Filetype with javascript and other types which prettier supports ?


Fix PRationalData comparison, tests

Fixes #618. In addition to fixing the issue, tests have been added to make sure comparison operators behave as expected, both on the reported failure cases and also more broadly.
I'm not sure why the failure in PEq happened: currently, this shouldn't be the case. It's possible we were working against a pre-normalization version of Rational in PlutusTx? Either way, I can't repro that issue, and the tests show no issues as far as I can tell.
@peter-mlabs - this should fix your issue, though I can't add you as a reviewer for some reason.

I'm not sure why it wouldn't be reproducible though. Probably we want to construct the rational from Plutrach(and not from PlutusTx)?

I'm not sure how we would construct it from Plutarch. As far as I can tell, you can only use pconstant to lift a Rational.

Yes, I agree. I can't find any function exposed that allows creation of non-reduced Rational values.
The changes made in POrd instance and test suite is good


pkg/report: extract guilty files for rcu errors correctly

The existing code is broken - the console output does not contain a whitespace before the apic_timer_interrupt frame. Also, it makes sense to cut not from its first, but rather from its last occurence.

@dvyukov PTAL


Update build_conda_pkg.yaml

Pull Request Description
Attempt to fix build conda package github action error where it: failed with error: [Errno 28] No space left on device by using sparse-checkout.

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

No linked issues found. Please add the corresponding issues in the pull request description.  Use GitHub automation to close the issue when a PR is merged


fix use virtual-hosted style when s3('uri'='s3://xxx')

Signed-off-by: nextdreamblue<EMAIL_ADDRESS>Proposed changes
Issue Number: close #xxx
Problem summary
when S3 tvf, the S3 SDK uses the virtual-hosted style by default, but the code set forceVirtualHosted=false when uri='s3://bucket.endpoint/file' and will get error like 'parse file failed, path = s3://file'
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[x] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[x] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[x] No Need

Does it need to update dependencies:

[ ] Yes
[x] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[x] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.6 seconds
load time: 651 seconds
storage size:<PHONE_NUMBER>5 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230104135811_clickbench_pr_73554.html

Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.

s3默认使用的virtual-hosted style，即类似：s3://bucket.endpoint/file的形式，如果是s3://endpoint/bucket/file的形式应该会报你这种错误才对。您用的哪种方式呢

Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.

s3默认使用的virtual-hosted style，即类似：s3://bucket.endpoint/file的形式，如果是s3://endpoint/bucket/file的形式应该会报你这种错误才对。您用的哪种方式呢

ok, I got it. Is "use_path_style"="true" no use for s3://bucket.endpoint/file ? When I'm usings3://bucket.endpoint/file, it's ok whatever I set "use_path_style"="true" or "use_path_style"="false"

Hi, I still get error like 'parse file failed, path = s3://file' after using your changes, but using 'http://' is ok.

s3默认使用的virtual-hosted style，即类似：s3://bucket.endpoint/file的形式，如果是s3://endpoint/bucket/file的形式应该会报你这种错误才对。您用的哪种方式呢

ok, I got it. Is "use_path_style"="true" no use for s3://bucket.endpoint/file ? When I'm usings3://bucket.endpoint/file, it's ok whatever I set "use_path_style"="true" or "use_path_style"="false"

看社区本身的设计和之前的代码实现，应该就是s3协议只支持virtual-hosted style。非s3协议才由use_path_style来控制用什么方式去解析uri


Function suggestion: Password generator encrypted via openssl

I'm simply suggesting to add this function. I use it to store passwords (while ensuring they are never plain-text) in our local documents. I used the container Alpine and installed openssh.
Just replace the master password 'FancyPasswordHere' with yours.
1) fct_crypt
#Generate a 32 characters password
STEP1=$(docker run --rm alpine sh -c \
"cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1")
#Show the password
echo "$STEP1" && echo
#Encrypt the pass
STEP2=$(echo "$STEP1" | openssl enc -aes-256-cbc -a -k "FancyPasswordHere")
STEP1="null"
#Create a variable so it will be quick to copy-paste it later
STEP3="THIS_VAR='$STEP2'"
#Show the final string to copy in your files
echo "$STEP3" && echo
# Copy it in the clipboard (mac)
echo $STEP3 | pbcopy
echo "Just paste your encrypted password into your files."

2) fct_decrypt
Copy-paste back the output of the previous command. ie:
THIS_VAR='U2FsdGVkX19YVGj32Z0lb1YlJbD2S3/wa9GQAliwOwPWgvSXHQ1gLnkgNovzMdtd pk7atELI8iWVAql65rmMQQ==', then run
echo "$THIS_VAR" | openssl enc -aes-256-cbc -d -a -k "FancyPasswordHere"

Cheers!
https://twitter.com/askpascalandy

Hi Pascal - this looks interesting. Did you fancy creating a function for it?
You can start with the base alpine function, you shouldn't need to add anything other than your shell script afterwards.
Here's an example for getting started - https://github.com/alexellis/faas/tree/master/sample-functions/AlpineFunction
Alex

Hey Alex!
Right now I run this locally only. No functions per say. It would be great to put our master password within a secret as well.
At this point I'm lazy (a.k.a need to manage stuff in my prod first). If you don't want to adapt it leave it there and I'll jump on it on day :)
Cheers!

Adding label - revisit, will come back to this and close for now. Feel free to re-open and add to the discussion.

@alexellis I would like to work on this as well. Do you have any further instructions?

Derek add label: revisit

This was a community suggestion for a utility function. We could use your help with the SNS queue if you'd like to start there?
Marking issue as Revisit for future follow-up


Automatically create directories when saving file

Thought this would be a good addition. Tested and seems to work perfectly.

I think CreateDir can work without looping. Mind trying that first?

CreateDir is terrible. Would not recommend 0/5

I think CreateDir can work without looping. Mind trying that first?

Sure thing, I'll try that right now.

I tried this in an attempt to avoid looping through the whole of string.Explode("/", Line). It didn't work.
if not file.IsDir(Line, "DATA") then
	file.CreateDir(Line)
end

The issue is that we need to extract the path from the filename so we can just create the directories. I'm not sure how to do that without looping. I think my implementation is pretty good since it only ends up looping a few times and only runs when you save the file. If you have a better idea we can try that, but I don't.

string.GetPathFromFilename can do that

Tested newest code, seems to work.

Tested and works.


docs: a listing component

What type of PR is this? (check all applicable)

[ ] Feature
[ ] Refactor
[ ] Bug Fix
[ ] Optimization
[ ] Documentation Update
[x] Other (please specify here: Component creation)

Description of what this Pull Request does
(type your description here)
A listing component in my application that can display data passed as props. This would allow easy creation of dynamic lists of items in the landing page and customize the content and appearance of the list based on the data provided.

The listing component should accept a data prop which allows developers to pass data to be displayed in the list.
The listing component should be able to render the list items accordingly.
The listing component should display the data passed via the data prop in the format specified by the component's implementation

Related Tickets & Documents

Closes #37
Related Issues: (enter ticket numbers here)

QA Instructions, Screenshots, Recordings

Added/updated tests?

[ ] Yes
[ ] Not Yet, and here is why: explain here
[ ] I need help with tests
[x] No, and here is why: explain here

Can you add a screenshot of what the component looks like in the UI

Can you add a screenshot of what the component looks like in the UI

Will attached a screeenshot
@jjleigh

Looks good!
When you add screenshots to the PR description ensure that the formatting isn't disturbed. Not an issue but I just wanted to point out that the formatting for the text under the screenshot got a bit messed up.

Corrected

@ibimina You need to resolves conflicts my dear before I review; I am reviewing on my local app, not just on github

alright will do that

@msarit I have resolved the conflicts

@msarit @jjleigh @annonymousauthority  I have resolved conflicts

I think this is ready to be merged @msarit


Stop specifying a range of copyright years

The date of firts publication seems sufficient and it prevents us from having
to update a range of years every year.

:copyright:

oh god yes can we do this everywhere?

:+1:

Down with date ranges! Next step, specify all the years individually like gnu.org :wink:

Typo: firts

Oh it's fixed in the actual commit.

:+1:

Merged in d372ebd3adbb646dea2822c25d1530da5bc36c5f


Time only with am pm

Changed the TIME Display to use the AM PM indicator.
Added two new UI test, one for Time Only and another for Date Only.
Removed the .idea directory and updated the .gitignore file

thanks 👍 I merge then upload a new release


caravel db upgrade fails on postgres

On 3ee9a68c09750317f20eedd3968e9f36f6da10a1 trying to apply migrations on a fresh project on postgresql with
caravel db upgrade

gives:
2016-06-03 17:58:05,386:ERROR:flask_appbuilder.base:
Add Permission on Menu Error: (psycopg2.ProgrammingError) can't adapt type '_LazyString'
[SQL: 'SELECT ab_view_menu.id AS ab_view_menu_id, ab_view_menu.name AS
ab_view_menu_name \nFROM ab_view_menu \nWHERE ab_view_menu.name = %(name_1)s \n
LIMIT %(param_1)s'] [parameters: {'param_1': 1, 'name_1': l'Sources'}]

On a second look this looks really a fab issue, @dpgaspar any hint on what could be wrong here?

@xrmx I think it's on our side and related to translations

@mistercrunch so we should not use lazy_gettext when the string is going through sqlalchemy?

@mistercrunch could you please reopen this since installation of postgres is still broken?
Even with the patch applied caravel db upgrade hang applying the migrations
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 4e6a06bad7a8, Init

And the server logs
2016-06-06 14:37:14 CEST [6125-1] caravel@caravel LOG:  could not receive data from client: Connection reset by peer
2016-06-06 14:37:14 CEST [6125-2] caravel@caravel LOG:  unexpected EOF on client connection with an open transaction


C# follow up

Another follow up due to rebase and automated test using Mono.

@XVincentX Please use "zero" base indentation for the code block, or otherwise it will be indented everywhere. It is not needed.


aruco tracker

mediapipe may not make a great demo because it is quite finicky, and in particular gives shoddy results for anything but hands and face. Build an aruco tracker, and perhaps one that takes in a library of labels

Closing this because I think a priority would be integrating with a more robust pose estimation platform such as DeepLabCut or MMPose


survey-*: stop autocomplete add param to enable

Excludes survey-multi-select since it's highly
specialized and the parameter doesn't really
apply to it.
Issue #1064

Thanks very much @chrisbrickhouse, looks great!
If we do this for the plugins that use radio button plugins then I'm wondering if we should do this for checkboxes too (survey-multi-select) for consistency? This stackoverflow question suggests that Firefox remembers the state of input type="checkbox" when autocomplete is not off, and the MDN input checkbox docs say this:

Unlike other browsers, Firefox by default persists the dynamic checked state of an  across page loads. Use the autocomplete attribute to control this feature.

Confusingly, the more general MDN input element docs say that the autocomplete attribute shouldn't have any effect on inputs of type "radio" or "checkbox". And I can't get autocomplete to work in Firefox with either radio buttons or check boxes, so I'm not sure that the autocomplete attribute is doing anything in these cases. But I don't think it would hurt to make these autocomplete changes in all of the plugins that use forms.

@becky-gilbert thanks for those links! I didn't know Firefox had that behavior, so I agree adding the functionality to survey-multi-select is a good idea. I should be pushing a commit to implement that soon.
As for the conflicting documentation, IIUC, autocomplete can be an attribute of both <form> and <input> tags, but the behavior is somewhat different between the two. Setting <form autocomplete="off"> turns off autocomplete for all children, and I believe it's this attribute which controls Firefox's checkbox persistence. For input tags, autocomplete can take more specific values to give the browser hints on what to fill, and this behavior does not affect radio buttons or checkboxes.
So we could have <input type="text" autocomplete="given-name" /> which should allow autocomplete of first names, but <input type="radio" autocomplete="on" /> which should not have any effect. If both were wrapped in <form autocomplete="off"> presumably the parent attribute would take precedence and no autocomplete or persistence behavior should be seen.


Scala 2.13 Support

I've forked the project and added 2.13 support - https://github.com/henricook/cryptoutils
I was working on publishing this to Maven because i note the project is quite inactive, but i see you're still fairly active @Karasiq - would you rather I opened a PR here instead? (And dropped my fork)
Would you be able to publish a new version soon after the PR was merged?

FYI This is on maven as com.henricook % cryptoutils % "1.5.2"


Responding to altair chart selections

Does shiny support reactivity/events from making a selection in an interactive chart such as Altair, Plotly, Bokeh, etc? I can't seem to find any examples of this; the closest is the page on "Reacting to widget updates" seem to suggest that shiny only supports events from selections in charts that provide this via an already existing jupyter widgets interaction (such as on_click etc), is this correct?
In Altair, we recently added support for exactly this: chart selections can be accessed in notebooks through a jupyter widget chart object. Based on my understanding this would be supported in shiny too (and maybe even quarto dashboards since they also support jupyter widgets?), but I want to double check if this is the case before trying it out.
I think it could be valuable to add a section on chart selections driving events in the docs, just to be explicit about what works and what doesn't.
Somewhat related issue in https://github.com/posit-dev/py-shiny/issues/768

Whoops, this was closed by mistake


Clean TS install, View type errors.

After a clean install using yarn create r3f-app next app -ts, the View components in both app/page.tsx and app/blob/page.tsx show the following errors:
Type '{ children: Element; className: string; }' is not assignable to type 'IntrinsicAttributes & RefAttributes<unknown>'.
Property 'children' does not exist on type 'IntrinsicAttributes & RefAttributes<unknown>'.

Inside src/components/canvas/View.tsx the type errors above are present on the ForwardRef, as well as a type error on PerspectiveCamera.
The app still runs and does not output any errors to the terminal or browser console.
Node v16.14.2, MacOS v12.5.1

Same Error with me, it's a eslint error. bumping next and next-eslint-config both to 13.4.0 solves some of problems but linting ones remains the same

I did what @vegancat suggested and it seemed to fix most of the errors with the View component except for the error @PButcher was saying they had. I was able to fix the errors with the View component by typing it like this:
interface ViewProps {
  children: ReactNode
  orbit?: boolean
  className?: string
}

I'm not sure if that's the best approach, but it seems to get rid of the ts error, at least.

Same issue and even worse. This is supper horrible

@tunztunztunz's suggestion resolved the View component problems for me.
Installing @types/three resolved all the type issues on the Three components.
I also needed to add a color prop to the first Common component.

usePostprocess.tsx is also full of errors. even after updating next and next eslint.

Running into a bunch of type errors as well with a fresh typescript install. installing @types/three didn't solve it for me.
any solutions?

Running into a bunch of type errors as well with a fresh typescript install. installing @types/three didn't solve it for me.
any solutions?

The Common component should be typed like this:
type CommonProps = { color?: THREE.ColorRepresentation }

export const Common = ({ color }: CommonProps) => (...)

and the View component should be typed like this:
type ViewProps = HTMLAttributes<HTMLDivElement> & {
  orbit?: boolean
}

const View = forwardRef<HTMLElement, ViewProps>(({ children, orbit, ...props }, ref) => {..}

I would make a PR, but it seems like the ts flavored template is hidden.


styl to style

I think it's a spelling mistake
https://vitejs.dev/config/#css-preprocessoroptions

styl the extension for stylus. It's not a spelling mistake.

I'm sorry it just clicked right after I submitted it
sorry sorry

styl the extension for stylus. It's not a spelling mistake.

I'm sorry it just clicked right after I submitted it
sorry sorry

No worries 🙂


feat: add custom ignore dir/files.

Deafault config contains in autoload/ignore_tree/ignore_tree.vim. And can be customized by editing g:fuzzyy_custom_ignore:
Example:
   let g:fuzzyy_custom_ignore = {
       \ 'dir': ['.git', '.hg', '.svn', '.rebar', '.eunit'],
       \ 'file': ['*.beam', '*.so', '*.exe', '*.dll', '*.dump', '*.core', '*.swn', '*.swp'],
  \ }

Hi Thank you for the pull request. I was in vacation the last week and I will review it this couple days.

Hi Thank you for the pull request. I was in vacation the last week and I will review it this couple days.

Sure, no problem. I just thought, that github didn't sent notification.

Thanks a lot for this PR. I had the though about adding this feature long time ago but never had the determination.
I would like to make some changes to this PR.

I merge all the builders into a single file since it is only a small feature.
I change the name of the option for adding custome ignore.
I will probably remove the test since It seem to have a bug that cause it to fail in Windows OS. And it seem to be a bug from vim itself.

vim9script
def Case1(): void
    echom [has("win32"), has("win64")]
    echom [1, 0]
    echom [has("win32") || has("win64")]
    echom [1 || 0]
    if has("win32") || has("win64")
        echo "skipped, wrong OS:windows"
        return
    endif
enddef
Case1()

The above code will have output like this and I can't figure out why:
[1, 0]
[1, 0]
[false]
[true]

Thanks a lot for this PR. I had the though about adding this feature long time ago but never had the determination.
I would like to make some changes to this PR.

I merge all the builders into a single file since it is only a small feature.
I change the name of the option for adding custome ignore.
I will probably remove the test since It seem to have a bug that cause it to fail in Windows OS. And it seem to be a bug from vim itself.

vim9script
def Case1(): void
    echom [has("win32"), has("win64")]
    echom [1, 0]
    echom [has("win32") || has("win64")]
    echom [1 || 0]
    if has("win32") || has("win64")
        echo "skipped, wrong OS:windows"
        return
    endif
enddef
Case1()

The above code will have output like this and I can't figure out why:
[1, 0]
[1, 0]
[false]
[true]

Good!!
About tests: interesting bug, i never encountered it, but if you have it, that other users also can. Then it's ok to be removed.

Nice! Perhaps the search engine should also consider what is specified in the 'wildignore' option. That is, if a user has set wildignore+=**/foo/, then seeing displaying some/folder/foo/somefile.txt in the results would be weird.

Nice! Perhaps the search engine should also consider what is specified in the 'wildignore' option. That is, if a user has set wildignore+=**/foo/, then seeing displaying some/folder/foo/somefile.txt in the results would be weird.

Sounds like a good idea, that plugin feature must integrate more with standard vim env.
But now, i'm not sure, that's all supported search tools can support fully this wildpatterns.
Need to do little research about that.

Need to do little research about that.

If you use grep, perhaps it is enough to add each element of wildignore option to --exclude-dir flag of grep.
For example, as a basic idea one could define var wildignore_paths = split(&wildignore ,',') and add each element of the resulting list into a --exclude-dir flag.

Patterns ending with /*: These patterns will match directories. For example, the pattern **cache**/* exclude all the files included in folders and subfolders that contains the string cache in their name, thus excluding the whole directories that contains cache in their name.
Patterns without a trailing /*: These patterns will match files.

Good idea, it may be work.

Alternatively, perhaps one could also think to redesign g:fuzzyy_custom_ignore such that it only consider patterns instead of differentiating between paths and files. I think in .gitignore you specify ignore patterns without differentiating, if I am not wrong. What do you think? :)

The idea about splitting dir list and file was for ability to easily apply
separate logic for files and dirs, cause 'find' needed it, and i thout, that
others commands also behave similarly.
Anyway, we can make 1 list and differntiate files and dirs by '/' or '' at the
end of expression.

I opened a PR yesterday:)

Anyway, we can make 1 list and differntiate files and dirs by '/' or '' at the
end of expression.

This is not possible because of the way wildignore works (you can try yourself :) ).


Update WindowsBuild.md

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please smoke test and merge