Display some error message about the fact that some remote kernels are no longer usable to avoid long spinnings of kernels in kernel picker

Type: Bug
In my .ipynb I select a working kernel and can execute cells. However the kernel-picker spins for an additional 7+ minutes.
It turns out I had a dead/zombie kernel (http://<IP_ADDRESS>:6666/) -- removing it fixed the issue.
My own guess: When the kernel-picker is initially clicked, and the user has to choose which kernel to use, presumably VSCode attempts to communicate with each kernel in its list, in order to populate a list of available kernels. Maybe it launches a thread for each. And the kernel-picker icon is set to spin (🔄) until ALL have completed.
SUGGESTION 1: Before the user has made a selection, for each kernel in the list indicate the STATE of that kernel, maybe with icons: ✅ (ready to use), 🔄 (spinning), ❌ (failed)
SUGGESTION 2: When the user selects a kernel, any other threads should be cancelled, and the kernel-picker icon should reflect the status of the SELECTED kernel.
PS Issue was originally reported in https://github.com/microsoft/vscode-jupyter/issues/15911
I'm creating a fresh issue as there's a lot of fluff in that thread (as @DonJayamanne and I figured out which symptoms are (ir)relevant).
VS Code version: Code 1.92.0 (b1c0a14de1414fcdaa400695b4db1c0799bc3124, 2024-07-31T23:26:45.634Z)
OS version: Darwin arm64 23.5.0
Modes:

System Info

Item
Value

CPUs
Apple M2 (8 x 2400)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabledwebgpu: enabledwebnn: disabled_off

Load (avg)
1, 2, 2

Memory (System)
24.00GB (2.35GB free)

Process Argv
--crash-reporter-id f10d97cd-2115-4dba-a34a-07be9312995a

Screen Reader
no

VM
0%

Extensions (21)

Extension
Author (truncated)
Version

dvt-remote-ssh
ami
1.0.0

nestedtext
bma
2.0.0

githistory
don
0.6.20

copilot
Git
1.221.1038

copilot-chat
Git
0.19.2024073102

vsc-python-indent
Kev
1.18.0

rainbow-csv
mec
3.12.0

vscode-docker
ms-
1.29.1

debugpy
ms-
2024.10.0

python
ms-
2024.12.2

vscode-pylance
ms-
2024.8.1

jupyter
ms-
2024.7.0

jupyter-keymap
ms-
1.1.2

jupyter-renderers
ms-
1.0.19

vscode-jupyter-cell-tags
ms-
0.1.9

vscode-jupyter-slideshow
ms-
0.1.6

remote-containers
ms-
0.380.0

remote-ssh
ms-
0.113.1

remote-ssh-edit
ms-
0.86.0

remote-explorer
ms-
0.4.3

yaml-cloudformation-highlighter
paq
2.0.4

(1 theme extensions excluded)

A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
vscorecescf:30445987
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
vscaac:30438847
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonregdiag2:30936856
pythonmypyd1:30879173
2e7ec940:31000449
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
jg8ic977:31013176
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
dwnewjupytercf:31046870
impr_priority:31102340
refactort:31108082
ccplc:31103425
pythonrstrctxt:31112756
wkspc-onlycs-c:31111717
wkspc-ranged-c:31111712

@p-i-
The other issues had all of the information and that was actually useful and was also in the correct repo
Moving back to Jupiter extension

My thinking was that there is a maybe a problem more fundamental than the failing to report unresponsive kernels -- the problem that once the user has selected a working kernel, the picker-icon is not reflecting the state of the selected kernel. And I think logically it should be.
Yes it would also be nice to be notified of any unresponsive kernel.
When the user clicks the picker to select a kernel, presumably they want to see a list of available kernels. And next to each item, an icon showing the status of this item, which I imagine would be something like:

currently running (🟢?)
responsive (✅?)
querying-in-progress / spinning (🔄)
unresponsive / given-up-trying (⚠ ! etc.)
certified dead (❌?) (not sure if this makes sense)

And these stati would be refreshed in realtime.
And whichever kernel they click, the ongoing status is now represented by the picker-icon.
I didn't want to dilute the original issue, so posted a fresh one, linking back to the original. Maybe was my mistake to close the original one.
Anyway, I have nothing further useful to contribute to this topic, and so will leave it in the capable hands and minds of the VSCode team (thankyou for all your hard work!).


add provider for static routes

There are a few little things that need to be fixed but the PR looks good, thanks!

done.
Do you know why these errors were not shown into scrutinizer report ?

The configuration used for the scrutinizer inspections is rather old and may be incomplete/incorrect. I'll look into it.
Thanks for your PR btw :)


How to pass variables in a query?

How can I replace this agent's name with the one in this query? Instead of hardcoding umer, i want to use listing_agent._text from the query. I tried my best to understand the variables part here: https://gridsome.org/docs/querying-data/#query-data-in-templates - but can't seem to get it working. Please help. Thank you.

If I understand correctly, you want to use the value of predubai.listing_agent._text in the filter for allPredubai? The opposite is not possible, unfortunately.
When using an automated template, Gridsome will pick values from the current node based on a certain naming convention. You could try a variable named $listing_agent___text (three underscores) to get the node.listing_agent._text value as a variable.

Yes, It's working great now. I had tried this earlier, I am sure I must have missed the three underscores. Thank you once again. I am loving gridsome so much.

Would you mind posting the final query? I'd really appreciate seeing what the answer was.


backend step is commented out

Hello,
Why is the backend step commented out in droid.track()?
Also, according to the paper, the backend should run in a separate process, and not sequentially like in the code.
Thank you

Currently, backend runs in the following manner:

For every factor (which are typically 10-12), backend is run at every step.
For final bundle adjustment, backend is run in iterative routine(7+12 times).


Traits for java!

Have you ever used php? especialy Laravel. do you understand how good traits can help especially in initial stages where you have not really defined your model to normalize to actual inheritance structure, or you have constraints over different models that extend from different ancestors and you dont have ability to edit them as they are part of a library or api, yet you want to add a common spec on all of them.
Yes, this is a huge feature i have ever desired in java. I wish to see this project grow to enable us to add our own features to java of any level or version.

@caoccao Sorry for the wait, on the code sample of a trait on php here is a sample:
Traits in PHP are a mechanism for code reuse in single inheritance languages such as PHP. They allow you to define methods that can be reused across multiple classes. Here's a simple example demonstrating how to define and use a trait:
Defining a Trait
First, you define a trait. A trait is defined similarly to a class, but instead of extending another class, you use the use keyword within a class to include the trait's methods.
trait Reader {
    public function add($var1, $var2) {
        return $var1 + $var2;
    }
}

trait Writer {
    public function multiply($var1, $var2) {
        return $var1 * $var2;
    }
}

Using Traits in a Class
To use these traits in a class, you simply include them using the use keyword within the class definition.
class File {
    use Reader;
    use Writer;

    public function calculate($var1, $var2) {
        echo "Result of addition: ". $this->add($var1, $var2). "<br>";
        echo "Result of multiplication: ". $this->multiply($var1, $var2);
    }
}

Instantiating and Using the Class
Finally, you can instantiate the class and call its methods, which now include those defined in the traits.
$o = new File();
$o->calculate(5, 3);

This example demonstrates how traits can be used to encapsulate behavior that can be shared among multiple unrelated classes, enhancing code reuse and reducing duplication.
It is like I suppose macros, it almost simply copies your code in trait into your class, and also checks for conflicts and ignores duplicates, so you can use a trait twice or further down as long as you maintain the method signature. It is almost like an java's default method in interfaces, but consider you have ability to declare and access members. Issue with it It can be messy if misused, and i suppose that's the reason why it is not included in java with its strict nature, but however for an experienced dev, It can be a huge neat feature to use to structure your code.

Those ideas make sense. Thank you.
The problems on my desk are:

It's half way done.

[x] Code -> AST
[x] AST Transformation
[ ] AST -> Code (Tedious work, requires weeks/months)

The trait feature lacks IDE support.

[ ] Idea Extension
[ ] VS Code Extension

I expect it to be a community driven project which implies who needs that who implements that. Please feel free to share your feedbacks. Thank you.


[Bug]: CFBlobstore unavailable on push to EKS with Korifi 0.3.0

What happened?
Doing a cf push with --no-route (to get around a different bug...) uploads fine, but fails with a message about the blobstore but not enough information to resolve it.
What you expected to happen
The upload to work and a container build to occur.
Acceptance Criteria
GIVEN a logged in cf user with access to an org and space
WHEN I cf push --no-route and the upload succeeds
THEN I expect the upload to be stored, and a container build to occur
How to reproduce it (as minimally and precisely as possible)
❯ cf push firstflask --no-route
Pushing app firstflask to org MyNewOrg / space FirstSpace as arn:aws:iam::656667300755:user/cf-admin...
Packaging files to upload...
Uploading files...
 7.99 MiB / 7.99 MiB [=============================================================================================================================================================================================] 100.00%Unexpected Response
Response Code: 502
Code: 0, Title: , Detail: {"errors":[{"detail":"Error uploading source package to the container registry","title":"CF-BlobstoreUnavailable","code":150006}]}

FAILED

Anything else we need to know?
EKS cluster has a default storage class:-
❯ kubectl get storageclass -o wide
NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  24h

Registry is an ECR with access secret defined and linked to the environment.
Environment
Revision of codebase: 0.3.0 release tgz
Kubernetes version (use kubectl version):

Client Version: v1.25.2
Kustomize Version: v4.5.7
Server Version: v1.22.13-eks-15b7512
Cloud provider or hardware configuration: EKS
Others:

Got a little further debugging. Found this in the korifi-api-system logs:-
{"level":"info","ts":1664439414.366823,"logger":"http-logger","msg":"response","correlation-id":"d220301f-3262-4e06-899d-990c3520b4ea","status":201,"size":735,"durationMillis":36}
{"level":"info","ts":1664439415.171536,"logger":"http-logger","msg":"request","correlation-id":"f6582847-7536-4d7b-9868-96e3f264a641","url":"/v3/packages/630984ad-5d36-43b9-ba23-c4c4ae2b196d/upload","method":"POST","remoteAddr":"<IP_ADDRESS>:56428","contentLength":8377315}
time="2022-09-29T08:16:58Z" level=info msg="Could not save cache" error="open /.ecr/.config.json.tmp1821651150: no such file or directory"
{"level":"info","ts":1664439422.8753464,"logger":"PackageHandler","msg":"Error calling uploadSourceImage","correlation-id":"f6582847-7536-4d7b-9868-96e3f264a641","err":"pushing image ref '656667300755.dkr.ecr.eu-west-2.amazonaws.com/korifi-packages/630984ad-5d36-43b9-ba23-c4c4ae2b196d' failed: failed to upload image: Post \"https://656667300755.dkr.ecr.eu-west-2.amazonaws.com/v2/korifi-packages/630984ad-5d36-43b9-ba23-c4c4ae2b196d/blobs/uploads/\": EOF"}
{"level":"info","ts":1664439422.8753843,"logger":"PackageHandler","msg":"handler returned error","correlation-id":"f6582847-7536-4d7b-9868-96e3f264a641","error":"pushing image ref '656667300755.dkr.ecr.eu-west-2.amazonaws.com/korifi-packages/630984ad-5d36-43b9-ba23-c4c4ae2b196d' failed: failed to upload image: Post \"https://656667300755.dkr.ecr.eu-west-2.amazonaws.com/v2/korifi-packages/630984ad-5d36-43b9-ba23-c4c4ae2b196d/blobs/uploads/\": EOF"}
{"level":"info","ts":1664439422.8754134,"logger":"http-logger","msg":"response","correlation-id":"f6582847-7536-4d7b-9868-96e3f264a641","status":502,"size":131,"durationMillis":7703}

I'm going to presume that the local cache file error is a warning and ignore that, and work on the basis the ECR URLs are wrong. I note the install guide doesn't mention ECR, so I'll try and get a working format and report back here (then do a PR later etc.)

Might be this... testing...
https://natan.la/notes/docker-ecr-eof.html

I've now found how to fix the ECR login issue and tested it. Will document and submit a PR for the docs. Now having a 401 unauthorized access issue even though perms are set. Is either an IAM/ECR perms thing (chasing down now), or a URL generation error in CF (unlikely):-
{"level":"info","ts":1664443712.2366157,"logger":"http-logger","msg":"response","correlation-id":"f5a70ce6-f3b4-4dd2-b615-4b7689a06249","status":201,"size":735,"durationMillis":31}
{"level":"info","ts":1664443713.0264022,"logger":"http-logger","msg":"request","correlation-id":"5a510c35-7c4c-4ffb-a585-3f57200c8f25","url":"/v3/packages/e796b66e-5476-461f-8a0c-12277a29088d/upload","method":"POST","remoteAddr":"<IP_ADDRESS>:53166","contentLength":8377315}
{"level":"info","ts":1664443715.8273907,"logger":"PackageHandler","msg":"Error calling uploadSourceImage","correlation-id":"5a510c35-7c4c-4ffb-a585-3f57200c8f25","err":"pushing image ref '656667300755.dkr.ecr.eu-west-2.amazonaws.com/korifi-packages/e796b66e-5476-461f-8a0c-12277a29088d' failed: failed to upload image: HEAD https://656667300755.dkr.ecr.eu-west-2.amazonaws.com/v2/korifi-packages/e796b66e-5476-461f-8a0c-12277a29088d/blobs/sha256:3a647c1fbe83b8b2cf4761db7e6083d16f5978024b25d177addb2136da5ab9e9: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)"}
{"level":"info","ts":1664443715.8274162,"logger":"PackageHandler","msg":"handler returned error","correlation-id":"5a510c35-7c4c-4ffb-a585-3f57200c8f25","error":"pushing image ref '656667300755.dkr.ecr.eu-west-2.amazonaws.com/korifi-packages/e796b66e-5476-461f-8a0c-12277a29088d' failed: failed to upload image: HEAD https://656667300755.dkr.ecr.eu-west-2.amazonaws.com/v2/korifi-packages/e796b66e-5476-461f-8a0c-12277a29088d/blobs/sha256:3a647c1fbe83b8b2cf4761db7e6083d16f5978024b25d177addb2136da5ab9e9: unexpected status code 401 Unauthorized (HEAD responses have no body, use GET for details)"}
{"level":"info","ts":1664443715.8275518,"logger":"http-logger","msg":"response","correlation-id":"5a510c35-7c4c-4ffb-a585-3f57200c8f25","status":502,"size":131,"durationMillis":2801}

I was expecting korifi-packages:UUID not korifi-packages/UUID in the reference. Wondering if this means it's trying to create a sub repository and failing...

I had to abandon the attempted use of ECR and move to Google's equivalent. I do want to keep taking a look at this though. I suspect it's a permissions issue in ECR but cannot prove that just yet. The google equivalent gives more constructive error messages back to Korifi which is visible in the logs whereas ECR does not. I'll look into this one I issue PRs for the other two issues I've logged in the last week.

Blocking this while we figure out how to put our hands on an AWS account.


First continue not work

In the checkout page the first step continue not work..

We're making updates to this module for the latest supported versions of Magento ,the module in it's current state is tested and working on 2.3.x and 2.4.x (bar a few bugs which will be fixed in a coming release). If this is still an issue in supported versions please re-open the ticket.


JS autocomplete doesn't work for object literal shorthands

TS Template added by @mjbvz
TypeScript Version: 4.1.0-dev.20201026
Search Terms

suggest / suggestion
intellisense
object short hand

VSCode Version: 1.51.0-insider (7a3bdf4ee9588755d447aa1c3b5db4a123fc11a9)
OS Version: Arch Linux

Steps to Reproduce:

Suppose there are variables in the current context (be it local or global)
Create an object literal and try to use a variable in a shorthand style
No semantic autocompletion is offered, and textual autocomplete is only offered after 3 characters

GIF file to better demonstrate:

Does this issue occur when all extensions are disabled?: Yes

If you don't mind, I can work with this issue. FYI @andrewbranch

Sounds good @orange4glace, let me know if you get stuck. You have about a month and a half before we need to get a fix in.

@andrewbranch Thanks for the reply! I have one question for this.
I'm planning to change the code like this and with this code,
every tests are passed except completionListAtIdentifierDefinitionLocations_destructuring, which is,
/// <reference path='fourslash.ts' />

//// var [x/*variable1*/

//// var [x, y/*variable2*/

//// var [./*variable3*/

//// var [x, ...z/*variable4*/

//// var {x/*variable5*/

//// var {x, y/*variable6*/

//// function func1({ a/*parameter1*/

//// function func2({ a, b/*parameter2*/
                         ~
                         Getting Global Autocompletion

verify.completions({ marker: test.markers(), exact: undefined });

As l understood, the parenthesis { of the last line after the func2( is recognized as ObjectLiteral with any type rather than ObjectBinding since the line is getting following error,
'}' expected.ts(1005)
41259.ts(15, 16): The parser expected to find a '}' to match the '{' token here.
41259.ts(13, 16): The parser expected to find a '}' to match the '{' token here.

I'm curious that,

Is the test case valid one even though there's an syntax error? (Original PR for testcase : https://github.com/microsoft/TypeScript/pull/1767)
If so, Fixing the code to satisfy both original issue and the testcase seems pretty hard as I think.

I'm very new to this community and anything that can be helped would be much appreciated!

We definitely want test cases that have syntax errors because they reflect half-written code as though a user is actively editing, and we want completions to work in those cases. That said, any particular assertion in a test like this might become wrong and need to be updated. What completion is showing up there?

With my modified version, it autocompletes global symbols at,
....
var {x, y/*variable6*/
function func1({ a/*parameter1*/
function func2({ a, b/*parameter2*/
                    ~
                    Global symbols Autocompletion

whereas with Typescript v4.0.5, it autocompletes nothing.
Currently, the compiler parses the testcase code like this,
/* Statement #0 Starts (VariableStatement) */
var [x
/* Statement #1 Starts (VariableStatement) */
var [x, y
/* Statement #2 Starts (VariableStatement) */
var [.
/* Statement #3 Starts (VariableStatement) */
var [x, ...z
/* Statement #4 Starts (VariableStatement) */
var {x
var {x, y
function func1/* Statement #5 Starts (ExpressionStatement) */({ a
function func2({ a, b

The opening bracket { in the func2({ a, b is recognized as an ObjectLiteralExpression even though it looks like (or it should be) an ObjectBindingPattern, finally it autocompletes with global symbols.

I think that’s probably alright. Can you go ahead and open a draft PR so we can make a playground build?

Sure! I've just opened it. :)


Feature Request: start/stop/restart saved servers from CLI

Currently hotel and chalet allow a user/developer to start/stop each saved server from the daemon's web app (localhost:2000).
I would like to be able to start/stop/restart a saved server via the chalet CLI, as shown here:
chalet start --name myapp
chalet stop --name myapp
chalet restart --name myapp

Optionally Tail Server Output
For start and restart, I would like an option (enabled via a new CLI option such as --listen) to output the server's logs to STDOUT, so that I could run these commands from a terminal window (or VS Code terminal) and see the server logs right there. This listening/tailing of the server logs could be terminated by SIGTERM or SIGINT (ctrl-C), ending the CLI process and returning to the shell, but keeping the server running (until stopped in the Chalet daemon web app or via chalet stop --name myapp).
If Server Already Running
Running the start command and specifying a server that is already running should cause an error message to be emitted to STDERR:
"Server '<name>' is already running"

However, if the --listen option is given, then the CLI process should output the server's logs to STDOUT and continue to tail the logs until terminated by SIGTERM or SIGINT (ctrl-C).
Preserve Current Daemon Management via start/stop/restart
If the -n/--name argument is omitted, then start/stop/restart would manage the daemon, as they do now, so current usage of chalet start/stop/restart would not be affected in any way.

I just realized/remembered that the add and run commands use the name of the current directory as the server name if the -n/--name option is not given. This makes me think it might be better to add new CLI commands for the behaviors described above, such as start-server/stop-server/restart-server. However, those look awkward, and newcomers to hotel/chalet would likely get confused between the daemon commands (e.g. start) and these server commands.
I'm open to alternative ideas for command syntax, but I still think it's highly desirable to be able to start/stop/restart servers and to see their log output from the CLI.


Issue#21 Add Partition Pruner as a fallback when HMS pruning fails

When we pass few kind of predicates like in predicates to Hive Metastore Servie to fetch partitions it fails. As a fallback:

all the partitions are fetched
after fetching all partitions from Hive Metastore, they will be pruned. This will cause unneeded partitions are not processed.
Other small fix associated with Partition Pruning:
a. Don't allow predicates of form to be sent to HMS: <partition_column> != null <partition_column> = null

Approved

thanks @sourabh912


Drop extended_networking_minimal

With the merge of https://github.com/openshift/openshift-ansible/pull/7373 we now seem to be successfully using the NetworkPolicy in the GCE tests (eg https://openshift-gce-devel.appspot.com/build/origin-ci-test/logs/test_branch_origin_extended_conformance_gce/2669/), so extended_networking_minimal is pretty redundant, and since it's currently disabled anyway due to dind-related flaking, we should just kill it. (We still need the full extended_networking test to exist, so we can test all three networking plugins when needed, but we should eventually port that to use ansible rather than dind.)
I don't know the details of how CI works so I don't know if this PR is right at all...

@danwinship can you also delete references to it in openshift/release/cluster/ci/config/prow/config.yaml?

@danwinship can you rebase and I'll merge?

@stevekuznetsov done


Add isInternal and authDomain to user object to keep track of auth domain that authenticated the user

Description
Opening up a Draft PR to solicit feedback. This change will add 2 new fields to a User:

isInternal - Flag that indicates if this is an internal user
authDomain - this is the name (key in the authc section) of the auth domain that authenticated the user

This change also modifies the /_plugins/_security/authinfo API to return user_is_internal and user_auth_domain.

Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation)

New feature
Issues Resolved
Related to: https://github.com/opensearch-project/security/issues/2846
Testing
Tests added in the integration test framework for various different backends
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Closing this PR for now and operating under the assumption that if a username can be found in the internal user list that it is the internal user. I plan to re-open this as more work in done in figuring out how to resolve roles adhoc for different external IdPs.


[Documentation] Gateway Manager Design doc

Design Document for Gateway Manager

Codecov Report

Merging #517 (cc0c30f) into master (62e7728) will increase coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #517   +/-   ##
=========================================
  Coverage     35.76%   35.77%
  Complexity     1165     1165
=========================================
  Files           468      468
  Lines         11343    11343
  Branches       1460     1460
=========================================
+ Hits           4057     4058    +1
+ Misses         6732     6731    -1
  Partials        554      554

Impacted Files
Coverage Δ
Complexity Δ

...alcor/elasticipmanager/dao/ElasticIpAllocator.java
63.81% <0.00%> (+0.27%)
48.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62e7728...cc0c30f. Read the comment docs.

LGTM


rotate not backward compatible

Hi,
Recently I upgraded to version 0.2.0 and noticed that the code for setting the Y axis label no longer works:
 svg.append('text')
                  .attr('class', 'axis-label y')
                  .attr('text-anchor', 'middle')
                  .attr('x', myX)
                  .attr('y',myY)
                  .attr('transform', 'rotate(-90)')
                  .attr("font-size", 14)
                  .text('my y label');

I think this may have worked if I removed the x and y attrs and used a translate in  the transform. But shouldn't the library be backward compatible?
Likewise, the following code to rotate the x-axis tick labels had an odd behavior. When the page loads, the tick labels are properly aligned., But as soon as I redraw the graph (because data changes dynamically), the x axis tick tables go above the x axis
.attr("transform", function(d) {
                        // slightly slant the tick labels to avoid overlapping when they are too close by
                       return "translate(" + (focusAreaProperties.padding.left + d3.mean(metaGroups[2][d])) + "," +
                               (focusAreaProperties.y2 + (focusAreaProperties.offset.y * 2)) + ")rotate(45)"

Again, the problem is solved if I continue to use the older version. Unfortunately, I cannot tell you what the older version is because some other developer has added that file
It would be much appreciated if you could take a look.
Regards,
John

Hi there,
This project is no longer being maintained so this issue is being closed. Thank you for using r2d3 and taking the time to report this issue.


Updating readxl to read 1000 rows before guessing col_type like readr

I've got some fugly spreadsheets that I work with that have a lot of missing data and now that readr reads 1000 rows, it doesn't guess wrong very often, but readxl is still only using 100 rows. I believe I've found them all and changed them from 100 to 1000. I've tested this on various xlsx and xls documents and haven't been let down yet.

Thanks!
Observations: 4,289
Variables: 33

Apologies for being unable to share the file. 🙁

Adding bullet point to NEWS shortly.

Thanks for this. Closing because this will be addressed by #223.


feat: add useAddPasskey hook

Pull Request Checklist

[ ] Did you add new tests and confirm existing tests pass? (yarn test)
[ ] Did you update relevant docs? (docs are found in the site folder, and guidelines for updating/adding docs can be found in the contribution guide)
[ ] Do your commits follow the Conventional Commits standard?
[ ] Does your PR title also follow the Conventional Commits standard?
[ ] If you have a breaking change, is it correctly reflected in your commit message? (e.g. feat!: breaking change)
[ ] Did you run lint (yarn lint:check) and fix any issues? (yarn lint:write)
[ ] Did you follow the contribution guidelines?

PR-Codex overview
This PR introduces a new useAddPasskey hook in the Alchemy library for React applications. It allows users to add a passkey to an Embedded Account.
Detailed summary

Added useAddPasskey hook with mutation function and state handling
Created documentation for useAddPasskey including usage and parameters
Updated sidebar to include useAddPasskey link
Updated types.ts with BaseHookMutationArgs
Added ComponentWithAddPasskey example component

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

oh meta note on all of these (and I need to do this in my hooks as well). We need to support devs to pass onSuccess, onError and onSettled to the props.
Maybe we should have our hooks follow a format like:
const useSomeHook = (props: UseSomeHookProps, mutationOpts?: Omit<UseMutationOpts, "mutationFn" | "mutationKey">

I think this looks good. I'm not going to block on this because we haven't explicitly aligned as a team but I'd really like us to get in the habit of jsdocs. Viem does a good job and there is probably little work given AI can generate most of the text for us.
Example https://github.com/wevm/viem/blob/9cae9b77b31c0a1388795de8056704cfde004260/src/actions/wallet/sendTransaction.ts#L86

chatted here, will follow up after the stack merges with jsdocs everywhere


include either "BrowserAnimationsModule" or "NoopAnimationsModule" in your application

I am facing the issue with ng-pick-datetime. I have populated the date input field but whenever i click on the field it give me the following error.
Found the synthetic property @backdropAnimate. Please include either "BrowserAnimationsModule" or "NoopAnimationsModule" in your application.
How can i resolve this error ?

Import either those module in your app module


Icon reverts to original colour when refocused while validation has not passed

Noticed the below on the Log In example.
Click "Log In" without populating any of the fields and observe the validation error.

Click back into the Username field and observe that the icon reverts to blue.

Thanks,

Thank you! Added 52e05d5 only for textboxes for now. I will update other controls when I have some free time. Also notice the validation template has a different background color which is annoying. If it's not updated from the toolkit we have to hack it manually.

Great! Thanks!
I really need to become more familiar with the toolkit.


Zollnar - The number of jobs posted on https://solr.peviitor.ro (18) is not the same as the number of jobs posted on the company's website (21)

ENVIRONMENT: production
URL: https://solr.peviitor.ro/
Browser: Chrome
Device: Laptop
OS: Windows 11
STEPS TO REPRODUCE:

Open the URL in the browser
Execute Query for "Zollnar" company (18)
Check the number of jobs
Open https://www.zollner.ro/ro/cariera/posturi-disponibile (21)
Count the jobs
EXPECTED RESULTS:
The number of jobs is the same
The city of the jobs is "Romania" instead of "Satu Mare"
See the excel file with Zollnar's jobs and their locations
joburi Zollnar.xlsx

ACTUAL RESULTS:
The number of jobs posted on https://solr.peviitor.ro (18) is not the same as the number of jobs posted on the company's website (21)
On peviitor.ro page is the same, the number of jobs is not the same as on the company website
The locality must be the same as in the job advertisement on the Zollnar website, namley Satu Mare

Fixed!

Verificat, OK!


[FEATURE] Component examples should use crates/libraries

Affected project(s)

[ ] documentation
[X] examples
[ ] wasmCloud host
[ ] wasmCloud CLI (wash)
[ ] wasmCloud dashboard UI (washboard)
[ ] capability providers
[ ] provider bindgen
[ ] control interface client
[ ] other / not sure

Is your feature request related to a problem? Please describe.
No response
Describe the solution you'd like
Our examples should establish best practices. There are libraries that are now enable more ergonomic workflows when working with WASI API's like wasi-rs and wasmcloud_component).
Describe alternatives you've considered
No response
Additional context
No response

YES PLEASE

We're doing a much better job with this with wasmcloud_component, and the remaining work here is transitioning examples over to use that crate. For folks looking to contribute, this is a great first issue since it's constrained to just refactoring one example!


Make it easier to resize the visualisation window

Visualizations should have resize areas defined as presented below:

With the 1:1 zoom level, these areas should be 16px wide inside of visualization and should span to 3px wide outside of the visualisation.
Ideally there should be a debug-switch that allows for runtime changing of the 16px and 3px to allow for testing which feels more correct/natural.
On zoom out, these areas should grow with the equation area_size_multiplier = 1 + (1/zoom_level - 1) * 0.5. Moreover, if zoom level < 0.5, these areas should not be active anymore. For example, if zoom level is 1.0, then areas are defined as presented above. If zoom level is 0.5 (the nodes are 2x smaller than normal), the size of areas is multiplied by 1.5.
Similarly, on zoom in, these areas should string with the equation area_size_multiplier = 1 - (zoom_level - 1) * 0.5, and should not string more if zoom level > 2.

Refinement notes:

The configuration of area sizes could be done by editing top-level CSS variables.
The different scaling of draggable edges will be in a separate ticket.


Username option for scraping tweets

Hi!
I wanted to add another option for username as well, and I did end up adding it in the python file that I forked but whenever I host it, the changes are not updated.
What change do I need to make for the same to be reflected when I host it?

I tried to host it on streamlit, it's not working correctly but still somewhere, I will create pr so that you can see the change as well.
YEAH! It's a huge issue that snscrape is not working or limits user to the number of tweets it can scrape. Somehow, the site is working for me but the results are inaccurate/requires a lot of data cleaning.  Twitter has recently blocked all third party applications and requires people to buy twitter api in order to access data. None of the python libraries work as well.

Hey Aditi! Thanks for creating an issue.... Otherwise I wouldn't have touched it any sooner.... Now I have made a complete overhaul... Have a look at my app now and do ping me if you encounter any other issues...
Thanks and regard,
Nirmal


join the Binance-volatility-trading-bot discord

I wasn't sure where to post this, maybe CyberPunkMetalHead wants to post the link in the readme.txt
I've been using the bot and i'd like to discuss what values etc work well for other people. or for general questions
https://discord.gg/SqajQ8Tc
Regards

Thank you @CK92149  for your service <3


state: track substate ComputedVar that depends on parent Var

Follow on from #840, that introduces substate dependence checking in __getattribute__ so that each state tracks which variables its substates depend on. When marking a state dirty, if any dirty vars are dependencies of substate ComputedVar, then pass the dirty var down to the substate and also mark it dirty.
Fix #839
All Submissions:

[x] Have you followed the guidelines stated in CONTRIBUTING.md file?
[x] Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

New Feature Submission:

[x] Does your submission pass the tests?
[x] Have you linted your code locally prior to submission?

Changes To Core Features:

[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your core changes, as applicable?
[x] Have you successfully ran tests with your changes locally?

Edit 2023-04-23: rebased after 840 merged, diff should be clean now and ready for review!
Thanks for taking my previous patch! 🙏

@masenf I put up a PR to speed up the dependency management: https://github.com/pynecone-io/pynecone/pull/864/files
Once that's merged, can you rebase this logic into the new format? Appreciate all your contributions!!

Now that computed vars dependencies are tracked all the way to the base state, there's no need to track explicit substate var dependencies...can just push all of the dirty_vars down into each substate and let the substate determine which vars need to be recomputed.
This might still result in calculating some parent computed var multiple times, especially if lots of other variables depend on it. Would need to poke around and see if computed var values get cached already or not. The problem is solvable because the nature of substate tree means parent computed vars will always be recalculated before substate is called.

@picklelo i think we can get away with removing dirty_substate tracking altogether!

The issue that I was having with dirty_substates was how to know when a grandchild that depended on a parent var needs to update?
I was passing all of the dirty vars into each substate (to pass into their substates) because the child state doesn't know what the grandchild computed var dependencies are without checking (and it was getting complicated).
I ended up simplifying by removing dirty_substates because my approach basically put all substates in the dirty set, so it made more sense up exhaustively iterate, and have each state only responsible for its own computed var deps.
I'll try to rework this with more perf consideration in mind, but I don't want to do it by making the code excessively complicated, unless a clear perf benefit is noted.

@masenf I just tested it out on our pcweb repo since it has a huge state tree, and the process time for state updates went from around 0.0010 to 0.0026 so I think there is some perf impacts.
The mark_dirty method recursively calls self.parent_state.mark_dirty() so the grandchild state should mark all the states up to the parent dirty. Can we add the grandchild computed var in the parent var dependencies - and when the parent var updates we can mark the grandchild as dirty?

I'm working on a "deep state" test that has 1000+ state nodes for doing deterministic perf testing.
I'm wondering if the extra time on pcweb is due to inefficiency in the algorithm, or the fact that substate delta was just not recomputed at all before this change.

Yeah I don't think substate computed vars that only depended on parent vars were computed before. If the performance impact is too large, maybe this limitation is okay since it may be a rare use case. But if there's a clean solution would be good to include.

@picklelo the latest update reduces the process time by an order of magnitude over my previous patch. still not ready to go in (docstrings, tests, etc), but I wanted to post my progress thus far. The main wins comes from calculating the explicit var -> substate dependencies after tracking during __init__ and pushing those relationships into each parent state. Then when mark_dirty is called, it only has to mark substates dirty that actually depend on the changed vars, which makes a significant difference for variables that only have 1 deep dependency.
One of the things that's going to really hurt us in real world perf is recalculating the ComputedVar everytime it's accessed by some dependency. For high-level, expensive computed vars, this could make things really slow if a lot of substates depend on it.

I'm pretty happy with how this one turned out. With this PR (including #916 and #917), there appears to be no regression processing state updates in the pcweb project!
Give it a shot and let me know what you think.


[CLI]: Issue with API api.runs() not providing all data

I am trying to get the data from each run/step in a certain project. In order to do so I am using wandb.Api().
My issue is that I can succesfully get an object with all of my runs but for most of the columns all of the values are NaN except the last one. This strikes me as a bit odd since that data is being plotted in the website: https://wandb.ai/ so it should exist somewhere.

import wandb

api = wandb.Api()

runs = api.runs("NAME_OF_PROJECT")
runs[0].history().val_f1

0           NaN
1           NaN
2           NaN
3           NaN
4           NaN
         ...
135         NaN
136         NaN
137         NaN
138         NaN
139    0.809524
Name: val_f1, Length: 140, dtype: float64

Environment
WandB version: 0.13.6
OS: MacOS Darwin
Python version: 3.10.8

Thanks for your answer. My project name is "Master Thesis_knock"
I was able to get the info I want with this
api = wandb.Api()
runs = api.runs("Master Thesis_knock")
history = runs[0].scan_history()
# Note that not all steps log "val_f1" since some of them belong to a training step
history = [row for row in history if "val_f1" in row.keys()]


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-k8s-26-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2463

Questions ?
Please refer to the Backport tool documentation


Rename menu item "Drivers and Integrations" to "Ecosystem Catalog"

About
After all the tutorials have been refactored into the CrateDB Guide with https://github.com/crate/crate-clients-tools/pull/82 recently, the enumeration of catalog items here became a bit of a lost place.
This patch concludes the renovation by effectively repurposing that documentation section into a (preliminary) ecosystem software catalog/gallery, in the spirit how others are running them.
References

https://github.com/crate/crate-clients-tools/pull/107

Merge or close?

I agree with. Lets simply close this.

I would not rename the section, "Drivers and Integrations" sounds good to me.


Option to include max word count for description

Hi,
I noticed when book descriptions get too long, the YAML is not converted to preview mode. May I suggest to include an option to limit the word count for the book descriptions?
this is what happens:

Thanks for all your hard work!

I have also come across this problem and would like to see a solution.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target/pull/13 is merged into live branch.
Auto archive content to https://github.com/OPS-E2E-PPE/docs-archive-test-target.git

Docs Build status updates of commit e9f1084:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

archvieSourceFolder/TOC.yml
:white_check_mark:Succeeded

archvieSourceFolder/content/someContent.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Do not encourage to publish package-lock in the source code version control as a general practice

Opening this here again as the repository of the earlier reported issue got archived.
https://github.com/npm/npm/issues/20603
The npm documentation mentions it clearly that package-lock is always meant to be committed, leaving library authors unaware of the risk that their consumers might install different dependency versions than what they're using.
The documentation should be changed to mention that since package-lock files are ignored in published npm packages, library authors should exercise caution when choosing to commit them to source code.
Examples:

https://docs.npmjs.com/cli/v6/configuring-npm/package-locks#using-locked-packages

It is highly recommended you commit the generated package lock to source control

https://docs.npmjs.com/cli/v10/configuring-npm/package-lock-json#description

This file is intended to be committed into source repositories

Article with reasoning:
https://gajus.medium.com/stop-using-package-lock-json-or-yarn-lock-909035e94328
cc: @gajus

I disagree with this suggestion. While it is true that when a package is a dependent you are going to have a different tree, there is a large benefit to consistent development environments.
Yes, there is inconsistency when people install the package, but just because there are scenarios where there is a lack of consistency doesn't mean there is no value in a development environment that is mostly reproducible. Having consistency when collaborating across teams, reviewing PRs, and debugging 3rd party bug reports removes a ton of randomness and edge cases.
It is definitely possible to have a test run in CI that includes --package-lock=false if you want to test what happens when you install w/o the lock, but imho that is far more valuable as a daily task rather than the default scenario. Every tree is going to end up with a different layout / resolution of the tree based on the various other dependencies (prod / dev / peer), so trying to test out for all potential iterations is impossible... what is possible is ensuring you have consistency during development.
I'll leave this open for more discussion, but I would not approve any changes to the current documentation that would remove this recomendation.

I make an .npmrc in all my projects with package-lock=false and i gitignore lockfiles for this reason.
Defaulting a lockfile was the wrong decision in npm 5 and it’s the wrong decision now. “A consistent development environment” is a bug - in the few scenarios where you need reproducibility, you can use `--before to obtain it just as easily as you could use package-lock to disable the lockfile, but the better default is the experience consumers get, even if it’s inconsistent.

The documentation should be changed to mention that since package-lock files are ignored in published npm packages, library authors should exercise caution when choosing to commit them to source code.

Wow, I never considered that. In this context, would a shrinkwrap file be recommended? That page says:

It's strongly discouraged for library authors to publish this [shrinkwrap] file, since that would prevent end users from having control over transitive dependency updates.

Publishing a shrinkwrap file is exceedingly user-hostile, because it prevents end users from getting transitive dependency updates.
In an app - something not published - shrinkwrap and package-lock are identical, and should always exist.

Publishing a shrinkwrap file is exceedingly user-hostile, because it prevents end users from getting transitive dependency updates.

But it only prevents end-users from getting that libraries transitive dependencies, right? That's my interpretation and if it's incorrect I think the documentation should clarify (but that's unrelated to this issue). If I am correct, I don't understand how that can affect the end user, unless the library failed in some other way (e.g., should have used peer dependencies but didn't).

Yes, that's right - it affects the end user because the dependencies of your library can't be deduped, or upgraded, which means if there's a bug or security vulnerability, they're stuck with what's in the shrinkwrap file.

Yes, that's right - it affects the end user because the dependencies of your library can't be deduped, or upgraded, which means if there's a bug or security vulnerability, they're stuck with what's in the shrinkwrap file.

Ah, gotcha. That makes sense.

Just to chime in here --- the right move is to check in the lockfile always 100% of the time, for all projects: both packages and applications.

“A consistent development environment” is a bug

I'm sorry but I have to chuckle a bit here. You gotta admit, it is a little funny to say that stability and order is a bug. Sure, you want to catch incompatibilities sooner. But entertain the opposing position for a second --- some developers want to develop without surprises. Maybe stability is a good thing?
Let's also explore what is the easier option:

where you need reproducibility, you can use --before to obtain it just as easily ...

Maybe using --before is easy, but it's not just as easy a having a lockfile. You can't commit it or share it as easily. You have to get your CI to run it somehow. I assume we're talking about debugging certain dependency combinations. This is where a lockfile shines, because a lockfile is an easy way to snapshot a dependency tree in time. Create the dependency tree with --before, then capture it in a lockfile, and push it to a branch. Golden.
It's actually easy on the other end, if you have a lockfile and want to install bleeding edge deps. Just delete the lockfile and install from scratch. That's just as easy.
Moreover, sorry if this is preachy, but if you want to catch errors, you should do it in an organized way:

decide that you want to catch errors
install bleeding edge deps and look for errors

The alternative is this:

try to do work
be surprised by some random breakage
be forced to deal with it

Which one is more organized? If npm removes the lockfile by default, it would make users to randomly deal with bleeding edge bugs. Is it worth it?
You also said something that I need to pick at:

the better default is the experience consumers get

The question is: which consumers?

Consumers that just created a new project and are getting the bleeding edge versions of all deps?
Consumers that have existing projects and are adding your package?
Consumers that have already installed your package at some previous point in time, and are installing or updating other dependencies?

Consumers have lockfiles too, so your consumers can have any variation or combination of deps. Also, consumers don't install your devDependencies, so if you want your library project's node_modules to mirror a consuming project, it's going to be hard. You might as well set up a downstream project just to test the package.
The point is: given the variability in consumer installation scenarios, a lockfile is the best way to capture each scenario in a branch, so it can be developed and tested in an orderly way.
TLDR: lockfiles take the W. Always commit 100% of the time.

@matthias-ccri you can put before=2024-08-27 in .npmrc and commit it, and it'll work in CI too. so yeah, it's just as easy.


Add a convenience value method for numeric literals

From https://github.com/ruby/yarp/issues/1098#issuecomment-1613821135
For instance this is necessary for https://github.com/ruby/yarp/issues/1237#issuecomment-1684896663

Can you add some tests? Particularly just around the rational/imaginary ones.


Explore Avalanche node implementation

Explore Avalanche node Implementation.

Following is an overview of Avalanche node implementation, as of version v1.10.5 (July 2023).
General Structure
AvalancheGo is a client for the Avalanche network -- a blockchains platform. It's implemented in the Go programming language. The implementation ubiquitously uses the dependency injection pattern and abstract interfaces (Go's structural typing). Individual components interact mostly by calling each other's methods, often following the observer pattern. Go channels are sometimes used, but mostly internally withing smaller components. To protect component state from concurrent access, the code mostly relies on traditional synchronization mechanism, such as mutexes, reader-writer mutexes, condition variables, and atomic memory primitives. Active concurrent components usually provide a method named Dispatch or Start to spawn their background goroutines. To avoid repeating time-consuming operations, the code uses several types of generic caches provided by the cache package, which implement the Cacher interface.
The central piece of AvalancheGo is the Node structure, representing an instance of an Avalanche node. Node is initialized with the Initialize method, given the node configuration and logging interfaces. The method creates and initializes different high-level components; in particular, it creates the P2P networking and Chain Manager components, as well as finally starts the Platform Chain. The Dispatch method starts node's servers, in particular, P2P networking.
Communication
Avalanche nodes communicate with each other by establishing mutually TLS-authenticated peer-to-peer TCP connections. The networking layer is represented with the Network interface, which is implemented by the network structure. Node creates an instance of Network using the NewNetwork constructor. The supplied message.Creator interface is used for deserializing inbound messages and creating network maintenance messages; ExternalHandler is used to dispatch inbound messages for handling by other components, as well as to notify about connected and disconnected peer nodes. Node supplies throttled Listener and Dialer instances, which limit the number of connections per second; ChainRouter (possibly wrapped) is used as ExternalHandler for dispatching inbound messages.
To support sending messages, Network extends the ExternalSender interface; the Send method  allows sending messages to a specific set of nodes, whereas the Gossip method sends a message to a random group of nodes in a subnet. Both method accept the subnet ID and an instance of the subnets.Allower interface to determine nodes that are allowed to connect to the subnet; they return a set of node IDs to which the message was successfully enqueued for sending.
The network layer initiates connections to some other nodes when the Track or ManuallyTrack method is called; the former happens when discovering new peers through gossiping peer lists, the latter during node bootstrapping. The outbound connections are established by calling the dial method, which spawn a new goroutine attempting to dial the remote peer node. Inbound connections are handled in a loop executed by the Dispatch method, which continuously accepts new connections and spawns a new goroutine for each new connection.
Once a connection with a remote peer is established, the upgrade method is called to establish a mutually authenticated TLS connection on top of the plain TCP connection. The networking layer tries to avoid duplicate connections. Finally, it calls the peer.Start constructor function to create a new instance of the peer structure, which implements the Peer interface. The function expects an instance of the MessageQueue interface as parameter, which gets created with the NewThrottledMessageQueue function. The queue is unbounded, but drops pushed messages using the supplied instance of the OutboundMsgThrottler. Before returning an instance of Peer, the constructor spawns three goroutines running the readMessages, writeMessages, and sendNetworkMessages methods.
The (peer).readMessages method reads messages from the connection, deserializes them, and calls the handle method to handle the received messages. The handle method handles network maintenance messages by calling the corresponding methods of peer and dispatches all other messages to the instance of the InboundHandler interface that was supplied to (peer).Start as the Router field of the peer.Config structure. This is the same as the ExternalHandler instance passed to NewNetwork.
The (peer).writeMessages method, after having sent a handshake message to the remote peer, enters a loop retrieving messages from the peer message queue and writing them to the connection using the writeMessage method. Messages get pushed into the message queue by the Send method, which is called internally by some peer methods for sending network maintenance messages and by the send method of network used to implement its Send and Gossip methods.
The (peer).sendNetworkMessages method takes care of sending network maintenance messages. It enqueues a PeerList message whenever the StartSendPeerList is called by the network layer. It also periodically sends Ping messages.
Consensus Protocol Implementation
Avalanche uses Snowman as the default consensus protocol, a variant of the Avalanche consensus protocol for linear chains. The high-level architecture of the Avalanche chain implementation is described in snow/README.md. The core Snowman consensus logic is represented by the snowman.Consensus interface implemented by the snowman.Topological structure. Consensus is driven by an instance of the snow/engine/common.Engine interface implemented by the snowman.Transitive structure. The Engine interface mostly defines methods for handling different messages received from the network and other components. Transitive engine receives an instance of Consensus as a field in the Config structure when it gets created by the snowman.New function. For sending messages, it also receives an instance of the Sender interface as a field in Config. It initializes Consensus in its Start method.
The consensus engine tracks the state of other validator nodes, outstanding requests, and unfulfilled block and vote dependencies; it relies on Consensus to determine the preferred and accepted blocks. It uses events.Blocker to keep track of missing dependencies. The (Blocker).Register registers a pending item, which has unfulfilled dependencies. The item is represented with the Blockable interface; the issuer and voter structures in the snow/engine/snowman package implement this interface. The (Blocker).Fulfill method is used to notify the pending items about a fulfilled dependency, calling their Fulfill methods. Similarly, the (Blocker).Abandon notifies the pending items about an abandoned dependency, calling their Abandon method.
The consensus engine is completely single-threaded. Asynchronously received messages and other events are handled by the handler structure from the snow/networking/handler package, implementing the Handler interface. It synchronizes access to the consensus engine by taking the consensus lock, the Lock filed of the snow.Context structure, which is also embedded into ConsensusContext. The SetEngineManager method assigns handler an instance of EngineManager, which provides access to the consensus engine.
The handler.New function creates a new instance of Handler. It creates two instances of MessageQueue and stores them in the syncMessageQueue and asyncMessageQueue fields. Those queues are used to hold messages to be processed synchronously and asynchronously, respectively. External messages get pushed to the corresponding queue, depending on the message type, by the Push method. The message queues are unbounded. They temporarily postpone messages whose senders caused excessive CPU usage as reported by a CPU resource tracker. Messages have an expiration time; expired messages get dropped immediately after being popped from a message queue by the popUnexpiredMsg method.
When started with the Start method, handler spawns three goroutines executing the dispatchSync, dispatchAsync, and dispatchChans methods.
The (handler).dispatchSync method retrieves messages from syncMessageQueue and passes them to the handleSyncMsg method, which dispatches them to the corresponding method of the consensus engine holding the consensus lock.
The (handler).dispatchAsync method retrieves messages from asyncMessageQueue and passes them to the handleAsyncMsg method, which requests execution of the executeAsyncMsg method on a worker pool. The executeAsyncMsg method dispatches the message to the corresponding method of the consensus engine without holding the consensus lock because those messages are not handled by the consensus engine itself, but forwarded to the VM.
The (handler).dispatchChans method dispatches internal messages generated by the handler itself and the VM. Messages from the VM are received from the Go channel supplied as parameter to  handler.New. Internal gossip requests are generated periodically. Internal timeouts are generated after calls to the RegisterTimeout method. Internally generated messages are passed to the handleChanMsg method, which dispatches them to the corresponding method of the consensus engine holding the consensus lock.
Consensus engines and handlers are created and managed by the chains.manager structure, which implements the chains.Manager interface; it is a part of Node. It creates the Sender for the consensus engine using the New function from  the snow/networking/sender package, which is a wrapper around an instance of the ExternalSender interface.
The wrapper directs messages for own node to an instance of the Router interface rather than via the wrapped ExternalSender. It also registers outbound requests with the Router, so that it fires a timeout if there is no response to the request. Chain Manager uses the Net and Router fields of its ManagerConfig as ExternalSender and Router, respectively. Node supplies its instance of Network and the ConsensusRouter field from its Config when creating Chain Manager, whereas ChainRouter is used as ConsensusRouter.
ChainRouter implements the ExternalHandler interface and is responsible for handling inbound messages received by the network layer. Sender registers outbound requests with ChainRounter by calling the RegisterRequest method. ChainRounter keeps track of outstanding requests and registers them with timeout.Manager. When an expected response or failure message is received by the HandleInbound method, it registers the response or removes the request from the Timeout Manager; otherwise, Timeout Manager call the supplied closure artificially delivering the timeout message passed by Sender to RegisterRequest. Inbound messages are passed to the corresponding chain Handler registered with the AddChain method by the Chain Manager.
Tracing and Controlling Execution
AvalancheGo implements a structured diagnostic logging mechanism in the utils/logging package, based on the go.uber.org/zap package. It also supports distributed tracing functionality implemented in the trace package, based on the go.opentelemetry.io/otel/trace package. Various metrics are collected using the github.com/prometheus/client_golang/prometheus package. Individual components implementing the health.Checker interface can be monitored for health status if registered with the Health service. The snow/networking/tracker package implements tracking of CPU and disk resource usage.


How to get the size of physical address and virtual address in RISCV?

Hello everyone，I recently saw that the cpuinfo in Linux for RISC-V architecture does not include the number of physical and virtual address bits supported by the current CPU like X86.

I found that the X86 CPU can use the cpuid instruction to obtain the physical address and virtual address information supported by the CPU,

but there seems to be no similar instruction in RISC-V. Will we add similar instructions in the future or use other methods to pass this information?

The OS can figure out which virtual address sizes are supported by attempting to program satp and seeing if the write succeeds.  It can then remember this information in a variable in memory.
Probing for the physical address size is trickier; it requires filling in a PTE and seeing if accessing it raises an exception.  It's still doable, though.  But this information isn't really useful, anyway.  The important thing is that the OS knows where memory is located.  (This might come from e.g. the devicetree.)

Is this something that we want to document in guides? maybe a tip?

In practice, I don't think that knowing the maximum physical address is very useful; the more important thing is for the OS to know what devices live at which addresses.  That one's a platform issue more than it is an ISA issue.

The OS can figure out which virtual address sizes are supported by attempting to program satp and seeing if the write succeeds. It can then remember this information in a variable in memory.
The size of a physical address isn't really interesting; the important thing is that the OS knows which physical addresses contain which memories, devices, etc. (This information might come from devicetree.) The physical address width is immaterial.

Thanks for your reply. Can I assume that RISCV currently has no intention to add an instruction like X86's cpuid to obtain address width information? In addition, do we have some Spec-specified method to obtain the address of the device, or is it just determined by the platform (such as device tree)?

AFAIK, there are no current plans to add a CPUID-like instruction.  The memory map is a platform matter, rather than an ISA matter, and yes, the platform might say something like use devicetree.

Adding to Andrew's response, a few years ago the addition of a CPUID-like instruction was explored, but ultimately the plan was to develop what is called the Unified Discovery extension for RISC-V standard "low-level" discovery (e.g. by M-mode firmware).  This information can then be used, for example, to populate industry-standard Device Tree or ACPI structures that are passed to an OS.  As far as discovery between an OS/hypervisor and user applications, that would be based on standard OS mechanisms such as ones in Linux.

Closing this issue then. Please reopen if needed.


[clusters] Network Commissioning Server

Problem
We don't have network commissioning cluster on the server side.

Summary of Changes
Add clusters for network commissioning to the lighting app server side. After this is merged and tested, will scale to all example apps.
Also, the AddThreadNetwork temporary does not work, it will be fixed by full support of bytes string.
Test
chip-device-ctrl > connect -ip fd12::2 12345678
...
Device temporary node id (**this does not match spec**): 269170
chip-device-ctrl > zcl NetworkCommissioning AddWiFiNetwork 269170 1 0 ssid=test credentials=12345678 breadcrumb=0 timeoutMs=1000
...
chip-device-ctrl > zcl NetworkCommissioning EnableNetwork 269170 1 0 networkID=test breadcrumb=0 timeoutMs=1
...

The device can connect to the network now.

Fixes #3945

Hi, pull latest code and compiling  chip-device-ctrl:
zcl  NetworkCommissioning AddWiFiNetwork 269170 1 0 ssid=test credentials=12345678 breadcrumb=0 timeoutMs=1000
An exception occurred during process ZCL command:
Parsing Error: value should be encoded in encoding:encodedvalue format

No clue to solve it.

Hi, pull latest code and compiling chip-device-ctrl:
zcl  NetworkCommissioning AddWiFiNetwork 269170 1 0 ssid=test credentials=12345678 breadcrumb=0 timeoutMs=1000
An exception occurred during process ZCL command:
Parsing Error: value should be encoded in encoding:encodedvalue format

Test it in many ways. No clue to solve it.

zcl  NetworkCommissioning AddWiFiNetwork 269170 1 0 ssid=str:test credentials=str:12345678 breadcrumb=0 timeoutMs=1000
I will add some more documents about this.


Decrypting smime.p7m attachments

Hi,
I have troubles getting the content of an email that is encrypted and signed (see attachment). The email itself only contains one smime.p7m attachment.
I use following code:
FileInputStream f = new FileInputStream(new File("email.eml"));

FileInputStream key = new FileInputStream(new File("keystore.p12"));
FileInputStream key2 = new FileInputStream(new File("keystore.p12"));

SmimeKeyStore smimeKeyStore = new SmimeKeyStore(key, "pw".toCharArray());
String alias = smimeKeyStore.getPrivateKeyAliases().iterator().next();

Pkcs12Config yourPkcs12Config = Pkcs12Config.builder()
        .pkcs12Store(key2) // path, File or InputStream
        .storePassword("pw")
        .keyAlias(alias)
        .keyPassword("pw")
        .build();

Email mergedEmail = EmailConverter.emlToEmail(f, yourPkcs12Config);
List<AttachmentResource> list = mergedEmail.getDecryptedAttachments();

for (AttachmentResource r : list) {
    play.Logger.info(r.readAllData());
}

But the output of r.readAllData() gives me binary data only.
When I open the email in Thunderbird it is decrypted correctly, it contains text only.
Can you please help me to get the actual content?
email.txt

Hi, my apologies for the late reply, but I'm currently unable to look into this as I have recently become father to our second child. Takes rather lot of attention I'm finding out. On top of that, the topic here is a rather complicated and needs some proper debugging and research.
I'm always open to suggestions and pull requests though. Sorry I don't have better news for the moment...

Congratulations!!! The customer has to wait then lol :)

When I open this file in Thunderbird, I don't see any attachments...

Probably because you cannot decrypt it? There is definitely an attachment in it.

I'm wrong, sorry.
If you look at the content of the file (email.txt) you can see a Content-Disposition: attachment; filename="smime.p7m" part.
That's the actual content (= text) of the email. But it's encrypted.
Thunderbird is able to the email without problems and shows the content (private key provided). The library is not.
You can understand the problem now?

Unless you provide the private key along with the encrypted email, I can't do a root-cause analyses without the private key. The sample emails included in Simple Java Mail's tests all work fine.
Can you provide the private key? Perhaps email it to me directly so I can analyse? I won't publish it in the GIT repo.

I have been provided with a matching keystore and have been analysing the issue. It turns out that S/MIME support was incomplete for the backwards compatible legacy signing protocol: "smime-type=signed-data". Nowadays signing is done using certificate added to an attachment, but legacy style is that the entire attachment content is wrapped similarly to how encrypted content still is wrapped (enveloped content).
I've fixed the support (that was pretty complex!), but I've got it working nicely now. Wrapping up on the code quality and then I'll be shipping out a new release...

Fix released in 6.4.1. Again, greatly appreciated your trust and cooperation, I could not have analysed and fixed this without it!


Design: Design API Programmers Guide on the RS Website

User story
As a sender, I need an accessible, updated, online Programmer's Guide, so that I can successfully connect to the ReportStream API.
Background & Context
This Programmer's Guide ticket captures the final design  and documentation work needed to handoff to the WebX engineering team. The design effort leverages the work already completed in Wireframe the API Programmer's Guide #8481, and will fill in the remaining gaps (namely, pixel perfect documentation of new components, and other technical specs).
What you need to know

Figma: Programmer's Guide Wireframes
Programmer's Guide Design Brief
Programmer's Guide Content

Acceptance criteria
High fidelity design of new components

[x] Code snippet, in-line
[x] Code snippet, multi-line
[x] Tip callout
[x] Prose style: H4
[x] Prose style: Body (small)

Documentation for WebX handoff

[x] Style guide of new components for WebX eng handoff
[x] Annotated high-fidelity page template with spacing, type styles
[x] Annotated wireframes for WebX eng handoff, outlining existing Trussworks components, external links, etc.
[x] Location on RS website, IA structure

Review

[ ] Review with content
[ ] Design lead (via tag in ticket)
[ ] Help WebEx engineering team scope the necessary engineering work and create tickets in #5814

@chris-kuryak I wrote this ticket and gave it 8pts. I recommend capturing the content finalization + review work in a separate ticket (cc @audreykwr).

@jillian-hammer looks great & agree with the points

I previewed this deliverable with the eng team and it fulfills their needs for a smooth handoff!
This is ready for content and design review! @audreykwr and @lizzieamanning can you give these a once over? did I miss anything? any inaccuracies/super off?

Full prototype (design + wireframes, annotated)
Style guide for new components & type styles

AI: to be reviewed by Audrey and Lizzy

@Andrey-Glazkv thanks for the bump! @jillian-hammer left a few minor comments, otherwise looks good to me.

Putting the content docs here.
Copy doc table format with design components noted
Traditional doc format Note: H tags in this doc do not currently match Figma components

@audreykwr how do you intend these two different docs being used? Is the first one the new format you were trying out?

hey @jpandersen87, feel free to review the links above ahead of the Friday hand-off call!


Upgrade Node to version 16

Needs resolution of problem with node-gyp (see Pacts / test at Actions.)

Needs resolution of problem with node-gyp (see Pacts / test at Actions.)

In this PR yarn.lock got deleted. Maybe not intended... :smile:

In this PR yarn.lock got deleted. Maybe not intended... 😄
😁 I've chosen the strategy to refresh it entirely with yarn install.

Anyways, I reinserted the old yarn.lock and it was still not working.
I installed nodegit (that uses node-gyp) as a dependecy, upgraded the lock file, and removed nodegit, and although it works now locally it doesn't in Actions.
There is an old issue about nodegit and node 16. https://github.com/nodegit/nodegit/issues/1840

Okay, I see. We need to go through the dependencies anyway and update them... Maybe we can replace nodegit?!

Okay, I see. We need to go through the dependencies anyway and update them... Maybe we can replace nodegit?!

It is not our direct dependency, but a dependency from a dependency. Fortunately, one can find it only at inyono/changelog and Jonas has already released the version 0.4.0 of it, which does not use nodegit.  I've upgraded it and it works now.


Lab03

Nice date/time handling and a good exercise in defining functions.
Make sure  you explore more options and print some intermediate results.


added "HiddenSlides" property to app.xml.json

I've added this property to 1.1.0. Thanks for the PR.


Tags are not being well generated

I am using @material-ui/core version 4.11.4 and React version 17.0.1. I am facing some issues with the tags of Material components. I want to have my component under the tag breadcrumb-XXX, but is not working. Here is my code:
const breadcrumb = createGenerateClassName({
  seed: "breadcrumb"
});

export const Breadcrumb = ({ icon, children, style }: IBreadcrumbProps) => {
  const classes = useStyles();

  return (
    <StylesProvider generateClassName={breadcrumb}>
      <Grid
        container
        alignItems="center"
        className={classes.breadcrumb}
        style={style}
      >
        {icon}
        {children}
      </Grid>
    </StylesProvider>
  );
};

The tag generated:
<div class="breadcrumb-MuiGrid-root jss18 breadcrumb-MuiGrid-container breadcrumb-MuiGrid-align-items-xs-center">
...
</div>

As you can see, there is a class jss18 generated without any tags. Because of that, I am having issues with overlapping styles on my portal. Is there a way to fix this? Am I doing something wrong?

It doesn't look like this bug report has enough info for one of us to reproduce it.
Please provide a CodeSandbox (https://material-ui.com/r/issue-template-next), or a link to a repository on GitHub.
Here are some tips for providing a minimal example: https://stackoverflow.com/help/mcve

Closing since no reproduction was provided within 14 days.


expcertificates: add requestmanager controller

What this PR does / why we need it:
This builds on top of #2810 and adds the 'requetmanager' component as per the design document.
Special notes for your reviewer:
This is WIP and I've still not written unit tests for this controller, but pushing it up so I can see e2e tests running in CI with this enabled!
Release note:
NONE

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/kind feature

/lgtm


M3-1748 Remove expanding panels for DNS landing and align add buttons

this PR also adjusts mobile styles for this display

Coverage remained the same at 27.746% when pulling fa47cfd88c935bd348c243591ad78333b1b1a292 on alioso:M3-1748 into 3d94f9acd53387b87f6480ee3631900cbd772bac on linode:develop.


Break-point of collapsible panel

Description
According to the docs, the ui config for UDashboardPanel looks like this:
{
  wrapper: 'flex-col items-stretch relative w-full',
  border: 'border-b lg:border-b-0 lg:border-r border-gray-200 dark:border-gray-800 lg:w-[--width] flex-shrink-0',
  grow: 'flex-1',
  collapsible: 'hidden lg:flex',
  slideover: 'lg:hidden'
}

The ui attribute for UDashboardPanel however only exposes this:
(property) ui?: Partial<{
    wrapper: string;
    border: string;
    grow: string;
}> | undefined

In other words, collapsible and slideover can't be configured. Why would I need that: To control the break-point. In my case, I'd like one panel to only become visible (i.e., flex) at the xl break-point, not at lg.
Not sure if this is a bug or intentional behavior?
Additional context
No response

This has been implemented in https://github.com/nuxt/ui-pro/commit/d27cbf55d50a9805ca1190f6cad421565db0a980 but not released yet. Will try to make a release next week :)


Add rabbitmq module to ballerina distribution

Purpose

$title

Check List

[x] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #19527 into ballerina-1.0.x will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff                @@
##           ballerina-1.0.x   #19527   +/-   ##
================================================
  Coverage            15.73%   15.73%
================================================
  Files                   47       47
  Lines                 1265     1265
  Branches               197      197
================================================
  Hits                   199      199
  Misses                1053     1053
  Partials                13       13

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c059112...d5c7f3f. Read the comment docs.


[Feat] Add Document Export Functionality

What this PR does / why we need it:
This pull request introduces a new document export feature for CodePair. Currently, users can write Markdown documents in CodePair, but there was no functionality to export these documents into various file formats. With this update, users can now export their Markdown documents as PDF, TXT, and DOCX files.
Which issue(s) this PR fixes:

Fixes #191
Special notes for your reviewer:
The implementation uses local storage to save the document title for file naming purposes. I would appreciate your feedback on whether there are better alternatives for this approach.
Does this PR introduce a user-facing change?:

Added document export functionality to CodePair.
Users can now export their Markdown documents to PDF, TXT, and DOCX formats.
This feature allows users to save and share their Markdown documents in multiple formats.

Additional documentation:

Checklist:

[x] Added relevant tests or not required
[x] Didn't break anything

Summary by CodeRabbit

New Features

Introduced a DownloadMenu component for exporting files in PDF, TXT, and DOCX formats.
Enhanced DocumentCard to save document names in local storage.
Added PreviewRefProvider to manage Markdown preview references.

Improvements

Integrated DownloadMenu in the DocumentHeader for easier access to download options.
Implemented local storage utility for managing document names.

Bug Fixes

Cleaned up local storage when the Preview component unmounts to prevent data retention issues.

This is a problem I learned from the additional modification.
If the client creates a markdown and extracts it as a PDF, there is a problem that the image cannot be rendered.
I can solve it if I make a PDF on the server, but can I modify the backend code together and solve it in the pull request now?

Please check formatting and lint rules.
You can check it by,
Frontend

Formatter: npm run format (It also perform auto formatting.)
Linter: npm run lint
Backend
Formatter: npm run format(It also perform auto formatting.)
Linter: npm run lint

Notify
This is a problem I learned from the additional modification. If the client creates a markdown and extracts it as a PDF, there is a problem that the image cannot be rendered. I can solve it if I make a PDF on the server, but can I modify the backend code together and solve it in the pull request now?

While modifying to convert Markdown text to PDF, it is necessary to check if there is an option to render images. If it is determined to be impossible on the frontend, backend code modifications are possible. However, the first option to consider should be to make changes on the frontend alone, as this seems optimal.

May I proceed with the code review?
If you are ready, please click the 'Re-request review' button.

@minai621
Since html-pdf-node has Puppeteer as a dependency, it requires the installation of Chromium. For now, we will add this to the installation script, but we should consider implementing an alternative solution in the future.


[Blazor] Semi-colon inside @() raises syntax errors

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
Inside a razor page:

foo

This works as expected.

bar

This raises CS0116 and CS0115 errors.
I don't know if this is intended behaviour, but the errors thrown aren't very indicative of the error at hand.
Expected Behavior
Not entirely sure, either a better error message or for it to compile properly.
Steps To Reproduce
See description above.
Exceptions (if any)
CS0115, CS0116
.NET Version
8.0.302
Anything else?
No response

Behavior is intended but agree we should have a razor error here if possible.

Related: https://github.com/dotnet/razor/issues/10383


0.9.4 WAL death spiral

Hi,
Trying to see if Influx can replace a TSDB cluster.  Right now, just one decently spec'd hw system and no clustering.
The wal starts growing, then RAM usage goes up until swapping and it's unusable.  ZFS and the disks are bored.  It looks like one 'shard' is being overused but I haven't looked into how WAL and boltdb work/dont't work together yet.
I'm wondering if we could perhaps skip things like compression and compaction on filesystems like ZFS?
root@influx-1:/data/influxdb/data/llnw-phx7-move/default # ls -lh /data/influxdb/data/llnw-phx7-move/default/16
-rw-r--r--  1 influxd  influxd    12G Sep 15 18:19 /data/influxdb/data/llnw-phx7-move/default/16

[opentsdb] 2015/09/15 18:38:30 failed to write point batch to database "llnw-phx7-move": write failed: engine: write points: write throughput too high. backoff and retry
[write] 2015/09/15 18:38:30 write failed for shard 16 on node 1: engine: write points: write throughput too high. backoff and retry
[opentsdb] 2015/09/15 18:38:30 failed to write point batch to database "llnw-phx7-move": write failed: engine: write points: write throughput too high. backoff and retry

[data]
  dir = "{{ datadir }}"

  # The following WAL settings are for the b1 storage engine used in 0.9.2. They won't
  # apply to any new shards created after upgrading to a version > 0.9.3.
  max-wal-size = 104857600 # Maximum size the WAL can reach before a flush. Defaults to 100MB.
  wal-flush-interval = "10m" # Maximum time data can sit in WAL before a flush.
  wal-partition-flush-delay = "2s" # The delay time between each WAL partition being flushed.

  # These are the WAL settings for the storage engine >= 0.9.3
  wal-dir = "{{ waldir }}"
  wal-enable-logging = true

  # When a series in the WAL in-memory cache reaches this size in bytes it is marked as ready to
  # flush to the index
  # wal-ready-series-size = 25600

  # Flush and compact a partition once this ratio of series are over the ready size
  # wal-compaction-threshold = 0.6

  # Force a flush and compaction if any series in a partition gets above this size in bytes
  # wal-max-series-size = 2097152

  # Force a flush of all series and full compaction if there have been no writes in this
  # amount of time. This is useful for ensuring that shards that are cold for writes don't
  # keep a bunch of data cached in memory and in the WAL.
  # wal-flush-cold-interval = "10m"

  # Force a partition to flush its largest series if it reaches this approximate size in
  # bytes. Remember there are 5 partitions so you'll need at least 5x this amount of memory.
  # The more memory you have, the bigger this can be.
  # wal-partition-size-threshold = 20971520

[opentsdb]
  enabled = true
  # bind-address = ""
  database = "llnw-phx7-move"
  # retention-policy = ""

Looks very similar to https://github.com/influxdb/influxdb/issues/4096, except I can assert that I have gobs of I/O to spare.

What are your input data rates? Check out 'SHOW STATS' for counts on your openTSDB input.

batches_tx
batches_tx_fail
connections_active
connections_handled
http_connections_handled
points_tx
tl_bad_float
tl_bad_line
tl_bytes_rx
tl_connections_active
tl_connections_handled
tl_points_rx
tl_read_err

18312
119097
10823
200101
141
18312000
1136205
6
11149264986
10807
199944
138551149
6

Adding some color: Looks to be stuck around 11k/s on mean(connections_active|tl_connections_active); some spikes to 25k/s (which sustaining that would be more ideal...).
That's "visible" with _internal using opentsdb.

I can offer to arrange a time for you to login since this may be an interesting scale test system

Thanks for the numbers.
I'll leave it to @pauldix since he made the latest changes to the WAL.

Just watching system during WAL recovery, something is amiss in this release.  It's using around 30GB RSS without any metrics coming in (the services are not even up yet) for 8GB in the data dir.

Working on the same cluster with @kev009 and @christianchristensen.
Today: Trying to tune the WAL settings (reducing them by an order of magnitude) so that disk writes occur earlier, the memory usage is up to 40G.
Likewise, the system is mostly idle -- and processing the ~70MB (!) WAL takes >15 minutes, even when there's a decent amount of memory to spare, the CPUs and disks are mostly idle.
How does one make WAL processing at startup faster at the expense of system resources?

@xmj how are you getting into this state? How many points are you writing across how many series? Do you know what the total throughput is?
I'm shocked that the WAL is getting up to 70MB if you have the memory settings low. With the new code it should start failing writes and putting pressure on the clients well before it gets to that size.
Any extra info you can give would be helpful.

@xmj also, what kind of hardware are you running on? Particularly the disks and their IOPS.

the CPUs are two Intel(R) Xeon(R) CPU E5620  @ 2.40GHz, 96MB RAM.
Disk space is provided by a 32 disk-array on ZFS as striped mirror (RAID10 if you will), with to SSDs for slog (ZIL) purposes.
Disk I/O is in the low single digits, and it looks like there's 370/630 r/w operations per seconds.
As for total throughput -- see @kev009's post from earlier. Given that InfluxDB just came up again the load isn't at full capacity yet.

@pauldix I've changed the memory settings before restarting, but even now WAL is growing. 27MB and counting.
That's with
[data]
  dir = "/data/influxdb/data"

  # The following WAL settings are for the b1 storage engine used in 0.9.2. They won't
  # apply to any new shards created after upgrading to a version > 0.9.3.
  max-wal-size = 1048576 # Maximum size the WAL can reach before a flush. Defaults to 100MB.
  wal-flush-interval = "30s" # Maximum time data can sit in WAL before a flush.
  wal-partition-flush-delay = "2s" # The delay time between each WAL partition being flushed.

  # These are the WAL settings for the storage engine >= 0.9.3
  wal-dir = "/data/influxdb/wal"
  wal-enable-logging = true

  # When a series in the WAL in-memory cache reaches this size in bytes it is marked as ready to
  # flush to the index
  wal-ready-series-size = 4096

  # Flush and compact a partition once this ratio of series are over the ready size
  wal-compaction-threshold = 0.1

  # Force a flush and compaction if any series in a partition gets above this size in bytes
  wal-max-series-size = 1048576

  # Force a flush of all series and full compaction if there have been no writes in this
  # amount of time. This is useful for ensuring that shards that are cold for writes don't
  # keep a bunch of data cached in memory and in the WAL.
  wal-flush-cold-interval = "1m"

  # Force a partition to flush its largest series if it reaches this approximate size in
  # bytes. Remember there are 5 partitions so you'll need at least 5x this amount of memory.
  # The more memory you have, the bigger this can be.
  wal-partition-size-threshold = 1048576

It's growing and still accepting writes? That doesn't seem right. Can you
verify the version by checking a response header?
On Wed, Sep 16, 2015 at 2:42 PM, Johannes Meixner<EMAIL_ADDRESS>wrote:

@pauldix https://github.com/pauldix I've changed the memory settings
before restarting, but even now WAL is growing. 27MB and counting.
That's with
[data]
dir = "/data/influxdb/data"
The following WAL settings are for the b1 storage engine used in 0.9.2. They won't
apply to any new shards created after upgrading to a version > 0.9.3.
max-wal-size = 1048576 # Maximum size the WAL can reach before a flush. Defaults to 100MB.
wal-flush-interval = "30s" # Maximum time data can sit in WAL before a flush.
wal-partition-flush-delay = "2s" # The delay time between each WAL partition being flushed.
These are the WAL settings for the storage engine >= 0.9.3
wal-dir = "/data/influxdb/wal"
wal-enable-logging = true
When a series in the WAL in-memory cache reaches this size in bytes it is marked as ready to
flush to the index
wal-ready-series-size = 4096
Flush and compact a partition once this ratio of series are over the ready size
wal-compaction-threshold = 0.1
Force a flush and compaction if any series in a partition gets above this size in bytes
wal-max-series-size = 1048576
Force a flush of all series and full compaction if there have been no writes in this
amount of time. This is useful for ensuring that shards that are cold for writes don't
keep a bunch of data cached in memory and in the WAL.
wal-flush-cold-interval = "1m"
Force a partition to flush its largest series if it reaches this approximate size in
bytes. Remember there are 5 partitions so you'll need at least 5x this amount of memory.
The more memory you have, the bigger this can be.
wal-partition-size-threshold = 1048576
—
Reply to this email directly or view it on GitHub
https://github.com/influxdb/influxdb/issues/4119#issuecomment-140838636.

I've patched the FreeBSD port to use 0.9.4 -- https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202981.
If I'm not mistaken it should be at d81618c

@pauldix here is the block and heap output from pprof https://gist.github.com/kev009/cbe1323cbeb71c70926b

Just for good measure: X-Influxdb-Version: 0.9 and SHOW DIAGNOSTICS returns "0.9"

Closing this out since it's for a version of InfluxDB that's no longer being actively maintained. Please feel free to open a new issue if you find an error with the current version.


DRAFT: experiment of #9637

Which issue does this PR close?

Closes #9637.
Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Continues on #9768


UnityContent is not a constructor on gatsby build, v7.x

Describe the bug
I'm trying to build a website with a unity game from 2019 inside it. For this website I am using Gatsby with React and Typescript. I'm able to build a development version with gatsby develop fine. However, once I run gatsby build it says:
Building static HTML failed for path "/projects/planetpolluter/"

See our docs page for more info on this error: https://gatsby.dev/debug-html

   9 | const UnityPlanetPolluter = () => {
  10 |     // @ts-ignore
> 11 |     const unityContent = new UnityContent(
     |                          ^
  12 |         unityBuildJson, unityLoaderJs
  13 |     );
  14 |     return (

  WebpackError: TypeError: react_unity_webgl__WEBPACK_IMPORTED_MODULE_1__.UnityContent is not a constructor

Here's the code I'm using:
import React from 'react';
import Unity, { UnityContent } from "react-unity-webgl";

const unityBuildPath = "/data/planetpolluter/PlanetPolluterBuild/";
const unityBuildJson = unityBuildPath + "PolluterWeb.json";
const unityLoaderJs = unityBuildPath + "UnityLoader.js";

const UnityPlanetPolluter = () => {
    // @ts-ignore
    const unityContent = new UnityContent(
        unityBuildJson, unityLoaderJs
    );
    return (
        <Unity unityContent={unityContent} />
    );
}

To Reproduce
Steps to reproduce the behavior:

Build a project with Unity 7.x and Gatsby

Expected behavior
To build and generate a website.
Screenshots
If applicable, add screenshots to help explain your problem.
Details (please complete the following information):

React Unity WebGL Version: 7.x
Device and OS: Dell XPS 15 and Windows 10
Browser: n/a
NodeJS Version: v14.17.5
Bundler: webpack (used by Gatsby)

Additional context
N/A

Sorry, I cannot help you with Gatsby specific problems. I'll leave this open for anyone else who might be able to help you. The only thing I can think of is, are you sure Gatsby is installing the right version of the module? Since UnityContent has been refactored to UnityContext in version 8.x.

I checked node_modules and my version is 7.1.13. The strange thing is I am able to run the app when testing with a development server through gatsby develop. To my understanding gatsby build doesn't have access to a browser when building unlike develop.
Here's a link to my repository on the gatsby branch in case anyone wants to try building it.

I just found a solution. You need to construct UnityContent after the component is mounted. I did this by using the hook useEffect(). You have to handle the case where the component is null since that's its state prior to mounting. Here's my working code:
import React from 'react';
import Unity, { UnityContent } from "react-unity-webgl"

const unityBuildPath = "/data/planetpolluter/PlanetPolluterBuild/";
const unityBuildJson = unityBuildPath + "PolluterWeb.json";
const unityLoaderJs = unityBuildPath + "UnityLoader.js";

const UnityPlanetPolluter = () => {
    const [unityContent, setUnityContent] = React.useState(null);
    React.useEffect(() => {
        setUnityContent(new UnityContent(
            unityBuildJson, unityLoaderJs
        ));
    })
    if (unityContent == null) {
        return (<>
        <h4>
             Unity is not working yet.
        </h4>
        </>);
    }
    else {
        return (
            <Unity unityContent={unityContent} />

        );
    }
}

export default UnityPlanetPolluter;


Add iOS 10 instruction in README to add NSMicrophoneUsageDescription to plist

Full error message:
This app has crashed because it attempted to access privacy-sensitive data without a usage description.  The app's Info.plist must contain an NSMicrophoneUsageDescription key with a string value explaining to the user how the app uses this data.

You should add this :
<key>NSMicrophoneUsageDescription</key>
<string>This sample uses the microphone to record your speech and convert it to text.</string>

to your info.plist file. At the top level dict ( Xcode edit source of info.plist )
We should probably add this to the README.

Yep was about to add that that's what I did :) thanks!

Thanks for reporting this !

no problem, thanks for providing the components!


Remove unnecessary O(log|V|) dictionary lookup

At Graph instantiation time, we sort vertices in "key order", so that the vertices are exactly the integers from 1 to |V|.  In this way, for search every Dictionary>>at: is replaced by Array>>at:.  However, as the code stands right now, we are not taking advantage of this and still performing the Dictionary>>at: O(log|V|) lookup.

Fixed in 2deb36, a605064.


Add endpoints to manage staff users

Add GraphQL endpoints to manage staff users of the dashboard.
Queries

[x] permissions() - list of all permissions
[x] groups() - list of all permission groups
[x] staffUsers() - list of all staff users

Mutations
Managing Django's permission groups:

[ ] groupCreate()
[ ] groupUpdate()
[ ] groupDelete()

Managing staff users:

[x] staffUserCreate()
[x] staffUserUpdate()

We've decided that it'll be easier for us to fetch all users (both staff and customers) from one endpoint. That's because we allow staff users to log in to the storefront and place the order so they can actually have both roles. I've added optional is_staff filter to the query in case one needs to filter them.

I've opened a separate issue for permission groups: #2289