auth_time in id_token should be the same in the newly issued id_token when refreshing

This issue is imported from pivotal - Originaly created at Dec 7, 2020 by Bart Geesink
When refreshing a token, the obtained id_token should  contain the same "auth_time". Now it gets a new auth_time, which is incorrect

unable to reproduce, closing (Bart Geesink - Dec 8, 2020)


mWeb - chat - In offline, in text+attachment message edit comment option is not shown

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.26
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: N/A
Issue reported by: Applause - Internal Team
Action Performed:

Go to https://staging.new.expensify.com/home
Go offline
Open a chat
Enter a text and upload an image and send it
Long press the message to open context menu

Expected Result:
In offline, in text+attachment message edit comment option must be shown
Actual Result:
In offline, in text+attachment message edit comment option is not shown
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/user-attachments/assets/8dd2aa53-bf11-43a2-928e-6cc19cfd0dce
View all open jobs on GitHub

@johncschuster  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Proposal
Please re-state the problem that we are trying to solve in this issue.
mWeb - chat - In offline, in text+attachment message edit comment option is not shown
What is the root cause of that problem?
Right here we disable if the comment created from the optimistic action meaning from Onyx not yet created on the BE
https://github.com/Expensify/App/blob/e77f20458ad13d94c7b0188acc2ed1a91bd8ccbd/src/libs/ReportUtils.ts#L3028-L3042
What changes do you think we should make in order to solve the problem?
We can remove the !reportAction?.isOptimisticAction check
So it will be (!reportAction.isAttachmentWithText && !reportAction.isAttachmentOnly)
What alternative solutions did you explore? (Optional)

This is intentional behaviour the markdown for text+attachment will be only ready after the server's response.


Add MacPorts install info

https://ports.macports.org/port/jwt-cli/

Summary
Add information on how to install jwt-cli via MacPorts on macOS.
Preflight checklist

[ ] Code formatted with rustfmt
[ ] Relevant tests added
[x] Any new documentation added

Thanks @mike-engel

Very sorry for the delay, but this is in 6.0.0 now


Starting select parent folder from workspace directory

Now when user adding new worktree, the parent folder selector starts selecting from current workspace directory, instead of user home. It also keeps the last parent workspace folder in runtime.
I tried to save last selected parent folder in workspace settings, but it requires declaring settings in package.json, what cannot be hidden from user settings. So for now this selection keeps only for runtime, may be later it can be stored in own dot-file in workspace or .vscode folder.
Known issues:

KDE file choser has ignoring option defaultUri.
Workarounds:

use VSCode simple dialog for file choser.
use GTK file choser (set env GTK_USE_PORTAL=0)

Suggestion for choosing a parent directory: Show file choser only when setting vscode-git-worktree .parent-dir (for example) is empty, and save selected path in workspace settings

Hi, sorry for the delay. I should have some time this week to review. Thanks for your contribution!


Add playlist blocks

This builds on #1159 to add the necessary components and adjustments in both back- and frontend to allow creating and editing playlist blocks.
These are already a lot of changes, so in order to limit the scope of this PR and make it somewhat reviewable, a feature still missing is a management area to create, delete and edit playlists themselves (i.e. adding and removing videos).

Oh and one additional idea: in the list/details view of playlist blocks, it would be nice to show the series a video belongs to. Similarly to how it's done in search results (visually, I mean).

Oh and this one bug still exists: the default order that I set for the playlist block is not respected when showing the block. Like, if I say "oldest first" in the block settings, save, and then reload the page, the playlist is still shown in original order.

Oh and one additional idea: in the list/details view of playlist blocks, it would be nice to show the series a video belongs to. Similarly to how it's done in search results (visually, I mean).

I saw this in the corner of my eyes and I have two quick comments you feel free to ignore because they might be hasty:

Generally, showing the series of a video is not as exciting/revealing as you might think, because in most situations the series title will be the same as the episode title, with the overall majority of videos coming from automated lecture recording (cf. https://video.ethz.ch/lectures/d-math/2021/autumn/401-2003-00L.html). And yes, this is about our opposite perspectives on the video portal, too. If this burns real estate or makes a UI cramped, we shouldn't expose that information.
Aren't you already making assumptions/decisions about the relationship of "playlists" and "series" if you show the series in the playlist block?

I saw this in the corner of my eyes and I have two quick comments you feel free to ignore because they might be hasty:
* Generally, showing the series of a video is not as exciting/revealing as you might think, because in most situations the series title will be the same as the episode title, with the overall majority of videos coming from automated lecture recording (cf. https://video.ethz.ch/lectures/d-math/2021/autumn/401-2003-00L.html). And yes, this is about our opposite perspectives on the video portal, too. If this burns real estate or makes a UI cramped, we shouldn't expose that information.

* Aren't you already making assumptions/decisions about the relationship of "playlists" and "series" if you show the series in the playlist block?

This definetively needs to be discussed. Also the fact, that at the moment when you access a video via a playlist, not only the series containing this video is revealed but also all videos of this series are shown below (see screenshot). This might be not wanted in some usecases (e.g., when a lecturer has a series for all his/her educational videos and distributes them using playlists).

Also the fact, that at the moment when you access a video via a playlist, not only the series containing this video is revealed but also all videos of this series are shown below (see screenshot). This might be not wanted in some usecases (e.g., when a lecturer has a series for all his/her educational videos and distributes them using playlists).

This is already a fixed semantic behavior that we can't really change anymore. Regarding the "listed status", a series and all its videos form one unit. You can always reach a video's series via the video itself, and reach all videos of that series then. (Of course, all of the videos you are allowed to see.) The quick answer to this is: that lecturer should not use a series for that use case then.

In fact, when accessin a video via the playlist "A" I would expect to become the information that this video is part of this playlist "A" and below the video a list of all videos of this playlist "A" is shown.

However, that makes sense to me. Mind you, the series link would still be shown in metadata, but the videos at the bottom would indeed be from the playlist. (Implementation note: I would do that via URL parameter like youtube does it.)


"Video unavailable" for unknown reason with oauth

Describe the bug
Getting "Video unavailable" for unknown reason.
"\u001b[91m---ZOY1kzAc is unavailable\u001b[0m"
To Reproduce
Steps to reproduce the behavior:

yt = YouTube(f"https://www.youtube.com/watch?v={video_id}",use_oauth=True, allow_oauth_cache=True)
Get this printed:

Video ID: ---ZOY1kzAc
Please open an issue at https://github.com/JuanBindez/pytubefix/issues and provide this log output.

Desktop (please complete the following information):

Pytubefix Version 6.2.2

Hello, I ask you to test this version pip install pytubefix==6.3rc1 which is currently being tested by @felipeucelli in #111, before testing read the commit carefully

Thank you! The error to the console is now fixed, but there is something else.
yt.streams.get_by_itag(402)
Will throw "---ZOY1kzAc is unavailable" when oauthed.
I am not sure if get_by_itag is supposed to fail like this.

Update. I think the account I was using was banned.
Tried with a new account and get this error: 'HTTP Error 428: Precondition Required'

Thank you! The error to the console is now fixed, but there is something else. yt.streams.get_by_itag(402) Will throw "---ZOY1kzAc is unavailable" when oauthed. I am not sure if get_by_itag is supposed to fail like this.
Edit: And it does work when not oauthed.

For me, this video is working correctly, perhaps there is some regional blocking.

Update. I think the account I was using was banned. Tried with a new account and get this error: 'HTTP Error 428: Precondition Required'

This error usually refers to session token expiration.

Huh, now no auth is getting these errors on the new update.

@JuanBindez I am working on a fix, will have a pull request for you in by tonight.


Define behaviour for reserved RV32 permission encoding

The spec says

Valid capabilities must not have the permissions field set to a reserved value according to Table 3 when XLENMAX=32.

I believe this can't ever happen for tagged capabilities but CGetPerm works for non-tagged capabilities. What should it return for this? Same question for CAndPerm.

Thanks for pointing this out. I agree that there is a gap in the spec regarding CGetPerm. However, CAndPerm does describe what happens in this case:

The AP field is not able to encode all combinations of permissions when XLENMAX=32. If
permissions that cannot be encoded are indicated, CANDPERM outputs a capability with
all architectural permissions cleared.

@PeterRugg had some comments about this behaviour, so there is a TODO note on it and should be picked up for discussion.

Hmm, that's a good question, and links into how we define the reserved permission bits in a way that maximises compatibility in all cases. I imagine the interpretation of all the encoded permission bits could potentially change when the reserved bits are set by future or non-standard extensions, i.e. effectively interpreting some of them as format bits. That means software shouldn't rely on the interpretation of the existing bits if the reserved bits are set. It still needs to be defined, though, so maybe all zero?

I'm happy with all zeroes - as that doesn't exclude using the reserved value for a purpose other than representing permissions.

Ok that makes sense; I'll make a PR.


Refactor shapely.ops to not use lgeos

Replacing direct lgeos calls with pygeos python functions in the shapely.ops module.
The remaining ones to do are polygonize, polygonize_full and nearest_points. For polygonize I already have a PR in pygeos, the others still need to be added there.

Coverage decreased (-0.3%) to 77.718% when pulling 0cf7349f368c4924c88d2cb6ddd37ebb8b2d5f79 on jorisvandenbossche:refactor-ops into d2b37278ce77492661462ef9b975d86f09b073df on Toblerity:shapely-2.0.

This is ready now: it fully removed all remaining usage of ctypes/lgeos from ops.py (and then remaining usage is in strtree.py)


Shift public input by 4 bytes

What ❔
Shift public input by 4 bytes
Why ❔
Needed for correct public input generation here
Checklist

[ ] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.
[ ] Code has been formatted via zk fmt and zk lint.

I’ll tentatively approve, let’s wait for proof generation anyway


Checksum contractAddress of contract fetching and propagate type

Summary
In order to maintain a strict type and checksummed address throughout the project, we should validate the incoming addresses. In newer controllers, we started doing this by adding an AddressSchema to address. This begins adding it to "existing" controllers.
This checksums the incoming contractAddress of ContractsController['getContract'] and propagates the stricter (0x${string}) type throughout the project accordingly
Changes

Add validation pipe checksum to incoming contractAddress of ContractsController['getContract']
Make the contractAddress type of ContractsService['getContract'] stricter
Checksum the sender and recipient addresses in Erc20TransferMapper/Erc721TransferMapper
Checksum address mapping values in Erc20TransferMapper, Erc721TransferMapper, SettingsChangeMapperand TransactionDataMapper
Make the contractAddress type of ITransactionApi['getContract'] (and it's implementation) stricter
Make the address type of AddressInfoHelper['get' | 'getOrDefault' | 'getCollection' | '_getFromSource'] stricter
Make the contractAddress type of IContractsRepository['getContract'] (and it's implementation) stricter
Make contracts cache dir contractAddress type stricter
Update types/tests accordingly

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

6 of 6 (100.0%)  changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.04%) to 92.861%

Totals

Change from base Build<PHONE_NUMBER>:
0.04%

Covered Lines:
7116

Relevant Lines:
7370

💛  - Coveralls


(PDB-3306) Remove clj cli import/export (superceded by rust)

Test FAILed

CLA signed by all contributors.

@rbrw need to update acceptance tests here, I think one of them is checking for the CLI

Test PASSed

Test PASSed

@rbrw There's a reference to the old command in anonymization.markdown that needs cleaning up

otherwise :+1: from me

Test PASSed

Test PASSed

Merging, given Russ's approval for everything but the docs, and a separate ticket for that: https://tickets.puppetlabs.com/browse/PDB-3499


Fixes inconsistent behavior when controlling multiple devices with brightnessctl

Fixes a small issue when multiple devices are controlled by brightnessctl and the current and max values are coming from Device 1 but changes are applied to Device 2.
This should probably be handled by addressing all devices at once and changing them to the desired percentages.
For example by changing all devices with the backlight class: brightnessctl -c backlight s 20%.
This sadly seems to be broken in brightnessctl v5.0 though.

Looks good, thanks!


Add new job-id annotation to assign globally unique job index to each job

Fixes #649

/assign @kannon92 @ahg-g please take a look when you have a sec, thanks!

/lgtm


Don't use personal GitHub tokens

On the admin and API repositories we currently use personal GitHub tokens to authenticate with the API.
This is not very robust as people come and go.
Look for another auth mechanism, maybe GitHub Apps?

related
https://github.com/cds-snc/notification-api/pull/1315
https://github.com/cds-snc/notification-admin/pull/1074
https://github.com/cds-snc/notification-document-download-api/pull/42
https://github.com/cds-snc/notification-document-download-frontend/pull/59
https://github.com/cds-snc/notification-documentation/pull/84


Easier to configure, easier to version-control, more tightly integrated node pools

This is an implementation of #238 from @redbaron especially what I've described in my comment there https://github.com/coreos/kube-aws/issues/238#issuecomment-272128944, and an answer to the request "3. Node pools should be more tightly integrated" of #271 from @sasso .
I believe this also achieves what was requested by @andrejvanderzee in https://github.com/coreos/kube-aws/issues/176#issuecomment-268117511.
After applying this change:

All the kube-aws node-pools sub-commands are dropped
You can now bring up a main cluster and one or more node pools at once with kube-aws up
You can now update all the sub-clusters including a main cluster and node pool(s) by running  kube-aws update
You can now destroy all the AWS resources spanning main and node pools at once with kube-aws destroy
You can configure node pools by defining a worker.nodePools array in cluster.yaml`
workerCount is dropped. Please migrate to worker.nodePools[].count
node-pools/ and hence node-pools/<node pool name> directories, cluster.yaml, stack-template.json, user-data/cloud-config-worker for each node pool are dropped.
A typical local file tree would now look like:

cluster.yaml
stack-templates/ (generated on kube-aws render)

root.json.tmpl
control-plane.json.tmpl
node-pool.json.tmpl

userdata/

cloud-config-worker
cloud-config-controller
cloud-config-etcd

credentials/

*.pem(generated on kube-aws render)
*.pem.enc(generated on kube-aws validate or kube-aws up)

exported/ (generated on kube-aws up --export --s3-uri <s3uri>)

stacks/

control-plane/

stack.json
user-data-controller

<node pool name = stack name>/

stack.json
user-data-worker

A typical object tree in S3 would now look like:

<bucket and directory from s3URI>/

kube-aws/

clusters/

<cluster name>/

exported/

stacks/

control-plane/

stack.json
cloud-config-controller

<node pool name = stack name>/

stack.json

Implementation details:
Under the hood, kube-aws utilizes CloudFormation nested stacks to delegate management of multiple stacks as a whole.
kube-aws now creates 1 root stack and nested stacks including 1 main(or currently named "control plane") stack and 0 or more node pool stacks.
kube-aws operates on S3 to upload all the assets required by all the stacks(root, main, node pools) and then on CloudFormation to create/update/destroy a root stack.
An example cluster.yaml  I've been used to test this looks like:
clusterName: <your cluster name>
externalDNSName: <your external dns name>
hostedZoneId: <your hosted zone id>
keyName: <your key name>
kmsKeyArn: <your kms key arn>
region: ap-northeast-1
createRecordSet: true
experimental:
  waitSignal:
    enabled: true
subnets:
- name: private1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: private2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: public1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
- name: public2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
controller:
  subnets:
  - name: public1
  - name: public2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: public1
  - name: public2
worker:
  nodePools:
  - name: pool1
    subnets:
    - name: asgPublic1a
  - name: pool2
    subnets: # former `worker.subnets` introduced in v0.9.4-rc.1 via https://github.com/coreos/kube-aws/pull/284
    - name: asgPublic1c
    instanceType: "c4.large" # former `workerInstanceType` in the top-level
    count: 2 # former `workerCount` in the top-level
    rootVolumeSize: ...
    rootVolumeType: ...
    rootVolumeIOPs: ...
    autoScalingGroup:
      minSize: 0
      maxSize: 10
    waitSignal:
      enabled: true
      maxBatchSize: 2
  - name: spotFleetPublic1a
    subnets:
    - name: public1
    spotFleet:
      targetCapacity: 1
      unitRootVolumeSize: 50
      unitRootvolumeIOPs: 100
      rootVolumeType: gp2
      spotPrice: 0.06
      launchSpecifications:
      - spotPrice: 0.12
         weightedCapacity: 2
         instanceType: m4.xlarge
        rootVolumeType: io1
        rootVolumeIOPs: 200
        rootVolumeSize: 100

cc @redbaron @sasso @andrejvanderzee As I remember this is most requested by you, would you mind taking a look into this? Tests are failing but it is already working with the above cluster.yaml.
Thanks for your supports!
TODOs

[x] Drop worker nodes from the main stack
[x] Create node pool(s) by default
[x] Update Documentation/*.md
[x] Fix integration tests
[x] Add integration tests
[x] Fix E2E tests
[x] Pass E2E tests

List of planned changes until this PR is finished:

[x] Rename nodePools[].nodePoolName to just nodePools[].name
[x] Rename nodePools to worker.nodePools
[x] Remove prefixes of worker* e.g. worker.nodePools[].workerRootVolumeSize to [ ] worker.nodePools[].rootVolumeSize
[x] Flatten worker.nodePools[].experimental.* to worker.nodePools[]. For example worker.nodePools[].experimental.waitSignal would be just worker.nodePools[].waitSignal

Things won't be addressed in this PR:

Transform Fn::ImportValues into Defaults of stack parameters and put Refs in place instead (#195 from @icereval, https://github.com/coreos/kube-aws/pull/284#issuecomment-275365832 from @redbaron)
Transform Fn::ImportValues into Refs of stack parameters while providing the parameters via the root stack (https://github.com/coreos/kube-aws/issues/238#issuecomment-272050366 from @redbaron)
Large, fundamental refactoring of existing code
Default settings for worker.experimental.* from the top level experimental.*
Default settings for worker.nodePools[].experimental.* from the upper level worker.experimental.*
Default settings for worker.nodePools[].experimental.* from the top level experimental.*
Selectively deprecate or drop all the former worker* keys in the top-level of cluster.yaml

Update the description several times to include all the topics in my mind.
@redbaron @sasso @andrejvanderzee I wish you like this. I'd appreciate your feedbacks if any!

Before I dive into code and testing. Do we need to keep the worker or could we only use the node pools to make things simple?

@sasso I'm believing/assuming that we don't need workers in a main cluster anymore. Node pools can completely replace workers in a main cluster and are more flexible.

just saw the comment in the template 😄

after a quick test I'm stuck on
(model.Worker) {
 AutoScalingGroup: (model.AutoScalingGroup) {
  MinSize: (int) 0,
  MaxSize: (int) 0,
  RollingUpdateMinInstancesInService: (int) 0
 },
 ClusterAutoscaler: (model.ClusterAutoscaler) {
  MinSize: (int) 0,
  MaxSize: (int) 0
 },
 SpotFleet: (model.SpotFleet) {
  TargetCapacity: (int) 0,
  SpotPrice: (string) "",
  IAMFleetRoleARN: (string) "",
  RootVolumeType: (string) "",
  UnitRootVolumeSize: (int) 0,
  UnitRootVolumeIOPS: (int) 0,
  LaunchSpecifications: ([]model.LaunchSpecification) {
  }
 },
 Subnets: ([]model.Subnet) <nil>
}
Error: Failed to initialize cluster driver : file cluster.yaml: public subnets required by default for node pool missing in cluster.yaml.
define one or more public subnets in cluster.yaml or explicitly reference private subnets from node pool by specifying nodePools[].subnets[].name
exit status 2

nodePools:
- name: "general"
  subnets:
    - name: default-private.us-east-1a
    - name: default-private.us-east-1c
    - name: default-private.us-east-1d
    - name: default-private.us-east-1e
  workerInstanceType: "c4.large"

Seems that subnets are not getting populated (I'm testing with existing subnets). Will debug more tomorrow...

wow, that is a great improvement, thanks for doing this work.
cluster.yaml looks exactly how I imagined it, with one exception: can we allow userdata file to be configurable per nodepool? doing that, not only worker nodes can be removed, controllers can be nodepool'ed too then

@sasso Thanks for the feedback! I'll investigate it soon :bow:

@redbaron Thanks for checking in!

can we allow userdata file to be configurable per nodepool?

Probably via introducing a new setting key e.g. controlplane.templatePath so that we can override which template other than the default stack-templates/controlplane to be used by kube-aws:
controlplane:
  templatePath: my-stack-templates/controlplane.json.tmpl`

doing that, not only worker nodes can be removed, controllers can be nodepool'ed too then

Sounds interesting!
Could you clarify your use-case(s) a bit?
Are you willing to auto-scale controller nodes with CA by having 1 controller node pool per AZ or anything else?

List of planned changes in this PR:

[ ] Rename nodePools[].nodePoolName to just nodePools[].name
[ ] Rename nodePools to worker.nodePools
[ ] Remove prefixes of worker* e.g. worker.nodePools[].workerRootVolumeSize to [ ] worker.nodePools[].rootVolumeSize
[ ] Flatten worker.nodePools[].experimental.* to worker.nodePools[]. For example worker.nodePools[].experimental.waitSignal would be just worker.nodePools[].waitSignal

@mumoshu I'm trying out this branch, currently getting this:
> kube-aws render stack
panic: runtime error: integer divide by zero

goroutine 1 [running]:
panic(0x6488a0, 0xc4200121d0)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/coreos/kube-aws/core/controlplane/config.Cluster.Config(0xc4200d5540, 0x1a, 0x73366a, 0x9, 0x0, 0x0, 0x0, 0x0, 0xc4201267d0, 0xb, ...)
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/core/controlplane/config/config.go:589 +0x9d1
github.com/coreos/kube-aws/core/root.StackAssetsRendererFromFile(0x7356fc, 0xc, 0x5f1bc0, 0xc420126639, 0xc420147c97, 0x4)
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/core/root/renderer.go:13 +0x87
github.com/coreos/kube-aws/cmd.runCmdRenderStack(0xb4e260, 0xb6e9b0, 0x0, 0x0, 0x0, 0x0)
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/cmd/render.go:71 +0x3d
github.com/coreos/kube-aws/vendor/github.com/spf13/cobra.(*Command).execute(0xb4e260, 0xb6e9b0, 0x0, 0x0, 0xb4e260, 0xb6e9b0)
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/vendor/github.com/spf13/cobra/command.go:632 +0x23e
github.com/coreos/kube-aws/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xb4e480, 0x0, 0x0, 0x0)
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/vendor/github.com/spf13/cobra/command.go:722 +0x367
github.com/coreos/kube-aws/vendor/github.com/spf13/cobra.(*Command).Execute(0xb4e480, 0xc420147f50, 0xc4200001a0)
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/vendor/github.com/spf13/cobra/command.go:681 +0x2b
main.main()
	/Users/cknowles/Projects/go/src/github.com/coreos/kube-aws/main.go:9 +0x2d
make: *** [kube-aws-render-new] Error 2

Culprit seems to be this line:
subnetIndex := etcdIndex % len(config.Etcd.Subnets)
I setup a brand new stack and did very little customisation to cluster.yaml. The main part I added was trying to add 3 private subnets against existing NAT Gateways ready for workers to be deployed:
subnets:
  - name: ManagedPrivateSubnetWithExistingNGW1
    private: true
    availabilityZone: eu-west-1a
    instanceCIDR: "<IP_ADDRESS>/24"
    natGateway:
      idFromStackOutput: vpc-nat-gateway-1-stg
  - name: ManagedPrivateSubnetWithExistingNGW2
    private: true
    availabilityZone: eu-west-1b
    instanceCIDR: "<IP_ADDRESS>/24"
    natGateway:
      idFromStackOutput: vpc-nat-gateway-2-stg
  - name: ManagedPrivateSubnetWithExistingNGW3
    private: true
    availabilityZone: eu-west-1c
    instanceCIDR: "<IP_ADDRESS>/24"
    natGateway:
      idFromStackOutput: vpc-nat-gateway-3-stg

I haven't checked it fully but I guess etcd subnets are not defaulting when some subnets are specified. However, cluster.yaml states this in etcd section - # If omitted, public subnets are created by kube-aws and used for etcd nodes.

@c-knowles Thanks as always for your feedback!
I'm sorry for confusing you but the comment # If omitted, public subnets are created by kube-aws and used for etcd nodes. applies only when the subnets key is completely missing in cluster.yaml.
If you do define subnets: with only the private subnets, you have to explicitly pass which private subnet(s) etcd nodes gets created like:
etcd:
  subnets:
  - name: ManagedPrivateSubnetWithExistingNGW1
  - name: ManagedPrivateSubnetWithExistingNGW2
  - name: ManagedPrivateSubnetWithExistingNGW3

I guess we can add more validation(s) and/or more kind error messages (of course wihout panicing!) but I don't have a concrete idea about those yet.

@mumoshu I can confirm as soon as I set etcd.subnets to something the stack rendered. I'm not sure how much access the etcd nodes need so I set to the same private subnets with NAT for now.

@mumoshu I think for now, it's enough for make the cluster.yaml clear as below. For future, could kube-aws do the defaulting where possible? I mean, do we have to specify all subnets if we just want to specify one?
# If both `etcd.subnets` and the top level `subnets` are omitted, public subnets are created by kube-aws and used for etcd nodes. If the top level `subnets` is present then `etcd.subnets` must be specified.

I guess the same confusion causes this for API ELB on up, this time on the controllers though render works but it fails during stack creation:
11:33:45 UTC+0800	CREATE_FAILED	AWS::ElasticLoadBalancing::LoadBalancer	ElbAPIServer	Either AvailabilityZones or SubnetIds must be specified

@c-knowles

I think for now, it's enough for make the cluster.yaml clear as below

Seems like so! 👍

@mumoshu I also found that a combination of existing VPC, no IGW specification and some managed public subnets means that an IGW is created by kube-aws as expected but it is not attached to the VPC. As far as I can tell this is supported in theory just the VPCGatewayAttachment definition inside controlplane is inside the {{if not .VPCID}} block.

@c-knowles I really appreciate your support here!
Sorry for the confusion again but specifying vpcId means that an user is responsible (1) to create an appropriate VPC gateway attachment for the existing VPC specified by vpcId and (2) to provide an appropriate internetGatewayId for the managed subnets or (3) to provide unmanaged, exisiting subnets with routes to the IGW already configured instead.
The general rule behind it is that kube-aws tries its best not to modify existing AWS resources - which is the VPC specified by vpcId in this case.
All kube-aws will do when chosen managed public subnets + existing vpc combination are creating managed subnets and associating the internet gateway specified by internetGatewayId to those subnets.
I'd like to add more validations and/or documentation(s) on demand. Please tell me here or more better submit another github issues to express demands!

@mumoshu sure yes, I thought that's what you were going for it's just not entirely clear in the current cluster.yaml I think. I'll file some issues shortly once I'd gone through a few more parts.

BTW, this is looking awesome! I'd held off on using node pools until this point but now it's so much easier to manage the various parts.

doing that, not only worker nodes can be removed, controllers can be nodepool'ed too then
Sounds interesting!
Could you clarify your use-case(s) a bit?
Are you willing to auto-scale controller nodes with CA by having 1 controller node pool per AZ or anything else?

Not so much for autoscaling, but unifying all but etcd node management configuration under the same config structure and probably unify some code path in kube-aws codebase too.

Just finished TODOs in https://github.com/coreos/kube-aws/pull/315#issuecomment-279264129 via 	3b65538
@c-knowles @sasso fyi nodePools are now moved to worker.nodePools. A working example of cluster.yaml for me is now:
clusterName: <myclustername>
externalDNSName: <myexternaldnsname>
hostedZoneId: <myhostedzoneid>
keyName: <mykeyname>
kmsKeyArn: <mykmskeyarn>
region: ap-northeast-1
createRecordSet: true
experimental:
  waitSignal:
    enabled: true
subnets:
- name: private1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: private2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: public1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
- name: public2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
controller:
  subnets:
  - name: public1
  - name: public2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: public1
  - name: public2
worker:
  nodePools:
  - name: "nodepool1"
    availabilityZone: "ap-northeast-1a"
    subnets:
    - name: public1
      availabilityZone: "ap-northeast-1a"
      instanceType: "c4.large"
#- name: "nodepool2"
#  availabilityZone: "ap-northeast-1b"
#  subnets:
#  - name: public2
#    availabilityZone: "ap-northeast-1b"

@sasso The bug which had been preventing subnets from being populated seems to be fixed according to the integration tests. Would you mind trying it again?

@mumoshu did a quick run
Error: Failed to initialize cluster driver : failed to load node pool #0: failed to render worker cloud config: template: userdata/cloud-config-worker:212:12: executing "userdata/cloud-config-worker" at <.Worker.SpotFleet.En...>: can't evaluate field Worker in type *config.ComputedConfig
exit status 2

@sasso Thanks for testing! It seems like you need to rm user-data/* and re-run kube-aws render

@redbaron

Not so much for autoscaling, but unifying all but etcd node management configuration under the same config structure and probably unify some code path in kube-aws codebase too.

It turned to be true! Almost everything which was assumed to be specific to worker node pools are now generalized into the same config structure:
type NodePoolConfig struct {
  AutoScalingGroup  AutoScalingGroup  `yaml:"autoScalingGroup,omitempty"`
  ClusterAutoscaler ClusterAutoscaler `yaml:"clusterAutoscaler"`
  SpotFleet         `yaml:"spotFleet,omitempty"`
  Subnets           []Subnet `yaml:"subnets,omitempty"`
  SpotFleet         SpotFleet         `yaml:"spotFleet,omitempty"`
  Count             int               `yaml:"count,omitempty"`
  CreateTimeout     string            `yaml:"createTimeout,omitempty"`
  InstanceType      string            `yaml:"instanceType,omitempty"`
  RootVolume        `yaml:",inline"`
  SpotPrice         string   `yaml:"spotPrice,omitempty"`
  SecurityGroupIds  []string `yaml:"securityGroupIds,omitempty"`
  Tenancy           string   `yaml:"tenancy,omitempty"`
}

Hence each worker node pool is now represented in code as:
type WorkerNodePoolConfig struct {
  model.NodePoolConfig `yaml:",inline"`
}

In the future, we could make another refactoring(s) to make controller node pool'ed by something like:
type ControllerodePoolConfig struct {
  model.NodePoolConfig `yaml:",inline"`
}

Integration tests WITHOUT real AWS services are now passing on Travis
Integration tests WITH real AWS services are now passing on my laptop

@mumoshu I've tried out the latest branch, so far no major additional issues. Is it possible to set a node pool ASG min to zero? I think it used to be possible when we added the ASGs but I can't seem to get it to be set now. I've tried worker.nodePools[].autoScalingGroup.minSize and worker.nodePools[].count.

I think there could be an issue with worker nodepools with wait signal in this branch. If I enable the wait signal, then CF MaxBatchSize is always zero which is not allowed on AWS. Setting nodepools[].waitSignal.maxBatchSize does not seem to solve it. I think the smallest worker config to reproduce is:
worker:
  nodePools:
    - name: worker-pool-a
      availabilityZone: eu-west-1a
      subnets:
        - name: ManagedPrivateSubnetWithExistingNGW1
      waitSignal:
        enabled: true
        # maxBatchSize: 1 # adding/removing this does not work around the issue

@c-knowles Thanks for reporting!
I've fixed the above two issues.
You can choose one of the below to have no worker nodes per node pool:

set worker.nodePools[].count to zero
set worker.nodePools[].autoScalingGroup.minSize to zero

Also, setting waitSignal.maxBatchSize would just work now.

@mumoshu great, that was quick! I will try it out soon.

@mumoshu , why did you decide to move nodepools under worker key? is there going to be anything else but nodepools?

@redbaron Thanks for the good question 👍
I've moved nodePools to worker.nodePools so that we can be more explicit about that the node pools are just for worker nodes "for now". A node can actually be either a worker or a controller and therefore just saying "node pools` is a bit ambiguous.
And in the future we can also:

If necessary, introduce new keys like worker.defaults.* or even worker.* to configure default settings used among all the worker node pools and
If necessary, introduce controller.nodePools to make controllers to be managed as node pools while differentiating those from worker node pools to e.g. allow enabling cluster-autoscaler (#168)

Node pools can naturally be extended to controllers due to the fact that both workers and controllers are "nodes" in k8s. IMHO only the differences are how they're tainted or not and which pods are scheduled onto them.

E2E tests passed 😮

It turned out I'd broken worker node pools powered by spot fleets.
However, they're now fixed and working as before.

I believe that everything is now implemented as described in the updated description of this PR.
Please tell me if anything is wrong!

Codecov Report

Merging #315 into master will decrease coverage by -14.44%.
The diff coverage is 24.11%.

@@             Coverage Diff             @@
##           master     #315       +/-   ##
===========================================
- Coverage   55.21%   40.78%   -14.44%
===========================================
  Files           6       19       +13
  Lines        1389     1974      +585
===========================================
+ Hits          767      805       +38
- Misses        506     1056      +550
+ Partials      116      113        -3

Impacted Files
Coverage Δ

core/controlplane/config/stack_config.go
74.19% <ø> (ø)

core/controlplane/config/tls_config.go
72.15% <ø> (ø)

model/root_volume.go
0% <ø> (ø)

model/subnet.go
0% <ø> (ø)

model/asg.go
0% <ø> (ø)

model/spot_fleet.go
0% <ø> (ø)

model/identifier.go
0% <ø> (ø)

model/nat_gateway.go
0% <ø> (ø)

cfnstack/provisioner.go
11.11% <20%> (-8.21%)
:x:

model/node_pool_config.go
20.58% <20.58%> (ø)

... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9dffba1...0569e9f. Read the comment docs.

Just curious, is it worth  to keep using separate CF stack per nodepool? It might be easier just to create ASG per nodepool in a main CF stack, no?

BTW I'm not satisfied with the packages names like core, config, model, cluster, cfnstack at all.
However not to keep this open too long, I decide not to make further renames and refactorings now. I'll probably work on these later.
IMHO:

model should be renamed to config and
The config packages under nodepool, controlplane and root should be dropped and merged into the top-level config(former model) respectively
The cluster packages under core/nodepool and core/controlplane should be dropped and merged into core/nodepool and core/controlplane respectively. I don't like cluster.Cluster and cluster.Config!
core should be named stack

Just curious, is it worth to keep using separate CF stack per nodepool? It might be easier just to create ASG per nodepool in a main CF stack, no?

It might be easier but I believe separating stacks would be better in longer term mainly because doing so (1) makes each stack template less complex hence manageable and (2) allows kube-aws to scale to dozens of node pools.

Finished rebasing to the latest master and adding various follows up for recently added features.

Added a minor fix for kube-node-label.service about an issue got worse via some changes in this PR.

Wow, @mumoshu unbelievable piece work done and developed so quickly. thank you!!

This PR is amazing, great work @mumoshu. It has been a busy week but I can't wait to spin this up!


Feature request: provide tabbed or dual view for site preview configured content types.

Ive just started working with this module so apologies if I have missed some configuration details on how to do this.
By following the  introductory configuration for content types lose the Drupal core output of a node only presented with the Next JS preview.
Is the only way to restore the Drupal core view to remove the configuration at /admin/config/services/next/entity-types?
Would it be possible to render both the Next JS output and the core Drupal output and have them available as separate tabs (or some other display type).
Thanks

yes, exactly that. presently the views are mutually exclusive and i have to delete configuration to restore the drupal core display. would it be possible to have them both?

no its not a blocker thanks for asking.
at this stage i am just testing capaibilities seeing how everything works and fits together.

This option could be useful.

I've opened #241 to share what we've done to date. I think we'd want to extend it to

Allow the HtmlRenderer intervention to be disabled (so Drupal could still render a frontend if it wanted to.
Support non-Node entities.

OK so some updates here.
I implemented a POC for the headless tab (uses a new route entity.ENTITY_TYPE.headless_preview) and ran some tests. It looks like while this works for the default entity view, it introduces some complexity when dealing with non-canonical routes (example revision routes or preview routes. See @Elendev solution at #132).
Example: node/12/headless renders the latest revision for node 12 but there's no way to easily handle revisions here.
I'm thinking of other ways we could solve this and I'm leaning towards a block-based solution.

The Headless Preview will be available as a block.
Instead of taking over HtmlRenderer, you will be able to enable the preview block and use visibility filters to turn it on/off.
The block can come with a site switcher toggle where you can select between the default Drupal view (Twig) and headless views (Next.js)
The site switcher can be its own block as well for more flexibility.

What do you think?

As the site select is an optional block, what is the default display under the view tab? Something would need to be displayed if the block is not visible.
Would this be configurable by next_drupal, the block, or the node?

We can show the Twig preview by default (if available) or make this configurable by entity type.

+1 for configurable options.
Is the Twig Preview the Drupal default display?

Is the Twig Preview the Drupal default display?

Yes or whatever the main content block render. Example the default display on a node view route or a revision display on a revision route.

I moved this to the 2.0.0 release for now but if we can figure out how to implement this in a non-breaking way, we could consider it in a dot release.

Being a block, would it only work on an entity view page? This wouldn't work in scenarios where Drupal was a hybrid CMS unless you relied solely on the site selector and viewed the site directly (no iframe, new tab) through that?

@fiasco You'll be able to place the block on any canonical route for any entity type.
When in hybrid mode, the block will default to the default html renderer so you will see the Twig template. You can then use the site selector to switch sites.


Minor: refactor List PartialOrd

Which issue does this PR close?

Follow #8290
Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Closed in favor of #8253


I am confused about func “_get_targets_single” in anchor_head when i use SSD;

hi
I am confused about func “_get_targets_single” in anchor_head when i use SSD;
when i run this func, i found the shape of created label is not match the predict featuremap size;
the wrong tips show as below:
"Expected input batch_size (21896) to match target batch_size (21856)"
the reason of ”21856“ is the code
"
inside_flags = anchor_inside_flags(flat_anchors, valid_flags, img_meta['img_shape'][:2], self.train_cfg.allowed_border)
anchors = flat_anchors[inside_flags, :]
"
the inside_flag is not all true;
could you please give me some help！

It is normal that the inside_flag is not all true because some anchors is out of the real image sizes and should be ignored.


This document is partial and not self explanatory

Questions:

Why would I want to use advanced edit mode in the first place? (what can I achieve in doing so? or what can't I achieve without it?)
Where in the "capabilities.json" does "advancedEditModeSupport" reside?
After setting this mode - How do I use it in the custom visual? (an example would be very useful)
3.1. Next steps? Where is the next document that comes after this one, that explains actually how to use this feature?
What does "in focus mode" mean?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 518254a4-e001-3e16-1220-9bc0358475bb
Version Independent ID: a445a855-c10e-8b16-9808-26b9e9db5ca1
Content: Advanced edit mode in Power BI visuals - Power BI
Content Source: powerbi-docs/developer/visuals/advanced-edit-mode.md
Service: powerbi
Sub-service: powerbi-custom-visuals
GitHub Login: @KesemSharabi
Microsoft Alias: kesharab

Thanks for your comment, @tomer-agam.
I've assigned this issue to the Power BI visual support team for review.

advanced edit mode allows to disable focus mode functionality.
you need to define advancedEditModeSupport in the root of capabilities.json.
no further steps are needed. PowerBI will handle this flag.
you can read about focus mode here.

#please-close @kfollis


[Bug]: Battery powered devices is reported as node neighbors for powered devices

Is your problem within Home Assistant (Core or Z-Wave JS Integration)?
NO, my problem is NOT within Home Assistant or the ZWave JS integration
Is your problem within ZWaveJS2MQTT?
NO, my problem is NOT within ZWaveJS2MQTT
Checklist

[X] I have checked the troubleshooting section and my problem is not described there.

[X] I have read the changelog and my problem was not mentioned there.

Describe the bug
What causes the bug?
Not sure.
What do you observe?
Battery powered devices (sleeping nodes) is reported as neighbors for powered devices during a network heal. For what I can see all the listed devices is also sleeping on the time when they get listed as a neighbor.
Example of the first healing node in the network:
2022-04-27T23:09:44.023Z CNTRLR   [Node 002] healing node...
2022-04-27T23:09:44.023Z CNTRLR » [Node 002] refreshing neighbor list (attempt 1)...
2022-04-27T23:09:44.031Z SERIAL » 0x01050048024efe                                                     (7 bytes)
2022-04-27T23:09:44.032Z DRIVER » [Node 002] [REQ] [RequestNodeNeighborUpdate]
                                    callback id: 78
2022-04-27T23:09:44.035Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:44.243Z SERIAL « 0x010500484e21dd                                                     (7 bytes)
2022-04-27T23:09:44.245Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:44.248Z DRIVER « [REQ] [RequestNodeNeighborUpdate]
                                    callback id:   78
                                    update status: UpdateStarted
2022-04-27T23:09:44.728Z SERIAL « 0x010500484e21dd                                                     (7 bytes)
2022-04-27T23:09:44.731Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:44.733Z DRIVER « [REQ] [RequestNodeNeighborUpdate]
                                    callback id:   78
                                    update status: UpdateStarted
2022-04-27T23:09:46.237Z SERIAL « 0x010500484e21dd                                                     (7 bytes)
2022-04-27T23:09:46.239Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:46.241Z DRIVER « [REQ] [RequestNodeNeighborUpdate]
                                    callback id:   78
                                    update status: UpdateStarted
2022-04-27T23:09:49.700Z SERIAL « 0x010a00040021029840be00b4                                          (12 bytes)
2022-04-27T23:09:49.704Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:49.706Z DRIVER « [Node 033] [REQ] [ApplicationCommand]
                                  └─[SecurityCCNonceGet]
2022-04-27T23:09:54.222Z SERIAL « 0x010500484e22de                                                     (7 bytes)
2022-04-27T23:09:54.226Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:54.229Z DRIVER « [REQ] [RequestNodeNeighborUpdate]
                                    callback id:   78
                                    update status: UpdateDone
2022-04-27T23:09:54.242Z CNTRLR « [Node 002] neighbor list refreshed...
2022-04-27T23:09:54.242Z CNTRLR » [Node 002] Deleting SUC return route...
2022-04-27T23:09:54.252Z SERIAL » 0x01110013210a9880e6d3587ddd41212a054f03                            (19 bytes)
2022-04-27T23:09:54.253Z DRIVER » [Node 033] [REQ] [SendData]
                                  │ transmit options: 0x05
                                  │ callback id:      79
                                  └─[SecurityCCNonceReport]
                                      nonce: 0xe6d3587ddd41212a
2022-04-27T23:09:54.255Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:54.260Z SERIAL « 0x0104011301e8                                                       (6 bytes)
2022-04-27T23:09:54.261Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:54.262Z DRIVER « [RES] [SendData]
                                    was sent: true
2022-04-27T23:09:54.281Z SERIAL « 0x011800134f00000200be7f7f7f7f000003000000000301000006              (26 bytes)
2022-04-27T23:09:54.282Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:54.284Z DRIVER « [REQ] [SendData]
                                    callback id:            79
                                    transmit status:        OK, took 20 ms
                                    routing attempts:       1
                                    protocol & route speed: Z-Wave, 100 kbit/s
                                    ACK RSSI:               -66 dBm
                                    ACK channel no.:        0
                                    TX channel no.:         0
2022-04-27T23:09:54.305Z SERIAL » 0x010500550250fd                                                     (7 bytes)
2022-04-27T23:09:54.306Z DRIVER » [Node 002] [REQ] [DeleteSUCReturnRoute]
                                    payload: 0x0250
2022-04-27T23:09:54.308Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:54.313Z SERIAL « 0x0104015501ae                                                       (6 bytes)
2022-04-27T23:09:54.314Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:54.315Z DRIVER « [RES] [DeleteSUCReturnRoute]
                                    was executed: true
2022-04-27T23:09:54.466Z SERIAL « 0x010500555000ff                                                     (7 bytes)
2022-04-27T23:09:54.468Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:54.470Z DRIVER « [REQ] [DeleteSUCReturnRoute]
                                    callback id:     80
                                    transmit status: OK
2022-04-27T23:09:54.487Z CNTRLR » [Node 002] Assigning SUC return route...
2022-04-27T23:09:54.502Z SERIAL » 0x010500510251f8                                                     (7 bytes)
2022-04-27T23:09:54.503Z DRIVER » [Node 002] [REQ] [AssignSUCReturnRoute]
                                    payload: 0x0251
2022-04-27T23:09:54.509Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:54.515Z SERIAL « 0x0104015101aa                                                       (6 bytes)
2022-04-27T23:09:54.517Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:54.518Z DRIVER « [RES] [AssignSUCReturnRoute]
                                    was executed: true
2022-04-27T23:09:55.104Z SERIAL « 0x010500515100fa                                                     (7 bytes)
2022-04-27T23:09:55.107Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:55.110Z DRIVER « [REQ] [AssignSUCReturnRoute]
                                    callback id:     81
                                    transmit status: OK
2022-04-27T23:09:55.123Z CNTRLR » [Node 002] deleting return routes (attempt 1)...
2022-04-27T23:09:55.145Z SERIAL » 0x010500470252ed                                                     (7 bytes)
2022-04-27T23:09:55.146Z DRIVER » [Node 002] [REQ] [DeleteReturnRoute]
                                    payload: 0x0252
2022-04-27T23:09:55.149Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:55.153Z SERIAL « 0x0104014701bc                                                       (6 bytes)
2022-04-27T23:09:55.154Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:55.155Z DRIVER « [RES] [DeleteReturnRoute]
                                    has started: true
2022-04-27T23:09:55.839Z SERIAL « 0x010500475200ef                                                     (7 bytes)
2022-04-27T23:09:55.843Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:55.845Z DRIVER « [REQ] [DeleteReturnRoute]
                                    callback id:     82
                                    transmit status: OK
2022-04-27T23:09:55.858Z CNTRLR » [Node 002] assigning return routes to the following nodes:
                                  1
2022-04-27T23:09:55.859Z CNTRLR » [Node 002] assigning return route to node 1 (attempt 1)...
2022-04-27T23:09:55.868Z SERIAL » 0x01060046020153ef                                                   (8 bytes)
2022-04-27T23:09:55.869Z DRIVER » [Node 002] [REQ] [AssignReturnRoute]
                                    payload: 0x020153
2022-04-27T23:09:55.871Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:55.881Z SERIAL « 0x0104014601bd                                                       (6 bytes)
2022-04-27T23:09:55.882Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:55.883Z DRIVER « [RES] [AssignReturnRoute]
                                    has started: true
2022-04-27T23:09:56.584Z SERIAL « 0x010500465300ef                                                     (7 bytes)
2022-04-27T23:09:56.589Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:56.592Z DRIVER « [REQ] [AssignReturnRoute]
                                    callback id:     83
                                    transmit status: OK
2022-04-27T23:09:56.603Z CNTRLR   [Node 002] healed successfully
2022-04-27T23:09:56.606Z CNTRLR » [Node 002] requesting node neighbors...
2022-04-27T23:09:56.614Z SERIAL » 0x01070080020000007a                                                 (9 bytes)
2022-04-27T23:09:56.615Z DRIVER » [REQ] [GetRoutingInfo]
                                    remove non-repeaters: false
                                    remove bad links:     false
2022-04-27T23:09:56.619Z SERIAL « [ACK]                                                                   (0x06)
2022-04-27T23:09:56.622Z SERIAL « 0x01200180ed9329a0115800000000128285dc607208000000000040995f0300000 (34 bytes)
                                  0b6
2022-04-27T23:09:56.623Z SERIAL » [ACK]                                                                   (0x06)
2022-04-27T23:09:56.624Z DRIVER « [RES] [GetRoutingInfo]
                                    node ids: 1, 3, 4, 6, 7, 8, 9, 10, 13, 16, 17, 20, 22, 30, 32, 33, 37, 44, 4
                                  5, 47, 82, 85, 90, 96, 97, 99, 104, 107, 108, 109, 111, 112, 118, 119, 122, 12
                                  5, 126, 127, 132, 183, 185, 188, 189, 192, 193, 194, 195, 196, 197, 199, 201,
                                  202
2022-04-27T23:09:56.636Z CNTRLR « [Node 002] node neighbors received: 1, 3, 4, 6, 7, 8, 9, 10, 13, 16, 17, 20, 2
                                  2, 30, 32, 33, 37, 44, 45, 47, 82, 85, 90, 96, 97, 99, 104, 107, 108, 109, 111
                                  , 112, 118, 119, 122, 125, 126, 127, 132, 183, 185, 188, 189, 192, 193, 194, 1
                                  95, 196, 197, 199, 201, 202
2022-04-27T23:09:56.637Z CNTRLR   [Node 008] added to healing queue for listening nodes
2022-04-27T23:09:56.637Z CNTRLR   [Node 020] added to healing queue for listening nodes
2022-04-27T23:09:56.637Z CNTRLR   [Node 022] added to healing queue for listening nodes
2022-04-27T23:09:56.638Z CNTRLR   [Node 090] added to healing queue for SLEEPING nodes
2022-04-27T23:09:56.638Z CNTRLR   [Node 104] added to healing queue for SLEEPING nodes
2022-04-27T23:09:56.638Z CNTRLR   [Node 132] added to healing queue for SLEEPING nodes
2022-04-27T23:09:56.638Z CNTRLR   [Node 193] added to healing queue for SLEEPING nodes
2022-04-27T23:09:56.639Z CNTRLR   [Node 003] healing node...

It happens to several devices during the heal. Search for line

2022-04-27T23:10:34.341Z CNTRLR
2022-04-27T23:13:54.262Z CNTRLR

What did you expect to happen?
Only devices that can route will be reported as neighbors.
Steps to reproduce the behavior:
Start a network heal and follow the debug log.
Device information
No response
How are you using node-zwave-js?

[ ] zwavejs2mqtt Docker image (latest)
[ ] zwavejs2mqtt Docker image (dev)
[ ] zwavejs2mqtt Docker manually built (please specify branches)
[ ] ioBroker.zwave2 adapter (please specify version)
[ ] HomeAssistant zwave_js integration  (please specify version)
[ ] pkg
[ ] node-red-contrib-zwave-js (please specify version, double click node to find out)
[ ] Manually built from GitHub (please specify branch)
[X] Other (please describe)

Which branches or versions?
zwavejs2mqtt version: 6.7.4.e6d17ad
zwave-js version: 9.0.4
Did you change anything?
no
If yes, what did you change?
No response
Did this work before?
Don't know, this is a new device
If yes, where did it work?
Can remember seeing this before, on version before v9.
Attach Driver Logfile
zwavejs_current.log

Fixed.
It doesn't seem to be a consistency to the «sleeping nodes» reported as neighbors. Did a reboot and a fresh network heal now, and node 2 now reports (even more) sleeping nodes:
2022-04-28T12:42:56.989Z CNTRLR « [Node 002] node neighbors received: 1, 3, 4, 6, 7, 8, 9, 10, 12, 13, 16, 17, 2
                                  0, 22, 30, 32, 33, 37, 44, 45, 47, 78, 81, 82, 85, 90, 96, 97, 99, 104, 107, 1
                                  08, 109, 111, 112, 118, 119, 122, 125, 126, 127, 132, 134, 182, 183, 185, 188,
                                   189, 192, 193, 194, 195, 196, 197, 199, 201, 202
2022-04-28T12:42:56.989Z CNTRLR   [Node 003] added to healing queue for listening nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 008] added to healing queue for listening nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 020] added to healing queue for listening nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 022] added to healing queue for listening nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 078] added to healing queue for sleeping nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 081] added to healing queue for sleeping nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 090] added to healing queue for sleeping nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 104] added to healing queue for sleeping nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 132] added to healing queue for sleeping nodes
2022-04-28T12:42:56.990Z CNTRLR   [Node 134] added to healing queue for sleeping nodes
2022-04-28T12:42:56.991Z CNTRLR   [Node 182] added to healing queue for listening nodes
2022-04-28T12:42:56.991Z CNTRLR   [Node 202] added to healing queue for sleeping nodes

zwavejs_current.log

Yes, they do all exists.

@AlCalzone

As discussed in Slack, I don't think this is actually a bug. Battery-powered nodes can be neighbors of mains-powered nodes. Unless the controller is reporting false info, but even then we can't do much about it.


Adding TypeScript to "Getting Started"

with reference to the closed PR #1800

@chalin @svrnm This seem to have not fixed the issue :/

/easycla

@chalin @svrnm This seem to have not fixed the issue :/

😡 ...
have you signed the CLA before, e.g. for any other CNCF project?

I think yes, I have contributed to Kubernetes before. Is it possible for me to try sign it again. I looked it up but didn't find any way to sign it again. (Nor getting option to sign for the first time contributor)

/easycla

I think I messed up the commit history.
Update: Got reply from the ticket. Can be caused by no of commits. Will open a new pr from new branch

Can be caused by no of commits

Wow. Heh. Thanks for investigating. Fingers crossed!


Unexpected token, expected ";" in correct code

TypeScript Version:  2.9.1

Search Terms:
Code
import {NetworkListenerEvent} from "../listeners/network-listener";

export enum NetworkControllerEnvironment {
    Unicast,
    Multicast,
    Broadcast,
    All
}

export default interface NetworkController {
    readonly endpoint: string;
    readonly environment: NetworkControllerEnvironment;
    readonly handle: (event: NetworkListenerEvent) => void;
}

Expected behavior:
compile successfully without error
Actual behavior:
compilation fails
Playground Link:
Related Issues:
Stack trace:
{ SyntaxError: D:\Projects\local-network\src\core\network-controller.ts: Unexpected token, expected ";" (10:25)

   8 | }
   9 |
> 10 | export default interface NetworkController {
     |                          ^
  11 |     readonly endpoint: string;
  12 |     readonly environment: NetworkControllerEnvironment;
  13 |     readonly handle: (event: NetworkListenerEvent) => void;
    at _class.raise (D:\Projects\local-network\node_modules\@babel\parser\lib\index.js:3861:15)
    at _class.unexpected (D:\Projects\local-network\node_modules\@babel\parser\lib\index.js:5190:16)
...

What version of the compiler are you using? my guess is you are using an older compiler version.

I wrote v2.9.1 in the top (output of tsc -v), if that's not the compiler version, how would I find it?

Sorry, did not notice the babel stack. this should be filed under https://github.com/babel/babel/issues


Permutive RTD module docs

Docs for Permutive RTD submodule
The docs are reflecting the latest update https://github.com/prebid/Prebid.js/pull/7001

Hey @bretg  - this creates the docs pages for the Permutive RTD submodule. As discussed, it's pretty much the same content as the updated readme from https://github.com/prebid/Prebid.js/pull/7001

@patmmccann are you also happy with the changes? Would be great to get this merged. Thanks!

yes ty!


Fix include when provided in array format from request

@maximebeaudoin Merge it when you're ready.


feat(gui): basic functionalities of buttons and Keys

lgtm, pressing the keys prints the according messages :)


WI-54641 Add type declarations to magic method signatures

https://wiki.php.net/rfc/magic-methods-signature

Why no mixed type for arguments and return values?

@b1rdex it's actually not real mixed. Real mixed doesn't allow void return types (and this is how PHPStorm's treats declared mixed, search for mixed|void in https://wiki.php.net/rfc/mixed_type_v2). However, it's allowed to have void return type declaration: https://3v4l.org/IqCnR

The real checks performed by PHP Engine are here: https://github.com/php/php-src/pull/4177/files#diff-246862e485897a77cdcef1c52d473a52


Installation instructions outdated

Hello, I was trying to build and install a bookstack using your image. I faced a number of issues.
First, the version uploaded to docker hub is rather old, with bookstack v22.04. I could install but I needed the newer features, so I came to build one myself.
Second, the "docker run" commands in the README.md don't work with the newer version of bookstack. They lack the -e APP_KEY=whatever\ line. Even if I add that, the app can't start properly, instead of login, I get an "unexpected error" page.
Furthermore, you cannot mount the .env file because the startup scripts would want to rename it and fail. So the modified docker run command also doesn't work as suggested in the README.md file.
As opposed to the compose yaml, where there is mysql 8.3, we still get the old 5.7.21 version of mysql in README.md "docker run" commands.
With docker compose, the server will start up and show a login page, but the default<EMAIL_ADDRESS>user name is not accepted because the new bookstack version demands a real e-mail address.
Anyway thanks for your hard work, I hope this helps to improve the documentation. Workaround is to compose and then enter:
docker exec -u root -it docker-bookstack-master_bookstack_1 bash
root@60*******a61:/var/www/bookstack# php artisan bookstack:create-admin
But I didn't yet find the correct way to do the installation steps mentioned.

First, the version uploaded to docker hub is rather old, with bookstack v22.04. I could install but I needed the newer features, so I came to build one myself.

New versions are uploaded to docker hub as can be seen here, and latest points the the current release, and the commands in the readme & compose refer to the latest version.
It's just that docker hub shows an old version by default for some reason.

but the default<EMAIL_ADDRESS>user name is not accepted because the new bookstack version demands a real e-mail address.

This should not be the case, at least that's not something that's changed in BookStack.


Refactor symmetry

change file structure
change type hierarchy
generalize cyclic symmetries

I decided to move the whole symmetrization out of this package to its own: SpinSymmetry.jl
The new code is more general and more correct than the removed one.


Return better errors from exec

Also cleans up some of the API side of exec.
Was writing the header twice (two different headers).
Closes #14876

LGTM

LGTM


⚠ Adacta-fintech.com has degraded performance

In 113c33b, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1239 ms

Resolved: Adacta-fintech.com performance has improved in 2e697d2.


Adding reasoning field - tasks1-50

I wrote a script for separating the reasoning field in the tasks and updated the first 50 tasks. I'll update all the files in the next PRs.

All the PRs on reasoning field: Is it about separating the categories with the word 'reasoning' and storing in a different field? If this is the case, we might not need to review. @yeganehkordi

All the PRs on reasoning field: Is it about separating the categories with the word 'reasoning' and storing in a different field? If this is the case, we might not need to review. @yeganehkordi

Yes, I've separated the reasoning field and sorted the keys.


About plug-ins

Sorry，The settings are not displayed successfully as required，Excuse me, what is the reason?

Might be the note me not very clear, your birthday.md have to match the file, if you put it in a folder then it have to be like this: 00 - System/birthdayNode.md


fix(cli): add missing recipe type exports

Closes #1317
📝 Description
Adds RecipeConfig and RecipeVariantRecord to the type exports.
⛳ Current behavior (updates)

Please describe the current behavior that you are modifying

Types targeting recipe configs are not exported in @pandacss/dev. Slot recipes do not face this issue as they are exported.
🚀 New behavior

Please describe the behavior or changes this PR adds

Adding recipe-related types fixes this issue in particular (and my own use case).
💣 Is this a breaking change (Yes/No):
No

📝 Additional Information
That being said, PatternConfig and Parts might want to be exported too (let me know if that's needed too and I'll make the changes)

LGTM, just needs a changeset like
---
'@pandacss/dev': patch
---

Add missing types (PatternConfig, RecipeConfig, RecipeVariantRecord) to solve a TypeScript issue (The inferred type of
xxx cannot be named without a reference...)


exposed CSS code

Steps to Reproduce
See https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/pages/c1/c1s1/
I'm not sure how widespread this in. If we can identify the issue, then it would be good to do a search through studio data.
If it's just a couple of pages, we can ask some to remove it manually through studio.

@pdpinch It looks like this raw CSS is in some legacy courses, I was able to do a quick search and filter out some courses.
Not sure if this list is considered long, or medium but I think this is something that will not occur in new courses so maybe we can do it manually.
If we decide to remove it manually then here are the links to studio (of pages) containing this raw CSS:
Note:
studio RC link can be replaced with prod.
It also contains some resources link: for images, there is this raw CSS in the image caption.
Format
Course Name
Studio site link

6-006-introduction-to-algorithms-fall-2011

https://ocw-studio-rc.odl.mit.edu/sites/6-006-introduction-to-algorithms-fall-2011/type/page/edit/2e91de4c-23a9-f323-866d-265121e71f07/
https://ocw-studio-rc.odl.mit.edu/sites/6-006-introduction-to-algorithms-fall-2011/type/page/edit/3e392c68-f217-c957-d2c8-70f39c63c084/

21a-550j-dv-lab-documenting-science-through-video-and-new-media-fall-2012

https://ocw-studio-rc.odl.mit.edu/sites/21a-550j-dv-lab-documenting-science-through-video-and-new-media-fall-2012/type/page/edit/d61a4e55-ca51-6fb4-fe89-4ab0e52d9a13/

6-004-computation-structures-spring-2017
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/24191c34-3d57-2641-409b-fd54b3d9b2dc/?q=1.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/76b26f4e-38da-1486-221a-ccb9b343c762/?q=10.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/2db1eadc-5344-f2de-9462-f7932dabec44/?q=11.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/29840bcc-41c4-bfab-a071-981783ab0a1b/?q=12.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/7fba5e1d-2fa4-056c-150f-90508b97f937/?q=13.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/76ea0269-3b06-ecaa-e0bb-d8ca4be4bb7e/
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/61c8d864-831f-2145-e0a9-922732a385ec/
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/435fa3f2-0748-7b97-ace6-65391f3bff5f/?q=17.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/5e79a00f-7377-2e87-4bc8-390515e63a0d/?q=18.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/afb5e6a8-476a-7f76-4e0a-4be0d22685e4/
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/f36c8cab-16b9-624b-b4fe-c82e9ddf0f73/?q=2.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/866cbcd7-fe65-4979-53fa-90baa0c3f6da/?q=20.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/9a439586-e23c-50b5-9bf2-cfbc1289a41e/?q=21.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/d77171d7-bbaa-8eb5-d974-6a2895701253/?q=3.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/b39a5b15-c1c8-c348-7bc5-29f6a8d2e699/
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/ba996719-fd35-c417-5993-5d31177dfcb9/?q=5.1+Annotated+Slides
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/1ce0a254-f5ec-8356-abea-088ff937a0ac/
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/a97ebc62-7536-6091-9846-90b005674a5d/
https://ocw-studio-rc.odl.mit.edu/sites/6-004-computation-structures-spring-2017/type/page/edit/c8a85f9c-129e-5b36-f9c5-2a3365574124/?q=8.1+Annotated+Slides

18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018
https://ocw-studio-rc.odl.mit.edu/sites/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/type/resource/edit/7d689bdb-8da9-848c-569f-85e93ce9b9d4/
https://ocw-studio-rc.odl.mit.edu/sites/18-650-statistics-for-applications-fall-2016/type/resource/edit/74db4119-31d8-54f5-d226-78b891acf562/?q=18-650f16-th.jpg
https://ocw-studio-rc.odl.mit.edu/sites/18-650-statistics-for-applications-fall-2016/type/resource/edit/de1506da-6268-0127-2f3f-35bb8412e638/?q=18-650f16.jpg
https://ocw-studio-rc.odl.mit.edu/sites/6-849-geometric-folding-algorithms-linkages-origami-polyhedra-fall-2012/type/page/edit/a370594e-3650-963e-bb62-70f5dc3b68ed/

If you need more info like studio admin links or UUIDs, let me know.

@MAbdurrehman12 can you just go ahead and fix these pages? You can do it through Studio, or some other way if you like.

can you just go ahead and fix these pages? You can do it through Studio, or some other way if you like.

Sure

These pages have been fixed on both, RC and PROD.
However, some pages had missing descriptions (page description as seen by search engines) which is a required field and we can not save the content without it so I also added them (added what seemed the most suitable to me) but they can be updated (if required).
Pages of this course 6-004-computation-structures-spring-2017 mentioned above had missing descriptions.


Change config.zsh to keep previously set values of config vars

Change variable assignments to:
: ${ZSH_AUTOSUGEST_VAR:=value}

https://unix.stackexchange.com/questions/122845/using-a-b-for-variable-assignment-in-scripts#comment685330_122848
See https://github.com/zsh-users/zsh-autosuggestions/commit/bcbdad83e940917db9bbd0afd62057229446c00f#r29294614

Fixed on develop branch


[cppgraphqlgen] Update cppgraphqlgen and PEGTL to 3.0.0

Cppgraphqlgen 3.0 depends on PEGTL 3.0, and both projects have moved to C++17. So to maintain backwards compatibility with the previous version of PEGTL (2.8.0) which used C++11 I moved that into a pegtl-2 package. I'm not doing same with cppgraphqlgen-2 because the binary files collide in the shared bin/lib directories, so the two versions can't coexist in vcpkg like PEGTL, which is a header-only library.

This follows the guidance in #5337. Once official releases are available for both projects I'll go ahead and send the PR.

I'm jumping the gun a little bit because I want to write the documentation for cppgraphqlgen 3.0 assuming that you can get PEGTL 3.0 from vcpkg. Using the 3.0.0-pre version numbers should make it fairly easy to drop in a new update later with 3.0.0.


json: unsupported value: NaN on getnetworkinfo

I'm on bitcoin testnet and I get this error message when trying to use getnetworkinfo.
~ lncli getnetworkinfo
unable to decode response:  json: unsupported value: NaN

Can you provide any further details to aide us in investigating this issue? Perhaps a log dump when the command is run?

I guess there was none, I tailed the logs right above, I would have noticed if there was one, but let me double-check.
It was a fresh installation from latest master. I think it was even before I added any funds.

This is the whole startup log in debug loglevel. (Trace dumps whole blocks and my ssh connection fails to handle that nicely :D)

12:25:42 2017-05-24 [INF] LTND: Version 0.2.1-alpha
12:25:42 2017-05-24 [INF] CHDB: Checking for schema update: latest_version=1, db_version=1
12:25:42 2017-05-24 [INF] LTND: Primary chain is set to: bitcoin
12:25:43 2017-05-24 [INF] DISC: Opened wallet
12:25:44 2017-05-24 [INF] LNWL: Established connection to RPC server localhost:18334
12:25:46 2017-05-24 [INF] DISC: The wallet has been unlocked without a time limit
12:25:46 2017-05-24 [DBG] DISC: Checking for previous saved block with height 1125563 hash 00000000ad1252fe57832b08579fb84823221575d5129b5eab73bba7b0c43d0c
12:25:46 2017-05-24 [INF] LTND: LightningWallet opened
12:25:46 2017-05-24 [DBG] DISC: Found matching block.
12:25:46 2017-05-24 [INF] DISC: Started rescan from block 00000000ad1252fe57832b08579fb84823221575d5129b5eab73bba7b0c43d0c (height 1125563) for 6 addresses
12:25:46 2017-05-24 [INF] DISC: Finished rescan for 6 addresses (synced to block 00000000ad1252fe57832b08579fb84823221575d5129b5eab73bba7b0c43d0c, height 1125563)
12:25:47 2017-05-24 [INF] LNWL: Established connection to RPC server localhost:18334
12:25:47 2017-05-24 [INF] NTFN: New block epoch subscription
12:25:47 2017-05-24 [INF] DISC: Authenticated Gossiper is starting
12:25:47 2017-05-24 [INF] NTFN: New block epoch subscription
12:25:47 2017-05-24 [INF] CRTR: FilteredChainView starting
12:25:47 2017-05-24 [INF] LNWL: Established connection to RPC server localhost:18334
12:25:47 2017-05-24 [INF] CRTR: Prune tip for Channel Graph: height=1125563, hash=00000000ad1252fe57832b08579fb84823221575d5129b5eab73bba7b0c43d0c
12:25:47 2017-05-24 [INF] CRTR: Filtering chain using 0 channels active
12:25:47 2017-05-24 [DBG] CRTR: Updating chain filter with new UTXO's: []
12:25:47 2017-05-24 [INF] CMGR: Server listening on [::]:5656
12:25:47 2017-05-24 [INF] RPCS: gRPC proxy started
12:25:47 2017-05-24 [INF] RPCS: RPC server listening on <IP_ADDRESS>:10009

@Roasbeef Maybe this has something to do with it: https://github.com/btcsuite/btcd/pull/947
But it should not, since I'm attaching it to the btcd built with your personal branch. Maybe you updated the getnetworkinfo struct to comply with bitcoind?

That btcd RPC call is completely unrelated to the issue you're reporting here. Dug in a bit last week and found the source, will push the fix in a minute here. Thanks for reporting this!


Allow toggling of SDK logs in e2e's

Summary
Similar to how you can change the cometbft logs from info to debug, I'd like to be able to switch sdk logs from info to debug

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged/assigned

Slightly looking into it, sdk config doesn't have a log level option to toggle like comet does, maybe we should bring it up to em.

I think you can control the log level with cli flags, but would be nice if it was in the config as well


Add inscribed_ fields for PriorLesion and AdjacentSpecimen

AttributeError: type object 'PriorLesion' has no attribute 'inscribed_clinicalCore_participant_ID'
AttributeError: type object 'AdjacentSpecimen' has no attribute 'inscribed_biospecimen_identifier'

Do you really need inscribed_clinicalCore_participant_ID on PriorLesion? I figure you'd just make them as you parse the |-separated lesions in a clinicacore spreadsheet; you never will encounter them separately or out-of-order, right @asitang?

Do you really need inscribed_clinicalCore_participant_ID on PriorLesion? I figure you'd just make them as you parse the |-separated lesions in a clinicacore spreadsheet; you never will encounter them separately or out-of-order, right @asitang?


L2: Support multiple Proxy deployments in the same transaction

Currently if there are more than one Proxy deployment in the same transaction they will not be decoded propertly.
There are some ways this can be done for proxies deployed using our official ProxyFactory :

Check ProxyCreation events on the transaction and match the order with the list of decoded data.
For saltNonce deployments calculate the deployment address using the decoded saltNonce and initializer.

@JagoFigueroa I created the test transaction on Sepolia as I didn't have funds on Polygon, but this should be tested on Polygon

Tested and verified, thank you! I have created a couple of safes on polygon modifying your script a little bit and can confirm that all good 😄


Add support for Microsoft authentication

Fixes https://github.com/grafana/grafana-infinity-datasource/issues/768
This PR adds native Azure authentication, provided by github.com/grafana/grafana-azure-sdk-go.
It supports:

ClientCredentials (this was possible via OAuth2)
ManangedIdentity (credential-less authentication inside Azure Virtual Machines)
WorkloadIdentity (credential-less authentication inside Kubernetes)

I fixed the conflict.
@yesoreyeram let me know If I can help/assist here.

@yesoreyeram
I may ask myself, how this may can work, if os.LookupEnv does not work.
At the end, grafana/grafana-azure-sdk-go is using Azure/azure-sdk-for-go is expect some OS environment variables.
Ref:

https://github.com/grafana/grafana-azure-sdk-go/blob/21e2891b4190eb7c255c8cd275836def8200faf8/aztokenprovider/retriever_workloadidentity.go#L74C32-L74C61
https://github.com/Azure/azure-sdk-for-go/blob/a45cf26fd136c65f9ca4f00405c5ccce7b3941e3/sdk/azidentity/workload_identity.go#L71

I figure out that Grafana admins needs to opt-in to allow such authentication.
https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#azure
Grafana pass the azure settings to plugins context, and the grafana-azure-sdk can be make aware of them

https://github.com/grafana/grafana/blob/82f910c0c5e1c9b0c903dc40d0094e3a00b3b2f7/pkg/services/pluginsintegration/pluginconfig/envvars.go#L63
https://github.com/grafana/grafana-azure-sdk-go/blob/21e2891b4190eb7c255c8cd275836def8200faf8/azsettings/env.go#L17

I had rewrite the plugin datasource config. The plugins will take care of the azure settings for now. It will allow only the specific authentications, if the Grafana admin has allowed it via grafana.ini.

Seems like something broke after a merge commit.

@yesoreyeram It seems like the error you get is coming from Grafana SDK, if user_identity_enabled = true is set on Grafana instance.
The errors comes from the Grafana SDK: https://github.com/grafana/grafana-azure-sdk-go/blob/c271284be0be9f85c64b6d2c8d00d22e458dc80f/azsettings/env.go#L86

Some changes I make:

I bump https://github.com/grafana/grafana-azure-sdk-go to v2
Rename Microsoft to Azure. While Microsoft would be technically correct, the Grafana SDK and eco system is using the term Azure and I follow the terms for constantcy.
Renaming jsonData properties. It has the benefit that grafana-azure-sdk-go has an builtin function to natively read datasource data and provide return an azureCredentials struct - https://github.com/grafana/grafana-azure-sdk-go/blob/c271284be0be9f85c64b6d2c8d00d22e458dc80f/azcredentials/builder.go#L10
I removed all Azure related types on the backend and fully using the Grafana Azure SDK capabilities. Building an own type and struct feels worth-less if Grafana Azure SDK could directly read the jsonData of an datasource.
I removed any custom validation. If a health check url is defined, all settings are validated through Grafana Azure SDK.

I will do a rebase after the next review cycle.

Great Work! looking forward for this PR to be merged

Hey @yesoreyeram
any new about this?

I see that the CI is failing because of spell check. It seems like some azure terms are unknown

great, guessing it is ready to merge now?

Done.
It seems like additional check (lint) where introduced.
If the CI would automatic start, I could fix the CI issues on my own. However I have to ask someone each time to trigger the CI which makes to process terrible slow.

tests are green here now

Hey @yesoreyeram can we merge this

Hey @yesoreyeram
what are the next steps here?

Hi @jkroepke this is blocked on our side, due to internal discussions on authentication architecture, which may take a little while. We understand that you put a lot of effort in this PR and we are sorry it is taking so long.

Hi @jkroepke @yesoreyeram @ivanahuckova
Found this PR when i tried to solve my issue: currently blocked since i cannot integrate Azure Key Vaults into Grafana because of problems with OAuth2 integration (cannot set the "Scopes" to https://vault.azure.net/.default). Seems that the OAuth2 part will not be taken over into any queries. Can you tell me if this would be solved with this PR too?
Background information:
We want to scrape the Key Vault Endpoints for getting expiry dates for proper monitoring, alerting, can improve release plannings of PROD environments (automotive company with automotive customers). Would be very happy to also have this native support provided (and hope that it will also solve my problem).

Can you tell me if this would be solved with this PR too?

Yep, within this PR, scopes can be configured.
https://github.com/grafana/grafana-infinity-datasource/pull/785/files#diff-337c34e7ed021093c90e6727c53d96c01e4023e9c15b193f1cf794eb907d2d35R114-R136

Can you tell me if this would be solved with this PR too?

Yep, within this PR, scopes can be configured. You can also use Managed Identities to gain access to KeyVault password-less.
https://github.com/grafana/grafana-infinity-datasource/pull/785/files#diff-337c34e7ed021093c90e6727c53d96c01e4023e9c15b193f1cf794eb907d2d35R114-R136

This would be awesome! Would solve a lot of problems and simplify several things, nowadays i would need to write custom prometheus exporter for doing this but which would lead to more effort instead of using alternatives like this one here <3

nowadays i would need to write custom prometheus exporter

Not sure, if it helps: https://github.com/webdevops/azure-keyvault-exporter

Not sure, if it helps: https://github.com/webdevops/azure-keyvault-exporter

Thanks for the link, already familiar with it :) Markus Blascchke is a former colleague of mine :) This looks really good but the issue here this that you also would need to dockerize it, create a helm chart, deploy it, maintain it, patch it....and you have to think about where to deploy it later. Makes it pretty difficult if you do not have one single environment but several ones without any multi tenancy setup. All about processes: In enterprise companies you then have also to take care about OSS, license and vulnerability such as security stuff.
Converting that into money would mean high costs. That is why it would be nice to use the plugin for doing that. You can automate the whole deployment via terraform, starting with Grafana, then the dashboards and last one the datasources (like this one here: https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/data_source.html) Since already mentioned above in Grafana would make the setup more attractive since i would then only need to take care about Grafana and not where i put also then the exporter -> there i would think about using Kubernetes or any other Container Solution with Prometheus integration.

@gwdawson @yesoreyeram

@mrmiyagietas we wrote a custom proxy between grafana-infinity-datasource and the upstream URL which injects the authentication and may do other things, e.g. pagination.


datepicker re-rendering doesn't work properly

When re-rendering an input with the datepicker-popup directive, ui-datepicker doesn't seem to take into account the previously selected model value.
Please take a look to this plunker, pick a date, then toggle the checkbox two times to remove and recreate the datepicker form field.
You'll notice that the form field value changed (date formatting is lost), which to form validation issues
Opened a stackoverflow question

@hamfastgamgee thanks for the fix! Maybe you can prepare a pull request?
Would be great to have the fix in the official repository.

Finally figured out how to get git all set up on my side.  Yay for corporate firewalls.  I've submitted a PR with the "if" check, but it turns out angular 1.2 "worked" without that (in that it "failed" but immediately auto-recovered in a way that angular 1.3 doesn't).
The PR does not fix the datepicker to be fully 1.3-compliant.  I may have some time to help with that as well, but not necessarily immediately.

This has been fixed by 2c2dba6d145c95500efe7f6be6248d56ae1aebee.


Adding Filebase to list of IPFS Pinning Services

Adding Filebase to list of IPFS Pinning Services

Since we merged #1358 , can we close this PR @jamongeon ?


[pulsar-client-tools] Add resource close in CmdBrokerStats and some minor fixes

Motivation
Resources in CmdBrokerStats should be closed.
Modifications
Use try-with-resources and add description for load-report in admin-cli doc.

run cpp tests


Bob stolzberg test

Updated the Vormetric KB article

Made changes to Vormetric KB article.
Added new Bitnami KB articles.

Added How to publish Partner Template kb


NIFI-8294 - Functional processor and service for sending flowfile content to Azur…

…e ADX cluster.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
This processor and service enable NiFi to send flowfile content of different formats to an Azure ADX cluster directly.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically main)?

[ ] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[x] Have you verified that the full build is successful on JDK 11?
[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

@markap14 - can you please help to merge this long pending PR or let us know if anything is required to be fixed? Some of our(Microsoft) customers have been waiting for this change to be merged.


babel 5 error

Hi, I'm getting the following error when compiling a project with redux-batched-subscribe:
ERROR in ./~/redux-batched-subscribe/lib/index.js
Module build failed: ReferenceError: [BABEL] /MYPATH/node_modules/redux-batched-subscribe/lib/index.js: Using removed Babel 5 option: /MYPATH/node_modules/redux-batched-subscribe/.babelrc.stage - Check out the corresponding stage-x presets http://babeljs.io/docs/plugins/#presets

Any idea? Thanks
UPDATE:
removing redux-batched-subscribe/.babelrc let me compile without errors.
Is it safe to remove it? In that case, should it be included in the npm package?

Thanks. Published version 0.1.6 which uses a files whitelist instead of npmignore - 765bcf70af8c40952d1dee5ddcecb22b0907500b