Private registry not reachable with crictl

Problem
Currently I'm facing a problem with reaching a private registry.
When we want to pull images from the registry I get a forbidden error.
It seems that this is caused by the TLS settings, that aren't applied correctly.
I've tested several scenario's, where the ContainerD based scenario's fail, while others succeed.
Below the scenario's, configs and environment information are shown.
Scenario's

Pull the image by applying a Kubernetes pod manifest, without success;

apiVersion: v1
kind: Pod
metadata:
  name: testimage
spec:
  containers:
  - name: testimage
    image: privateregistry/path/image:1.0.0

Pull image with crictl with the private registry settings, without success;

crictl pull privateregistry/path/image:1.0.0

Pull the image with docker (based on docker config), with success;

docker pull privateregistry/path/image:1.0.0

Pull the image with ctr, with success.

ctr i pull privateregistry/path/image:1.0.0 -u username:password -k

Configs
registries.yaml
mirrors:
  privateregistry:
    endpoint:
      - "privateregistry"
configs:
  "privateregistry":
    auth:
      username: username
      password: password
    tls:
      insecure_skip_verify: true

Environment information

k3s version 1.23.6+k3s1
ContainerD version 1.5.11-k3s2
Crictl version: 1.22.0-k3s1
Docker version: 20.10.12

Are you using k3s with an external docker container runtime, or with the embedded containerd? Are you restarting k3s after changing registries.yaml?

mirrors:
 privateregistry:
   endpoint:
     - "privateregistry"

This isn't a valid endpoint. First of all, if the hostname of the private registry is the same as the name you're using, you don't need to specify an endpoint - for any registry myregistry.com, there is an implicit default endpoint of https://myregistry.com/v2. Second, endpoints must be a URL, not just a hostname.
Additionally, registries need to have a . or : to be recognized as a registry, and not just part of the image name. privateregistry has neither, so the actual image you're pulling in your example is docker.io/privateregistry/path/image:1.0.0

Hi,
I didn't want to share the real url, so I've put privateregistry instead in my example.
So the real url is like privateregistry.domain.com.
I'm using the embedded ContainerD.

Check your syntax, make sure you've restarted k3s, and check the contents of /var/lib/rancher/k3s/agent/etc/containerd/config.toml and /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

Hi,
I've did the check on the config.toml file ans see that all the necessary content exists.
It looks like this:
`[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."privateregistry.com"]
endpoint = ["privateregistry.com"]
[plugins.cri.registry.mirrors."privateregistry.com".auth]
username = "username"
password = "password"
[plugins.cri.registry.mirrors."privateregistry.com".tls]
insecure_skip_verify = true`

The problem is solved.
It was caused by the proxy.
It seems that the setting NO_PROXY should be used instead of / or combination with CONTAINERD_NO_PROXY.

Ah, indeed. You didn't mention anything about using a proxy in your issue description. That would have been useful information to include.


⚠ vk-s has degraded performance

In a56cd7b, vk-s ($VK_API_STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 7283 ms

Resolved: vk-s performance has improved in 78310a7 after 49 minutes.


ActiveStorage: NoMethodError: undefined method `rails_blob_path'

Per Rails guides, a project may choose to disable ActiveStorage routes by using the setting:
config.active_storage.draw_routes = false

This causes the path method rails_blob_path to not exist.
In Rails Admin, if a user exports a model to CSV and includes the active storage field, they will then receive the error:
/gems/rails_admin-2.2.1/lib/rails_admin/config/fields/types/active_storage.rb:37 in resource_url
/gems/rails_admin-2.2.1/lib/rails_admin/config/fields/types/file_upload.rb:31 in block in <class:FileUpload>
/gems/rails_admin-2.2.1/lib/rails_admin/config/configurable.rb:75 in instance_eval
/gems/rails_admin-2.2.1/lib/rails_admin/config/configurable.rb:75 in block in register_instance_option
/gems/rails_admin-2.2.1/lib/rails_admin/config/proxyable/proxy.rb:25 in method_missing
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:98 in block in generate_csv_row
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:97 in collect
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:97 in generate_csv_row
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:80 in block (2 levels) in generate_csv_string
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:71 in block (2 levels) in find_each
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:71 in each
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:71 in block in find_each
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:138 in block in find_in_batches
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:245 in block in in_batches
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:229 in loop
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:229 in in_batches
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:137 in find_in_batches
/gems/activerecord-<IP_ADDRESS>/lib/active_record/relation/batches.rb:70 in find_each
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:79 in block in generate_csv_string
vendor/ruby-3.0.2/lib/ruby/3.0.0/csv.rb:1281 in generate
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:76 in generate_csv_string
/gems/rails_admin-2.2.1/lib/rails_admin/support/csv_converter.rb:47 in to_csv
/gems/rails_admin-2.2.1/lib/rails_admin/config/actions/index.rb:76 in block (4 levels) in <class:Index>
/gems/actionpack-<IP_ADDRESS>/lib/action_controller/metal/mime_responds.rb:214 in respond_to
/gems/rails_admin-2.2.1/lib/rails_admin/config/actions/index.rb:44 in block (2 levels) in <class:Index>
/gems/rails_admin-2.2.1/app/controllers/rails_admin/main_controller.rb:22 in instance_eval
/gems/rails_admin-2.2.1/app/controllers/rails_admin/main_controller.rb:22 in index
/gems/rails_admin-2.2.1/lib/rails_admin/config/actions/export.rb:21 in block (2 levels) in <class:Export>
/gems/rails_admin-2.2.1/app/controllers/rails_admin/main_controller.rb:22 in instance_eval
/gems/rails_admin-2.2.1/app/controllers/rails_admin/main_controller.rb:22 in export
...

Suggested Changes
When config.active_storage.draw_routes is false:

Make ::Types::ActiveStorage#resource_url and ::Types::MultipleActiveStorage#resource_url return nil

Export the attachment filename instead of a URL

Does export_value { '--' } in your config for those models not work? I believe that's working for us in our project.


[BUG] Sorting in config list

https://github.com/openbullet/openbullet/commit/c227ffed2b78b50d542af414d9b528300e19c05f

Looks like we can't sort by creation date, you can only sort by ascending order. when you click the column again it doesn't sort in descending order. Some other columns have the same behavior... maybe it's something expected ??
EDIT:
probably the same bug as described here ?
https://github.com/openbullet/openbullet/issues/149

Duplicate https://github.com/openbullet/openbullet/issues/149


Time is up for incomplete_utterance_rewriting.

Time is up for incomplete_utterance_rewriting.

@Henryflee I have prepared the code and now working on providing pretrained model weights for the new clean code version. And the expected time will be after 1~2 days. Please stay tuned.

looking forward to seeing your codes

@Henryflee @luciencho Remaining scripts & corresponding pretrained weights are uploaded. Now the main structure of our code has been released. Please see here for a guide.

Almost all code done. Hope you enjoy the code & pretrianed models.


Allow debug mode to be toggled

So wanted to just export ucldebug but gofmt went crazy on it. This formats all the code and makes that simple change.
If you want me to remove the gofmt portion I can.

Exportable debug toggling is ok. Can you remove the gofmt changes; I'll do a gofmt after you push the change over as there are some commenting in there that I will correct after a local gofmt.
thanks.

Closing. New PR is up.


Can iD pre-fill entries with open data?

This might already exist, and I'm just to new to have realised, but can iD pre-fill or auto-fill data if the data is openly available?
For example, in the UK, the Post Office have made available (via a Freedom of Information request) a CSV dataset that lists all of the open Post Office's, which includes the address, GPS location and opening hours of each Post Office.
This data is currently being used for a Post Office Progress Tool - created by one Robert Whittaker, but I do wonder, if I were to add a Post Office node / way using iD, could iD then use the Post Office dataset to grab the address and opening hours data automatically?

I don't know about your example, but if you add a link to a Wikipedia article on an element, it will fill out the information from that, which is pretty awesome :)

Hi @UKChris-osm. This is an interesting idea, but I think it's outside the scope of iD right now. If the post office progress tool can already match authoritative data to existing data, I'd recommend running an automated edit to sync them.
Eventually we'd like to add an assisted-editing workflow to iD that helps mappers manually review and copy over features from external datasets (similar to RapiD). But attempting to pre-populate attributes sounds like a big lift with unclear value.


feat: replace Individual and Organization roles with enums (fix #701)

fix: resolve issue with array key not existing in experiences

@michelled I am going to merge this— I am quite confident that it all works and test are passing with 100% coverage of the changed code. For QA next week, can you test selecting and editing roles of individuals and community orgs?

@greatislander I tested and the only bug I found is https://github.com/accessibility-exchange/platform/issues/788


ACT-712 Add description field (large text) to Stakeholder Type model

The stakeholder type model needs a large text field called Description so that users can provide details about the Stakeholder Type.


No /oauth route exist

[x] I was not able to find an open or closed issue matching what I'm seeing.
[x] This is not a question. (Questions should be asked on slack (Signup for Slack here) or our forums.)

After configuring Expressive 3 and oauth2 (0.4.0) with the actual docs, when i make a request for token i got a 404 error.
I selected zend-servicemanager, FastRoute in the default config.
Code to reproduce the issue
http -f POST localhost:8080/oauth grant_type=password client_id=client_test username=user_test password=test
HTTP/1.1 404 Not Found
Connection: close
Content-type: text/html; charset=UTF-8
Date: Fri, 23 Mar 2018 15:28:03 +0100
Host: localhost:8080
X-Powered-By: PHP/7.1.15-1+ubuntu16.04.1+deb.sury.org+2
Cannot POST http://localhost:8080/oauth
Expected results
JWT token

If i manually add a route in routes.php file, it is fine!
$app->route('/oauth', Zend\Expressive\Authentication\OAuth2\OAuth2Middleware::class, ['GET', 'POST'], 'oauth');

@mbourquin you right, we need to add this route manually as you suggested. I'm fixing the documentation. Thanks!


Initial PCIe Advanced Error Reporting (AER) support.

AER information is only included in the JSON output.
AER output is not included by default, use the -a or --aer optional
command line parameter to include the AER details.
AER information is only provided for devices that support the PCIe
AER capability and the Linux kernel being used must also provide
support for PCIe AER statistics, supported devices will have various
aer_* files under /sys/bus/pci/devices//.

README.md: Updated usage message.
pci_lib/pci_lib.py: Added AER support.
pcicrawler/cli.py: Added AER support including support for command line.

Candidate AER Support for pcicrawler
In regards to the "Support for AER issues" Issue I've put together some initial changes that will optionally capture the PCIe AER stats/counts in the pcicrawler JSON formatted output.  Before investing more time in this I figured it would be good to make a sanity check and get some feedback.  What has been provided in the commit may be sufficient as-is.
As referenced in @bluecmd's original "Support for AER issues" request I used the various per-device pseudo filesystem files provided by the kernel under /sys/bus/pci/devices//.  For each PCIe device that supports AER the aer_dev_correctable, aer_dev_fatal and aer_dev_fatal files provide key/value pairs of error names and their count, the root port files aer_rootport_total_err_cor, aer_rootport_total_err_fatal and aer_rootport_total_err_nonfatal provide counts of errors (correctable, fatal and non-fatal) as observed on a root port.
Given the potentially large volume of AER content I made the AER gathering optional and not included by default.  The related CLI usage text I've proposed is:
  -a, --aer / --no-aer            Include PCIe Advanced Error Reporting (AER)
                                  information when available - only provided
                                  in JSON output

As for the JSON output: When AER information is requested on the command line and AER information is available for a device the information is associated with the device's JSON field name "aer".  If there's no AER data available then the device's JSON output will not be polluted with an empty/null "aer" entry.  There are two sets of AER information that may be available, "device" information and "rootport" information.  Sample JSON output can be found below.
AER "device" information: The names (aer_dev_correctable, aer_dev_fatal and aer_dev_nonfatal) come directly from the filenames referenced on the pseudo filesystem and the text and numbers in the key/value pairs come directly from those files, one key/value pair per line.
AER "rootport" information: The rootport device's JSON field names (aer_rootport_total_err_cor, aer_rootport_total_err_fatal and aer_rootport_total_err_nonfatal) correspond to the filenames referenced on the pseudo filesystem and the error counts come directly from those files.
Testing and sample output:
The AER support relies on the per-device pseudo filesystem files located under /sys/bus/pci/devices//.  For testing purposes reading an assortment of zero value error counts is not particularly satisfying and does not help in validating that the correct information sources are being referenced.
For basic testing I used the aer_inject kernel module and the aer-inject command to fake-out an assortment of PCIe errors on a root port on a Dell T30 running CentOS 8 (and later openSUSE Leap 15.2).  Note that I needed to append "pcie_ports=native" to the kernel command line and reboot before I was successful injecting AER errors on this platform.
Test system details:
[allan@centos8 ~]$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 8.2.2004 (Core)
Release:        8.2.2004
Codename:       Core
[allan@centos8 ~]$
[allan@centos8 ~]$ uname -a
Linux centos8 4.18.0-193.14.2.el8_2.x86_64 #1 SMP Sun Jul 26 03:54:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[allan@centos8 ~]$

Here's the pcicrawler results from CentOS 8:
Start by showing pcicrawler detail of the specific device we'll be looking at (address 0000:00:01.0):
[allan@centos8 ~]$ sudo /usr/local/bin/pcicrawler -x --addr 0000:00:01.0
0000:00:01.0, PCIe root_port, Intel Corporation (8086) Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (1901)
[allan@centos8 ~]$

For reference here's the pcicrawler JSON output without the AER information command line option for that device:
[allan@centos8 ~]$ sudo /usr/local/bin/pcicrawler -x --addr 0000:00:01.0 -j | python3 -m json.tool
{
    "0000:00:01.0": {
        "vendor_id": "8086",
        "device_id": "1901",
        "class_id": "060400",
        "subsystem_vendor": "1028",
        "subsystem_device": "07c5",
        "addr": "0000:00:01.0",
        "express_type": "root_port",
        "cur_speed": "2.5GT/s",
        "cur_width": 0,
        "capable_speed": "8GT/s",
        "capable_width": 16,
        "target_speed": "8GT/s",
        "slot": 1,
        "presence": false,
        "power": null,
        "attn_led": "unsupported",
        "path": []
    }
}
[allan@centos8 ~]$

Now here's the JSON output with the AER command line option, note that PCIe errors where injected against this root port/device (address at 0000:00:01.0) via aer_inject (3 correctable Bad TLP, 2 uncorrecteable non-fatal Completion Abort and 1 uncorrectable fatal Malformed TLP) - hence there are a few non-zero AER error entries:
[allan@centos8 ~]$ sudo /usr/local/bin/pcicrawler -x --addr 0000:00:01.0 -j -a | python3 -m json.tool
{
    "0000:00:01.0": {
        "vendor_id": "8086",
        "device_id": "1901",
        "class_id": "060400",
        "subsystem_vendor": "1028",
        "subsystem_device": "07c5",
        "addr": "0000:00:01.0",
        "express_type": "root_port",
        "cur_speed": "2.5GT/s",
        "cur_width": 0,
        "capable_speed": "8GT/s",
        "capable_width": 16,
        "target_speed": "8GT/s",
        "slot": 1,
        "presence": false,
        "power": null,
        "attn_led": "unsupported",
        "path": [],
        "aer": {
            "device": {
                "aer_dev_correctable": {
                    "RxErr": 0,
                    "BadTLP": 3,
                    "BadDLLP": 0,
                    "Rollover": 0,
                    "Timeout": 0,
                    "NonFatalErr": 0,
                    "CorrIntErr": 0,
                    "HeaderOF": 0,
                    "TOTAL_ERR_COR": 3
                },
                "aer_dev_fatal": {
                    "Undefined": 0,
                    "DLP": 0,
                    "SDES": 0,
                    "TLP": 0,
                    "FCP": 0,
                    "CmpltTO": 0,
                    "CmpltAbrt": 0,
                    "UnxCmplt": 0,
                    "RxOF": 0,
                    "MalfTLP": 1,
                    "ECRC": 0,
                    "UnsupReq": 0,
                    "ACSViol": 0,
                    "UncorrIntErr": 0,
                    "BlockedTLP": 0,
                    "AtomicOpBlocked": 0,
                    "TLPBlockedErr": 0,
                    "TOTAL_ERR_FATAL": 1
                },
                "aer_dev_nonfatal": {
                    "Undefined": 0,
                    "DLP": 0,
                    "SDES": 0,
                    "TLP": 0,
                    "FCP": 0,
                    "CmpltTO": 0,
                    "CmpltAbrt": 2,
                    "UnxCmplt": 0,
                    "RxOF": 0,
                    "MalfTLP": 0,
                    "ECRC": 0,
                    "UnsupReq": 0,
                    "ACSViol": 0,
                    "UncorrIntErr": 0,
                    "BlockedTLP": 0,
                    "AtomicOpBlocked": 0,
                    "TLPBlockedErr": 0,
                    "TOTAL_ERR_NONFATAL": 2
                }
            },
            "rootport": {
                "aer_rootport_total_err_cor": 3,
                "aer_rootport_total_err_fatal": 1,
                "aer_rootport_total_err_nonfatal": 2
            }
        }
    }
}
[allan@centos8 ~]$

So as expected, for this root port both the device and root port information (with expected error counts) are available and reported.  An inspection of the JSON output for a non-root port PCIe device shows only the expected AER "device" entries with all error counts of zero (device address here is 0000:02:00.0):
[allan@centos8 ~]$ sudo /usr/local/bin/pcicrawler -x --addr 0000:02:00.0 -j -a | python3 -m json.tool
{
    "0000:02:00.0": {
        "vendor_id": "104c",
        "device_id": "8240",
        "class_id": "060400",
        "subsystem_vendor": "0000",
        "subsystem_device": "0000",
        "addr": "0000:02:00.0",
        "express_type": "pci_bridge",
        "cur_speed": "2.5GT/s",
        "cur_width": 1,
        "capable_speed": "2.5GT/s",
        "capable_width": 1,
        "target_speed": null,
        "location": "slot 2, root_port",
        "path": [
            "0000:00:1c.0"
        ],
        "aer": {
            "device": {
                "aer_dev_correctable": {
                    "RxErr": 0,
                    "BadTLP": 0,
                    "BadDLLP": 0,
                    "Rollover": 0,
                    "Timeout": 0,
                    "NonFatalErr": 0,
                    "CorrIntErr": 0,
                    "HeaderOF": 0,
                    "TOTAL_ERR_COR": 0
                },
                "aer_dev_fatal": {
                    "Undefined": 0,
                    "DLP": 0,
                    "SDES": 0,
                    "TLP": 0,
                    "FCP": 0,
                    "CmpltTO": 0,
                    "CmpltAbrt": 0,
                    "UnxCmplt": 0,
                    "RxOF": 0,
                    "MalfTLP": 0,
                    "ECRC": 0,
                    "UnsupReq": 0,
                    "ACSViol": 0,
                    "UncorrIntErr": 0,
                    "BlockedTLP": 0,
                    "AtomicOpBlocked": 0,
                    "TLPBlockedErr": 0,
                    "TOTAL_ERR_FATAL": 0
                },
                "aer_dev_nonfatal": {
                    "Undefined": 0,
                    "DLP": 0,
                    "SDES": 0,
                    "TLP": 0,
                    "FCP": 0,
                    "CmpltTO": 0,
                    "CmpltAbrt": 0,
                    "UnxCmplt": 0,
                    "RxOF": 0,
                    "MalfTLP": 0,
                    "ECRC": 0,
                    "UnsupReq": 0,
                    "ACSViol": 0,
                    "UncorrIntErr": 0,
                    "BlockedTLP": 0,
                    "AtomicOpBlocked": 0,
                    "TLPBlockedErr": 0,
                    "TOTAL_ERR_NONFATAL": 0
                }
            }
        }
    }
}
[allan@centos8 ~]$

From the testing above we see that when the optional AER CLI parameter is not provided we get the preexisting/original pcicrawler output and when the AER CLI option is provided we get AER information, in the pcicrawler JSON output, for PCIe devices that support AER.

Approved @allan-young @bluecmd


Include c4 instance types

With the announcement of the new c4 instance types, (http://aws.amazon.com/ec2/instance-types/?sc_ichannel=em&sc_icountry=global&sc_icampaigntype=launch&sc_icampaign=em_130722160&sc_idetail=em_2057599741&ref_=pe_411040_130722160_17#compute-optimized), we should be including c4 instance types in the table.

We also have more info in the pricing page.
It is worth noting (not just for c4 instances, but for all instances) that there are different prices listed for different Linux installations.  I don't think I was aware that Red Hat & SUSE cost more than Amazon or Ubuntu flavors...

Changes in #92 are live. Thanks @stefanor!

w00t!


Auto terminate unused cloud(AWS,Google,Azure) resources

Create a utility script to auto terminate the cloud resources, if not used within the last 1 hr. This will improve our resource usage efficiency and reduce costs.
For instance, we can create a Lambda function on AWS which checks whether a particular resource(say, for example an EMR cluster) is unused for the last 1 hr and then terminate it automatically.

Did a quick POC and was able to launch aws-nuke on our AWS account, as shown below. Next step is to integrate it with our staging airflow instance and run it daily at specified period of the day.
``
(base) phanikv@Phanis-MacBook-Pro nuke % ./aws-nuke-v2.17.0-darwin-amd64 -c config.yml --profile default
aws-nuke version v2.17.0
Do you really want to nuke the account with the ID<PHONE_NUMBER>08 and the alias 'aws-oss-main'?
Do you want to continue? Enter account alias to continue.

aws-oss-main

WARN[0116] skipping request: DNS lookup failed for globalaccelerator.us-east-2.amazonaws.com; assuming it does not exist in this region
us-east-2 - KMSAlias - alias/aws/dynamodb - [Name: "alias/aws/dynamodb"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/ebs - [Name: "alias/aws/ebs"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/elasticfilesystem - [Name: "alias/aws/elasticfilesystem"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/es - [Name: "alias/aws/es"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/glue - [Name: "alias/aws/glue"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/kinesisvideo - [Name: "alias/aws/kinesisvideo"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/rds - [Name: "alias/aws/rds"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/redshift - [Name: "alias/aws/redshift"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/redshifttest - [Name: "alias/aws/redshifttest"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/s3 - [Name: "alias/aws/s3"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/ssm - [Name: "alias/aws/ssm"] - cannot delete AWS alias
us-east-2 - KMSAlias - alias/aws/xray - [Name: "alias/aws/xray"] - cannot delete AWS alias
WARN[0118] skipping request: DNS lookup failed for globalaccelerator.us-east-2.amazonaws.com; assuming it does not exist in this region
us-east-2 - AthenaWorkGroup - primary - [ARN: "arn:aws:athena:us-east-2:475538383708:workgroup/primary", Name: "primary"] - cannot delete primary athena work group
us-east-2 - EC2DHCPOption - dopt-059bbfad9f92078f6 - [DefaultVPC: "true"] - would remove
us-east-2 - GlueDatabase - default - would remove
us-east-2 - GlueDatabase - providers_db - would remove
WARN[0119] skipping request: DNS lookup failed for worklink.us-east-2.amazonaws.com; assuming it does not exist in this region
us-east-2 - Route53ResolverRule - rslvr-autodefined-rr-internet-resolver (Internet Resolver) - [ID: "rslvr-autodefined-rr-internet-resolver", Name: "Internet Resolver"] - Filtering DomainName "."
us-east-2 - CloudFormationStack - eksctl-providers-team-eks-cluster-cluster - [CreationTime: "2022-04-14T18:00:22Z", LastUpdatedTime: "2022-04-14T18:00:22Z", Name: "eksctl-providers-team-eks-cluster-cluster", tag:alpha.eksctl.io/cluster-name: "providers-team-eks-cluster", tag:alpha.eksctl.io/eksctl-version: "0.92.0", tag:eksctl.cluster.k8s.io/v1alpha1/cluster-name: "providers-team-eks-cluster"] - would remove
us-east-2 - CloudWatchEventsTarget - Rule: AutoScalingManagedRule Target ID: autoscaling - would remove
INFO[0121] TagOptionNotMigratedException: TagOption Migration not complete
us-east-2 - EC2Subnet - subnet-0d9b95c165c481808 - [DefaultForAz: "true", DefaultVPC: "true"] - would remove
us-east-2 - EC2Subnet - subnet-0815bfc21b6b4c71a - [DefaultForAz: "true", DefaultVPC: "true"] - would remove
us-east-2 - EC2Subnet - subnet-083a7e2a7c3dc82a0 - [DefaultForAz: "true", DefaultVPC: "true"] - would remove
us-east-2 - CloudWatchEventsRule - Rule: AutoScalingManagedRule - would remove
us-east-2 - EMRCluster - j-LXCZH4ZP5GHO - already terminated
``

AWS nuke ran fine on staging, need to check why it is failing on astro-cloud

AWS nuke ran fine on astro-cloud #323

In case of GCP, we are using following resources:

GCS Buckets
Compute Instances
DataProc Cluster
Kubernetes cluster

Upon exploring available options, there is no direct library available to destroy those resources as per our need. We will need to write a custom shell script to list and destroy the resources.
We also need to check how to securely store the service account key file json needed for authentication & to be used by the shell script as part of our DAG/CI.

Phani unblocked me with assigning permissions to the service account for GCP nuke.
Started writing DAG for nuking. Connection based approach does not work as secrets do not come. Using Variable based approach. Able to authenticate and execute gcloud commands with that

Need additional permissions to create above resources to my dev account (this is different from the service account) to be able to test the nuke.

PR is ready for the above, tested nuking the resources created in GCP example DAGs: https://github.com/astronomer/astronomer-providers/pull/563

GCP nuke DAG is deployed and it is running fine on Astro Cloud.

Found Azure nuke in open source https://github.com/ekristen/azure-nuke Originally based on the source code from aws-nuke fork and aws-nuke original need suggestion on this

In Azure these are the following resource used as part of example DAG

Azure Data factory pipeline
Service Bus
Data Storage blob container

there is different ways to terminate the resource in azure

Use Azure cli commands in the custom shell script
Use Azure PowerShell commands
Use the Open source library for Azure nuke https://github.com/ekristen/azure-nuke

Started with Azure nuke with https://github.com/ekristen/azure-nuke added example DAG, nuke-config.yml file, Testing is pending

Nuke is complete for AWS & GCP. For Azure, we have a separate ticket: https://app.zenhub.com/workspaces/airflow-providers-team-61a89ee7fbf048001dc8378d/issues/astronomer/astronomer-providers/570


Fix KeyError when PGCR result incomplete

Long story short

Checks

[x] I have run nox tests and all pipelines have passed.
[ ] I have wriiten tests according to the code i added, modified or deleted.

Related issues

Thanks for the report but can you provide more information about what raised the KeyError? Using .get on non optional fields is not fixing the issue + The type checker should error Immediately since this type safe.

Traceback that raised the error.
The endpoint that caused the error.
Character and membership IDs that didn't return those fields.

Thanks for the report but can you provide more information about what raised the KeyError? Using .get on non optional fields is not fixing the issue + The type checker should error Immediately since this type safe.

Traceback that raised the error.
The endpoint that caused the error.
Character and membership IDs that didn't return those fields.

Huge thanks for your work!
When I deserialize post activity, I noticed some of activity will raise KeyError. After I requested the api without using aiobungie, I think this Exception caused by Bungie.
Deserialize the standard return data of Destiny2.GetPostGameCarnageReport won't raise KeyError, Bungie's docs doesn't say these field is nullable. But sometimes there're only 4 fields (5 fields missing) in entries.player.destinyUserInfo: "crossSaveOverride", "isPublic", "membershipType" and "membershipId". Value of "genderHash", "raceHash", "classHash" are also null in player's data. When using fuction deserialize_destiny_membership and _deserialize_post_activity_player, it will raise KeyError because of the unexpected fields lost.
Here are two sample instance id, you can test it by yourself :)
[Fields Missing]<PHONE_NUMBER>1
[Standard]<PHONE_NUMBER>

I tested this locally and yeh it looks like its Bungie's fault for not returning those fields. No idea why. This can be fixed at our end however your PR is still missing stuff,

The PostActivityPlayer object's typehints needs to be changed to typing.Optional[T] since they're either an int or None. This is only for the fields that raised the KeyError.
Using .get might lead to some funky behavior, You should assign a default and use the walrus operator making an if check, Example, For fields that raised KeyError.

An example
player = payload["player"]
class_hash: typing.Optional[int] = None
if (class_hash := player.get("classHash")) is not None:
    class_hash = class_hash # This is None by default

# Same thing for all 5 fields
return Object(class_hash=class_hash)


Please support invidio.us

https://github.com/omarroth/invidious
$ yt https://invidio.us/channel/UC23qupoDRn9YOAVzeoxjOQA
[generic] UC23qupoDRn9YOAVzeoxjOQA: Requesting header
WARNING: Falling back on generic information extractor.
[generic] UC23qupoDRn9YOAVzeoxjOQA: Downloading webpage
[generic] UC23qupoDRn9YOAVzeoxjOQA: Extracting information
ERROR: Unsupported URL: https://invidio.us/channel/UC23qupoDRn9YOAVzeoxjOQ

@dstftw
Single video works but playlist doesn't work?
Test: https://invidio.us/playlist?list=PLDIoUOhQQPlXr63I_vwF9GD8sAKh77dWU


Altcoin Tests CanSignTransactions() problem with Stratis Altcoin wallet confirmations

This may be a behaviour difference between Bitcoin and Stratis wallets.
When the P2SH coin spend is attempted:
// Let's try P2SH with 2 coins
aliceAddress = alice.PubKey.ScriptPubKey.GetScriptAddress(builder.Network);
txid = rpc.SendToAddress(aliceAddress, Money.Coins(1.0m));

The Stratis node throws and exception saying "No spendable transactions found."
It looks like this is because the previous transaction sent to the wallet doesn't have enough confirmations yet so the coins are still locked.
If I change the blocks generated to this:
node.Generate(builder.Network.Consensus.CoinbaseMaturity + 3);

Instead of coinbase+1 then I don't have problems with the confirmation as there are enough coins for the transactions.
If this is a valid change let me know and I can submit a PR.

Which mean that the CoinbaseMaturity setting of stratis in NBitcoin.Altcoins is incorrect

However, this is strange as there is another test testing the correctness of this setting. I think there is another explanation.

Ok, I will investigate further. I suspect maybe it is to do with the way the change for a transaction is being handled in Stratis vs Bitcoin. Will close this out until I can find some more concrete explanation.


open file from windows explorer with existing vs code instance open, doesn't open file!

VSCode Version: 1.11.1
OS Version: Windows 10.0.14393 Build 14393

Steps to Reproduce:

open vs code from powershell
double click or right click and 'open in vs code'

opening code from shortcut, or from cmd console is ok

this seems to be an issue using conemu around powershell. closing :)


Add VectorGrid support

Fixes #107

Tested locally,  getting good vector tiles. Thanks!

FWIW setting "interactive": True on the VectorGridProtobuf layer causes errors due to an upstream bug https://github.com/Leaflet/Leaflet.VectorGrid/issues/273 Not sure there is anything to do about it.

This has a merge conflict now @blackary

@kotfic If the bug is on the underlying Leaflet library, it will likely be difficult to fix in this package (not necessarily impossible, but much more difficult, since this is two layers above leaflet -- leaflet -> folium -> streamlit-folium). Do you have an example streamlit script which shows the issue?

@blackary yeah sorry not suggesting any action item on this PR.  There are 4 or 5 issues on Leaflet.VectorGrid describing the problem,  seeing as the last real commit on that library was July 2019 I don't have much hope of it being fixed.  Its just a bummer that it has the function (so the duck typing passes)  but the function is broken. It is equally a bummer that leaflet/javascript doesn't support determining the layer type so onLayerClick can't easily skip implement a code path that avoids the exception.


feat: pkg-util-kubectl Enhancement

Signed-off-by: xian-jie.shen<EMAIL_ADDRESS>Pre-Checklist
Note: please complete ALL items in the following checklist.

[x] I have read through the CONTRIBUTING.md documentation.
[x] My code has the necessary comments and documentation (if needed).
[x] I have added relevant tests

Description

try to add ut for the package

Related Issues
see https://github.com/devstream-io/devstream/issues/833 for the details
New Behavior (screenshots if needed)
[going@dev kubectl]$ go test -race -cover  -coverprofile=./coverage.out -timeout=10m -short -v
=== RUN   TestKubeCreate
=== RUN   TestKubeCreate/base
2022-07-18 09:04:50 !! [ERROR]  Failed to exec: < /usr/local/bin/kubectl create -f nginx-deployment.yaml >.
2022-07-18 09:04:50 !! [ERROR]  Exec logs: < The connection to the server localhost:8080 was refused - did you specify the right host or port?
 >. Got error: exit status 1.
--- PASS: TestKubeCreate (0.07s)
    --- PASS: TestKubeCreate/base (0.07s)
=== RUN   TestKubeApply
=== RUN   TestKubeApply/base
2022-07-18 09:04:50 !! [ERROR]  Failed to exec: < /usr/local/bin/kubectl apply -f nginx-deployment.yaml >.
2022-07-18 09:04:50 !! [ERROR]  Exec logs: < The connection to the server localhost:8080 was refused - did you specify the right host or port?
 >. Got error: exit status 1.
--- PASS: TestKubeApply (0.06s)
    --- PASS: TestKubeApply/base (0.06s)
=== RUN   TestKubeDelete
=== RUN   TestKubeDelete/base
2022-07-18 09:04:50 !! [ERROR]  Failed to exec: < /usr/local/bin/kubectl delete -f nginx-deployment.yaml >.
2022-07-18 09:04:50 !! [ERROR]  Exec logs: < error: the path "nginx-deployment.yaml" does not exist
 >. Got error: exit status 1.
--- PASS: TestKubeDelete (0.05s)
    --- PASS: TestKubeDelete/base (0.05s)
=== RUN   Test_kubectlAction
=== RUN   Test_kubectlAction/base
2022-07-18 09:04:50 !! [ERROR]  Failed to exec: < /usr/local/bin/kubectl apply -f nginx-deployment.yaml >.
2022-07-18 09:04:50 !! [ERROR]  Exec logs: < The connection to the server localhost:8080 was refused - did you specify the right host or port?
 >. Got error: exit status 1.
--- PASS: Test_kubectlAction (0.06s)
    --- PASS: Test_kubectlAction/base (0.06s)
PASS
coverage: 81.8% of statements

@hxcGit @steinliber my local environment doesn't have kubectl cmd so that the test case can't cover the left part, how do you think about this circumstance, should we try to cover the left part or leave it to something like the e2e test?

@jxs1211  I think we can leave it alone. First, the function is very simple with no complicated and error-prone operations. Besides, it's hard to handle with unit test.


Progress bar

A progress bar for process_api_requests would be nicer to look at than log messages that spam the console.

Closing because package is in maintenance mode


@-mentions count not positioned correctly

The count for the total @-mentions displayed in the Top Bar of the homepage is not positioned correctly.
The screenshot below is taken from a pixel 4 emulator. The positioning was also there when tested on oneplus 7T.
Screenshots:

Thanks for the report @SilentCruzer !
Related discussion: https://chat.zulip.org/#narrow/stream/243-mobile-team/topic/unread.20count.20is.20not.20visible.20properly.20in.20overlay.2E

Hey @alya, If it's a bug then Can I give it a try to this one. Thanks 😊

Sure @alya I'll let you know my plan. Thanks 😊

Hey @alya!, So My plan is like this:

We are using a component name CountOverlay and CountOverlay is using ComponentWithOverlay which has a prop of overlaySize. So we just need to change the size of the overlay.

Previously, overlaySize was 15 and I made it to 19 and It looks pretty good to me. But, if you know any perfect size for this icon then we can also do this. But I don't think it is necessary, because it looks very reasonable.  I've done the work.

Hoping for your feedback. Then, I'll raise the PR. Thanks 😊
Here is the Preview, Please have a look and let me know your feedback,
Previously
https://github.com/zulip/zulip-mobile/issues/5278#issue-1162010470
Now

Thanks, @thisisnitish! Could you please post a screenshot that shows this badge and the unread-PMs badge (in the bottom tab bar) at the same time? You can make the unread-PMs badge appear by having an unread PM, or by changing this line in src/main/MainTabsScreen.js, e.g., like so:
-            tabBarBadge: unreadPmsCount > 0 ? unreadPmsCount : undefined,
+            tabBarBadge: 1,

I think ideally we'd want the badges to match closely, so it's helpful to see them both at the same time.

Yeah, Sure @chrisbobbe!, Even I checked this scenario and it was perfectly working. I'll post the screenshot. Thanks 😊

Thanks, @chrisbobbe!, I've posted a Screenshot for both the badges at the same time. Please have a look. Thanks 😊

Thanks, that's helpful! There's a pretty noticeable difference in the badge's position relative to the icon. Could you see if there's a nice way to fix that? Ideally we'd look at the implementation of the bottom-tabs badge in node_modules/@react-navigation/bottom-tabs and try to match that, I think.
From the inspector, it looks to me like the center of the badge lines up with the top-right corner of the person-icon's rectangle. (This could be wrong, it's just how it looks to me without reading code, playing around with system font size, etc. 🙂)

Sure @chrisbobbe!, I didn't realised that.
One question I've, The badge we are seeing on person icon in bottom tabs. I think, we need the same kind of visual on top at mention icon....right??

Hey @chrisbobbe!, I just revisited to the code of MainTabsScreen and HomeScreen and found that, we are using react-navigation for the Bottom Tabs in MainTabsScreen.

But, in HomeScreen we have our own component named TopTabButtonGeneral. It has wrapped the IconUnreadMentions component.

When I further dived into the code I found that It is using CountOverlay which is again using ComponentWithOverlay where we pass prop overlayPosition. When I checked the overlayPosition It was give option top-right, which I feel is correct.

The style we have given to the top-right prop is also I feel is correct.

This was the update so far I have. I'm still trying to fix the badge on mention icon at proper position. But, I'm looking for your suggestions also. Thanks 😊

Hey @chrisbobbe!, I made some changes in ComponentWithOverlay, actually I was playing with position of the badge.
Currently, we have given:
'top-right': {
    right: 0,
    top: 0,
  },

But, I changed to:
'top-right': {
    right: -10,
    top: 10,
  },

Below is the preview:

Thanks! It looks more similar now. I think that approach is probably fine for the time being, then we can look closer when we reorganize this part of the UI in #4851 (our latest thinking for that is here, I think).

Thanks @chrisbobbe!, I checked #4851 and our latest thinking. If you confirm, I'll raise the PR and we can use the above approach for the time being. Thanks 😊

Yep, sounds good, thank you!

Thanks @chrisbobbe!, I've raised the PR. Please have a look #5301. Thanks 😊


Publish GPG fingerprints and public keys used to verify TUF archives

Description of issue or feature request:
It would be nice to have an official record of the GPG public keys used to verify TUF archives.
Current behavior:
Right now, I'm able to find only one reference to this key.
Expected behavior:
Public keys should be published not only on GPG key servers, but also on other channels, such as the GitHub repository, web site, and Keybase. Full fingerprint should also be used in any reference.

The maintainers doc includes the full GPG fingerprints for each maintainer in the project.  The roadmap doc indicates the signer for the latest release.
I think the installation instructions in the README explain how release archives can be verified.  They also link to the maintainers and roadmap docs.

Nice, I missed this! Do you think it would be helpful to point to these in the installation instructions?

Oops, I missed this, thanks for the pointer!
Think it would be helpful to publish an official copy of these public keys?

Ahh, I should probably also link to the maintainer doc from the README.  Let me fix that.
How do you recommend we publish an official copy of the public keys?  On our lab's website?  Twitter account?
All of the public keys in maintainers.txt should be on GPG servers.

Certainly I think a copy should be on the GitHub repo as well as the lab web site. Python has a copy on its web site, for example.
But that doesn't help too much with trust and revocation. I would also recommend looking into Keybase, but that is entirely optional.

We now publish our PGP fingerprints on GitHub and the website.  All of our PGP keys can also be found on key servers.
I'll add our full public keys to the website soon.


Preverse error when return from failed precheck

Resolves #23

Waiting on @nhooyr to agree with these changes before merging.

Commented in #23

@nhooyr this is ready


TypeError: argument of type 'function' is not iterable

Logger: aiohttp.server
Source: /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:448
First occurred: 21:53:34 (3 occurrences)
Last logged: 21:57:58

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 477, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 559, in _handle
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 77, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 85, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 222, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 74, in wrapper
    return await method(view, request, data, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 370, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 417, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 575, in _async_handle_step
    result = await self.async_finish_flow(flow, result.copy())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1484, in async_finish_flow
    await self.config_entries.async_add(entry)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1821, in async_add
    await self.async_setup(entry.entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1937, in async_setup
    result = await async_setup_component(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/setup.py", line 159, in async_setup_component
    return await existing_setup_future
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'function' is not iterable

Seems like that error happens only when trying again. When deleting all entries and restarting HA, this is the first error I get trying to set it up:
Logger: aiohttp.server
Source: /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:448
First occurred: 22:11:07 (1 occurrences)
Last logged: 22:11:07

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 477, in _handle_request
    resp = await request_handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_app.py", line 559, in _handle
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 77, in forwarded_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 85, in ban_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware
    return await handler(request)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin
    return await func(self, request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 222, in post
    return await super().post(request, flow_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 74, in wrapper
    return await method(view, request, data, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 122, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 370, in async_configure
    result = await self._async_configure(flow_id, user_input)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 417, in _async_configure
    result = await self._async_handle_step(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 575, in _async_handle_step
    result = await self.async_finish_flow(flow, result.copy())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1484, in async_finish_flow
    await self.config_entries.async_add(entry)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1821, in async_add
    await self.async_setup(entry.entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1937, in async_setup
    result = await async_setup_component(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/setup.py", line 185, in async_setup_component
    await future
  File "/usr/src/homeassistant/homeassistant/setup.py", line 165, in async_setup_component
    result = await _async_setup_component(hass, domain, config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/setup.py", line 343, in _async_setup_component
    processed_config = conf_util.async_drop_config_annotations(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/config.py", line 1268, in async_drop_config_annotations
    if integration.domain in config and integration.domain != HOMEASSISTANT_DOMAIN:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'function' is not iterable

Thanks for the update - I'm on it

Code Updated to V1.02 to resolve the following:
Problem Summary
The integration setup was failing during the OpenAI API key entry step in the UI, with the error message indicating an unknown issue or connection failure. The failure was traced to changes in the config_flow.py and init.py files that were introducing additional validation and configuration requirements.
Solution Breakdown
The following steps resolved the issue:
Revert config_flow.py to the Old Version:
The new version of config_flow.py contained more rigorous validation for the OpenAI API key, including a direct validation call (validate_openai_api_key) which caused errors when setting up the integration.
To resolve this, the entire config_flow.py was reverted to the old version, which simply checked if the OpenAI API key was provided without additional validation during setup.
Key Changes in config_flow.py:
Removed additional API key validation logic that attempted to contact OpenAI's servers during the setup process.
This removed the need for handling API errors during the initial configuration flow, which simplified the setup process and prevented connection issues from blocking the integration.
Remove CONFIG_SCHEMA from init.py:
The new version of init.py contained the line: CONFIG_SCHEMA = cv.config_entry_only_config_schema. This introduced stricter schema validation requirements, which caused issues during integration setup.
Removing this line allowed the integration to avoid unnecessary validation checks, making it more flexible during the configuration phase.
Update coordinator.py API Call:
In the new version, the API call to OpenAI was using response = openai.chat.completions.create(...), which caused an error.
Reverting this line to the old version (response = openai.Engine.list()) resolved the issue and allowed the integration to connect to the OpenAI API successfully.
Result
After applying the above changes, the integration was able to successfully complete the setup without errors. The removal of the additional validation logic and configuration schema requirements simplified the setup process, ensuring compatibility with the current Home Assistant setup.
Steps to Apply the Fix
Replace config_flow.py:
Use the old working version of config_flow.py that does not include extra API key validation.
Edit init.py:
Remove the line: CONFIG_SCHEMA = cv.config_entry_only_config_schema.
Edit coordinator.py:
Change the line of code responsible for creating the OpenAI API request back to the old version: response = openai.Engine.list().
These changes should fix the 'unknown error' during setup and allow the integration to function as expected.


Setting up the Fitbit component doesn't work

Home Assistant release (hass --version):
0.53.0
Python release (python3 --version):
3.5.2
Component/platform:
fitbit
Description of problem:
I'm getting the following exception after authorizing Home Assistant using the /auth/fitbit URL:
Traceback (most recent call last):
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/ha/.local/lib/python3.5/site-packages/homeassistant/components/sensor/fitbit.py", line 272, in setup_platform
    if int(time.time()) - expires_at > 3600:
TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'

My fitbit.conf file does not contain a last_saved_at key. The Fitbit component does not appear to be functional after this.
Expected:
No exception.

I had to manually add last_saved_at to my fitbit.conf as a workaround.

I have the same trouble here https://community.home-assistant.io/t/fitbit-authentication/21158/9?u=bertrumuk
Added last_saved_at after my refresh token and before the access token and that sorted it
, "last_saved_at":<PHONE_NUMBER>, "access_token":
Thanks @gunnarbeutner for providing this quick fix

It was working fine but has stopped working again. @gunnarbeutner is yours still working ?

Yeah, the Fitbit component seems to be working just fine here.

How strange. Can you share your last saved at number please ?

{... "last_saved_at":<PHONE_NUMBER>}

That is for today's date. Are you changing it everyday for it to work?
Where are you putting it in the conf?

Nah, I only had to add it once to finish the setup process in the frontend ("last_saved_at": 0 in ~/.homeassistant/fitbit.conf).

I'm going to start again with a new key when I get home from work. There is a link on the Fitbit API page that can authorise your key with HA. Going to try my luck with that first before getting a new key.
Issue still there though so this needs to stay open.


Azure migrate - Add Discovery SOurce: Linux root user has issues "Failed to fetch BIOS GUID for the server 'XXXXXXXXX'. BIOS GUID is important to uniquely identify and discover a server."

Tried the "https://docs.microsoft.com/en-us/azure/migrate/troubleshoot-appliance"
cat /sys/class/dmi/id/product_uuid  ----------works
dmidecode | grep -i uuid | awk '{print $2}' --------dmidecode command not recognized when issued as a root user.
Root being a super user, not sure why I see unable to fetch BIOS GUID error. Definitely it is not a permission issue.  So what else is the issue here.  Please guide.
Below is error message from Azure Migrate Registration Validation failed.
Source validation failed
Details
Failed to fetch BIOS GUID for the server 'XXXXXXXXX'. BIOS GUID is important to uniquely identify and discover a server.
Recommendation
Ensure that the user account provided on the appliance have the required permissions as documented here.
Connect to the impacted server from the appliance and run the commands documented here to check if you can fetch BIOS GUID for the server.
If the issue persists, submit a Microsoft support case, providing the appliance machine ID (available in the footer of the appliance configuration manager).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a39b2848-3db5-7fa8-b361-d0c20afec7ea
Version Independent ID: ee5a0d4d-3163-12a4-5dba-bb6a4725272d
Content: Troubleshoot the Azure Migrate appliance - Azure Migrate
Content Source: articles/migrate/troubleshoot-appliance.md
Service: azure-migrate
GitHub Login: @Vikram1988
Microsoft Alias: vibansa

@Asha-Hiremath Thank you for reaching out to us. Please note that this channel (the Azure Docs repository) is only for driving improvements related to the Azure documentation. Your query needs deeper investigation. I recommend you to create a technical support ticket. The ticket enables you to work closely with the support engineers and get a quick resolution to your issue.
Here is the link - https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request to create support case.


Use a Trigger Mid Flow?

Article states triggers can be used mid-flow to delay actions. I've been unable to utilize this functionality and can't find it mentioned anywhere else. Is this a new/limited feature or is it an error?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f2533fa7-a9c2-5203-9d33-a04288246d0c
Version Independent ID: 9465133e-bafd-b35d-c228-6587970da6ee
Content: Use wait conditions to delay flows - Power Automate
Content Source: articles/dataverse/wait-conditions.md
Service: power-automate
GitHub Login: @MSFTMan
Microsoft Alias: Deonhe

@EmEnglish Thanks for report this. A trigger can only be used at the start of a flow. I've fixed the typo in the sentence. You'll see the update in a few hours. Thanks!

Thank you!


Webservice instead of hard coded url

Hello,
I would like to use the fetchupdate method, but instead of pointing to a content_url i would like to use a webservice that returns the json. Is this possible?
Kind regards,
Mitch

What happens if you call chcp.fetchUpdate({'config-file': 'https://my-rest-api.com/chcp'}) ? the url should point directly to the chcp.json file, I don't think it matters whether is a static hosting or like a REST api


Unsubscribe to state when not active route?

Hi!
I changed the formulation of the problem.
When navigating to a new page, the previous page in the router stack still subscribes to the state which results in unnecessary rebuilds which affect performance.
Is there a way to only subscribe to the state if it is the active one in the router stack?

It just hit me that I should break out that part so that it is not part of the tree...

@martinory can't you just use a Route Observer?
https://master-api.flutter.dev/flutter/widgets/RouteObserver-class.html


Support CMake 2.8 instead of CMake 3.0

RHEL 7 only supports CMake 2.8, so ensure tcmu-runner can
be built within a RHEL 7 build environment.
Signed-off-by: Jason Dillaman<EMAIL_ADDRESS>

thanks! 👍


Issue/140: Using RFC2136 dns challenge does not allow me to create multi domain certs.

Removal of RFC2136_ZONE from help text
Query nameserver directly to find zone we have to update
During insert, make sure the new record is the ONLY challenge.
(I had a few panics, hence 3 challenges left. Not good.)

Looking good. Just two nits with the DNS message headers and the unit test needs to be updated with the changes, too.

The findZone() function would probably be suited as shared between all the DNS providers. It is pretty much what I was aiming for in #142. You could go one step further and adjust lookupNameservers() to use query the NS of the result of findZone().
CloudFlare would certainly benefit from being able to directly lookup the zone. Rackspace also supports that in their API (which I was hoping to implement after the restructure).

@LukeHandle Interesting, I would have thought that finding the zone would be more useful as a new function on the dns challengers. So they can each implement in a way they see fit? But glancing over the current implementations, they all iterate over the list of zones they manage, to pick out the right one. Offering the zone to them in Present() might indeed improve life for them.

I don't believe this single query SOA answer approach is working out.
Reason: invalid domains mess things up. We were trying to use the SOA of the authoritative zone in the AUTHORITY section of the reply. This works fine for valid, existing domains. For invalid domains, they return "nl." or "com." or even just "." as /that/ NS has authoritative knowledge over the non-existence. Sadly however it is NOT the NS we want to talk to... But I have no way of discerning both type of answers. Using the number of dots > 1 is no good, invalid-domain.co.uk. would slip through.
So lookupZone("invalid-domain.co.uk") > "co.uk"; lookupNss("co.uk") = ["nsa.co.uk"...] and this is where we send the dyn-updates to and query to check if the propagation has succeeded... Not getting  enthusiastic :)
Am I missing anything?
PS. The current implementation of lookupNameservers also misses the boat for .co.uk domains.

Ahum. Turns out there is a thing called https://godoc.org/golang.org/x/net/publicsuffix. I'm using that the verify the NXDOMAIN SOA record is not on that list.

@middelink Each of them would benefit from this as at some point or another they have to establish which zone to use. With this method we can say without any doubt which zone the record should be added to. Iterating through the returned list is inefficient when you have hundreds of hosted domains and DNS should always give the actual answer.
CloudFlare and Rackspace APIs also allow you to specify the zone name in the lookup simplifying things there.
If we use the SOA method then I would agree a new function would make sense in dns_providers. The lookupNameservers() function can call the lookup zone and perform an NS query on the zone vs. iterating through each level. Supplying it with the Present would also make sense considering they would all benefit from it.

@LukeHandle I agree with you fully :) The approach I'm taking now (having found a solution to the invalid domains; see above) is the one of least resistance and have findZone() cache, so if dns providers call it again, it does not have to go out and do the dns thing again.
The other, slightly more intrusive approach would be to get the zone in Solve() and plumb it through to all the providers. then no caching would be needed (making testing a tad easier too)
As I have no strong feelings about one over the other, feel free to +1  the later ^^

For invalid domains, they return "nl." or "com." or even just "." as /that/ NS has authoritative knowledge over the non-existence. Sadly however it is NOT the NS we want to talk to...

@middelink
Thats a good example for why setting the recursion desired flag when talking to an authoritative NS is a bad idea (see my comment above). Without the recursion flag the SOA method in any case either returns the appropriate NS or nothing at all.
Valid domain

dig +norecurse www.rancher.com SOA @emma.ns.cloudflare.com

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39531
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;www.rancher.com.		IN	SOA
;; AUTHORITY SECTION:
rancher.com.		86400	IN	SOA	brad.ns.cloudflare.com. dns.cloudflare.com.

=> lookupZoneByFqdn() would return the correct zone name
Invalid subdomain

dig +norecurse nonexistant.subdomain.rancher.com SOA @emma.ns.cloudflare.com

;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 26264
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
;nonexistant.subdomain.rancher.com. IN	SOA
;; AUTHORITY SECTION:
rancher.com.		86400	IN	SOA	brad.ns.cloudflare.com. dns.cloudflare.com.

=> lookupZoneByFqdn() would return the correct zone name
Invalid root domain

dig +norecurse nowaythatdomainexists.com SOA @emma.ns.cloudflare.com

;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 3982
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;nowaythatdomainexists.com.	IN	SOA

=> lookupZoneByFqdn() would return an error

@middelink Personally, :+1: the latter :stuck_out_tongue:
I think that each provider benefits from it and it reduces duplicated code as they each try and find out which zone to use.

@janeczku True, it won't help if we do not know the authoritative nameservers. So what would only work for the rfc2136 provider (even assuming all the domains I want to request a cert for are under the same RFC2136_NAMESERVER btw). And of course it would break 100% reliable in split view nameserver setups.
In all other cases we still do the chopping off labels thing (lookupNameservers) which than still ends up misdiagnosing invalid domains.

@LukeHandle I decided to not go for the plumbing in this PR, it is already touching too much code and changing the interface on Present/Cleanup does not seem like a hot idea right now.

For invalid domains, they return "nl." or "com." or even just "." as /that/ NS has authoritative knowledge over the non-existence. Sadly however it is NOT the NS we want to talk to...

To which nameserver do you want to talk regarding a non-existent domain??? The authority for nonexistandomain.nl is the nl. TLD nameserver.

PS. The current implementation of lookupNameservers also misses the boat for .co.uk domains.

I can't reproduce any of the behavior you were describing in relation to the current implementation of lookupNameservers() (to check propagation). Were you referring to the ability of the logic used in lookupNameservers to lookup the zone of the domain?

I still think that this PR is on the right track.
Here is the RFC recommendation on how Dynamic DNS update clients should discover the zone name:
https://tools.ietf.org/html/rfc4697#section-2.8.1
Dynamic update agents SHOULD send SOA or NS queries to progressively
   higher-level names to find the closest enclosing zone for a given
   name to update.

To which nameserver do you want to talk regarding a non-existent domain??? The authority for nonexistandomain.nl is the nl. TLD nameserver.

Sure. badly phrased sentence. My bad. The point was we don't have a authoritative nameserver when we do Solve(). lookupNameservers() tries to find them by doing a recursive NS search on the recursive nameserver, and cutting off the labels till it gets an answer. It only stops when it has less than two labels left, so it stops at xxx.TLD. Therefore "in-valid.co.uk" will result in the "co.uk" nameservers as it will cut away the "in-valid" part and recurses since dns.LabelCount("co.uk.") >= 2.
So while I don't object to the code, though it will result in potentially multiple DNS queries, it cutoff methodology is too simple. Hence my use of publicsuffix.

Here is the RFC recommendation on how Dynamic DNS update clients should discover the zone name: https://tools.ietf.org/html/rfc4697#section-2.8.1

Funny how the RFC is about "Observed DNS Resolution Misbehavior" :)
So what are you saying, should we not use the SOA name in the authoritative section to directly jump to the zone we look for? It seems to pass the dns_challenge_test.go fine.

Funny how the RFC is about "Observed DNS Resolution Misbehavior" :)

It seems we are not the only ones who get their mind blown by the pitfalls and intricacies of dealing with DNS. :laughing:

So what are you saying, should we not use the SOA name in the authoritative section to directly jump to the zone we look for?

Good question. Could the RFC statement relate to the cases where the authoritative NS is not known in advance and you have to do a iterative query down from the root domain?
I think with this requirement (https://tools.ietf.org/html/rfc2308):
Name servers authoritative for a zone MUST include the SOA record of
the zone in the authority section of the response when reporting an
NXDOMAIN or indicating that no data (NODATA) of the requested type exists
we can expect to always receive the SOA in the authority section on the first query, right? So my answer would be that we should use the SOA record in the first answer and we don't need to chop of names.

Well, I guess my PR stands than ^^
BTW, if this is in, I will do some work on more hermetic testing in dns_challenge_test, up and including adding in-valid.co.uk :)


How to custom format a time range

Have I missed something in the docs to custom format a range of time?
Date.range('00:00', '04:30').seconds() this is fine but I want to understand it in hours and mintues, or even in days, hours, minutes and seconds!
Date.range('00:00', '04:30').format('%Hh %Mm') produces a Date.range(...).format is not a function error.
so how do I present a range of time in a custom format using Sugar, thanks.

Hi, sorry for the delay... I'm showing Date.range('00:00', '04:30').hours() as working fine... a format method on a Date range isn't a thing now but it certainly could be! I'll mark this as an idea for a new feature.

no problem on the delay, thanks for responding.  Of course I could use just hours, just minutes or just seconds.  But in a real world application, for example a countdown timer would look silly displaying something like there is 24 hrs 600 min and 12654 seconds remaining  it doesn't work like that.  Can you imagine a marathon display board, you finished in 2hrs 934 min and 522345 seconds lol!!!  You'd expect it to be like there is 24hrs 15 min and 5 seconds remaining.  Maybe I'm missunderstanding something about how to use the library but I don't see any way to get the formatted output like that.  So currently I get the time remaining result in seconds and then use another function to calculate the resulting hours minutes and seconds.  Leaving out any zero results, like when we're down to less than 1 hour it puts out just the min and sec.

Gotcha... well in that regard all I can say is that the DateRange object itself is maybe not as mature as it potentially could be, and I'll definitely take this use case into consideration for the next iteration. For now you I'd say you can achieve what you want with some app-side logic and I'll see what I can do about taking this use case into consideration for an upcoming version... but to answer your immediate question yes for now you're going to have to roll this yourself...


[BUG] TEST

Description
(Description about the issue)
Use Case
(Use cases and benefits)
Screenshots
(Screenshots of the issue or reference)
Additional Context
(Any other context about the problem)

/label add TEST


refactor(ast): put assert_layouts.rs behind debug_assertions

#4615 was showing as green on all checks, but failed CI on main once merged because of errors in "Test wasm32-wasip1-threads" job, which only runs on main. This change will make sure any such problem gets caught at the PR stage in future.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#4621  👈
#4620
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rzvxa and the rest of your teammates on  Graphite

@rzvxa Does this definitely solve the problem? Don't look like WASM is tested at all on PRs.

@rzvxa Does this definitely solve the problem? Don't look like WASM is tested at all on PRs.

We used to have one, It is changed in #4547.
What if we create a new task that would run wasm tests only if not running on "main"?

Maybe we should just remove/comment out the 32-bit layout assertions until we solve this? We don't really need them at present anyway. It's not ideal to run risk turning CI red by accident. Could make life difficult when trying to git bisect a bug later on.

Maybe we should just remove/comment out the 32-bit layout assertions until we solve this? We don't really need them at present anyway. It's not ideal to run risk of turning CI red by accident. Could make life difficult when trying to git bisect a bug later on.

Or we can remove the condition of check wasm-unknown-unknown, We should be fine if we run a cargo check on the wasm.
This used to be unconditional after we got rid of ci-optimize. so there shouldn't be an issue if we make it unconditional once again.
  test-wasm32-unknown-unknown:
    name: Check wasm32-unknown-unknown
    # if: ${{ github.ref_name == 'main' }}
    runs-on: ubuntu-latest
`

Or we can remove the condition of check wasm-unknown-unknown

Agreed. Let's merge this PR and do that in a follow-up.


Update README.md with info that Avahi daemon and Chrome on Linux should be stopped

Follow-up on #66

Probably, you need to change MulticastDNS=yes to MulticastDNS=no in /etc/systemd/resolved.conf if you're using systemd-resolved.

I came to this after running into issues with avahi-daemon.service and avahi-daemon.socket. I think printing a log::info statement in the std environment would be helpful as right now there's no level of logging that actually reveals the root issue (since StdIoError is kind of opaque).
Additionally, stopping avahi is not as trivial as sudo systemctl stop|disable <service> as it will retrigger itself. I had to do sudo systemctl mask <service> for each of them to completely stop them.

I came to this after running into issues with avahi-daemon.service and avahi-daemon.socket. I think printing a log::info statement in the std environment would be helpful as right now there's no level of logging that actually reveals the root issue (since StdIoError is kind of opaque).

Which is why we also have #67

Additionally, stopping avahi is not as trivial as sudo systemctl stop|disable <service> as it will retrigger itself. I had to do sudo systemctl mask <service> for each of them to completely stop them.

Yes, this can go into the readme.

@ivmarkov what are your thoughts on adding a zeroconf feature for hosted systems in order to support using mDNS with Avahi rather than adding directions on how to disable?

@ivmarkov what are your thoughts on adding a zeroconf feature for hosted systems in order to support using mDNS with Avahi rather than adding directions on how to disable?

If somebody (you?) want to spend time on making it work - by all means do. I was unable to find a Rust crate that can talk to Avahi. astro-dnssd (the one we use for MacOSX) claims to do so, but fails at runtime.
IMO, for real (embedded) use cases, this does not bring much value, as you want a tightly controlled, minimal Linux stack, possibly created with buildroot underneath your embedded software. I don't think Avahi has a place there.

Of course, in my case I'm looking at implementing a Matter bridge on my home server for a non-Matter set of devices so I'm not really resource constrained but I also don't want to prevent other processes from making use of mDNS at the same time.
The zeroconf library claims to support avahi and depends on avahi-sys (it also supports Bonjour, but I don't have a Mac system to test on).

Maybe I misunderstood this PR but as far as I can tell this didn't fix the issue for me.
Still panics without disabling the demon on Ubuntu 22.04:
[2023-10-06T20:04:44Z INFO  rs_matter::transport::core] Transport: waiting for incoming packets
Error: Error::StdIoError: Address already in use (os error 98) {
0: rs_matter::error::Error::new_with_details
at ./rs-matter/src/error.rs:111:24
1: <rs_matter::error::Error as core::convert::Fromstd::io::error::Error>::from
at ./rs-matter/src/error.rs:165:9
2: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Resultcore::convert::Infallible,E>>::from_residual
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/core/src/result.rs:1962:27
3: rs_matter::transport::udp::async_io::UdpListener::new::{{closure}}
at ./rs-matter/src/transport/udp.rs:58:40
4: rs_matter::mdns::builtin::MdnsService::run::{{closure}}
at ./rs-matter/src/mdns/builtin.rs:141:10
5: <core::pin::Pin as core::future::future::Future>::poll
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/core/src/future/future.rs:125:9
6: <&mut F as core::future::future::Future>::poll
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/core/src/future/future.rs:113:9
7: <embassy_futures::select::Select3<A,B,C> as core::future::future::Future>::poll
at /home/nikolas/.cargo/registry/src/index.crates.io-6f17d22bba15001f/embassy-futures-0.1.0/src/select.rs:111:33
8: async_io::driver::block_on
at /home/nikolas/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-io-1.12.0/src/driver.rs:146:33
9: dimmable_light::run
at ./rs-matter/../examples/dimmable_light/src/main.rs:165:5
10: core::ops::function::FnOnce::call_once
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/core/src/ops/function.rs:250:5
11: std::sys_common::backtrace::rust_begin_short_backtrace
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/sys_common/backtrace.rs:154:18
12: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/thread/mod.rs:529:17
13: <core::panic::unwind_safe::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/core/src/panic/unwind_safe.rs:271:9
14: std::panicking::try::do_call
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/panicking.rs:526:40
15: std::panicking::try
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/panicking.rs:490:19
16: std::panic::catch_unwind
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/panic.rs:142:14
17: std::thread::Builder::spawn_unchecked::{{closure}}
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/thread/mod.rs:528:30
18: core::ops::function::FnOnce::call_once{{vtable.shim}}
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/core/src/ops/function.rs:250:5
19: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/alloc/src/boxed.rs:2007:9
20: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/alloc/src/boxed.rs:2007:9
21: std::sys::unix::thread::Thread::new::thread_start
at /rustc/9f5fc1bd443f59583e7af0d94d289f95fe1e20c4/library/std/src/sys/unix/thread.rs:108:17
22: start_thread
at ./nptl/pthread_create.c:442:8
23: clone3
at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
}
Steps to reproduce:
Boot Ubuntu 22.04
Open Browser to clone rs-matter repository
Build and run OnOff_Light example

I would recommend copying the example code into an application and then initializing with the zeroconf feature.
Sidenote: @kedars what do you think about splitting the examples on a per platform basis? There's a lot of extra noise in the example in making the same source file support esp32 and Linux.

I just on the master commit 3d8fc59596c0413a7ded32bb401a3858b34a14e1, also met this issue.   On arch linux.

I just on the master commit 3d8fc59, also met this issue. On arch linux.

This issue is no longer relevant because the current main branch now does support Avahi. You just need to compile with cargo build --features async-io,zeroconf --examples

This issue is no longer relevant because the current main branch now does support Avahi. You just need to compile with cargo build --features async-io,zeroconf --example

On mine node,
sudo lsof -i -P -n | grep 5353
systemd-r     457 systemd-resolve  15u  IPv4     18562      0t0  UDP *:5353
systemd-r     457 systemd-resolve  16u  IPv6     18563      0t0  UDP *:5353

cargo build --features async-io,zeroconf --examples
And the error log is that:
2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key D val 250
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key CM val 1
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key DN val OnOff Light
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key VP val 65521+32768
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key SII val 5000
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key SAI val 300
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key PH val 33
[2024-03-21T08:28:35Z INFO  rs_matter::mdns::builtin] mDNS TXT key PI val
[2024-03-21T08:28:35Z DEBUG zeroconf::linux::service] Registering service: AvahiMdnsService { client: None, poll: None, context: AvahiServiceContext { name: Some("135098307ADC8921"), kind: "_matterc._udp", port: 5540, group: None } }
[2024-03-21T08:28:35Z ERROR rs_matter::mdns::builtin] Encountered error registering mDNS service: could not initialize AvahiClient

Should i also need disable it with MulticastDNS=no?

On mine node,
sudo lsof -i -P -n | grep 5353
systemd-r     457 systemd-resolve  15u  IPv4     18562      0t0  UDP *:5353
systemd-r     457 systemd-resolve  16u  IPv6     18563      0t0  UDP *:5353

What you have running on your node is not Avahi, but systemd-resolve - which - to my understanding - is a "very minimal" mDNS-like client which we cannot / do not yet integrate with.
So it is normal that the app outputs:

[2024-03-21T08:28:35Z DEBUG zeroconf::linux::service] Registering service: AvahiMdnsService { client: None, poll: None, context: AvahiServiceContext { name: Some("135098307ADC8921"), kind: "_matterc._udp", port: 5540, group: None } }
[2024-03-21T08:28:35Z ERROR rs_matter::mdns::builtin] Encountered error registering mDNS service: could not initialize AvahiClient

... because what you have is not Avahi in the first place.
I think what you should do is
(a) stop this systemd-resolve
(b) Build and run the rs-matter stack without Avahi/zeroconf support, as you started originally. I.e. cargo build --features async-io --examples

Should i also need disable it with MulticastDNS=no? look like We should also zeroconf for mdns in systemd-resolved. Because systemd-resoloved and avahi conflict on the mdns.
Now I just disable mdns in systemd-resolved, and build without zeroconf, now example run ok.

Exactly. Until (if ever) we also have an integration with systemd-resolved, you need to disable it, and then run rs-matter with its own built-in mDNS responder.


[Bug]: resolve math with triple multi-value + references is not working

What happened?
issue when three things are combined:

math expression
multi-value (with spaces to separate them)
token references

{
  "scale": {
    "value": 4,
    "type": "spacing"
  },
  "2xs": {
    "value": "{scale}*1.5",
    "type": "spacing"
  },
  "foo": {
    "value": "{2xs} {2xs} {2xs}",
    "type": "spacing"
  }
}

the output of foo, the math is not resolved in the 2nd and 3rd value, we get 6px 4px*1.5 4px*1.5
Reproduction
repro
Expected output
we expect 6px 6px 6px
Version
0.9.10

Thank you so much @jorenbroekema !

You're welcome, this one was quite a pain to figure out haha

@jorenbroekema I've upgraded to version 0.10.3 but it still seems to have the same issue for me? Do I need to provide an extra transform?

If you click the reproduction link I added in the first comment in this issue, it shows to be fixed. If it's still broken for you in some way, can you post the link to a reproduction on your end e.g. in the configurator?

The configurator doesn't handle multiple files very well, which may be the issue. I've tried to replicate here but didn't quite work: https://configurator.tokens.studio/#project=hVRdb5swFP0rV2hSPkSgrbaXZJ0mrer6OK3TXsa0OmCCO7CRbbpGiP++6w9IIGmbJ3w/zj33+DhtcHG5UrrJmIi0+Eu5ih6V4ME6aBMOkAQqJSVNgjXYs4k8kbKxkfdhH9L72kawvCYp47skMKku9Bh98Ajl6lkdnSfISdDaud3yMvqQBOFx0QuzcJodmfAuCIOLq5WiFeGapercXudZvcTsDDvXG2FtB6+cRtzf4D/sMGyRCp6z3fRCRCNTi/Cr32S5jJfHWzrE3179uiQ6F7JSI/1TNd4SiUnClSn8KkVTO4q2KjzU1JLm7NnTz0apbcPK7BvRhcvaY4z98agqZyVVR+TNb6R0EmRUacaJZmYRA/XniUhGttgYjfl4SKRM9MA3HqrPCIuqnJpF0h1Tmsr7m9uGp2aukREVZ1UtpIZ7vS/pDbMZIveQS1HBTJnoKhvCs03CfUMLPeSPXlMFne/77C7KP7pYZatBeGUwEh7HMIqGcMUzqIkkFUVMwCCI2vKElHDYUiBZRjPbeU8pZCJVayi0rtU6jndMF802SkUVvzI64aec55PNFz09+kyquqRgnwQMICH8K1hawGOjNEqgG4kMdYEVZi4QBQznTECjk7lzawjzTtaHZ2cvnaMCJlbtv+AIUQ0tPo8+SAsq1zC3Exdw/Qla9y80kJymnR8cW0d0A7jix9Xq0GOoc0qtxs42C6+EMx8UtKypVO6Kp85IOD5kZWxh3H9HSUZl6Ds1zX72fkWLXE/NFrmyO4c/1R/t7nFelvXW5p2mTr9ZL59Lzaw+PR1UJ/evYN4edggt97D3XbfohetXE42uG/2d5lRSnvplfPVmpPHDu/agw9x+d4tuLYXQBhSzJ8rMfQy548qzEI6JTScjGF7Qw2a4KvwKuv8=
I have however sent my build.js on Slack which may help. You already have all the tokens from our previous discussion.

The configurator doesn't handle multiple files very well, which may be the issue. I've tried to replicate here but didn't quite work. It won't let me paste the link on GitHub.
I have however sent my build.js on Slack which may help. You already have all the tokens from our previous discussion.

Had to fix your link a bit https://configurator.tokens.studio/#project=hVRdb5swFP0rV2hSPkSgrbYXsk6TVnV9nNZpL2NaHTDBHdjINl0ixH/ftQ0ESD/yhO/HueceH6fxLi43StcpE4EWfylXwaMS3Iu8JuYAsacSUtDYi8CeTeSJFLWNvPf7kD5WNoLlFUkY38eeSbV+h9EHRyhXBzU6z5Bjr7Fz2/Vl8KE6xJ4/LnthGs6zQ2Peer53cbVRtCRcs0Q9t5kByoQ443DO6xlubmyAtS28cprwfoP7wH/YIBE8Y/v5dYhaJhbhV7/Feh2uxxs6xN+d9lVBdCZkqSbqJ2q6JRKThCtT+FWKunIUHWhnj9EyiCppxg7dIukktatZkX4jOndZewxxXjipylhB1WgN85toHnspVZpxoplZyUD9eSKSkR02Bpa/P6035Il2pWbeUP2MxKjPuWUk3TOlqby/ua15YuYaQVF7VlZCarjXx4LeMJsh8giZFCUslIlu0iG82Ma8a2igh/zRq6ug7fo+T9QNVboZrkAZjJiHIUyiPlzxFCoiSUkREzAIorI8ISEcdhRImtLUdt5TCqlIVAS51pWKwnDPdF7vgkSU4SujY37OeTnbfNXTowdSVgUF+zhgAPHhX86SHB5rpVECXUtkqHOsMHOBKGA4ZwYanM1dWkOYFxOdHqC9dI4KmFh5/IIjRDm0dHn0QZJTGcHSTlzB9Sdo3L/RQHKedn5wbB3RLeCKHzebU4+hzim1GjvbrDolnPkgp0VFpXJXPHdGzPFJK2ML4/47SlIq/a5T0/Rn71e0yPXcbIEru3P4c/3R7h3Oy7Le2rzT1Om36OVzqYXVp6eD6mTdK1g2px18y93vfdeueuH61UStq1p/pxmVlCfdMl31dqLxw7vmpMPSfrerNpJCaAOK2TNlll0MuePKCx/GxOaTEQwv6GE7XBV+ee1/

Hmm I wonder if the problem is that both objects are called spacing. When I tried to change it in your new link, that failed. Obviously it's the output from Tokens Studio so I am unable to change it on the input. Could it be the parser is struggling with both objects called the same thing?

Yes, in your version the token has the exact same name as what it's referencing in the value, so then when you do this reference {spacing.2xs} inside spacing.2xs itself, it is creating an infinite loop of referencing itself.

Can you share the repository for me to troubleshoot it manually? Or perhaps create a reproduction repo for me to check?

Ah thanks I appreciate that @jorenbroekema here's a repo that I think will cover everything you need https://github.com/ShaneHudson/design-tokens-replicate-bug
It's  --token-semantic-spacing-inset-default and the similar groupings that is broken.

@jorenbroekema That's incredibly useful, thank you!

I am so sorry... I've just realised that while your recommendations are helpful and appreciated, the issue I was having was due to being on the previous version of Style Dictionary itself!
So for anyone else that may find this, make sure you're on 3.8.0 or higher (of Style Dictionary, not sd-transforms).

Actually, style-dictionary configurator is relying on a browser patch of style-dictionary that's based on 3.1.1 I think, from the top of my head. So 3.8.0 is not a requirement, pretty sure anything on the v3 major will work just fine.


stem anatomy and morphology traits-(syn: core)

Proposed new terms: core anatomy and morphology trait >
core size: size of the core measured in
core shape: shape of core measured in
core length: length of core measured in
these traits are for brassicas
Reported by: lauramooray
Original Ticket: obo/plant-trait-ontology-to-requests/156

Created a new child term: stem weight (TO:0000952): A stem anatomy and morphology trait (TO:0000361) which is associated with the weight of a stem (PO:0009047).
Added Xrefs to: weight (PATO:0000128); TO_GIT:157
Added narrow synonym: stalk weight

Created new child term: stem number (TO:0000953): A stem anatomy and morphology trait (TO:0000361) which is associated with the number of stems (PO:0009047).
Added Xrefs to: number (PATO:0001555); TO_GIT:157
Added narrow synonym: stalk weight
Created new child term: stem size (TO:0000954):A stem anatomy and morphology trait (TO:0000361) which is associated with the size of a stem (PO:0009047).
Added Xrefs to: size (PATO:0000117); TO_GIT:157
Added narrow synonym: stalk size

stem number (TO:0000953): Revised parent and def'n
Was: stem number (TO:0000953): A stem anatomy and morphology trait (TO:0000361) which is associated with the number of stems (PO:0009047).
Xrefs to: number (PATO:0001555); TO_GIT:157
narrow synonym: stalk weight
Revised:
stem number (TO:0000953): A shoot system anatomy and morphology trait (TO:0000077) which is associated with the number of stems (PO:0009047).

removed narrow synonym: stalk weight

Fixed duplicated stem size term which was added accidentally- deleted term stem size TO:0000954.  Moved synonym stalk size (related) to TO:0000892.


JsonFieldProcessor: support is*() getter for boolean fields

revised https://github.com/bluelinelabs/LoganSquare/pull/9

Great stuff, thanks. Have a few updates I'm planning to do over the next day or two, but I'll include this in the next release.

@EricKuck Sweet! Thank you for the cool library.


Exclude expense categories from trial balance graph?

I really love the idea of the graph at the top of the trial balance page, its a great visulisation. However, tax accounts for a significant chunk of my graph meaning everything else is squashed in the corner and harder to visualise. It'd be great if there was some way to either zoom in or entirely exclude certain expense categories (e.g. Expenses:Tax) so I can get better visualisation of things actually in my control.
Thoughts?

I had the same problem. A quick fix is to write a plugin to rename accounts so they get out of the Expense category:
https://github.com/redstreet/beancount_plugins_redstreet/tree/master/rename_accounts
If one can stretch principles a bit, taxes can be considered as reducing income, and thus a part of income. Of course, this is just a big hack, and it's even worse when applied to other accounts. But it's there if you want something now that works.
As you point out, a better way would be to simply tag expense accounts with metadata like fava-exclude-from-treemap. An even better way would be to be able to this dynamically and interactively.

@redstreet Thanks for the suggestion of moving tax to income, feels like I'm cheating myself a bit but it works well. I'm not sure I understand the point of the plugin, I just renamed my accounts with a search and replace, I can't think of some reasoning I'd want it done on the fly?

Taxes don't really belong in the Income category. The point of the plugin is to avoid making a permanent change, and to instead give you the option of easily and temporarily moving taxes or any other account to different hierarchies depending on the types of treemaps you want to see in fava.
In other words, while the plugin itself is a hack, searching and replacing my source files is an even bigger hack that I would not want to do. :)

I guess this issue applies to all reports with treemaps (or any chart maybe). We should have a report, probably just named "Charts", where you show the treemap for a specified list of accounts (or maybe a regular expression to filter the accounts to display). Similar for other charts. A sunburst chart (ref #198) would be an even better visualization for this kind of hierarchical data (and one of the sunburst chart examples also has some cool interactions).

Perhaps a more generally useful way would be to display a treemap/sunburst of an arbitrary query assuming the output is valid to display as a treemap/sunburst.

Could we achieve this with an extension to the Query view? So when the results have a certain "shape", we display charts for them?

With #339 merged, the original intent of this issue is now possible using a FROM filter.
It's not clear to me if a more general way to display query result is feasible and it's a bit unrelated to the original report, so I'll close this.


Enable dynamic versions in pip-tools

Description
This PR configures setuptools-scm to get the package versions dynamically from git tags in the pip-tools option.
Inspired by https://github.com/EnergySystemsModellingLab/MUSE_OS/pull/596
Type of change

[ ] Documentation (non-breaking change that adds or improves the documentation)
[ ] New feature (non-breaking change which adds functionality)
[ ] Optimization (non-breaking, back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Breaking change (whatever its nature)

Key checklist

[ ] All tests pass (eg. python -m pytest)
[ ] Pre-commit hooks run successfully (eg. pre-commit run --all-files)

Further checks

[ ] Code is commented, particularly in hard-to-understand areas
[ ] Tests added or an issue has been opened to tackle that in the future. (Indicate issue here: # (issue))

Having said that, it seems setuptools is not happy with the dynamic version. We might need to switch to something more modern, like hatchling. Check the official docs.

Having said that, it seems that setuptools is not happy with the change.

Having said that, it seems that setuptools is not happy with the change.

Ok, I've fixed the tests. It required initialising the git repo before installing the package, and then a few adjustments...


[Enhancement]: Audit log for dp decommission

Contact Details
<EMAIL_ADDRESS>Is there an existing issue for this?

[X] I have searched all the existing issues

What would you like to be added?
Add audit logs to the key steps of dp decommission  process to facilitate issue identification.
Why is this needed?
No response
Anything else?
No response

closed by #3563


docs: Fix typo in README

Thank you for publishing very interesting library!
I ran sample code in the README and got an error Tantiny::UnsupportedField: Can't search the "title," field with this query. .
This patch seemed to fix it.

Ah, classic typo. Thanks.


Add TLS reload

What problem does this PR solve?

Support TiKV client TLS reloading: A backport from #597 #589
Support JDBC connection retrying.

What is changed and how it works?
TiKV client TLS reloading
Auto reloading certificates. There are two configurations to control this work.

spark.tispark.tikv.tls_reload_interval : The interval time of checking if there is any reloading certificates. If your certificate needs to be updated frequently, you can actively set this parameter. The default is 10s (10 seconds).
spark.tispark.tikv.conn_recycle_time: The interval time of cleaning the expired connection with TiKV. Only take effect when enabling cert reloading. The default is 60s (60 seconds).

spark.tispark.tikv.tls_reload_interval = 10s
spark.tispark.tikv.conn_recycle_time = 60s

JDBC connection retrying
Add a retry policy for all JDBC SQL execution. If a SQL execution met a communication exception, the retry policy would take effect. And the connection in this SQL execution would rebuild several times in 10 seconds until the connection is successful. If we met other exceptions such as SQLSyntaxErrorException, the retry policy will not be triggered.
// Retry policy
  public <RespT> RespT executeWithRetry(Callable<RespT> proc) throws SQLException {
    while (true) {
      try {
       // execute plan
        result = proc.call();
      } catch (CommunicationsException e) { // Only retry when CommunicationsException happened
        // If retry time is over 10 seconds, this will throw a runtime exception
        boolean retry = handler.handleRequestError(backOffer, e);
        if (retry) {
          continue;
        }
      } catch (SQLException e) { // do not retry when other exception happened.
        throw e;
      } catch (Exception e) {
        throw new RuntimeException(e);
      }
    }

Check List
Tests

Unit test
Manual test (add detailed scripts or steps below)

Code changes

Has exported variable/fields change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

Need to update the documentation
Need to be included in the release note

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/merge

/run-all-tests

/merge