ambient: add tests for components restarting

This adds tests for restarting ztunnel and workloads. Depends on https://github.com/istio/ztunnel/pull/1176 to pass with 100% success rate.
I have been stress testing this on my local machine. We have currently zero tolerance for any errors. This might be too sensitive, in which case we can re-evaluate. But it works fine on my end.
This change includes some logic into the echo server to gracefully shutdown. This is required outside of Istio to properly gracefully shutdown an application running in Kubernetes.

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

/retest

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

integ-telemetry-mc_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-mc

gencheck_istio
8140309b32551346c04a38171f10968365407980
link
true
/test gencheck

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests-arm64_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests-arm64

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

lint_istio
8140309b32551346c04a38171f10968365407980
link
true
/test lint

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

gencheck_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-helm_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-helm

integ-ambient_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-ambient

integ-telemetry-istiodremote_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-istiodremote

unit-tests_istio
8140309b32551346c04a38171f10968365407980
link
true
/test unit-tests

integ-telemetry_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry

integ-telemetry-discovery_istio
8140309b32551346c04a38171f10968365407980
link
true
/test integ-telemetry-discovery

gencheck_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test gencheck

lint_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test gencheck

lint_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test lint

integ-ambient_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test integ-ambient

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test gencheck

integ-ambient_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test integ-ambient

lint_istio
1c412758204f9d3cc944938a72982af739da77b5
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-ambient_istio
771a1bdd6eb60a31a8886915ac28ced25dabbacc
link
true
/test integ-ambient

lint_istio
1c412758204f9d3cc944938a72982af739da77b5
link
true
/test lint

gencheck_istio
1c412758204f9d3cc944938a72982af739da77b5
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
1c412758204f9d3cc944938a72982af739da77b5
link
true
/test lint

gencheck_istio
1c412758204f9d3cc944938a72982af739da77b5
link
true
/test gencheck

integ-basic-arm64_istio
ac9815557f1d5a4a5a9a8648e5c98bfaf79c2ac6
link
true
/test integ-basic-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Important to have in some form IMO, especially for node-critical dataplane stuff (ztunnel) and user workloads.

/test integ-basic-arm64

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-ambient_istio
ac9815557f1d5a4a5a9a8648e5c98bfaf79c2ac6
link
unknown
/test integ-ambient

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.


switch ssntp payload from yaml to json for marshall/unmarshall

The yaml parser is much slower than the json parser.  Switching our ssntp payload to be json rather than yaml will cut the marshal/unmarshal time in half.

I wonder if this should be tagged "enhancement" versus eg: "technical debt cleanup"

I've added the "janitorial" label on this.  It's a modestly complex effort from an integration / validation standpoint, but in terms of code complexity and requisite initial skill to accomplish it should be a good starting point for a new contributor.

Have we definitely agreed that it's a good idea to do this?  IIRC the yaml parsing was slow but it wasn't on the critical path.  Moving to json would make things less readable.  Perhaps we could try to speed up the yaml parser.


Added US states

Thanks for contributing @joelacus !
These files are programmatically updated on a monthly base using the
https://github.com/lutangar/cities.json/blob/master/convert.js script, so I can't accept this PR as is.
Thus, an acceptable PR for #2 would be an update of the convert script. Have a look at build process npm run build for more details.

Ah, I just used python to edit cities.json, I didn't realise it was updated every month with node.js.


Subaccount creation REST API

Tech Deisgn Docs 📜

REST Action
HTTP Verb
Path

Create
POST
/v0/api/account/:id/sub

ETA: 2023-10-07

Will close this as "not planned" for now.


Issue with Duo Yubikey auth

Output:
➜  ~ aws-okta exec developer --mfa-device u2f --debug -- echo "hello"
DEBU[0000] Parsing config file /Users/vho/.aws/config
DEBU[0000] Using aws_saml_url from profile: okta
DEBU[0000] using okta provider
DEBU[0000] Failed to reuse session token, starting flow from start
DEBU[0000] Step: 1
DEBU[0001] Step: 2
INFO[0001] Requesting MFA. Please complete two-factor authentication with your second device
DEBU[0001] 009PSPwKCb5AZJyoPhNcOYWoFKCe85gJOuXZf_UFqN
DEBU[0001] Okta Factor Provider: DUO
DEBU[0001] Okta Factor ID: <REDACTED>
DEBU[0001] Okta Factor Type: web
DEBU[0001] Host:api-<REDACTED>.duosecurity.com
Signature:TX|<REDACTED>
StateToken:<REDACTED>

DEBU[0001] challenge u2f
INFO[0001] Sending Push Notification...
DEBU[0002] Facet: https://api-<REDACTED>.duosecurity.com
Touch the flashing U2F device to authenticate...

INFO[0002] Authentication succeeded, continuing
INFO[0002] Device: u2f
INFO[0004] Err: Prompt request failed: 403
Failed Duo challenge. Err: Prompt request failed: 403

OS: macOS 10.13.6
Yubikey: YubiKey 5C Nano
aws-okta version: 0.19.5 (I installed via brew)
I disabled all other 2FA options on Okta and Duo just to isolate the problem (but fwiw Duo Push notifications work well)

On the master branch I get the same:
➜  aws-okta git:(master) go run main.go --debug --mfa-duo-device u2f --mfa-factor-type web --mfa-provider DUO exec zimride-sudo-developer -- echo "success"
DEBU[0000] Parsing config file /Users/vho/.aws/config
DEBU[0000] Using aws_saml_url from profile: okta
DEBU[0000] using okta provider
DEBU[0000] Failed to reuse session token, starting flow from start
DEBU[0000] Step: 1
DEBU[0001] Step: 2
INFO[0001] Requesting MFA. Please complete two-factor authentication with your second device
DEBU[0001] 00nSXYp3XCpYYfj3BYjBA1zV8LxpUxzdo6595zJoBQ
DEBU[0001] MFAConfig: {DUO web u2f}

DEBU[0001] {<REDACTED> web DUO {{   {{}}}}}

DEBU[0001] Using matching factor "DUO web" from config

DEBU[0001] Okta Factor Provider: DUO
DEBU[0001] Okta Factor ID: <REDACTED>
DEBU[0001] Okta Factor Type: web
DEBU[0002] Host:<REDACTED>.duosecurity.com
Signature:TX|<REDACTED>
StateToken:<REDACTED>

DEBU[0002] challenge u2f
INFO[0002] Sending Push Notification...
DEBU[0002] Facet: https://api-c162f2e2.duosecurity.com
Touch the flashing U2F device to authenticate...

INFO[0004] Authentication succeeded, continuing
INFO[0004] Device: u2f
INFO[0007] Err: Prompt request failed: 403
Failed Duo challenge. Err: Prompt request failed: 403
exit status 1

This is a big ask (at least until #129), but I think we might need to do some HTTP-level debugging.

Getting the same issue myself

We don't really use u2f at Segment, so this is going to be hard to repro.
I did some experiments to show how you can get raw HTTP traffic: https://github.com/segmentio/aws-okta/wiki/HTTP-debugging Maybe that could shed some more light.

unfortunately I'm on a MacOS client so capturing that traffic wouldn't be terribly easy (though maybe it's reproducible in a container) Is it possible for you to use a software U2F key to try to reproduce? https://github.com/github/SoftU2F

I actually have a u2f key; it's more that the Okta set up would be tough. I might be able to get to it next week.
The TLS key log method can work with a native client. It requires patching the HTTP clients and building a special bin though.

I'm not sure if it helps or hurts but I'm doing U2F through duo. Also it fails harder when the debug flag isn't set. Debug mode actually makes it get farther, which is unexpected. Can get logs to show the difference tomorrow.

@vivianho / @smiller171 I ran into this same issue and I think I found the problem. Can you take a look at #135 and see if that fixes it for you?

@austinylin I'm trying to build it now. I usually build Go stuff in Docker but since I need a native Mac binary (can't pass USB devices into containers with Docker for Mac) It's taking me a bit.
I'm attempting to cross-compile from within Docker, but if that fails I'll have to actually set up a Go environment on my system

with your changes @austinylin
❯ aws-okta exec -m u2f redacted -- aws s3 ls
INFO[0001] Requesting MFA. Please complete two-factor authentication with your second device
INFO[0002] Sending Push Notification...
Touch the flashing U2F device to authenticate...

INFO[0006] Authentication succeeded, continuing
INFO[0006] Device: u2f
INFO[0007] Err: Prompt request failed: 403
Failed Duo challenge. Err: Prompt request failed: 403

❯ aws-okta exec -m u2f -d redacted -- aws s3 ls
DEBU[0000] Parsing config file /Users/smiller/.aws/config
DEBU[0000] Using aws_saml_url from profile: okta
DEBU[0000] using okta provider
DEBU[0000] Failed to reuse session token, starting flow from start
DEBU[0000] Step: 1
DEBU[0001] Step: 2
INFO[0001] Requesting MFA.
DEBU[0002] Facet: https://redacted.duosecurity.com
Touch the flashing U2F device to authenticate...

INFO[0005] Authentication succeeded, continuing
INFO[0005] Device: u2f
INFO[0006] Err: Prompt request failed: 403
Failed Duo challenge. Err: Prompt request failed: 403

Unless I built the binary incorrectly, nothing seems to have changed

@austinylin thank you so much for this change, it worked for me!!
➜  aws-okta git:(master) ✗ go run main.go --debug --mfa-duo-device u2f --mfa-factor-type web --mfa-provider DUO exec developer -- echo "success"
DEBU[0000] Parsing config file /Users/vho/.aws/config
DEBU[0000] Using aws_saml_url from profile: okta
DEBU[0000] using okta provider
DEBU[0000] Failed to reuse session token, starting flow from start
DEBU[0000] Step: 1
DEBU[0001] Step: 2
INFO[0001] Requesting MFA. Please complete two-factor authentication with your second device
DEBU[0001] <REDACTED>
DEBU[0001] MFAConfig: {DUO web u2f}

DEBU[0001] {<REDACTED> web DUO {{   {{}}}}}

DEBU[0001] Using matching factor "DUO web" from config

DEBU[0001] Okta Factor Provider: DUO
DEBU[0001] Okta Factor ID: <REDACTED>
DEBU[0001] Okta Factor Type: web
DEBU[0001] Host:<REDACTED>.duosecurity.com
Signature:TX|<REDACTED>

DEBU[0001] challenge u2f
INFO[0001] Sending Push Notification...
DEBU[0002] Facet: https://<REDACTED>.duosecurity.com
Touch the flashing U2F device to authenticate...

INFO[0003] Authentication succeeded, continuing
INFO[0003] Device: u2f
DEBU[0008] Step: 3
DEBU[0010] Writing session for developer to keyring
DEBU[0010]  Using session I75I, expires in 59m59.534742s
success

Working for me as well, but I'm not sure why the -m flag no longer works (--mfa-duo-device is much more annoying)


Methods list display is out of order

When viewing a list of methods (methods(f)), the displayed list is out of order from within the pop-up result display.
I’m on my phone right now, but I can get a MWE later if needed.

What did you mean by "the pop-up result display" ?
The datatips that would appear on hover ?

Fixed by https://github.com/JunoLab/Atom.jl/pull/189.


subtract returning incorrect difference

take special note of the comment block for the subtract function, and the tests to figure out what is wrong.

we got this one


RenderCached/AsCache Issues

As of the latest version, I am attempting to bundle my CSS/JS in my Global.asax file as a cached version. However; when I attempt to retrieve the bundled information, I get an error telling me that the key is not defined in the dictionary.
In the mean time, I can use the Named flavoring, but I can't say I understand why this is happening. So I can only fathom it is a bug. Any insight?

I have a site which is using the latest version of SquishIt with AsCached and RenderCached just fine. Can you show me how you are using it?

I must say that I am a bit confused about how AsCached and RenderCached are working. My mission is:
In debug mode - everything is acting as normal - no caching and bundling.
In release mode - I want squishIt to bundle a number css files, cache them in 365days also add a statuscode to 304, with an version number on the bundled file. I have tried this url  https://github.com/jetheredge/SquishIt/wiki/Using-SquishIt-programmatically-without-the-file-system with no success. Is there any test proj that has this that I am asking about?

I am also having this issue: Global.asax.cs:
// SquishIt
Bundle.Css()
.Add("~/Content/Site.css")
.AsCached("main", "~/assets/css/main");
In _layout.cshtml:
@Bundle.Css().MvcRenderCachedAssetTag("main")
Gives compilation error:
CS1061: 'SquishIt.Framework.Css.CSSBundle' does not contain a definition for 'MvcRenderCachedAssetTag' and no extension method 'MvcRenderCachedAssetTag' accepting a first argument of type 'SquishIt.Framework.Css.CSSBundle' could be found (are you missing a using directive or an assembly reference?)
Version is 0.8.5

I do have the same issue as imjoshdean, even tough I call AsCached("key", "/styles/key") in Application_Startup in Global.asax I can't retrieve anything from the cache later on.
I'm not using MVC, so I have a HttpHandler that does the same thing as the MVC Controller: it just outputs the result of  Bundle.Css().RenderCached("key").
It was however working perfectly in the previous version.

I'll look into this as soon as possible. One question, are you running in debug mode? If you are, the bundle is never rendered into the cache. You'll need to make sure you use ForceRelease on both ends in order to force the bundle to compress. The necessity to call "ForceRelease" on both sides will be fixed in the next release. Let me know if that fixes your issue.
As an alternative to called ForceRelease, you could set Debug="false" in the web.config.

I've been setting Debug="false"

The error that you were seeing was around a reference to MvcRenderCachedAssetTag. This is likely because you have not imported the proper namespaces in your app. Unless your error has changed. MvcRenderCachedAssetTag is an extension method in the SquishIt.Mvc dll.

Calling ForceRelease on the render side of things made it work; thanks for your answer!


Flowsim bugs on GLOBECOM

a flow is not removed when clicking the remove button, and the active flow entry count decreases to negative.
a dec TTL action cannot be added to apply_actions instruction; the ADD button is not available

fixed


multiple versions of json_pure gem installed cause issues

Hi,
Not sure if there is anything we can do, however thought I would flag it.
I had both json_pure-1.8.1 and json_pure-1.8.2 installed.
When running:
puppet apply tests/create.pp --noop
I would get a
#<MultiJson::AdapterError: Did not recognize your adapter specification (can't activate json_pure-1.8.1, already activated json_pure-1.8.2).>
To fix I had to uninstall json_pure-1.8.1

I wasn't able to recreate this and I've not heard the issue arrise from other users. Certainly odd. I'll close, but keep a look out if I see it elsewhere. The move to puppet-agent should make this sort of conflict much less likely all being well.


Added 'get resource count' endpoint to resource_router

Done. Thanks for recommendations, actually I just build the container, I first did not install the dependencies locally, but I did it now. On WSL2 there were some issues with mysql headers, I'll hopefully create an issue with a proposed solution in the future.


Add support for is_created in on_model_change for inline fields

Hi I need to send an email when a certain inline model is created and the only way I could figure out to do that is to use on_model_change. Unfortunately for inline fields is_created isn't supported. Is there a better way to go about firing that email and/or would you accept a PR that essentially duplicated what you have done in the migration path for adding the is_created flag?

PR created at #1039


Check for alt attribute on all non-text elements

Hi! :smile_cat:
In the todo section in the README, the following task was listed: "Alternative text for other elements (e.g., object, embed)". I looked up the elements where an alt attribute is recommended (rather than a name attribute, etc) and implemented a new method for this.
A few things of note:

For backwards compatibility, I've left in the img alt test. But if you'd prefer to just replace this method with my new one and bump this to a major version, let me know.

I created new test content for the dummy app to the best of my grokking abilities.

There was a weird jquery version incompatibility issue preventing the tests from running locally or within Travis, so I went ahead and stepped down the version tolerance to solve this temporarily.

Thank you so much! :grin:

Sorry for taking so long to look at this.

For backwards compatibility, I've left in the img alt test. But if you'd prefer to just replace this method with my new one and bump this to a major version, let me know.

Let's keep the back-compat (even though we're still pre-1.0), but can you add a TODO comment so that I remember to go clean that up when I get around to releasing 1.0?
Also, would appreciate it if you could squash down your commits a bit, seems like they should be able to condense into 2-3 commits.
Thanks a bunch!

@trentmwillis I am a bit of a drip feed committer and often forget before pushing, thanks for reminding me to squash :stuck_out_tongue_winking_eye:
I added the TODO comment above the method itself, hope that's an okay position for it.

I am a bit of a drip feed committer

No worries, that's a great thing when working locally.
There's two more minor adjustments to make, then I'll merge!

Thanks! Will cut a new release with your recent improvements later today.


besthitsearch fails to report HLA-DQB1 as an exact match

When using besthitsearch, I get HLA-DQB1 as the best hit for itself, but the exact match bit is set to false.

Looking at other examples, it looks like the dash in the middle of the name is what is causing problems:
RNU6-280P > RNU6-280P
rp11-222g7.2 > RP11-222G7.2
rp11-654c22.2 > RP11-654C22.2
rp11-315f22.1 > RP11-315F22.1
rp11-399e6.1 > RP11-399E6.1
rnu1-62p > RNU1-62P
rp11-654c22.1 > RP11-654C22.1
rp11-399e6.2 > RP11-399E6.2
sertad4-as1 > SERTAD4-AS1
rp11-399e6.4 > RP11-399E6.4
rp4-667h12.4 > RP4-667H12.4
rp11-31f15.2 > RP11-31F15.2
rp11-380j14.1 > RP11-380J14.1
rp11-380j14.4 > RP11-380J14.4
rnu5a-8p > RNU5A-8P
rp5-879k22.1 > RP5-879K22.1
rp11-944l7.6 > RP11-944L7.6
rp11-244n20.7 > RP11-244N20.7

we really need to split the search (which currently all rely on _get_free_text_search() function) in two separate search and lookup methods


Makefile: build 32-bit binary on armv8l

armv8l is always 32-bit and should implement the armv7 ISA, so just use the same filename as for armv7.

Can you explain why you want to change it?

Does the improved commit message help?

Absolutely, thanks.
FWIW, the official binaries are built using goreleaser. It doesn't use the Makefile.


Comments

Sup. This is long, sorry. First I'd like to mention my appreciation that your ideas here are more uh... how do I put this... usually when I've seen ideas for standard extension they haven't been very coherent, and seem to be more interested in following the pack of other languages than seriously considering what problems Lisp actually has. So, nice.
Secondly, I'm going to mention previous/alternate versions of several things you mention in the README, that don't seem to be mentioned. I don't know if you know of them or not, but I think they are useful cross references.
Anyway:

LOOP: you probably know about [iterate](https://common-lisp.net/project/iterate/ which is extensible). You may or may not know that the Xerox implementation of LOOP used in several implementations is extensible. This extensibility is not really documented but it is there, and has been used for example to implement SBCL's for-as-sequence clause. Looking at how that's done would probably be good for any specification of an extensible LOOP.
Allow eval access to environment. This is an idea I very much support in principle but it would require some major changes. Environment objects in the standard are basically "compile-time", and do not really include things like the actual values of variable bindings. Actual runtime environments would be their own beasts. The most interesting system with really usable runtime environments is, I think, Kernel and I have a few partially baked ideas relating to a language with real runtime environment introspection and usage.
Compilation/block compilation. I have some even less baked ideas here. I think that COMPILE-FILE and COMPILE could be provided with a (compile-time kind) environment. Then, the semantic constraints in CLHS <IP_ADDRESS> would apply only with respect to that environment. This would allow precise control of what a compilation unit is actually dependent on. Many additional operators would be required, for example to construct one of these environments from a file (sort of like a runtime CFASL) and read and alter them finely.
Extensible sequences. I assume you are already aware of the existing sequences extension which is implemented by SBCL, Clasp, and ABCL. I think that it is pretty good though I have some minor issues with it, which I have commented on a bit on Clasp's implementation. I have also written a bit of a more general extension for "iterators", which allow the use of some sequence function on things that are not sequences (e.g. because they are infinite, like lazy lists, or unordered, like sets or hash tables). Iterators
Standardize the Meta-Object Protocol for CLOS. Something else I support in principle, but there is a major issue: The meta-object protocol as described in AMOP is not adequately specified and has some weird holes. I mean if you look at Closer-MOP you can see it has pretty extensive code - it's not just a portability library, it's trying to hack pretty different implementations into something uniform.
First-class macros. This is another thing I have been interested in and done some work on. In ancient lisp dialects they were called fexprs. They are interesting but there are some serious issues with them. First, fexpr forms cannot be compiled without specific per-fexpr support from the compiler. Second, if the implementation is not permitted to assume that unknown operators are functions (rather than macros or fexprs), it cannot compile much of anything. Third, they cause some weird syntax escaping, e.g. if in a scheme with fexprs you write (define (funcall1 op arg) (op arg)), (funcall1 a-fexpr x) will result in the fexpr receiving the symbol arg, from the definition of funcall, as an argument. Fourth, I haven't seen many uses for them that wouldn't be covered just by using a macro. Kernel, mentioned above, is the most extensive modern system with fexprs I know of.
Environments. See Trucler
Name conflicts. This is already implemented widely enough that it could probably be a CDR or whatnot.
Object system

I have some ideas about a replacement for defstruct that I'll elaborate on in my repo. The short version is that I would like to have structs that can be stored unboxed in arrays, or ideally, usable to implement your own arrays efficiently, with something like C's flexible array members.
Recursive types seem nice but involve subtle issues with a dynamic typing system with subtypes. One I have contemplated often: the proper list type is defined by proper-list = (or null (cons t proper-list)). We could also define a type for circular lists: circular-list = (cons t circular-list). This seems reasonably intuitive, but what is (subtypep 'circular-list nil)? With the (sorta) obvious definition of subtypep on recursive types, we would go, okay, let's assume it is, and then it's a subtype if (cons t circular-list) is a subtype of nil; this is a subtype if the cdr of the cons is a subtype of nil; we are asking whether (subtypep 'circular-list nil) is true while determining whether (subtypep 'circular-list nil) is, so by assumption it is, and so circular lists don't exist. Sort of the issue here is that familiar recursive types in Haskell and so on are isorecursive, while in Lisp equirecursive types seem more obvious. https://en.wikipedia.org/wiki/Recursive_data_type#Theory
The problem with specializing on types is that there is no obvious ordering of them. The ordering of classes is actually a very important but subtle part of CLOS: Every finalized class has a class precedence list which controls which methods are selected. You can see the problem here if you imagine having one method specialized on (integer 0 1) and another on (integer 1 2); which is called first? This is before even getting into the efficiency concerns.
Robert Strandh (beach) has developed a new system for performing generic function dispatch very efficiently. To summarize, the discriminating function is JIT-compiled. Once a generic function has been called a few times, the discriminating part is basically just a jump table. It's very fast.
I have some ideas on top of this to completely eliminate discrimination in some contexts; for example if a method is specialized on a class, and the method calls some accessor, the accessor could just be inlined. I don't think anything in the standard prevents this, but careful work needs to be done to ensure code isn't bloated, and it would introduce extensive interconnection between generic functions in order to ensure everything is updated when it needs to be.
Christophe Rhodes, the author of the sequences extension, also had some ideas about allowing MOP users to decide on their own how their generic functions should discriminate: Generalizers
Traits/typeclasses - I really don't see how these make sense outside of a statically typed system - and not Lisp's gradual typing system, which is inherently optional.

Syntax. I'm not sure I understand some of what you're getting at here, but I will mention that the standard already specifically defines which reader macro characters can be introduced by programmers. CLHS 2.1.4 explicitly reserves (for example) [] and {} for user use, while ~ is reserved to implementors. I will also mention that the reader implementation Eclector includes more hooks for the reader that could be incorporated into an extension, e.g. here.
Separation into libraries. This would be very difficult, because there are many cross-cutting concerns. For example practically every standard function can signal errors in some situation, meaning making the condition system optional is difficult.
Continuations. I'm fond of delimited continuations, which I think are better than undelimited ones. Some interesting resources here are this fella's work and the very complex continuation system in Racket, which includes some stuff that is probably intended to allay concerns like Pitman's.

I think I'm just currently addressing the part about extensible sequences (in a sense) as well as part about dispatching on types and typeclasses in my https://github.com/lisp-polymorph(btw we still need any help we can get with it) -- which is why I stopped adding stuff to this doc. Doesn't mean I abandoned it btw. Thank you for your input, if you think you can extract something and add it to the doc itself, I would appreciate it a lot.


updated-core: Channels do not work

I recently tried to figure out why no metrics are send to cassandra in the updated-core branch.
I figured out two points:
First: No data will be send to the channel when using the line pipeline becaue the condition in tcp.clj line 43 will not be (never be?) fulfilled:
https://github.com/pyr/cyanite/blob/feature/updated-core/src/io/cyanite/transport/tcp.clj#L43
Second: Despite of the first point there seems to being something wrong with concurrency/channels as the invocation of (a/alts chans) in engine.clj line 50 never returns/waits forever:
https://github.com/pyr/cyanite/blob/feature/updated-core/src/io/cyanite/engine.clj#L50

Hi @MichaelHierweck,
Thanks for reporting this. My naive approach to #80 has been udpated and changed to be more reliable (at the expense of a breaking schema change).


Bugfix: Retain slot capacity while type sharing

See 9020511
If we are sharing types, we should make sure the old type and new type has same slotCapacity. Added additional assert.
Also add a DBG only flag on DynamicType that will indicate if the type was cached for ChangePrototype feature. This will make debugging easy especially when one of the cached types are involved in failure in OOP codegen.

@pleath, can you please help with the review?

LGTM. Thanks.


allowedMentions is not work

GCommands version : ^7.0.0-dev.1630320763
Code :
const { Command } = require("gcommands");

module.exports = class ping extends Command {
    constructor(...args) {
        super(...args, {
            name: "ping",
            description: "DevTest",
        })
    }

    async run({ client, respond }) {
        respond({ content: `${client.ws.ping}ms!`, allowedMentions: {repliedUser: true} })
    }
}

I used allowed Mentions: {repliedUser: true} to mention but it doesn't work like below

Below is the message when it works normally.

Try
allowedMentions: { parse: ['users'] }

Still doesn't work, but thanks for the help :)

#159


[android] deep link doesn't work

App linking is broken for Android devices. Tested on iOS and works fine. Not quite sure why this is happening.
flutter doctor -v

flutter run -v and attempting to open an app link (none of my listeners are triggered, and the app ends up on the default route)

flutter run -d [device id]

I've looked around and it seems like a few others are experiencing similar issues, I've ran a flutter upgrade/ flutter pub upgrade, made sure package names in build.gradle/mainActivity/AndroidManifest are all the same, and ran flutter clean.

Hi @kendall-lu
can you please provide a reproducible minimal code sample.
Please refrain from posting pictures if not necessary.
Thank you

Hi @kendall-lu
can you please provide a reproducible minimal code sample.
Please refrain from posting pictures if not necessary.
If you are using firebase_dynamic_links
please open the issue in the dedicated github.
Thank you

Hey there, unfortunately, the process of recreating the issue would require the setup of app links. Here is an intent filter example similar to my current codebase: <intent-filter android:autoVerify="true">                 <action android:name="android.intent.action.VIEW" />                 <category android:name="android.intent.category.DEFAULT" />                 <category android:name="android.intent.category.BROWSABLE" />                 <data android:scheme="http" android:host="example.fullpath.net" android:pathPrefix="/signup/" />                 <data android:scheme="https" />             </intent-filter> Is there anything else I could provide you with?

Hi @kendall-lu
Can you please clarify if you're using firebase_dynamic_links package or you're referring to  Android App Links
Please provide more details
Thank you

Hi @kendall-lu
Can you please clarify if you're using firebase_dynamic_links package or you're referring to Android App Links
Please provide more details
If you're referring android app links, this is wrong place to file issue for that
Thank you

Using android app links

Hi @kendall-lu
This platform is not meant for assistance on personal code.
Please see https://flutter.dev/community for resources and asking questions like this, you may also get some help if you post it on Stack Overflow.
Closing, as this isn't an issue with Flutter itself,
if you disagree please write in the comments and I will reopen it
Thank you

Hi @kendall-lu
This platform is not meant for assistance on personal code.
Please see https://flutter.dev/community for resources and asking questions like this, you may also get some help if you post it on Stack Overflow.
Closing, as this isn't an issue with Flutter itself,
if you disagree please write in the comments and I will reopen it
Thank you

Hi @kendall-lu
it's not part of Flutter framework, it's native thing
There might be something in native side causing this
If you can give an example on how it is related Flutter, i will reopen the issue
Thank youi


having error creating accounts in eosio using cli--cleos in ubuntu 18.04

`cleos create account eosio myaccount
EOS6BX6eQH2JwKoMwh57wi1p76sr3CkqHNDBzjGCYWnMn4M3GH7fq EOS6BX6eQH2JwKoMwh57wi1p76sr3CkqHNDBzjGCYWnMn4M3GH7fq
418832ms thread-0   main.cpp:2756                 main                 ] Failed with error: Assert Exception (10)
response_content_length >= 0: Invalid content-length response
`
that is the command I run i don't understand what it means I'm new to eos and ubuntu

which version are u using?

Please start nodeos with --http-validate-host=false


Enum::iter() for fieldless enums

There are various crates that allow for iterating over the constants in an enum, but such a core feature should be part of the standard library.

Given that the linked crate has ~50k downloads over the course of a 1 1/2 years, I'm not inclined to believe that it's a "core feature". This should stay as an external crate.
Note: Rust has other "core features" as crates because it is so easy to link in a crate, and makes it easier to fix bugs. A bug in std is a bug forever (see std channels), but a bug in an external crate can be fixed.

FWIW, there are other crates that do similar, such as strum_macros. If someone was truly interested in pushing this forward, I'd encourage them to do a broad review of the ecosystem to find how many potential crates exist and how many uses they have, plus whatever feedback from people who implemented the iterator "by hand" without a crate.

Given that the linked crate has ~50k downloads over the course of a 1 1/2 years, I'm not inclined to believe that it's a "core feature". This should stay as an external crate.

If the logic for not including this was that it had too few downloads, I think it's worth mentioning that enum-iterator now has 2M+ downloads. I end up using it in all of my projects of sufficient scale.

Given that the linked crate has ~50k downloads over the course of a 1 1/2 years, I'm not inclined to believe that it's a "core feature". This should stay as an external crate.
Note: Rust has other "core features" as crates because it is so easy to link in a crate, and makes it easier to fix bugs. A bug in std is a bug forever (see std channels), but a bug in an external crate can be fixed.

strongly disagree

Given that the linked crate has ~50k downloads over the course of a 1 1/2 years, I'm not inclined to believe that it's a "core feature". This should stay as an external crate.

If the logic for not including this was that it had too few downloads, I think it's worth mentioning that enum-iterator now has 2M+ downloads. I end up using it in all of my projects of sufficient scale.

it is now 3,158,869 and still growing everyday

Given that the linked crate has ~50k downloads over the course of a 1 1/2 years, I'm not inclined to believe that it's a "core feature". This should stay as an external crate.

If the logic for not including this was that it had too few downloads, I think it's worth mentioning that enum-iterator now has 2M+ downloads. I end up using it in all of my projects of sufficient scale.

it is now 3,158,869 and still growing everyday

And strum_macros has 17.8 million. I would love to have this feature too.


Add GitHub Actions Support to Checkmarx 2MS Tool

Description:
The Checkmarx 2MS tool is a powerful secret leakage detection tool that helps developers identify sensitive data and other secrets that may have been unintentionally leaked within their code repositories. To integrate this tool effectively into the development workflow, we need to add support for GitHub Actions to Checkmarx 2MS.
Technical Details:
To add GitHub Actions support to Checkmarx 2MS, we will create a custom action that can be used within GitHub workflows. This action will leverage the Checkmarx 2MS tool to scan a specified code repository for potential secret leakage issues and provide detailed results to the user. The action should be configurable, allowing users to specify the repository to scan, the API key to use for authentication, and any other relevant options.
Once the custom action is created, we can add it to the GitHub Marketplace, making it easily accessible for users. Additionally, we will provide documentation on how to integrate this action into existing workflows and best practices for using the Checkmarx 2MS tool for secret leakage detection within the GitHub ecosystem.

Depends on #30

Can we integrate into https://github.com/Checkmarx/ast-github-action/ instead of maintaining yet another GH ?
CC @pedrompflopes

@kaplanlior I see people using this tool freely in their indie projects, without being Checkmarx customers.
Having said that, we can guide them on how to use ast-github-action for only 2ms.

I'm suggesting waiting for #66

I suggest let's do both,

creating a GitHub action for 2ms
contributing a PR for ast-github-action with the additions

this will be flexible for all users

Regarding ast-github-action, talk with Pedro.
Follow the kics-github-action flow.

Should be assigned to @ShimonMizrahi


Add json properties

Description
Following the Add MongoDb Issue, we are adding a properties directly on the connector to add JSON output options.
We have now to options :
relaxed : it's the standard way of JSON format
canonical [default] : it's the model extended of JSON
you can find here the differences.
Here the followings steps that i had :

I add the configurations in the MongoSourceConfig
Next i added the two test of witch type of parser we should use
And last i use the parser parseJSON() from the MongoDB doc

Checklist

[ ] This MR references an issue with a proper description
🍭 Features

[x] Add properties for json.format
[x] Add different output json type depending on the format

⭐ Code Quality

[ ] Tests have been added / changed
[ ] Code is commented where it is complex
[ ] Tests pass
[ ] Code was reviewed
[ ] branch was squashed/rebased to limit the number of commits
[ ] Branch is up to date with master

📖 Repo Documentation

[ ] Update the CHANGELOG
[ ] Update the README

Is there some documentation to update?

Does this project have (unit) tests? Should we write some?

Is there some documentation to update?

I haven't found the dedicated doc in their doc repo :/

Does this project have (unit) tests? Should we write some?

Yep there is some i'm gonna try to add some

This has been submitted upstream: https://github.com/mongodb/mongo-kafka/pull/22


File "/home/mona/anaconda3/lib/python3.7/site-packages/instaloader/instaloader.py", line 483, in login     self.context.login(user, passwd)   File "/home/mona/anaconda3/lib/python3.7/site-packages/instaloader/instaloadercontext.py", line 238, in login     "follow the instructions, then retry.".format(resp_json.get('checkpoint_url'))) instaloader.exceptions.ConnectionException: Login: Checkpoint required. Point your browser to https://www.instagram.com/challenge/25013168396/z2B79pMtFK/ - follow the instructions, then retry.

for the following piece of code:
import instaloader

loader= instaloader.Instaloader(download_comments=False,
                    download_videos=False,
                    download_geotags=False,
                    save_metadata=False)
loader.login('XYZ', 'ABC')
loader.download_hashtag('fashion', max_count=100, fast_update=False, profile_pic=False)

I get the following error:
(base) mona@mona:~/research/insta-scrape$ python instaloader_tester.py
Traceback (most recent call last):
  File "instaloader_tester.py", line 7, in <module>
    loader.login('XYZ', 'ABC')
  File "/home/mona/anaconda3/lib/python3.7/site-packages/instaloader/instaloader.py", line 483, in login
    self.context.login(user, passwd)
  File "/home/mona/anaconda3/lib/python3.7/site-packages/instaloader/instaloadercontext.py", line 238, in login
    "follow the instructions, then retry.".format(resp_json.get('checkpoint_url')))
instaloader.exceptions.ConnectionException: Login: Checkpoint required. Point your browser to https://www.instagram.com/challenge/25013168396/z2B79pMtFK/ - follow the instructions, then retry.

I followed this URL, logged on, and entered the code sent to my phone, and ran the python command again and got the same error. So I am not sure what is meant by then retry.".format(resp_json.get('checkpoint_url'))) since I am not sure where in the code I should exactly write it. Thanks for any tips.
$ instaloader --version
4.5.4
$ python
Python 3.7.6 (default, Jan  8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
$ lsb_release -a
LSB Version:	core-11.1.0ubuntu2-noarch:security-11.1.0ubuntu2-noarch
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.1 LTS
Release:	20.04
Codename:	focal

use the session file to login not the login function. Instagram hates bots so manully logging in will break a lot of the times it seems. follow this to create a season file for your account: https://instaloader.github.io/troubleshooting.html


Add union types concept

@neenjaw - Is this closer to what you're expecting? Future ones would be against empty concepts of course.


add upload schemas api

Coverage increased (+0.6%) to 71.697% when pulling 88cd99782ce04f2c9d64814d48c24a1cecf772b9 on aseTo2016:master into fa2904b7ecf07309db36891751dd41f1e91868cf on ServiceComb:master.

Codecov Report

Merging #128 into master will increase coverage by 0.52%.
The diff coverage is 76.67%.

@@            Coverage Diff            @@
##           master    #128      +/-   ##
=========================================
+ Coverage   66.68%   67.2%   +0.52%
=========================================
  Files          18      18
  Lines        3956    4184     +228
=========================================
+ Hits         2638    2812     +174
- Misses       1153    1195      +42
- Partials      165     177      +12

Impacted Files
Coverage Δ

server/service/microservices.go
72.3% <25%> (+0.36%)
:arrow_up:

server/service/schema.go
75.77% <78.36%> (+1.36%)
:arrow_up:

server/service/governservice.go
64.07% <0%> (-1.46%)
:arrow_down:

server/service/tag.go
71.01% <0%> (-0.49%)
:arrow_down:

server/service/rule.go
71.8% <0%> (+0.29%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa2904b...88cd997. Read the comment docs.


Fix typings pipeline + cleanup

Bug in the update typings pipeline - we were getting the tag name after switching back to the DT repo so it wasn't accurate.
Cleaning up linting issues flagged by DT in typings file.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.005%) to 42.205%

Totals

Change from base Build<PHONE_NUMBER>:
0.005%

Covered Lines:
27754

Relevant Lines:
61526

💛  - Coveralls


git hooks

I use this tool in many of my projects to manage git hooks that run various lints. The yaml file is roughly my standard for rust projects and the shell script just checks that the local tools are installed and sets up the git hooks.
Is this something you'd consider adding to the project?

If you don't want to make it a requirement for all contributors, it'd still be helpful for me personally. Write now, my current workflow is to cherry-pick this commit into any new branch I make, then drop the commit before pushing.

Appreciate the contribution idea, but I'm going to close this as I'm not interested in setting up git hooks or maintaining tooling for setting up git hooks at this time


Remove useMemo from useFormStatus example

Overview
The useMemo here breaks the rules of React, is unnecessary and makes a worse experience:

Breaks the rules of React: setting a timer in memo, setting state, and reading/writing to refs are all side effects in render, which is unsafe. useMemo must be a pure function with no side effects.
Unnecessary: the formData is automatically updated as state, so there's no need to store it in a ref ref, you can just read it. The showSubmitted state is also unnecessary since the pending state already tells you if it's being submitted.
Worse experience: by using a timer, there is an extra state update to show the submitted state (causing a multi-pass render update), and there is an artificial delay between when the form submission completes and when the "Submitted request" text dismisses. By using the pending and formData values directly, the "Submitted request` text is tied directly to the form submission transition "pending" state, so the disabled state and the text update in the same commit.

Additional improvements
I also made a couple more improvements:

CSS to space things out
Reset the form after submission
Increase form submission time to 2 seconds

Before
https://github.com/reactjs/react.dev/assets/2440089/7352d743-a9c7-4025-a16b-26185530af37
After
https://github.com/reactjs/react.dev/assets/2440089/e70f0603-7be0-44ed-be4a-4122374b65ed

I think I added this example and I don't recall why I didn't disable the input while submitting -- is there a reason why we changed that?

@lunaleaps if you don't disable the input, then the user can continue typing but when the action finishes the reset will blow away their changes, which is sucky
https://github.com/reactjs/react.dev/assets/2440089/8eff4a67-e087-47d9-9819-ad9aa70d4f54


cannot suppress warning

I have vext.pyqt5 installed in my virtualenv, and now I'm getting a warning on every call to a cli program in this virtualenv:
WARNING:vext:Could not add extra path: /usr/share/sip

how can I suppress the warning in a painless way?
I tried to configure the logger, but I think because vext plays with the import system this output happens before setting up the logger. any help would be appreciated.

The default level should probably be errors, try this fix  https://github.com/stuaxo/vext/pull/41  (it's untested, so it might need tweaking)

that does the trick, indeed :)
:+1:

Cheers for the report, fix will be in next version.


fix indentation

For everything related to coding standard, we are relying on PHP-CS-Fixer to automate the task. So, if something is not implemented by PHP-CS-Fixer, we don't update the code (and again, we would have to do it everywhere instead of just one place).


libvmi: Refactor cloudinit buiders

What this PR does
The existing cloud-init builders have too many permutations, making it
hard to maintain and use.
This change starts with having one builder per source type and the data
used is passed through a cloudinit dedicated builder.
The number of permutation is left to the callers now.
The library provides two libvmi main builders for cloudinit, one for NoCloud and one for ConfigDrive.
The specific building blocks of the cloudinit details are exposed through a dedicated cloudinit builders that the user can use per need.
Note, it is up to the caller to assure not to build a wrong spec.
The library does not attempt to correct it, mainly because the caller may want to build intentionally a bad configuration (e.g. for unit tests).

Fixes #
Why we need it and why it was done in this way
The following tradeoffs were made:
The following alternatives were considered:
Links to places where the discussion took place:
Special notes for your reviewer

Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note

NONE

/test all

/test all

change: Rebase
/sig code-quality

@nunnatsa , this is the alternative approach I manage to build. It seems simple enough not to make too much noise.
Adding something new should not require touching any existing builder, which I find nice.
I will appreciate you opinion on this and any other alternatives you can think of.
Feel free to take whatever you think is worth it, from here if you want to continue on your PR or some new one.
I'll hold this for now until you have a chance to look at it.
/hold

change: Rebase

change: Moved one builder between commits.

change: Rebase

Nice PR! other then a small comment which is basically a reminder to your own comment it looks good :)

@ShellyKa13 , could you please have another look?

/unhold

change: Attempt to fix failure of a storage e2e test.

/test pull-kubevirt-e2e-k8s-1.30-sig-storage

/lgtm


ComboBox Dictionary binding bug

Describe the bug
When binding a Dictionary<string, string> to a ComboBox having the SelectedValuePath="Key",

the SelectedItem is not being updated.
the SelectedItem set in the MainViewModel ctor is not being reflected in the UI

Steps to reproduce the bug
Here's a repo which reproduces the issue. Please ignore the other controls. They are meant to showcase other bugs I filled.
But if I delete it in the future, then here's some context:
Give a ComboBox like:
<ComboBox SelectedValuePath="Key" DisplayMemberPath="Value" SelectedItem="{x:Bind vm.SelectedItem, Mode=TwoWay}" ItemsSource="{x:Bind vm.MyItems, Mode=TwoWay}"/>
And a ViewModel like:
public Dictionary<string, string> MyItems { get; set; } = new Dictionary<string, string>()

public MainViewModel()
{
    MyItems.Add("item_1", "ITEM 1");
    MyItems.Add("item_2", "ITEM 2");
    SelectedItem = "item_1";
}

private string _selectedItem;
public string SelectedItem
{
    get => _selectedItem;
    set => SetProperty(ref _selectedItem, value);
}

When the user selects an item in the ComboBox, the SelectedItem is not updated with the....well....selected item.,,
When setting the SelectedItem in the ctor, the ComboBox is empty, so the value is not reflected in the UI.

Expected behavior

SelectedItem is updated with the selected item
SelectedItem has "ITEM 1" selected when updated from the ctor

Version Info
Win 10 November 2019 Update (18363)
NuGet package version:

Windows 10 version
Saw the problem?

November 2019 Update (18363)
Yes

May 2019 Update (18362)

October 2018 Update (17763)

April 2018 Update (17134)

Fall Creators Update (16299)

Creators Update (15063)

Device form factor
Saw the problem?

Desktop
Yes

Mobile

Xbox

Surface Hub

IoT

Additional context
Not sure if this is the right repo to fill the bug in since the ComboBox is not from WinUI but dunno which else if not...

I had the same issue as you. I know this was filed a long time ago, but here is the solution:
Here you are setting Selected**Item**. SelectedValuePath is completely ignored by design, as that is what the Selected**Value** property is for.
I think this works as designed and the issue can be closed @ranjeshj.


next: Gear db migration

original tables:

_auth
_password_history
_sso_custom_token
_sso_oauth

proposed tables:

core gear:

_user

contains auth info
_auth without password and provider_info

auth gear:

auth_password_history

replace _password_history

auth_provider_oauth

replace _sso_oauth

auth_provider_password

contains the auth data, which is needed when client call loginWithAuthData

auth_principal

contains all principal, each principal represents a method to login
each user has many principals
custom auth provider will have a custom provide value here

I prefer having another program or another set of commands to manage the db status, because there must be some problem if the required condition is not reached, which maybe schema not created or core table version not updated, and these are some higher level control.
Maybe something like this:
skydbctl version -dburl xxx -app xxx [-gear xxx]
skydbctl create -dburl xxx [-app xxx [-gear xxx]]
skydbctl drop -dburl xxx [-app xxx [-gear xxx]]

// in non-force update, skydbctl would resolve and update the core version required for the gear version.
// force update would be equal to current migrate.
skydbctl update -dburl xxx -app xxx -gear xxx [-f] <version>

Does it still hold that gear tables start with underscore, to distinguish from user created table? If yes, shall it be renamed, for example from auth_password_history to _auth_password_history?

Does it still hold that gear tables start with underscore, to distinguish from user created table? If yes, shall it be renamed, for example from auth_password_history to _auth_password_history?

Didn't consider this in this PR, and I'm ok with this, let me update.

@chpapa updated

@carmenlau updated

@chpapa @rickmak @carmenlau updated.
I've separate core migration paths and command from gears.


Fixes 16556 CI failures

Signed-off-by: John Howard<EMAIL_ADDRESS>Sorry, I broke some CI tests from Windows to Linux in https://github.com/docker/docker/pull/16556 which got merged. This should fix Windows CI again.  @icecrime

Close - still one test failing. Looking...

Fixed that one failure. Windows CI should be green now - waiting

Yup, Windows CI is green. Waiting on Janky and Experimental.

LGTM

LGTM

LGTM


Custom Songs Cover Art Not Showing Up In Spotify

Description
Cover Art for custom songs in spotify wont load, though the song information works just fine
and yes, i have used Spicetify and the custom cover still wont show up, did anyone get the same issue ?
Versions
Monstercat Visualizer Version : 2.1.0
Spicetify Version : 0.9.7

Rainmeter Version: 4.3.1
OS Version: Windows 8.1
Skin Version: 2.1.0
Spotify Version: 1.1.28
.721.g5b5ee660

Screenshots
non custom song

Custom Song issues

what the cover supposed to be

I'm having the same issue right now. Posted to r/rainmeter and I can't seem to find a fix.


Create STIX 2.0 Example: Campaign and Threat Actor

Initial version in https://github.com/oasis-open/cti-stix2-json-schemas/commit/0804d42001888057fcf043467f599301d3f4976b

This has been done


How the hell do I start this bot...

Before submitting an issue, make sure you have:

[x ] Updated to the latest version
[ x] Read the README and docs
[ x] Searched the bugtracker for similar issues including closed ones

Purpose of your issue?

[ ] Bug report (encountered problems/errors)
[ ] Feature request (request for a new functionality)
[x ] Question
[ ] Other

Paste the output of python -V here:
Python 3.7.5rc1

Describe your issue
I installed the bot via CMD line [Windows 10] (python install -U instabot) and went to /site-packages/instabot however, I don't see an examples folder. I'm a little confused on how I'm actually supposed to start the bot from CMD.

just take examples from here

Okay, but then how exactly do I start it…?

From: EightShift<EMAIL_ADDRESS>Sent: Monday, November 25, 2019 5:17:17 AM
To: instagrambot/instabot<EMAIL_ADDRESS>Cc: craigslong<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [instagrambot/instabot] How the hell do I start this bot... (#1133)
just take examples from herehttps://github.com/instagrambot/instabot/tree/master/examples
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/instagrambot/instabot/issues/1133?email_source=notifications&email_token=AF72AQSVDQQ6ITG7G5S2M63QVOX33A5CNFSM4JQ636ZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFCBLTY#issuecomment-558110159, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AF72AQX57Q65RIRYBOGUDOTQVOX33ANCNFSM4JQ636ZA.

pip install -U instabot
git clone https://github.com/instagrambot/instabot
cd examples
python multi_script_CLI.py

Please ask further questions in Telegram group: https://t.me/instabotproject


New version: Yandex.Browser version <IP_ADDRESS>0

Created by 🥟 Dumplings in workflow run #6686.

Log
Updated: <IP_ADDRESS>9 → <IP_ADDRESS>0
Submitting WinGet manifests
Creating manifests
Uploading manifests and making commits
Creating a pull request

Microsoft Reviewers: Open in CodeFlow

https://github.com/microsoft/winget-pkgs/issues/142854


[IOTDB-2310] grafana plugin NPE, Using aggregate queries with "group …

[IOTDB-2310] grafana plugin NPE, Using aggregate queries with "group by level=*"

Coverage increased (+0.009%) to 67.649% when pulling 8ad088831057d7bba33d3a060ce0b7121e691cc9 on CloudWise-Lukemiao:iotdb-2310 into c5283320ee3521ed0d4f85cad072ef370a7bf8b4 on apache:master.


CLI exploration of the swift package benchmark command doesn't exist

I found the commands to invoke benchmarks in the online docs, but I was hoping to be able to invoke something like: swift package benchmark --help to get a list of the commands and how to use them, or perhaps swift package plugin benchmark -help.
Getting to CLI arguments exposed is what I was after - the DocC compiler plugin exposes some of its help in this fashion. Is that something possible here?

Yes, it's definitely on my to-do. Basically, the plugin can't use Swift Argument Parser with the nice auto-generation of help, so I'll need to write up a page manually - looking at some minor usability improvements first but will be there soon, thanks for the ping!


update glob-parent and hosted-git-info test dependencies

PR Summary

update glob-parent and hosted-git-info test dependencies

first is due to a security alert
second was found using yarn audit

PR Context

PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.

Breaking changes

[x] None
OR
[ ] Experimental feature(s) needed

[ ] Experimental feature name(s):

User-facing changes

[x] Not Applicable
OR
[ ] Documentation needed

[ ] Issue filed:

Testing - New and feature

[x] N/A or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

Tooling

[x] I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
OR
[ ] I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:

[ ] Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
(which runs in a different PS Host).

[ ] Issue filed:

[ ] Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.

[ ] Issue filed:

[ ] Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).

[ ] Issue filed:

[ ] Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).

[ ] Issue filed:

LGTM


Next module tools doc

PR Details

Description

Related Issue

Motivation and Context

How Has This Been Tested

Types of changes

[x] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] My code follows the code style of this project.
[ ] I have updated changeset
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.10 out of 14 committers have signed the CLA.:white_check_mark: chenjiahan:white_check_mark: 9aoy:white_check_mark: zllkjc:white_check_mark: sanyuan0704:white_check_mark: xuchaobei:white_check_mark: JSerFeng:white_check_mark: caohuilin:white_check_mark: 10Derozan:white_check_mark: ahaoboy:white_check_mark: yimingjfe:x: Asuka109:x: KyrieLii:x: clChenLiang:x: targeralYou have signed the CLA already but the status is still pending? Let us recheck it.


Tko ramp: fix takeoff issue with large tko ramp time

fixes https://github.com/PX4/Firmware/issues/15408
Because the velocity starts with a positive value (down) to ramp-up slowly the motors, the position setpoint (integral) was going down into the ground, canceling the input of the velocity controller during the early takeoff phase.
Instead of a smooth ramp, the behavior was a delay followed by a quick takeoff or a land detection.
Setting the current altitude setpoint with the current altitude during this phase avoids the canceling effect and creates a takeoff purely velocity-based.
SITL (jMavSim) results:
MPC_TKO_RAMP_T = 5s for both tests.
Before, the trajectory_setpoint.z was going down during the early takeoff phase, delaying the thrust ramp:

Now, the thrust ramps properly since the beginning of the velocity ramp:

@ThomasRigi Could you please test this?
@julianoes This was also probably a reason why we sometimes had a land detection during takeoff

Not broken by this PR, but I'd just like to point out we have a big-ish spike in acceleration between the 1.5m/s and 3m/s climb speeds. We should probably lower the limit there in auto mode, since this high acceleration in Z isn't used in any other context I can think of, and Z accelerations can be scary from a prop noise and power consumption perspective.

@jkflying Well, it's a ramp, not a spike! :D But we can reduce the default jerk, yes.

Thanks @bresch for fixing the ramp!
I just tested with HEADLESS=1 make px4_sitl gazebo_standard_vtol and the ramp works : https://logs.px4.io/plot_app?log=9e20a1fb-ce21-423b-8a82-a8994cb4f130
There are large oscillations, but I don't think your PR is to blame for this. The Z velocity setpoint is good in any case.
Good to go in I'd say. :)

Thanks for the test @ThomasRigi . I still need to investigate if the CI SITL failure is real.

CI failures were due to older VTOL STIL issues (underpowered); rebasing solved the issue.


Add a new ddev clean command to remove unused or replaceable items

While on a quest to clear up some hard drive space on my local machine, I discovered that the ~/.ddev/ directory was hiding a large amount of database backups. After confirming with @rfay on Slack, I set about to delete them - when I was through I reclaimed over 20GB of hard drive space (granted, I've been using DDEV for a few years, and have a significant number of projects).
I think it would be a good idea to add a new command to automate the process. I suggested ddev clean above, but I'm not tied to that command name. Maybe ddev cache-clear or ddev reclaim or ddev gimme-my-hd-space-back.
Ideally, when run, the command should remove the following:

~/.ddev/<project-name>/mysql.saved/ - possibly from when the DB was moved from bind-mounted to a Docker volume.
~/.ddev/<project-name>/mysql_migrated.back/ - possibly from when the DB was moved from bind-mounted to a Docker volume(?)
~/.ddev/pantheon/<project-name>/ - database backups from each and every ddev pull...
Any other location where .ddev is squirreling away DB backups.

The command should also report how much disk space was reclaimed.
I would also suggest a couple of options on the command:

--dry-run - does everything except actually deleting any files.
--keep-recent - keeps any files less than 1 year (or some reasonable time frame) old.

ddev delete images cleans up docker images. It would be great to have a clean command that would remove everything that is replaceable.

images
images-built (if that isn't being cleaned up)
db snapshots?
test tarballs in ~/.ddev (only affects people who run tests locally)

ddev delete images gets rid of images in general if i understood the docs correctly? But in case of a DDEV update another measure would be really useful afterwards. a command deleting only images with version tags < the current version updated to (currently <1.5.1). My current approach after the update was:
docker images -a resulting in a list like:
drud/ddev-webserver               v1.15.1-d9gin-built       694d286e6653        13 seconds ago      1.27GB
<none>                            <none>                    b2eda55daa84        14 seconds ago      1.27GB
<none>                            <none>                    daca591c811a        14 seconds ago      1.27GB
<none>                            <none>                    c5cf75df96df        14 seconds ago      1.27GB
drud/ddev-dbserver-mariadb-10.3   v1.15.1-d9gin-built       b30f0e0e2010        4 minutes ago       498MB
<none>                            <none>                    a9f839b595ac        4 minutes ago       498MB
<none>                            <none>                    7b50491dfa66        4 minutes ago       498MB
<none>                            <none>                    ba6cfb5b5c63        4 minutes ago       498MB
drud/ddev-ssh-agent               v1.15.1-built             018b530fdfb3        4 minutes ago       117MB
<none>                            <none>                    413ca0092f5a        4 minutes ago       117MB
<none>                            <none>                    dd182e99ae45        4 minutes ago       117MB
<none>                            <none>                    e482a6c023fe        4 minutes ago       117MB
drud/ddev-webserver               v1.15.1                   d876b0fe987f        29 hours ago        1.27GB
drud/ddev-ssh-agent               v1.15.1                   f66b0ae743eb        29 hours ago        117MB
drud/ddev-router                  v1.15.1<PHONE_NUMBER>62        29 hours ago        204MB
drud/ddev-dbserver-mariadb-10.3   v1.15.1                   0cd436dc8868        29 hours ago        498MB
drud/ddev-dbserver-mariadb-10.2   v1.15.0-wordpress-built   20acd011937f        3 days ago          449MB
<none>                            <none>                    030c77a77427        3 days ago          449MB
<none>                            <none>                    55e200a03d77        3 days ago          449MB
<none>                            <none>                    beaf8dca8a34        3 days ago          449MB
drud/ddev-webserver               v1.15.0-d9gin-built       154c824745d7        4 days ago          1.26GB
drud/ddev-webserver               v1.15.0-wordpress-built   154c824745d7        4 days ago          1.26GB
<none>                            <none>                    ea93efe64cfd        4 days ago          1.26GB
<none>                            <none>                    cfaf4456b493        4 days ago          1.26GB
<none>                            <none>                    627719b86d63        4 days ago          1.26GB
drud/ddev-dbserver-mariadb-10.3   v1.15.0-d9gin-built       a20baa95e301        4 days ago          498MB
<none>                            <none>                    0ed493ff99be        4 days ago          498MB
<none>                            <none>                    581d34466a2b        4 days ago          498MB
<none>                            <none>                    18a8f8996ca9        4 days ago          498MB
drud/ddev-ssh-agent               v1.15.0-built             0f91de04108f        4 days ago          117MB
<none>                            <none>                    59aee9bf708c        4 days ago          117MB
<none>                            <none>                    9c14dee4fe98        4 days ago          117MB
<none>                            <none>                    643a551f63a1        4 days ago          117MB
drud/ddev-webserver               v1.15.0                   b18fef685762        2 weeks ago         1.26GB
drud/ddev-ssh-agent               v1.15.0                   8c8ddc08f20f        2 weeks ago         117MB
drud/ddev-router                  v1.15.0                   932e8559e711        2 weeks ago         204MB
drud/ddev-dbserver-mariadb-10.3   v1.15.0                   9b4ffe7e40ec        2 weeks ago         498MB
drud/ddev-dbserver-mariadb-10.2   v1.15.0                   fe46a4d1a5cf        2 weeks ago         449MB

if i try to copy and paste image ids and perform e.g.
docker rmi 154c824745d7
leading often for images to
Error response from daemon: conflict: unable to delete 154c824745d7 (cannot be forced) - image has dependent child images

so you i went stepwise through manually with:
docker rmi drud/ddev-webserver:v1.15.0-d9gin-built
so a big yes please for a more sophisticated and convenient delete command for images of especially older outdated ddev images.

So first @rpkoller , you can use ddev rmi -f to force image deletion, but first do a ddev stop.
ddev delete images tries to delete all ddev images that are not the current tagged versions. It may not be perfect.
Sometimes it's worth it just to delete all images, ddev poweroff && docker rmi -f $(docker images -q)
The other day I had 96GB remaining in the docker disk space after deleting everything though, so I just did a docker factory reset.

@rfay well i even do a ddev poweroff before i start do housecleaning tasks in docker.... and the -f option is a good idea/suggestion. i will try next time thanks!
if i understood correctly ddev delete images is bascially the thing i've suggested with --keep recent in https://github.com/drud/ddev/issues/2332#issuecomment-663309133 (guess i haven't understood its purpose and function in the docs) ...so with the current version 1.15.1 ddev delete imagesdeletes all images with version tags < 1.15.1.... i will also give that a try as well... normally i am reluctant to perform a factory reset and wipe everything clean. i try to keep the transfer volume down and not reload stuff if possible.

I don't blame you for fearing docker factory reset. However, the only real thing you lose is databases, and I typically keep dumps or snapshots of everything that's important. Testing new insider Docker versions keeps me on my toes on this and just forces me to do that more often than I'd like.

@mikebarkas if you'd like to take this one on, here is some strategy:

With this command, we're only trying to do things that have no direct harm. So avoiding destroying databases, etc. We never touch code.

Find al the things to delete and then display them with a util.Confirm. (The real problem people might face would be loss of snapshots, so important to point that out well)

Global: delete ~/.ddev/testcache and ~/.ddev/bin

Foreach project .ddev:

downloads
db_snapshots

Delete all images

Give guidance on other things that can be deleted that have real consequences (ddev delete --omit-snapshot for each project, all images: docker rmi -f $(docker images -q), all volumes (will lose all databases) docker system prune --volumes, deleting projects)

I'm torn about where to put this in the command hierarchy. @ultimike asked for ddev clean but since it's an unusual command it might be better stashed in ddev debug clean.
@ultimike 's suggestion of --dry-run is a good one, so let's add that. I think we might want to leave --yes off of this one.

Thanks @rfay , I will get started this week.