Error in tensorflow init: TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

TensorFlow version: v1.11.0-0-gc19e29306c 1.11.0
Python version:
3.6.4, via Anaconda (Anaconda3-5.3.0-Linux-x86_64), and also 3.6.7, and 3.7 via Anaconda
also 3.6.4 as a direct install
Centos7, in a Docker container, with all dependencies pip installed

During __init__ on import of tensorflow,  seeing ```TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'`` - exception trace below
try:
    print("Importing tensorflow", flush=True)
    import tensorflow as tf
    print("Imported tensorflow", tf, flush=True)

except:
    pass

The circumstances are unusual and difficult to reproduce. The code runs fine except in situ installed to support a Streams python map operator. What's needed here is some guidance as to what could cause this specific failure at this specific location in the __init__ method on import (specifically, at  import tensorflow.python.ops import array_ops), and what measures should be taken to address it.
Other info / logs
Importing tensorflow
02 Nov 2018 22:01:40.634-0400 [62] ERROR #splapplog,J[0],P[0],CandidateEvents.MinimalExtractedObjectsSRLNN,python M[splpy_general.h:importModule:352]  - CDIST0305E: Fatal error: missing module: CandidateEventsExtractor.
Traceback (most recent call last):
  File "/tmp/StorylineAggregator1.distributed/toolkits/com.ibm.btn.util.candidate_events_extractor/opt/python/streams/CandidateEventsExtractor.py", line 5, in <module>
    from SRLNN_Pipeline import SRLNN_Pipeline
  File "/tmp/StorylineAggregator1.distributed/toolkits/com.ibm.btn.util.candidate_events_extractor/opt/python/modules/SRLNN_Pipeline.py", line 9, in <module>
    from sentence_embeddings import read_corpus, do_faiss
  File "/tmp/StorylineAggregator1.distributed/toolkits/com.ibm.btn.util.candidate_events_extractor/opt/python/modules/sentence_embeddings.py", line 19, in <module>
    import tensorflow_hub as hub
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow_hub/__init__.py", line 21, in <module>
    import tensorflow as tf
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow/__init__.py", line 22, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 63, in <module>
    from tensorflow.python.framework.framework_lib import *  # pylint: disable=redefined-builtin
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow/python/framework/framework_lib.py", line 52, in <module>
    from tensorflow.python.framework.importer import import_graph_def
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 27, in <module>
    from tensorflow.python.framework import function
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 35, in <module>
    from tensorflow.python.ops import array_ops
  File "/opt/conda/envs/srl-nn/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 153, in <module>
    listdiff.__doc__ = gen_array_ops.list_diff.__doc__ + "\n" + listdiff.__doc__
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
02 Nov 2018 22:01:40.639-0400 [62] ERROR #splapplog,J[0],P[0],CandidateEvents.MinimalExtractedObjectsSRLNN,spl_pe M[PEImpl.cpp:logTerminatingException:2183]  - CDISR5033E: An exception occurred during the execution of the CandidateEvents.MinimalExtractedObjectsSRLNN operator. Processing element number 0 is terminating.
02 Nov 2018 22:01:40.639-0400 [62] ERROR #splapptrc,J[0],P[0],CandidateEvents.MinimalExtractedObjectsSRLNN,spl_operator M[PEImpl.cpp:handleOperatorFailure:661]  - CDISR5030E: An exception occurred during the execution of the CandidateEvents.MinimalExtractedObjectsSRLNN operator. The exception is: Unknown Python error

This turned out to be, as expected, an environment issue, resolved by proper setting of PYTHONPATH, among other steps.


Regarding backdoor.propensity_score_stratification

Hello Team,
I am new to doWhy!
I went through the research paper and documentation and started following the documentation.
I tried to simply replicate the example from https://microsoft.github.io/dowhy/example_notebooks/DoWhy-The Causal Story Behind Hotel Booking Cancellations.html in a Colab notebook.
Got two errors while running estimators and refute
estimate = model.estimate_effect(identified_estimand,
                                 method_name="backdoor.propensity_score_stratification",target_units="ate")
# ATE = Average Treatment Effect
# ATT = Average Treatment Effect on Treated (i.e. those who were assigned a different room)
# ATC = Average Treatment Effect on Control (i.e. those who were not assigned a different room)
print(estimate)

while running the above estimator getting warnings with sklearn as below
/usr/local/lib/python3.7/dist-packages/sklearn/utils/validation.py:760: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)
/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/usr/local/lib/python3.7/dist-packages/pandas/core/computation/expressions.py:204: UserWarning: evaluating in Python space because the '*' operator is not supported by numexpr for the bool dtype, use '&' instead
  f"evaluating in Python space because the {repr(op_str)} "

Also while running refute getting the same warning
refute1_results=model.refute_estimate(identified_estimand, estimate,
        method_name="random_common_cause")
print(refute1_results)

warning :
/usr/local/lib/python3.7/dist-packages/sklearn/utils/validation.py:760: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)
/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
/usr/local/lib/python3.7/dist-packages/pandas/core/computation/expressions.py:204: UserWarning: evaluating in Python space because the '*' operator is not supported by numexpr for the bool dtype, use '&' instead
  f"evaluating in Python space because the {repr(op_str)} "

Based on these findings I have a couple of questions.

Do I need to be worried about these warnings.
As per the warning for logistic regression it seems we need to standardize the data. Is it correct?
Also, does categorical and data of different scales need to be normalized or scaled for doWhy or it handles it explicitly.

Thanks in Advance

@Tejas003 you do not need to scale the data for most datasets. The benefit is that the causal effect is returned in the original scale.
In your case, however, there is a convergence warning which means that the model parameters are not fit properly. That is a big issue and needs to be resolved--you can follow the preprocessing/standardization steps to see if that works. You can ignore the rest of the warnings.
One question: are you using the same data as in the notebook?  Or did you make any changes?
For your specific questions:

Only convergence warning is a problem.
yes, you can try standardizing the data, and then (un)-standardizing the causal effect to find the effect in original data-scale.
DoWhy does not need to do that, since we are only interested in the effect of a particular variable keeping other constant. So it operates in the original data scale. It would not matter if other variables are at a higher or lower scale; since the only change we are interested in is in the treatment variable.


Using vecLib.framework instead of M1 AMX machine code?

So I have been thinking about the M1 AMX work in the chonker branch...
If we look at pytorch, It uses the apple accelerate framework(Accelerate.framework).
The Accelerate.framework relies on  vecLib.framework.
In the vecLib.framework we have cblas.h which has the ops tinygrad deals with.
There is of course some performance penalties to calling a function and having it do stuff compared to writing machine code. But to me it seams like the performance penalty might be worth it if the reduction in complexity is large enough.
Maybe using cblas makes fusing certain operations hard?

The main issue with using a BLAS library is operations after the SUM, say like a RELU. The BLAS library would have to write the matrix out to RAM, the RELU kernel would have to read it, then it would write it back.
While we could make things faster today with a BLAS library, this gives up one of the long term gains possible with tinygrad. The RELU fusing already works today, and real gains will happen when we start fusing in the norm.


Automated Spec Update

Automated Spec Update
333278e77e8104e31e544aecac5338a350537ed5

Add basic files
Forgot to save readme
Add logo to readme

Codecov Report

Merging #34 into master will decrease coverage by 4.53%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #34      +/-   ##
==========================================
- Coverage   53.44%   48.90%   -4.54%
==========================================
  Files           6        6
  Lines         683      683
==========================================
- Hits          365      334      -31
- Misses        318      349      +31

Flag
Coverage Δ

#integration
?

#unit
48.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/response.js
38.88% <0.00%> (-44.45%)
:arrow_down:

src/dropbox.js
44.82% <0.00%> (-31.04%)
:arrow_down:

src/utils.js
20.00% <0.00%> (-20.00%)
:arrow_down:

lib/routes.js
50.10% <0.00%> (-0.84%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63b4b3a...a08efa5. Read the comment docs.


Support KiCad 6

The stable version of KiCad 6.0.0 was released. I'll update to support with all PCB data.
As far as I know, it will no longer be backward compatible with KiCad 5.
https://www.kicad.org/blog/2021/12/KiCad-6.0.0-Release/
TODO

[ ] Corne Cherry
[ ] Corne Chocolate
[ ] Corne Classic
[ ] Corne-light
[ ] https://github.com/foostan/kbd/issues/7

Hi, any progress on this? :)


Failed to load file.

sorry not Idris2.
Hi.
I was following the TDD idris book (10.3.3 TestStore.idr) in AMD fx8300 windows 10 visual studio code. However, I saw 'Failed to load file'. Is there any way? (In Atom and other extensions it works.)

Hi, apologies for taking so long to get back to you.
It could be a number of things, I'd need a reproducible example to know for sure. If you're still facing this issue, can you check the extension settings, and make sure Idris2 Mode isn't on?

Hi, My English level is so bad. I’m forgot this issue. 😃 Idris1 only (not install Idris2)C:\Users\i>idris --version1.3.3 Vscode idris extenstionIdris Language v0.0.12  I solved it like this:One is to use atoms.Another is to add it to C:\idris\libs\libs\base\TDD .(like ‘import Data.Vect’ TDD was use mkdir. ) Windows용 메일에서 발송된 메일입니다. 보낸 사람: meraymond2보낸 날짜: 2022년 3월 30일 수요일 오후 5:10받는 사람: meraymond2/idris-vscode참조: -; Author제목: Re: [meraymond2/idris-vscode] Failed to load file. (Issue #78) Hi, apologies for taking so long to get back to you.It could be a number of things, I'd need a reproducible example to know for sure. If you're still facing this issue, can you check the extension settings, and make sure Idris2 Mode isn't on?—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>


Why does the Considerations say "but not Lists"?

In the Considerations section, there is the line:
you can subscribe to events from libraries (but not lists)

Why? This page is about subscribing to list notifications. If there's special meaning about the word events there, can this be clarified as to how this differs from list subscriptions?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2ade5442-a0dc-e57c-bde0-876396aba7f1
Version Independent ID: 50aa69e3-8a27-dac1-5c60-1d5309a60a33
Content: Subscribe to list notifications
Content Source: docs/spfx/subscribe-to-list-notifications.md
Product: sharepoint
Technology: sharepoint-framework
GitHub Login: @spdevdocs
Microsoft Alias: spdevdocs

You got to love when Microsoft says.
"Subscribe to list notifications"
then say
"you can subscribe to events from libraries (but not lists)"
Workarounds

Call the list every x seconds to check if it is updated (very bad for performance).
Create a Document Library and subscribe to it.
Save a dummy file on the Document Library every time you change a item on the list. (hide the list in shame for having to use so low level trick to get things to work).

In a interactive app with 15 persons accessing three lists at the same time and pinging several lists every two seconds making 450 calls each list every minute (1.250 call total) I would use the second solution.
Any other solution would be appreciated.
User Voice
https://sharepoint.uservoice.com/forums/329220-sharepoint-dev-platform/suggestions/39642082-subscribe-to-list-notifications-for-lists

Third workaround. Get version 1.11.
Get @microsoft/sp-list-subscription.
On sp-list-subscription.js line 3426
Changing the line from
this.transports = opts.transports || ['polling', 'websocket'];
to
this.transports = opts.transports || ['websocket','polling'];
This seams to be working on list's also.

Ping
This has been fixed in SPFX 1.12.1
https://docs.microsoft.com/en-us/sharepoint/dev/spfx/release-1.12.1

Re-opened accidentally.


Add type checking as part of tests

Recently, a commit broke something (see: #51 ) broke something that could've been caught if the tests also ran typechecking.
Please also run typechecking as part of tests - to make sure that new commits don't break anything. Thanks.

Agree, this would be very helpful! Will push it on the queue, but no promises on when it will get done. Feel free to put up a PR!


Update single-sign-on-saml-protocol.md

Fix typo

@tomatsue : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"

#sign-off


L4 AuthorizationPolicy not work properly

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
According to getting-started

It seems that AuthorizationPolicy not work properly.
 Proceeding.
2023-07-18T08:11:03.121829Z	info	install	Created CNI config /host/etc/cni/net.d/10-kindnet.conflist
2023-07-18T08:11:03.121969Z	info	install	Installation succeed, start watching for re-installation.
2023-07-18T08:11:22.850715Z	info	cni	Pod istio-system/ztunnel-qmlzl excluded because it only has 1 containers
2023-07-18T08:11:32.251127Z	info	ambient	Writing ambient config: {"ztunnelReady":true,"redirectMode":"iptables"}
2023-07-18T08:11:32.251722Z	info	ambient	active ztunnel updated to ztunnel-qmlzl
2023-07-18T08:11:32.251794Z	info	ambient	Node-level network rule cleanup started
2023-07-18T08:11:32.251803Z	info	ambient	Detecting iptables command
2023-07-18T08:11:32.318445Z	info	ambient	Using iptables command: iptables-nft
2023-07-18T08:11:32.321184Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.323059Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.325264Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.327302Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.329070Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.331382Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.333726Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.335737Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-PREROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.337558Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.339333Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-POSTROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.341330Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.343126Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-PREROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.345228Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.346743Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-POSTROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.348641Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.350383Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-FORWARD' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.352387Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.354034Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-INPUT' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.355750Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.357877Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-OUTPUT' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T08:11:32.359534Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T08:11:32.374222Z	warn	ambient	Error running command ip rule del priority 100: RTNETLINK answers: No such file or directory

2023-07-18T08:11:32.379558Z	warn	ambient	Error running command ip rule del priority 101: RTNETLINK answers: No such file or directory

2023-07-18T08:11:32.381651Z	warn	ambient	Error running command ip rule del priority 102: RTNETLINK answers: No such file or directory

2023-07-18T08:11:32.383675Z	warn	ambient	Error running command ip rule del priority 103: RTNETLINK answers: No such file or directory

2023-07-18T08:11:32.383762Z	warn	ambient	did not find existing inbound tunnel istioin to delete: Link not found
2023-07-18T08:11:32.383783Z	warn	ambient	did not find existing outbound tunnel istioout to delete: Link not found
2023-07-18T08:11:32.527109Z	info	ambient	Namespace kube-public is disabled from ambient mesh
2023-07-18T08:11:32.527154Z	info	ambient	Namespace kube-node-lease is disabled from ambient mesh
2023-07-18T08:11:32.527158Z	info	ambient	Namespace default is disabled from ambient mesh
2023-07-18T08:11:32.527161Z	info	ambient	Namespace local-path-storage is disabled from ambient mesh
2023-07-18T08:11:32.527163Z	info	ambient	Namespace istio-system is disabled from ambient mesh
2023-07-18T08:11:32.527172Z	info	ambient	Namespace kube-system is disabled from ambient mesh
2023-07-18T08:12:44.707133Z	info	cni	istio-cni cmdAdd podName: ztunnel-q4cvx podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:12:44.707173Z	info	cni	Pod istio-system/ztunnel-q4cvx excluded because it only has 1 containers
2023-07-18T08:15:16.204714Z	info	cni	istio-cni cmdAdd podName: details-v1-5ffd6b64f7-vlzn5 podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:15:16.204773Z	info	cni	Pod default/details-v1-5ffd6b64f7-vlzn5 excluded because it only has 1 containers
2023-07-18T08:15:16.584610Z	info	cni	istio-cni cmdAdd podName: ratings-v1-5f9699cfdf-zdn27 podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:15:16.584655Z	info	cni	Pod default/ratings-v1-5f9699cfdf-zdn27 excluded because it only has 1 containers
2023-07-18T08:15:16.938053Z	info	cni	istio-cni cmdAdd podName: reviews-v1-569db879f5-8mcjj podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:15:16.938096Z	info	cni	Pod default/reviews-v1-569db879f5-8mcjj excluded because it only has 1 containers
2023-07-18T08:15:16.962595Z	info	cni	istio-cni cmdAdd podName: reviews-v2-65c4dc6fdc-fnb6n podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:15:16.962635Z	info	cni	Pod default/reviews-v2-65c4dc6fdc-fnb6n excluded because it only has 1 containers
2023-07-18T08:15:17.337901Z	info	cni	istio-cni cmdAdd podName: reviews-v3-c9c4fb987-4rg5c podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:15:17.337948Z	info	cni	Pod default/reviews-v3-c9c4fb987-4rg5c excluded because it only has 1 containers
2023-07-18T08:15:17.658375Z	info	cni	istio-cni cmdAdd podName: productpage-v1-8b588bf6d-q44wx podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:15:17.658414Z	info	cni	Pod default/productpage-v1-8b588bf6d-q44wx excluded because it only has 1 containers
2023-07-18T08:16:17.332686Z	info	cni	istio-cni cmdAdd podName: sleep-78ff5975c6-zs2cr podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:16:17.332712Z	info	cni	Pod default/sleep-78ff5975c6-zs2cr excluded because it only has 1 containers
2023-07-18T08:16:30.380327Z	info	cni	istio-cni cmdAdd podName: notsleep-6d6c8669b5-dl5b2 podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:16:30.380372Z	info	cni	Pod default/notsleep-6d6c8669b5-dl5b2 excluded because it only has 1 containers
2023-07-18T08:19:19.653957Z	info	cni	istio-cni cmdAdd podName: bookinfo-gateway-istio-bcb5599b9-67zrs podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T08:19:19.654000Z	info	cni	Pod istio-system/bookinfo-gateway-istio-bcb5599b9-67zrs excluded because it only has 1 containers
2023-07-18T08:20:54.786423Z	info	ambient	Namespace default is enabled in ambient mesh
2023-07-18T08:20:54.792075Z	info	ambient	Adding pod 'details-v1-5ffd6b64f7-vlzn5/default' (0b5d1f2d-1c16-4ee0-8f59-2378ba5b96f5) to ipset
2023-07-18T08:20:54.802974Z	info	ambient	Adding route for details-v1-5ffd6b64f7-vlzn5/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.822874Z	info	ambient	Adding pod 'ratings-v1-5f9699cfdf-zdn27/default' (aa66d2e1-e25c-49f0-bb66-63d4198437f3) to ipset
2023-07-18T08:20:54.830119Z	info	ambient	Adding route for ratings-v1-5f9699cfdf-zdn27/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.840651Z	info	ambient	Adding pod 'reviews-v1-569db879f5-8mcjj/default' (6f487c20-bd44-43c5-b04b-42459c9655e0) to ipset
2023-07-18T08:20:54.846036Z	info	ambient	Adding route for reviews-v1-569db879f5-8mcjj/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.855985Z	info	ambient	Adding pod 'reviews-v2-65c4dc6fdc-fnb6n/default' (833ed718-edfb-46ff-a6aa-9252a52ea4a1) to ipset
2023-07-18T08:20:54.860246Z	info	ambient	Adding route for reviews-v2-65c4dc6fdc-fnb6n/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.871341Z	info	ambient	Adding pod 'reviews-v3-c9c4fb987-4rg5c/default' (99eaa66f-0554-4dd8-9f0e-84c0cf8fd963) to ipset
2023-07-18T08:20:54.876532Z	info	ambient	Adding route for reviews-v3-c9c4fb987-4rg5c/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.889111Z	info	ambient	Adding pod 'productpage-v1-8b588bf6d-q44wx/default' (f83e92fc-8dcb-4ea6-9eef-2a30dd065b11) to ipset
2023-07-18T08:20:54.894494Z	info	ambient	Adding route for productpage-v1-8b588bf6d-q44wx/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.906920Z	info	ambient	Adding pod 'sleep-78ff5975c6-zs2cr/default' (41924591-e353-4151-a613-8a7d67e18d87) to ipset
2023-07-18T08:20:54.911498Z	info	ambient	Adding route for sleep-78ff5975c6-zs2cr/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.922299Z	info	ambient	Adding pod 'notsleep-6d6c8669b5-dl5b2/default' (3726e9b9-42ce-488f-9482-da7ebb151a15) to ipset
2023-07-18T08:20:54.930359Z	info	ambient	Adding route for notsleep-6d6c8669b5-dl5b2/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.944848Z	info	ambient	Pod 'details-v1-5ffd6b64f7-vlzn5/default' (0b5d1f2d-1c16-4ee0-8f59-2378ba5b96f5) is in ipset
2023-07-18T08:20:54.949567Z	info	ambient	Route already exists for details-v1-5ffd6b64f7-vlzn5/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.955942Z	info	ambient	Pod 'ratings-v1-5f9699cfdf-zdn27/default' (aa66d2e1-e25c-49f0-bb66-63d4198437f3) is in ipset
2023-07-18T08:20:54.964580Z	info	ambient	Route already exists for ratings-v1-5f9699cfdf-zdn27/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.970689Z	info	ambient	Pod 'reviews-v1-569db879f5-8mcjj/default' (6f487c20-bd44-43c5-b04b-42459c9655e0) is in ipset
2023-07-18T08:20:54.977688Z	info	ambient	Route already exists for reviews-v1-569db879f5-8mcjj/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.983461Z	info	ambient	Pod 'reviews-v2-65c4dc6fdc-fnb6n/default' (833ed718-edfb-46ff-a6aa-9252a52ea4a1) is in ipset
2023-07-18T08:20:54.990274Z	info	ambient	Route already exists for reviews-v2-65c4dc6fdc-fnb6n/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:54.996499Z	info	ambient	Pod 'reviews-v3-c9c4fb987-4rg5c/default' (99eaa66f-0554-4dd8-9f0e-84c0cf8fd963) is in ipset
2023-07-18T08:20:55.011901Z	info	ambient	Route already exists for reviews-v3-c9c4fb987-4rg5c/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:55.017681Z	info	ambient	Pod 'productpage-v1-8b588bf6d-q44wx/default' (f83e92fc-8dcb-4ea6-9eef-2a30dd065b11) is in ipset
2023-07-18T08:20:55.022087Z	info	ambient	Route already exists for productpage-v1-8b588bf6d-q44wx/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:55.029727Z	info	ambient	Pod 'sleep-78ff5975c6-zs2cr/default' (41924591-e353-4151-a613-8a7d67e18d87) is in ipset
2023-07-18T08:20:55.034233Z	info	ambient	Route already exists for sleep-78ff5975c6-zs2cr/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T08:20:55.039256Z	info	ambient	Pod 'notsleep-6d6c8669b5-dl5b2/default' (3726e9b9-42ce-488f-9482-da7ebb151a15) is in ipset
2023-07-18T08:20:55.045529Z	info	ambient	Route already exists for notsleep-6d6c8669b5-dl5b2/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-18T09:07:37.684890Z	info	install	Detect changes to the CNI configuration and binaries, attempt reinstalling...
2023-07-18T09:07:38.136974Z	info	install	Copied istio-cni to /host/opt/cni/bin.
2023-07-18T09:07:38.137082Z	info	install	Directory /host/secondary-bin-dir is not writable, skipping.
2023-07-18T09:07:38.223921Z	info	install	write kubeconfig file /host/etc/cni/net.d/ZZZ-istio-cni-kubeconfig with:
# Kubeconfig file for Istio CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    server: https://[<IP_ADDRESS>]:443
    certificate-authority-data: <CA cert from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt>
users:
- name: istio-cni
  user:
    token: "<redacted>"
contexts:
- name: istio-cni-context
  context:
    cluster: local
    user: istio-cni
current-context: istio-cni-context

2023-07-18T09:07:38.237318Z	info	install	Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2023-07-18T09:07:38.237400Z	info	install	CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "log_uds_address": "/var/run/istio-cni/log.sock",
  "ambient_enabled": true,
  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "kube-system" ]
  }
}
2023-07-18T09:07:38.244494Z	info	install	CNI config file /host/etc/cni/net.d/10-kindnet.conflist exists. Proceeding.
2023-07-18T09:07:38.245177Z	info	install	Created CNI config /host/etc/cni/net.d/10-kindnet.conflist
2023-07-18T09:07:38.245197Z	info	install	CNI configuration and binaries reinstalled.
2023-07-18T09:58:05.147711Z	info	install	Detect changes to the CNI configuration and binaries, attempt reinstalling...
2023-07-18T09:58:06.253318Z	info	install	Copied istio-cni to /host/opt/cni/bin.
2023-07-18T09:58:06.254010Z	info	install	Directory /host/secondary-bin-dir is not writable, skipping.
2023-07-18T09:58:06.254430Z	info	install	write kubeconfig file /host/etc/cni/net.d/ZZZ-istio-cni-kubeconfig with:
# Kubeconfig file for Istio CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    server: https://[<IP_ADDRESS>]:443
    certificate-authority-data: <CA cert from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt>
users:
- name: istio-cni
  user:
    token: "<redacted>"
contexts:
- name: istio-cni-context
  context:
    cluster: local
    user: istio-cni
current-context: istio-cni-context

2023-07-18T09:58:06.257797Z	info	install	Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2023-07-18T09:58:06.258081Z	info	install	CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "log_uds_address": "/var/run/istio-cni/log.sock",
  "ambient_enabled": true,
  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "kube-system" ]
  }
}
2023-07-18T09:58:06.260572Z	info	install	CNI config file /host/etc/cni/net.d/10-kindnet.conflist exists. Proceeding.
2023-07-18T09:58:06.264114Z	info	install	Created CNI config /host/etc/cni/net.d/10-kindnet.conflist
2023-07-18T09:58:06.264427Z	info	install	CNI configuration and binaries reinstalled.
2023-07-18T10:41:34.609541Z	info	install	Detect changes to the CNI configuration and binaries, attempt reinstalling...
2023-07-18T10:41:34.949252Z	info	install	Copied istio-cni to /host/opt/cni/bin.
2023-07-18T10:41:34.949394Z	info	install	Directory /host/secondary-bin-dir is not writable, skipping.
2023-07-18T10:41:34.949527Z	info	install	write kubeconfig file /host/etc/cni/net.d/ZZZ-istio-cni-kubeconfig with:
# Kubeconfig file for Istio CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    server: https://[<IP_ADDRESS>]:443
    certificate-authority-data: <CA cert from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt>
users:
- name: istio-cni
  user:
    token: "<redacted>"
contexts:
- name: istio-cni-context
  context:
    cluster: local
    user: istio-cni
current-context: istio-cni-context

2023-07-18T10:41:34.949781Z	info	install	Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2023-07-18T10:41:34.949820Z	info	install	CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "log_uds_address": "/var/run/istio-cni/log.sock",
  "ambient_enabled": true,
  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "kube-system" ]
  }
}
2023-07-18T10:41:34.950249Z	info	install	CNI config file /host/etc/cni/net.d/10-kindnet.conflist exists. Proceeding.
2023-07-18T10:41:34.951064Z	info	install	Created CNI config /host/etc/cni/net.d/10-kindnet.conflist
2023-07-18T10:41:34.951126Z	info	install	CNI configuration and binaries reinstalled.
2023-07-18T11:14:00.154899Z	info	cni	istio-cni cmdAdd podName: ztunnel-xtbzm podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-18T11:14:00.154936Z	info	cni	Pod istio-system/ztunnel-xtbzm excluded because it only has 1 containers
2023-07-18T11:14:00.213810Z	info	ambient	Writing ambient config: {"ztunnelReady":false,"redirectMode":"iptables"}
2023-07-18T11:14:00.214542Z	info	ambient	active ztunnel updated, no ztunnel running on the node
2023-07-18T11:14:00.214575Z	info	ambient	Node-level network rule cleanup started
2023-07-18T11:14:01.212621Z	info	ambient	Writing ambient config: {"ztunnelReady":true,"redirectMode":"iptables"}
2023-07-18T11:14:01.212891Z	info	ambient	active ztunnel updated to ztunnel-xtbzm
2023-07-18T11:14:01.212922Z	info	ambient	Node-level network rule cleanup started
2023-07-18T11:14:01.215208Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.218264Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.220138Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.223054Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.226380Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.228538Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.232552Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.234631Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-PREROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.241234Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.245022Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-POSTROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.246779Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.248686Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-PREROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.250448Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.253944Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-POSTROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.255961Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.258878Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-FORWARD' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.260869Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.265061Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-INPUT' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.268395Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.271294Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-OUTPUT' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-18T11:14:01.275323Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-18T11:14:01.281142Z	warn	ambient	Error running command ip rule del priority 100: RTNETLINK answers: No such file or directory

2023-07-18T11:14:01.286222Z	warn	ambient	Error running command ip rule del priority 101: RTNETLINK answers: No such file or directory

2023-07-18T11:14:01.288836Z	warn	ambient	Error running command ip rule del priority 102: RTNETLINK answers: No such file or directory

2023-07-18T11:14:01.291159Z	warn	ambient	Error running command ip rule del priority 103: RTNETLINK answers: No such file or directory

2023-07-18T11:14:01.293486Z	warn	ambient	did not find existing inbound tunnel istioin to delete: Link not found
2023-07-18T11:14:01.293860Z	warn	ambient	did not find existing outbound tunnel istioout to delete: Link not found
2023-07-18T11:14:01.471279Z	info	ambient	Namespace kube-system is disabled from ambient mesh
2023-07-18T11:14:01.471356Z	info	ambient	Namespace kube-public is disabled from ambient mesh
2023-07-18T11:14:01.471374Z	info	ambient	Namespace kube-node-lease is disabled from ambient mesh
2023-07-18T11:14:01.471393Z	info	ambient	Namespace default is enabled in ambient mesh
2023-07-18T11:14:01.471429Z	info	ambient	Namespace local-path-storage is disabled from ambient mesh
2023-07-18T11:14:01.471446Z	info	ambient	Namespace istio-system is disabled from ambient mesh
2023-07-18T11:29:35.033773Z	info	install	Detect changes to the CNI configuration and binaries, attempt reinstalling...
2023-07-18T11:29:35.274036Z	info	install	Copied istio-cni to /host/opt/cni/bin.
2023-07-18T11:29:35.274132Z	info	install	Directory /host/secondary-bin-dir is not writable, skipping.
2023-07-18T11:29:35.274269Z	info	install	write kubeconfig file /host/etc/cni/net.d/ZZZ-istio-cni-kubeconfig with:
# Kubeconfig file for Istio CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    server: https://[<IP_ADDRESS>]:443
    certificate-authority-data: <CA cert from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt>
users:
- name: istio-cni
  user:
    token: "<redacted>"
contexts:
- name: istio-cni-context
  context:
    cluster: local
    user: istio-cni
current-context: istio-cni-context

2023-07-18T11:29:35.274722Z	info	install	Using CNI config template from CNI_NETWORK_CONFIG environment variable.
2023-07-18T11:29:35.274754Z	info	install	CNI config: {
  "cniVersion": "0.3.1",
  "name": "istio-cni",
  "type": "istio-cni",
  "log_level": "info",
  "log_uds_address": "/var/run/istio-cni/log.sock",
  "ambient_enabled": true,
  "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/ZZZ-istio-cni-kubeconfig",
      "cni_bin_dir": "/opt/cni/bin",
      "exclude_namespaces": [ "kube-system" ]
  }
}
2023-07-18T11:29:35.275133Z	info	install	CNI config file /host/etc/cni/net.d/10-kindnet.conflist exists. Proceeding.
2023-07-18T11:29:35.276762Z	info	install	Created CNI config /host/etc/cni/net.d/10-kindnet.conflist
2023-07-18T11:29:35.276825Z	info	install	CNI configuration and binaries reinstalled.

Affected product area

[X] Ambient
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Control Plane Revisions

Can you confirm that there is no waypoint installed in the default namespace?

Yep!

Thanks for the help. There‘s more Info:

run:
bin/istioctl pc log ds/ztunnel -n istio-system --level=ztunnel=debug

Then, exec:
kubectl exec deploy/notsleep -- curl -s http://productpage:9080/ | grep -o "<title>.*</title>"
However, I don't seem to see any logs related to the request in ztunnel. Please let me know if you need to turn on other settings.

> kis logs ztunnel-xtbzm
2023-07-18T11:14:00.559256Z  INFO ztunnel: version: version.BuildInfo{Version:"556e299c60ecba1fc2697810010a6a76c3ae6535", GitRevision:"556e299c60ecba1fc2697810010a6a76c3ae6535", RustVersion:"1.68.2", BuildStatus:"Clean", GitTag:"1.18.0-6-g556e299", IstioVersion:"unknown"}
2023-07-18T11:14:00.561696Z  INFO ztunnel: running with config: window_size: 4194304
connection_window_size: 4194304
frame_size: 1048576
socks5_addr: <IP_ADDRESS>:15080
admin_addr: <IP_ADDRESS>:15000
stats_addr: '[::]:15020'
readiness_addr: '[::]:15021'
inbound_addr: '[::]:15008'
inbound_plaintext_addr: '[::]:15006'
outbound_addr: '[::]:15001'
local_node: ambient-worker
proxy_mode: Shared
local_ip: <IP_ADDRESS>
cluster_id: Kubernetes
ca_address: https://istiod.istio-system.svc:15012
ca_root_cert: !File ./var/run/secrets/istio/root-cert.pem
xds_address: https://istiod.istio-system.svc:15012
xds_root_cert: !File ./var/run/secrets/istio/root-cert.pem
xds_on_demand: false
fake_ca: false
termination_grace_period:
  secs: 5
  nanos: 0
proxy_metadata:
  ISTIO_VERSION: 1.18.1
num_worker_threads: 2
enable_original_source: null
proxy_args: proxy ztunnel

2023-07-18T11:14:00.562172Z  INFO ztunnel::proxy::inbound: listener established address=[::]:15008 component="inbound" transparent=true
2023-07-18T11:14:00.562269Z  INFO ztunnel::proxy::inbound_passthrough: listener established address=[::]:15006 component="inbound plaintext" transparent=true
2023-07-18T11:14:00.562359Z  INFO ztunnel::proxy::outbound: listener established address=[::]:15001 component="outbound" transparent=true
2023-07-18T11:14:00.562437Z  INFO ztunnel::proxy::socks5: listener established address=<IP_ADDRESS>:15080 component="socks5"
2023-07-18T11:14:00.562514Z  INFO ztunnel::readiness: Task 'proxy listeners' complete (4.986502ms), still awaiting 1 tasks
2023-07-18T11:14:00.562583Z  INFO ztunnel::hyper_util: listener established address=<IP_ADDRESS>:15000 component="admin"
2023-07-18T11:14:00.562648Z  INFO ztunnel::hyper_util: listener established address=[::]:15020 component="stats"
2023-07-18T11:14:00.563648Z  INFO ztunnel::hyper_util: listener established address=[::]:15021 component="readiness"
2023-07-18T11:14:00.570531Z  INFO ztunnel::xds::client: sending initial request resources=0 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T11:14:00.570908Z  INFO ztunnel::xds::client: sending initial request resources=0 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T11:14:00.616698Z  INFO xds{id=1}: ztunnel::xds::client: Stream established
2023-07-18T11:14:00.617141Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=13
2023-07-18T11:14:00.617960Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=0
2023-07-18T11:14:00.618363Z  INFO ztunnel::readiness: Task 'workload manager' complete (60.836039ms), marking server ready
2023-07-18T11:14:01.361445Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=2
2023-07-18T11:18:59.674665Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T11:18:59.674697Z  INFO xds{id=1}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T11:18:59.674733Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=1
2023-07-18T11:21:31.138897Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=1
2023-07-18T11:21:31.140100Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=0
2023-07-18T11:21:31.140125Z  INFO xds{id=1}: ztunnel::workload: handling RBAC delete default/productpage-viewer
2023-07-18T11:21:42.713758Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=1
2023-07-18T11:21:42.717870Z  INFO xds{id=1}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T11:21:42.717931Z  INFO xds{id=1}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T11:23:00.606954Z  INFO ztunnel::telemetry: new log filter is ztunnel=debug,info
2023-07-18T11:45:31.136548Z DEBUG xds{id=1}: ztunnel::xds::client: XDS client terminated: gRPC error (Unknown error): error reading a body from connection: stream closed because of a broken pipe, retrying in 20ms
2023-07-18T11:45:31.165803Z  INFO ztunnel::xds::client: sending initial request resources=15 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T11:45:31.165880Z  INFO ztunnel::xds::client: sending initial request resources=1 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T11:45:31.174594Z DEBUG xds{id=2}: ztunnel::xds::client: connected established
2023-07-18T11:45:31.174646Z  INFO xds{id=2}: ztunnel::xds::client: Stream established
2023-07-18T11:45:31.174674Z  INFO xds{id=2}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=15
2023-07-18T11:45:31.174768Z DEBUG xds{id=2}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.workload.Workload" nonce="2987132a-eef7-4b69-8b4f-7eccdfa5c641" type=Ack
2023-07-18T11:45:31.174791Z  INFO xds{id=2}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T11:45:31.174811Z  INFO xds{id=2}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T11:45:31.174833Z DEBUG xds{id=2}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="dc3ff3bb-f7b4-41b0-8b9e-123485f7a31e" type=Ack
2023-07-18T11:45:31.174857Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-productpage"
2023-07-18T11:45:31.174866Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/bookinfo-gateway-istio"
2023-07-18T11:45:31.174876Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T11:45:31.174890Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/istiod"
2023-07-18T11:45:31.174900Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/ztunnel"
2023-07-18T11:45:31.174912Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-ratings"
2023-07-18T11:45:31.174922Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T11:45:31.174933Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/notsleep"
2023-07-18T11:45:31.174943Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-details"
2023-07-18T11:45:31.174951Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T11:45:31.174975Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/sleep"
2023-07-18T11:45:31.174989Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T11:45:31.175008Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T12:45:22.604816Z DEBUG xds{id=2}: ztunnel::xds::client: XDS client terminated: gRPC error (Unknown error): error reading a body from connection: stream closed because of a broken pipe, retrying in 40ms
2023-07-18T12:45:22.686390Z  INFO ztunnel::xds::client: sending initial request resources=15 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T12:45:22.686775Z  INFO ztunnel::xds::client: sending initial request resources=1 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T12:45:22.724664Z DEBUG xds{id=3}: ztunnel::xds::client: connected established
2023-07-18T12:45:22.725386Z  INFO xds{id=3}: ztunnel::xds::client: Stream established
2023-07-18T12:45:22.726233Z  INFO xds{id=3}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=15
2023-07-18T12:45:22.727190Z DEBUG xds{id=3}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.workload.Workload" nonce="d0a706d2-30b6-4b50-9422-14cc8662b609" type=Ack
2023-07-18T12:45:22.731882Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T12:45:22.733092Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/bookinfo-gateway-istio"
2023-07-18T12:45:22.734293Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/ztunnel"
2023-07-18T12:45:22.734378Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-ratings"
2023-07-18T12:45:22.734417Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-details"
2023-07-18T12:45:22.734458Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/notsleep"
2023-07-18T12:45:22.734495Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-productpage"
2023-07-18T12:45:22.734550Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T12:45:22.734616Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T12:45:22.734653Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/istiod"
2023-07-18T12:45:22.734706Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/sleep"
2023-07-18T12:45:22.734750Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T12:45:22.756843Z  INFO xds{id=3}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T12:45:22.757495Z  INFO xds{id=3}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T12:45:22.757946Z DEBUG xds{id=3}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="fc16610f-862d-454a-84ec-2c99915d43eb" type=Ack
2023-07-18T12:45:22.758282Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T14:32:50.827124Z DEBUG xds{id=3}: ztunnel::xds::client: XDS client terminated: gRPC error (Unknown error): error reading a body from connection: stream closed because of a broken pipe, retrying in 80ms
2023-07-18T14:32:50.918514Z  INFO ztunnel::xds::client: sending initial request resources=15 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T14:32:50.918582Z  INFO ztunnel::xds::client: sending initial request resources=1 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T14:32:50.926332Z DEBUG xds{id=4}: ztunnel::xds::client: connected established
2023-07-18T14:32:50.926381Z  INFO xds{id=4}: ztunnel::xds::client: Stream established
2023-07-18T14:32:50.926414Z  INFO xds{id=4}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=15
2023-07-18T14:32:50.926508Z DEBUG xds{id=4}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.workload.Workload" nonce="a36d1d11-b982-4a26-89c3-11383279e060" type=Ack
2023-07-18T14:32:50.926534Z  INFO xds{id=4}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T14:32:50.926544Z  INFO xds{id=4}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T14:32:50.926563Z DEBUG xds{id=4}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="5667067e-a31d-4e6f-94e3-a65bc10f4247" type=Ack
2023-07-18T14:32:50.926581Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-productpage"
2023-07-18T14:32:50.926598Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/istiod"
2023-07-18T14:32:50.926607Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T14:32:50.926615Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T14:32:50.926625Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/bookinfo-gateway-istio"
2023-07-18T14:32:50.926633Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-details"
2023-07-18T14:32:50.926643Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-ratings"
2023-07-18T14:32:50.926651Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/ztunnel"
2023-07-18T14:32:50.926660Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T14:32:50.926670Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/notsleep"
2023-07-18T14:32:50.926684Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/sleep"
2023-07-18T14:32:50.926692Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T14:32:50.926703Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T15:51:50.286659Z DEBUG xds{id=4}: ztunnel::xds::client: XDS client terminated: gRPC error (Unknown error): error reading a body from connection: stream closed because of a broken pipe, retrying in 160ms
2023-07-18T15:51:50.455367Z  INFO ztunnel::xds::client: sending initial request resources=15 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T15:51:50.455421Z  INFO ztunnel::xds::client: sending initial request resources=1 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T15:51:50.463693Z DEBUG xds{id=5}: ztunnel::xds::client: connected established
2023-07-18T15:51:50.463751Z  INFO xds{id=5}: ztunnel::xds::client: Stream established
2023-07-18T15:51:50.463801Z  INFO xds{id=5}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=15
2023-07-18T15:51:50.463903Z DEBUG xds{id=5}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.workload.Workload" nonce="01b309c1-28eb-414d-a449-7b8ed9e5a93d" type=Ack
2023-07-18T15:51:50.464014Z  INFO xds{id=5}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T15:51:50.464068Z  INFO xds{id=5}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T15:51:50.464088Z DEBUG xds{id=5}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="bd2bed07-3a76-46bf-88df-9229bb4adba8" type=Ack
2023-07-18T15:51:50.464116Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-productpage"
2023-07-18T15:51:50.464130Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-details"
2023-07-18T15:51:50.464141Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/ztunnel"
2023-07-18T15:51:50.464153Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/notsleep"
2023-07-18T15:51:50.464163Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T15:51:50.464173Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T15:51:50.464181Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/bookinfo-gateway-istio"
2023-07-18T15:51:50.464189Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/istiod"
2023-07-18T15:51:50.464202Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-ratings"
2023-07-18T15:51:50.464213Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/sleep"
2023-07-18T15:51:50.464227Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T15:51:50.464237Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T15:51:50.464253Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T19:16:31.393986Z DEBUG xds{id=5}: ztunnel::xds::client: XDS client terminated: gRPC error (Unknown error): error reading a body from connection: stream closed because of a broken pipe, retrying in 320ms
2023-07-18T19:16:31.760419Z  INFO ztunnel::xds::client: sending initial request resources=15 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T19:16:31.764797Z  INFO ztunnel::xds::client: sending initial request resources=1 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T19:16:31.821621Z DEBUG xds{id=6}: ztunnel::xds::client: connected established
2023-07-18T19:16:31.822223Z  INFO xds{id=6}: ztunnel::xds::client: Stream established
2023-07-18T19:16:31.823946Z  INFO xds{id=6}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=15
2023-07-18T19:16:31.825357Z DEBUG xds{id=6}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.workload.Workload" nonce="62725b75-8dab-40ed-9c5a-38119d12528c" type=Ack
2023-07-18T19:16:31.826414Z  INFO xds{id=6}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T19:16:31.829725Z  INFO xds{id=6}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T19:16:31.829827Z DEBUG xds{id=6}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="10ff6e13-1a57-49d0-8e00-6e846adf2d18" type=Ack
2023-07-18T19:16:31.829965Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/istiod"
2023-07-18T19:16:31.830042Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-details"
2023-07-18T19:16:31.830078Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T19:16:31.830113Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-productpage"
2023-07-18T19:16:31.830147Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T19:16:31.830576Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-ratings"
2023-07-18T19:16:31.830668Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/ztunnel"
2023-07-18T19:16:31.830738Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/bookinfo-gateway-istio"
2023-07-18T19:16:31.830788Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/sleep"
2023-07-18T19:16:31.830830Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T19:16:31.830871Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/notsleep"
2023-07-18T19:16:31.830960Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T19:16:31.831008Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T22:43:13.603399Z DEBUG xds{id=6}: ztunnel::xds::client: XDS client terminated: gRPC error (Unknown error): error reading a body from connection: stream closed because of a broken pipe, retrying in 640ms
2023-07-18T22:43:14.286751Z  INFO ztunnel::xds::client: sending initial request resources=15 type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T22:43:14.286995Z  INFO ztunnel::xds::client: sending initial request resources=1 type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T22:43:14.328627Z DEBUG xds{id=7}: ztunnel::xds::client: connected established
2023-07-18T22:43:14.338685Z  INFO xds{id=7}: ztunnel::xds::client: Stream established
2023-07-18T22:43:14.339011Z  INFO xds{id=7}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.workload.Workload" size=15
2023-07-18T22:43:14.339388Z DEBUG xds{id=7}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.workload.Workload" nonce="cfb57228-6977-48c0-bf08-0fde5d2f70f0" type=Ack
2023-07-18T22:43:14.342226Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/sleep"
2023-07-18T22:43:14.343367Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-productpage"
2023-07-18T22:43:14.343746Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/ztunnel"
2023-07-18T22:43:14.344139Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T22:43:14.344620Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-ratings"
2023-07-18T22:43:14.345080Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/bookinfo-gateway-istio"
2023-07-18T22:43:14.346032Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/istio-system/sa/istiod"
2023-07-18T22:43:14.346680Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T22:43:14.346760Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-details"
2023-07-18T22:43:14.346810Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/notsleep"
2023-07-18T22:43:14.346866Z DEBUG ztunnel::workload: prefetched cert for "spiffe://cluster.local/ns/default/sa/bookinfo-reviews"
2023-07-18T22:43:14.346933Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.workload.Workload"
2023-07-18T22:43:14.347812Z  INFO xds{id=7}: ztunnel::xds::client: received response type_url="type.googleapis.com/istio.security.Authorization" size=1
2023-07-18T22:43:14.347894Z  INFO xds{id=7}: ztunnel::workload: handling RBAC update default/productpage-viewer
2023-07-18T22:43:14.347987Z DEBUG xds{id=7}: ztunnel::xds::client: sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="d51ce763-5260-45a9-b3e9-2c937efe9cfc" type=Ack
2023-07-18T22:43:14.348072Z DEBUG ztunnel::xds::client: sending request type_url="type.googleapis.com/istio.security.Authorization"
2023-07-18T23:47:10.041098Z  INFO ztunnel::telemetry: new log filter is ztunnel=debug,info

Hmm yeah there doesn't appear to be any traffic flowing through the ztunnel; it looks like you're using kind?

@keithmattix
Yes, I am using kind. Are there any known issues?

@keithmattix
Also, I'd like to know if it's feasible to replace Iptables mode with ebpf mode for ambient's basic functionality in a Kind environment (and if it is, I'd like to try it out with a private backported ebpf version of istio 1.18).

@fyuan1316 I still can't reproduce this on kind. If you spin up another cluster with a fresh install, does this error still occur?

I recreated a Kind cluster yesterday and performed the previous actions, but the problem persists.
Sorry, I don't have another cluster right now, I'll try again later when I have an environment. Thanks again for your help.

I suspect the problem is related to the vm environment.
I set up a 2 node k0s cluster with vagrant to test this, although there were some issues that didn't reproduce the expected results in getting-started.  Proceeding.
2023-07-21T10:44:47.791584Z	info	install	Created CNI config /host/etc/cni/net.d/10-kuberouter.conflist
2023-07-21T10:44:47.791618Z	info	install	Installation succeed, start watching for re-installation.
2023-07-21T10:44:59.214739Z	info	cni	Pod istio-system/ztunnel-n6898 excluded because it only has 1 containers
2023-07-21T10:45:22.885690Z	info	ambient	Writing ambient config: {"ztunnelReady":true,"redirectMode":"iptables"}
2023-07-21T10:45:22.887064Z	info	ambient	active ztunnel updated to ztunnel-n6898
2023-07-21T10:45:22.887157Z	info	ambient	Node-level network rule cleanup started
2023-07-21T10:45:22.887303Z	info	ambient	Detecting iptables command
2023-07-21T10:45:22.922062Z	info	ambient	Using iptables command: iptables-nft
2023-07-21T10:45:22.925385Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.928279Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.930695Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.933593Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.936780Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.939066Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.941558Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.944245Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-PREROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.947196Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.950533Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-POSTROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.952884Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.955505Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-PREROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.958221Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.961149Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-POSTROUTING' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.963684Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.966912Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-FORWARD' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.969970Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.973019Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-INPUT' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.975666Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.978196Z	warn	ambient	Error running command iptables-nft: iptables v1.8.7 (nf_tables): Chain 'ztunnel-OUTPUT' does not exist
Try `iptables -h' or 'iptables --help' for more information.

2023-07-21T10:45:22.981113Z	warn	ambient	Error running command iptables-nft: iptables: No chain/target/match by that name.

2023-07-21T10:45:22.984971Z	warn	ambient	Error running command ip rule del priority 100: RTNETLINK answers: No such file or directory

2023-07-21T10:45:22.987929Z	warn	ambient	Error running command ip rule del priority 101: RTNETLINK answers: No such file or directory

2023-07-21T10:45:22.990976Z	warn	ambient	Error running command ip rule del priority 102: RTNETLINK answers: No such file or directory

2023-07-21T10:45:22.993110Z	warn	ambient	Error running command ip rule del priority 103: RTNETLINK answers: No such file or directory

2023-07-21T10:45:22.993272Z	warn	ambient	did not find existing inbound tunnel istioin to delete: Link not found
2023-07-21T10:45:22.993425Z	warn	ambient	did not find existing outbound tunnel istioout to delete: Link not found
2023-07-21T10:45:22.999331Z	error	controllers	error handling istio-system/ztunnel-n6898, retrying (retry count: 1): failed to get veth device: no routes found for <IP_ADDRESS>	controller=ambient
2023-07-21T10:46:11.703728Z	info	cni	istio-cni cmdAdd podName: details-v1-7c7dbcb4b5-m65nd podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:11.703742Z	info	cni	Pod default/details-v1-7c7dbcb4b5-m65nd excluded because it only has 1 containers
2023-07-21T10:46:11.846178Z	info	cni	istio-cni cmdAdd podName: ratings-v1-844796bf85-p4vz4 podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:11.846194Z	info	cni	Pod default/ratings-v1-844796bf85-p4vz4 excluded because it only has 1 containers
2023-07-21T10:46:12.010177Z	info	cni	istio-cni cmdAdd podName: reviews-v1-5cf854487-l6qb4 podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:12.010191Z	info	cni	Pod default/reviews-v1-5cf854487-l6qb4 excluded because it only has 1 containers
2023-07-21T10:46:12.093245Z	info	cni	istio-cni cmdAdd podName: productpage-v1-664d44d68d-n7jn8 podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:12.093435Z	info	cni	Pod default/productpage-v1-664d44d68d-n7jn8 excluded because it only has 1 containers
2023-07-21T10:46:12.105117Z	info	cni	istio-cni cmdAdd podName: reviews-v3-797fc48bc9-pshlj podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:12.105140Z	info	cni	Pod default/reviews-v3-797fc48bc9-pshlj excluded because it only has 1 containers
2023-07-21T10:46:12.198580Z	info	cni	istio-cni cmdAdd podName: reviews-v2-955b74755-fb5dp podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:12.198593Z	info	cni	Pod default/reviews-v2-955b74755-fb5dp excluded because it only has 1 containers
2023-07-21T10:46:14.623324Z	info	cni	istio-cni cmdAdd podName: sleep-9454cc476-nk6cb podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:14.623346Z	info	cni	Pod default/sleep-9454cc476-nk6cb excluded because it only has 1 containers
2023-07-21T10:46:15.209171Z	info	cni	istio-cni cmdAdd podName: notsleep-5ccf57569-4stzr podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:15.209187Z	info	cni	Pod default/notsleep-5ccf57569-4stzr excluded because it only has 1 containers
2023-07-21T10:46:26.379508Z	info	cni	istio-cni cmdAdd podName: bookinfo-gateway-istio-7bf96c9f7-v794j podIPs: [{IP:<IP_ADDRESS> Mask:ffffff00}]
2023-07-21T10:46:26.379547Z	info	cni	Pod istio-system/bookinfo-gateway-istio-7bf96c9f7-v794j excluded because it only has 1 containers
2023-07-21T10:52:21.293660Z	info	ambient	Namespace default is enabled in ambient mesh
2023-07-21T10:52:21.294132Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.294543Z	info	ambient	Adding pod 'ratings-v1-844796bf85-p4vz4/default' (38b3ce00-74b1-438a-b5dd-109d8a2b203d) to ipset
2023-07-21T10:52:21.294706Z	error	ambient	Failed to add pod ratings-v1-844796bf85-p4vz4 to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.322549Z	info	ambient	Adding route for ratings-v1-844796bf85-p4vz4/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.332754Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod ratings-v1-844796bf85-p4vz4: Cannot find device "istioin"

2023-07-21T10:52:21.333148Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.359078Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.359198Z	info	ambient	Adding pod 'reviews-v1-5cf854487-l6qb4/default' (7ab09f9b-1cf8-4bea-8c00-b351b8e3a140) to ipset
2023-07-21T10:52:21.359874Z	error	ambient	Failed to add pod reviews-v1-5cf854487-l6qb4 to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.372906Z	info	ambient	Adding route for reviews-v1-5cf854487-l6qb4/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.383937Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod reviews-v1-5cf854487-l6qb4: Cannot find device "istioin"

2023-07-21T10:52:21.384423Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.400319Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.400384Z	info	ambient	Adding pod 'reviews-v2-955b74755-fb5dp/default' (cfb8e477-5fb8-4383-b2ef-188c02bf0bde) to ipset
2023-07-21T10:52:21.400432Z	error	ambient	Failed to add pod reviews-v2-955b74755-fb5dp to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.414681Z	info	ambient	Adding route for reviews-v2-955b74755-fb5dp/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.427247Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod reviews-v2-955b74755-fb5dp: Cannot find device "istioin"

2023-07-21T10:52:21.428489Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.441786Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.442197Z	info	ambient	Adding pod 'reviews-v3-797fc48bc9-pshlj/default' (54bed8b5-e7f6-4557-9737-630eaffe6c36) to ipset
2023-07-21T10:52:21.442420Z	error	ambient	Failed to add pod reviews-v3-797fc48bc9-pshlj to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.455510Z	info	ambient	Adding route for reviews-v3-797fc48bc9-pshlj/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.464494Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod reviews-v3-797fc48bc9-pshlj: Cannot find device "istioin"

2023-07-21T10:52:21.464811Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.480166Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.481053Z	info	ambient	Adding pod 'productpage-v1-664d44d68d-n7jn8/default' (9c627954-b7a8-4599-a2d5-f52ec1221471) to ipset
2023-07-21T10:52:21.481270Z	error	ambient	Failed to add pod productpage-v1-664d44d68d-n7jn8 to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.493181Z	info	ambient	Adding route for productpage-v1-664d44d68d-n7jn8/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.502303Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod productpage-v1-664d44d68d-n7jn8: Cannot find device "istioin"

2023-07-21T10:52:21.502576Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.515370Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.515507Z	info	ambient	Adding pod 'sleep-9454cc476-nk6cb/default' (e2f0b64b-dd54-4a63-ae7e-7939b24d39ac) to ipset
2023-07-21T10:52:21.515931Z	error	ambient	Failed to add pod sleep-9454cc476-nk6cb to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.526966Z	info	ambient	Adding route for sleep-9454cc476-nk6cb/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.535321Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod sleep-9454cc476-nk6cb: Cannot find device "istioin"

2023-07-21T10:52:21.535715Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.549301Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.549690Z	info	ambient	Adding pod 'notsleep-5ccf57569-4stzr/default' (fe1a4cf0-f019-40d2-b4fb-41ba3d42ce56) to ipset
2023-07-21T10:52:21.549987Z	error	ambient	Failed to add pod notsleep-5ccf57569-4stzr to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.561751Z	info	ambient	Adding route for notsleep-5ccf57569-4stzr/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.569652Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod notsleep-5ccf57569-4stzr: Cannot find device "istioin"

2023-07-21T10:52:21.569714Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.579898Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.579946Z	info	ambient	Adding pod 'details-v1-7c7dbcb4b5-m65nd/default' (4599bcf5-4599-44a9-960c-dc610eaca4b8) to ipset
2023-07-21T10:52:21.580327Z	error	ambient	Failed to add pod details-v1-7c7dbcb4b5-m65nd to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.592805Z	info	ambient	Adding route for details-v1-7c7dbcb4b5-m65nd/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.603200Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod details-v1-7c7dbcb4b5-m65nd: Cannot find device "istioin"

2023-07-21T10:52:21.603870Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.613152Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.613199Z	info	ambient	Adding pod 'ratings-v1-844796bf85-p4vz4/default' (38b3ce00-74b1-438a-b5dd-109d8a2b203d) to ipset
2023-07-21T10:52:21.613263Z	error	ambient	Failed to add pod ratings-v1-844796bf85-p4vz4 to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.627025Z	info	ambient	Adding route for ratings-v1-844796bf85-p4vz4/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.640342Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod ratings-v1-844796bf85-p4vz4: Cannot find device "istioin"

2023-07-21T10:52:21.640415Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.647806Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.648219Z	info	ambient	Adding pod 'reviews-v1-5cf854487-l6qb4/default' (7ab09f9b-1cf8-4bea-8c00-b351b8e3a140) to ipset
2023-07-21T10:52:21.648498Z	error	ambient	Failed to add pod reviews-v1-5cf854487-l6qb4 to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.662407Z	info	ambient	Adding route for reviews-v1-5cf854487-l6qb4/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.670891Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod reviews-v1-5cf854487-l6qb4: Cannot find device "istioin"

2023-07-21T10:52:21.671104Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.676695Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.676915Z	info	ambient	Adding pod 'reviews-v2-955b74755-fb5dp/default' (cfb8e477-5fb8-4383-b2ef-188c02bf0bde) to ipset
2023-07-21T10:52:21.677274Z	error	ambient	Failed to add pod reviews-v2-955b74755-fb5dp to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.689198Z	info	ambient	Adding route for reviews-v2-955b74755-fb5dp/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.703742Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod reviews-v2-955b74755-fb5dp: Cannot find device "istioin"

2023-07-21T10:52:21.703875Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.710230Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.710308Z	info	ambient	Adding pod 'reviews-v3-797fc48bc9-pshlj/default' (54bed8b5-e7f6-4557-9737-630eaffe6c36) to ipset
2023-07-21T10:52:21.710475Z	error	ambient	Failed to add pod reviews-v3-797fc48bc9-pshlj to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.727161Z	info	ambient	Adding route for reviews-v3-797fc48bc9-pshlj/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.736342Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod reviews-v3-797fc48bc9-pshlj: Cannot find device "istioin"

2023-07-21T10:52:21.736431Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.743613Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.743669Z	info	ambient	Adding pod 'productpage-v1-664d44d68d-n7jn8/default' (9c627954-b7a8-4599-a2d5-f52ec1221471) to ipset
2023-07-21T10:52:21.743928Z	error	ambient	Failed to add pod productpage-v1-664d44d68d-n7jn8 to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.755585Z	info	ambient	Adding route for productpage-v1-664d44d68d-n7jn8/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.763437Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod productpage-v1-664d44d68d-n7jn8: Cannot find device "istioin"

2023-07-21T10:52:21.763613Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.770347Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.770579Z	info	ambient	Adding pod 'sleep-9454cc476-nk6cb/default' (e2f0b64b-dd54-4a63-ae7e-7939b24d39ac) to ipset
2023-07-21T10:52:21.770768Z	error	ambient	Failed to add pod sleep-9454cc476-nk6cb to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.781680Z	info	ambient	Adding route for sleep-9454cc476-nk6cb/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.788945Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod sleep-9454cc476-nk6cb: Cannot find device "istioin"

2023-07-21T10:52:21.789077Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.795835Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.795900Z	info	ambient	Adding pod 'notsleep-5ccf57569-4stzr/default' (fe1a4cf0-f019-40d2-b4fb-41ba3d42ce56) to ipset
2023-07-21T10:52:21.795942Z	error	ambient	Failed to add pod notsleep-5ccf57569-4stzr to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.808369Z	info	ambient	Adding route for notsleep-5ccf57569-4stzr/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.816201Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod notsleep-5ccf57569-4stzr: Cannot find device "istioin"

2023-07-21T10:52:21.816429Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>
2023-07-21T10:52:21.823780Z	error	ambient	Failed to list ipset entries: failed to list ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.823823Z	info	ambient	Adding pod 'details-v1-7c7dbcb4b5-m65nd/default' (4599bcf5-4599-44a9-960c-dc610eaca4b8) to ipset
2023-07-21T10:52:21.823863Z	error	ambient	Failed to add pod details-v1-7c7dbcb4b5-m65nd to ipset list: failed to add IP <IP_ADDRESS> to ipset ztunnel-pods-ips: no such file or directory
2023-07-21T10:52:21.834861Z	info	ambient	Adding route for details-v1-7c7dbcb4b5-m65nd/default: [table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]
2023-07-21T10:52:21.841583Z	warn	ambient	Failed to add route ([table 100 <IP_ADDRESS>/32 via <IP_ADDRESS> dev istioin src <IP_ADDRESS>]) for pod details-v1-7c7dbcb4b5-m65nd: Cannot find device "istioin"

2023-07-21T10:52:21.841739Z	warn	ambient	Failed to get device for destination <IP_ADDRESS>

I think you may be running into a bug that was patched on master. For whatever reason, I'm not experiencing it; could you try the latest dev build to see if you run into it?

Thanks, I'll give it a try.

Tried the latest dev build and cni got the same error.
Downgraded the OS version to ubuntu 18.04 and cni gets the same error.
2023-07-22T03:38:33.114750Z	info	ambient	Detected iptables-legacy
2023-07-22T03:38:33.117064Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.119810Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.124510Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.126750Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.128493Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.130567Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.132195Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.135167Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-PREROUTING':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.137507Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.139401Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-POSTROUTING':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.142615Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.145857Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-PREROUTING':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.148203Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.150809Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-POSTROUTING':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.153253Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.158591Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-FORWARD':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.161727Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.164916Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-INPUT':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.167159Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.169063Z	warn	ambient	Error running command iptables-legacy: iptables v1.8.7 (legacy): Couldn't load target `ztunnel-OUTPUT':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

2023-07-22T03:38:33.171485Z	warn	ambient	Error running command iptables-legacy: iptables: No chain/target/match by that name.

2023-07-22T03:38:33.174408Z	warn	ambient	Error running command ip rule del priority 100: RTNETLINK answers: No such file or directory

2023-07-22T03:38:33.176623Z	warn	ambient	Error running command ip rule del priority 101: RTNETLINK answers: No such file or directory

2023-07-22T03:38:33.179951Z	warn	ambient	Error running command ip rule del priority 102: RTNETLINK answers: No such file or directory

2023-07-22T03:38:33.183734Z	warn	ambient	Error running command ip rule del priority 103: RTNETLINK answers: No such file or directory

Can I get some information about the environment you are using to run the getting started operation?
For example, is the cluster running on vm or bare metal, and does ambient have any special operating system requirements, such as centos or ubuntu? Does the version matter?

2023-07-21T10:45:22.999331Z	error	controllers	error handling istio-system/ztunnel-n6898, retrying (retry count: 1): failed to get veth device: no routes found for <IP_ADDRESS>	controller=ambient

On a non-kindnet network, not being able to get the corresponding veth from the route table triggers the above error, which prevents it from working properly.

Ambient definitely has some CNI-level requirements, but someone like @bleggett would know better than I do

2023-07-21T10:45:22.999331Z	error	controllers	error handling istio-system/ztunnel-n6898, retrying (retry count: 1): failed to get veth device: no routes found for <IP_ADDRESS>	controller=ambient

On a non-kindnet network, not being able to get the corresponding veth from the route table triggers the above error, which prevents it from working properly.
@keithmattix Do we have plans to support this case?

Yes, this has been raised previously (e.g. in https://github.com/istio/istio/issues/46163) - ambient CNI does not support CNIs that do not use veth devices at this time.

I was having the exact same issue , in kind , thank you for raising this.


Add a first example ingestion script to the docs

This adds the example discussed in #271 to the getting started docs, aiming to provide a first example script that uses as little as possible of the terracotta API and not much core Python or other libraries either! Hopefully a good starting point for more novice users.

Fantastic, thanks!


AMBARI-25714 :  Upgrade commons-compress to 1.21 to resolve CVEs

Upgrade commons-compress to 1.21 to resolve CVEs
Verified it was built ok. Started the server and installed a few services.

+1 , lgtm.


chore: add deno to the providers in defaultProviders comment

Just a small comment change, mis-leading in editor suggestions.

As per https://github.com/jspm/generator/issues/367#issuecomment-2227502294, defaultProvider: 'deno' is very much not supported.
If we do want to support it, let's open a new feature request for that. jsr likely comes into the discussion at this point though?

The i think it should be changed to denoland right ?
Because i see we do have a denoland provider for the field
https://github.com/jspm/generator/blob/main/src/providers/index.ts#L55

updated it to denoland now 👍

I've started some further work on jsr: support in https://github.com/jspm/generator/pull/372. Closing for the reasons as mentioned in the comment.


[Snyk] Upgrade oro-regexp from 2.0.2 to 2.0.3

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade oro-regexp from 2.0.2 to 2.0.3.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.
The recommended version was released 21 days ago, on 2024-01-13.

Release notes

Package name: oro-regexp

2.0.3 - 2024-01-132.0.3 / 2024-01-13

Added coverage for testing.
Added watcher for coding.
Improved github cicd replacing npm install to npm ci.
Enhanced linter adding some extensions.
Enhanced prettier adding import-sorter.
Enhanced testing to achieve the 100% of coverage (yay!).
Updated dev libs:

@ babel/core from v7.23.3 to v7.23.7.
@ babel/preset-env from v7.23.3 to v7.23.7.
@ types/jest from v29.5.10 to v29.5.11.
@ typescript-eslint/eslint-plugin from v6.12.0 to v6.18.0.
@ typescript-eslint/parser from v6.12.0 to v6.18.0.
eslint from v8.54.0 to v^8.56.0.
eslint-plugin-unicorn from v49.0.0 to v50.0.1.
prettier from v3.1.0 to v3.1.1.
typescript from v5.2.2 to v5.3.3.

Added dev libs:

@ trivago/prettier-plugin-sort-imports
eslint-config-prettier
eslint-plugin-jest
nodemon

2.0.2 - 2023-11-222.0.2 / 2023-11-22

Fixed github action npm_publish_on_pr_merge_to_master.
Updated dev libs:

@ babel/core from v7.23.2 to v7.23.3.
@ babel/preset-env from v7.23.2 to v7.23.3.
@ babel/preset-typescript from v7.23.2 to v7.23.3.
@ types/jest from v29.5.6 to v29.5.10.
@ typescript-eslint/eslint-plugin from v6.9.0 to v6.12.0.
@ typescript-eslint/parser from v6.9.0 to v6.12.0.
eslint from v8.52.0 to v8.54.0.
eslint-plugin-unicorn from v48.0.1 to v49.0.0.
prettier from v3.0.3 to v3.1.0.
tsup from v7.2.0 to v8.0.1.

    from oro-regexp GitHub release notes

Commit messages

Package name: oro-regexp

91cdb16 Merge pull request #4 from oropesa/chore/add-coverage
70cad7e UPD v2.0.3 (see changelog)

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Fixed in #14


Implementation Guide Visualization

Changes in this PR:
Added a visualization for implementation guides to the capability tab on the shiny dashboard. Since all our stored endpoints do not currently have any implementation guides set, a blank table with text explaining that there are no implementation guides is displayed when there are no implmentation guides to be shown, otherwise, a bar chart will be displayed showing the counts of each type of implementation guide.
Pull requests into this repository require the following checks to be completed by the submitter and reviewers.
Submitter:

[x] Administrative tasks are complete.

This pull request describes why these changes were made.
The JIRA ticket links to this PR.
The JIRA ticket for this PR is: Lantern 257 JIRA Ticket

[x] Tests are complete.

Tests are included and test edge cases.
Tests have been run locally and pass.

[x] Code review has been performed.

Code diff has been done and checked to ensure only expected code is being committed.
A linter for the language has been run.

Primary Reviewer:

[ ] Code works.

Code has been run locally and the basic functionality affected by the PR has been checked.
Edge cases have been checked to ensure behavior is as expected.

[ ] Code review has been performed.

Code is maintainable, reusable, efficient, and correct.
Code accomplishes the tasks purpose.
Code follows style guidance appropriate for the language, including passing linter checks.
Code is well commented (which does not mean verbosely commented).

[ ] Tests are complete.

Tests are included and test edge cases.
Tests have been run locally and pass.

Secondary Reviewer:

[ ] Code works.

Code has been run locally and the basic functionality affected by the PR has been checked.
Edge cases have been checked to ensure behavior is as expected.

[ ] Code review has been performed.

Code accomplishes the tasks purpose.

[ ] Tests are complete.

Tests have been run locally and pass.

Merger:
Prior to merging, ensure that all checks have been completed. Delete the branch after the merge
is complete.
Create a new branch <branch name>_gomodupdate. Run make update_mods to update the go.mod
and go.sum files to point to master. Create a PR. Require one sanity check reviewer.

Looks good to me! I'll approve it once it's been rebased!