Stage 1 Saving

Hi,
I closed my Windows related question, since I found it easier to get up and running in WSL. I'm right near the end of getting out a colorized ply. Unfortunately, there's one tiny issue I'm hoping you can help me with.
I have the gui open, I'm at stage 1, everything looks good so I click on 'mesh'. The gui says that it saved the file called ngp_stage1_1.ply, I see the terminal and it says saved to a folder called mesh_stage0, but there's no ply file by the name given in the gui.

During stage 0, I got an exported uncolorized ply in the mesh_stage0/ folder, but now there's no stage1 ply to be found anywhere. I can see it in the gui though - a colorized mesh.
Here's the log:
[INFO] Trainer: ngp_stage1 | 2023-03-09_08-25-07 | cuda | fp16 | trial_syn_lego/
[INFO] #parameters: 18513142
Namespace(path='data/nerf_synthetic/lego/', O=True, workspace='trial_syn_lego/', seed=0, stage=1, ckpt='latest', fp16=True, test=False, test_no_video=False, test_no_mesh=False, camera_traj='', data_format='nerf', train_split='train', preload=True, random_image_batch=True, downscale=1, bound=1.0, scale=0.8, offset=[0, 0, 0], mesh='', enable_cam_near_far=False, enable_cam_center=False, min_near=0.05, enable_sparse_depth=False, enable_dense_depth=False, iters=30000, lr=0.01, lr_vert=0.0001, pos_gradient_boost=1, cuda_ray=True, max_steps=1024, update_extra_interval=16, max_ray_batch=4096, grid_size=128, mark_untrained=True, dt_gamma=0.0, density_thresh=10, diffuse_step=1000, background='random', enable_offset_nerf_grad=False, num_rays=4096, adaptive_num_rays=True, num_points=262144, lambda_density=0, lambda_entropy=0, lambda_tv=1e-08, lambda_depth=0.1, lambda_specular=1e-05, wo_smooth=False, lambda_lpips=0, lambda_offsets=0.1, lambda_lap=0.001, lambda_normal=0, lambda_edgelen=0, contract=False, patch_size=1, trainable_density_grid=False, color_space='srgb', ind_dim=0, ind_num=500, mcubes_reso=512, env_reso=256, decimate_target=300000.0, mesh_visibility_culling=True, visibility_mask_dilation=5, clean_min_f=8, clean_min_d=5, ssaa=2, texture_size=4096, refine=True, refine_steps_ratio=[0.1, 0.2, 0.3, 0.4, 0.5, 0.7], refine_size=0.01, refine_decimate_ratio=0.1, refine_remesh_size=0.02, vis_pose=False, gui=True, W=1000, H=1000, radius=5, fovy=50, max_spp=1, refine_steps=[3000, 6000, 9000, 12000, 15000, 21000])
NeRFNetwork(
(encoder): GridEncoder: input_dim=3 num_levels=16 level_dim=1 resolution=16 -> 2048 per_level_scale=1.3819 params=(6119864, 1) gridtype=hash align_corners=False interpolation=smoothstep
(encoder_color): GridEncoder: input_dim=3 num_levels=16 level_dim=2 resolution=16 -> 2048 per_level_scale=1.3819 params=(6119864, 2) gridtype=hash align_corners=False interpolation=linear
(sigma_net): MLP(
(net): ModuleList(
(0): Linear(in_features=16, out_features=32, bias=False)
(1): Linear(in_features=32, out_features=1, bias=False)
)
)
(color_net): MLP(
(net): ModuleList(
(0): Linear(in_features=32, out_features=64, bias=False)
(1): Linear(in_features=64, out_features=64, bias=False)
(2): Linear(in_features=64, out_features=6, bias=False)
)
)
(specular_net): MLP(
(net): ModuleList(
(0): Linear(in_features=6, out_features=32, bias=False)
(1): Linear(in_features=32, out_features=3, bias=False)
)
)
)
[INFO] Loading stage 0 model to init stage 1 ...
[INFO] loaded model.
[WARN] missing keys: ['vertices_offsets']
[INFO] Loading latest checkpoint ...
[WARN] No checkpoint found, abort loading latest model.
==> Saving mesh to trial_syn_lego/mesh_stage0
==> Finished saving mesh.

@tsugg Hi, the mesh button is actually from an old project, and I forget to remove it as it generate wrong results.
You need to use the command line to train and save stage1 mesh, which will be in obj format.

Ahhhh well it that case I might need some help here lol.
python main.py data/nerf_synthetic/lego/ --workspace trial_syn_lego/ -O --bound 1 --scale 0.8 --dt_gamma 0 --stage 0 --lambda_tv 1e-8

It seems to be related to OpenGL in WSL2, maybe you could check this?

Hi, I am fail to run with --gui option. Do you have any idea about the errors?
I post errors at https://github.com/ashawkey/nerf2mesh/issues/82 .
Thanks


[improve](dynamic table) refine SegmentWriter columns writer generate

Proposed changes
Dynamic Block consists of two parts, dynamic part of columns and static part of columns
static   dynamic
| ----- | ------- |
the static ones are original _tablet_schame columns
the dynamic ones are auto generated and extended from file scan.
**We should only consisder to use Block info to generte columns when it's a dynamic table load procudure.**
And seperate the static ones and dynamic ones

Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

related with: https://github.com/apache/doris/pull/16795/files
But this pr maybe more elegant.

fix #16658


App freezes because of Realm?

I am using Realm as the mobile database in one of my apps and while using app start freezing randomly. I was not sure what's happening so while debugging when I paused in XCode during app freeze I saw Realm in stack traces but couldn't make out of it anything. I am attaching here a screenshot. Anyone know what's happening here?

Realm framework version: 2.5.5
Xcode version: XCode 8.3
iOS/OSX version: iOS 10
Dependency manager + version: Cocoapods

So what was the problem?


Scenario benchmark upgrade

Bump the verion of the new nuget package
Generalize the Etw listener, so it can be used in the scenario case too
Impreved the scenario interface. It provides more options to the scenario user to customize the process that will be launched
When running an scenario, it will return the Process "Elapsed time"
When running Etw scenario, it will return the Etw process running time
When running Etw scenario, data for the specified process, its child processes, and modules will be collected
Added an Etw simple parser. It collects PMC counters per process, and per module
Small changes to the AssemblyModel, which was initially not planned to be exposed as public
Increase the default InstructionRetired.Interval from 100,000 to 1,000,000. This will reduce the overall etl file size and still keep the collected data meanigful
Created few interfaces such as PerformanceMonitorCounter and ProcessExitInfo to be exposed through the run scenario
Small bug fix: GC.SuppressFinalize was called before Dispose(true)
Adding a small test for BenchmarkIterationMeasurement.Dispose
Remove unnecessary package reference
Remove global.json file
Skip attempt to process csv file when there is no file (Benchmarks did not run. For example, they are all facts or disabled)
(We throw System.IO.FileNotFoundException exception when no benchmark ran.)
Print more Debug information on the ETW profiling.

Pending:

There is some cleanup I'm doing like removing duplicated code
Update the README

Hmm, that is a lot of changes. Anything you want me to focus on?
I looked at bit at the module attribution for PMC events. I am curious where this is going. Are you thinking of handling the Method Load & Unload events from the CLR too?

@AndyAyersMS In this PR I'm collecting PMC data from the scenarios via ETW. The plan is to extend it and collect user specified providers data too.
With these changes we will be able to collect PMC data from the TP and benchmarks scenario runs.

Yes, I will be adding managed and kernel data on next PR :)

In reply to: 332351324 [](ancestors = 332351324)

Overall, looks good.  I think there are a few more open issues around try/catch of user code, but perhaps they are all handled via the using blocks?

In reply to: 332326182 [](ancestors = 332326182)

Yes, I've tried making all resources implement the disposable interface and placing them in using blocks.

In reply to: 334887279 [](ancestors = 334887279,332326182)


Microsoft Onedrive Reset label referencing Outlook

microsoftonedrivereset)
name="Microsoft Outlook Reset"
type="pkg"
packageID="com.microsoft.reset.Outlook"
downloadURL="https://office-reset.com"$(curl -fs https://office-reset.com/macadmins/ | grep -o -i "href."."Outlook_Reset..pkg" | cut -d '"' -f2)
expectedTeamID="QGS93ZLCU7"
TeamID is correct, some minor updates are needed :) Thanks for the great work on this 🥇

you are right @rougegoat thankyou


(BRK-617) Print exception message for test cases

Some exceptions implement the #inspect method to only print the class
name or otherwise change the output to not include the error message in
question, which makes such exceptions very challenging to deubg. This
commit changes the test case exception printing method to explicitly
print the exception class and message so that there's always relevant
context available when an exception is raised.

Refer to this link for build results (access rights to CI server needed):
http://jenkins-beaker.delivery.puppetlabs.net//job/qe_beaker_btc-intn/1786/
Test PASSed.


Fix #11818 fix workloadSelector for Sidecars

This is a second attempt to fix #11818.
In comparison to the simple/naive attempt in #12620 this PR extends ServiceDiscovery with a new function GetProxyWorkloadLabels(*model.Proxy) that should return the labels associated with the workload that should be associated with the given proxy.
This way it should also be possible to define Sidecar instances for workloads coming from Consul, or even MCP (Idk if this makes sense though)
I didn't any tests yet, as I am not sure if this PR has a chance of being accepted, I can add them anytime during the review.
In my local, manual tests associating Sidecars worked for K8s Deployments with and without Services.
While going through the code I was also wondering if there is an error here:
https://github.com/istio/istio/blob/release-1.1/pilot/pkg/proxy/envoy/v2/ads.go#L669-L697
First EDS is pushed, and after that the SidecarScope I set.
But pushing EDS depends on the SidecarScope. So wouldn't it make more sense to set the sidecar scope first and then push EDS?

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: robertpanzer
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: costinm, mandarjog
If they are not already assigned, you can assign the PR to them by writing /assign @costinm @mandarjog in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS
pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @robertpanzer. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

@robertpanzer: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests.sh
75656cff10e96eeee32d944bc32ee472f17c9e6d
link
/test e2e-simpleTests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
75656cff10e96eeee32d944bc32ee472f17c9e6d
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
75656cff10e96eeee32d944bc32ee472f17c9e6d
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
75656cff10e96eeee32d944bc32ee472f17c9e6d
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

codecov check failed three times in a row
--- FAIL: TestCheckCoverage (7.79s)
    main_test.go:216: Coverage dropped:
        istio.io/istio/pilot/pkg/networking/core/v1alpha3/fakes/fake_service_accounts.go:-46.300000% (46.300000% to 0.000000%)
        istio.io/istio/pilot/pkg/serviceregistry/consul/controller.go:-16.100000% (80.300000% to 64.200000%)

I doubt we care about the first one (though it could be a clue), but the second one looks significant

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
75656cff10e96eeee32d944bc32ee472f17c9e6d
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
f16ee889c316f7fd9b4ab46c4c5d80d880b7f09d
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
f16ee889c316f7fd9b4ab46c4c5d80d880b7f09d
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
f16ee889c316f7fd9b4ab46c4c5d80d880b7f09d
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@duderino I added tests for the consul registry, the code coverage error for that one went away:
--- FAIL: TestCheckCoverage (9.71s)
    main_test.go:216: Coverage dropped:
        istio.io/istio/pilot/pkg/networking/core/v1alpha3/fakes/fake_service_accounts.go:-46.300000% (46.300000% to 0.000000%)
=== RUN   TestGoPath

I am not sure though how I could have affected code coverage of the service accounts fake, in particular to 0.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
f16ee889c316f7fd9b4ab46c4c5d80d880b7f09d
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: robertpanzer
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: mandarjog, nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @mandarjog @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS
pilot/OWNERS
tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-pilot-multicluster-e2e

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-pilot-multicluster-e2e

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-pilot-multicluster-e2e

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-pilot-multicluster-e2e

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-pilot-multicluster-e2e

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

prow/e2e-simpleTests-minProfile.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTestsMinProfile

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

prow/e2e-simpleTests-minProfile.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTestsMinProfile

prow/istio-pilot-multicluster-e2e.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

prow/e2e-simpleTests-minProfile.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTestsMinProfile

prow/istio-pilot-multicluster-e2e.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-multicluster-e2e

prow/e2e-simpleTests-cni.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests-cni

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

prow/e2e-simpleTests-minProfile.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTestsMinProfile

prow/istio-pilot-multicluster-e2e.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-multicluster-e2e

prow/e2e-simpleTests-cni.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests-cni

prow/e2e_pilotv2_auth_sds.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio_auth_sds_e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

prow/e2e-simpleTests-minProfile.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTestsMinProfile

prow/istio-pilot-multicluster-e2e.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-multicluster-e2e

prow/e2e-simpleTests-cni.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests-cni

prow/e2e_pilotv2_auth_sds.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio_auth_sds_e2e

prow/release-test.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test release-test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
b9125c27cae58f81e058ec2825ee76f1e3e6d110
link
/test istio-integ-k8s-tests

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e-simpleTests.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests

prow/e2e-simpleTests-minProfile.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTestsMinProfile

prow/istio-pilot-multicluster-e2e.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-multicluster-e2e

prow/e2e-simpleTests-cni.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-simpleTests-cni

prow/e2e_pilotv2_auth_sds.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio_auth_sds_e2e

prow/release-test.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test release-test

prow/e2e-dashboard.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-dashboard

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/istio-pilot-multicluster-e2e.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-multicluster-e2e

prow/e2e_pilotv2_auth_sds.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio_auth_sds_e2e

prow/release-test.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test release-test

prow/istio-integ-k8s-tests.sh
881745f0067ae32eb3e91b00101936310098ab7a
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-bookInfoTests-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-mixer-no_auth.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test e2e-mixer-no_auth

prow/e2e_pilotv2_auth_sds.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test istio_auth_sds_e2e

prow/release-test.sh
8436ce227d319dc1d7d01ca8b93edb15fd94fe0d
link
/test release-test

prow/istio-integ-k8s-tests.sh
881745f0067ae32eb3e91b00101936310098ab7a
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
881745f0067ae32eb3e91b00101936310098ab7a
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm at a high level, but I am not very familiar with this part of the code, you'll need approval from owners as well
Thanks for working on this

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: howardjohn, robertpanzer
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: mandarjog, nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @mandarjog @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS
pilot/OWNERS
tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

I was just looking at the code for adding the EnvoyFilter, and it seems like using the workload labels instead of the Service labels should be trivial now.

@duderino Regarding the drop in coverage to 0 in fake_service_accounts.go it looks like the corresponding interface was removed in c4c2baddf61c3f72a0cbeb088c991c8899d8ddc2.
The coverage without this PR seems to come from the fact that fakes.ServiceAccounts was a nested struct in fakes.ServiceDiscovery. I can't tell how that came, if it was really generated by counterfeiter like this or if this was a manual change.
However the method fakes.ServiceDiscovery.GetServiceAccounts() is called.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
881745f0067ae32eb3e91b00101936310098ab7a
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
8e63465d06c0c9a89bba56d22e25ede13f85075d
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e63465d06c0c9a89bba56d22e25ede13f85075d
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
8e63465d06c0c9a89bba56d22e25ede13f85075d
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Add a fix for #12957: Make EnvoyFilter also match against Pod Labels instead of Service labels.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8e63465d06c0c9a89bba56d22e25ede13f85075d
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
8e63465d06c0c9a89bba56d22e25ede13f85075d
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
8e63465d06c0c9a89bba56d22e25ede13f85075d
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Test failures look related to the change to EnvoyFilter? @robertpanzer

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
6ca73cabd61c19bf4f1d600ecb72b6c00fad7223
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn I have no real clue why these are failing. The tests are stable locally, and the test output doesn't really show much.
In fact there should be a lot more, locally I have some info logs, in the logs on k8s-gubernator I see nothing but a lot of empty whitespace.
Also I cannot view the full logs.
I would be glad for any help, in particular as I cannot run these specific test scripts like prow/istio-unit-tests.sh locally. make works for me.

@howardjohn I found the logs now and found out that I forgot to check in the test configuration.
prow/istio-unit-tests.sh is green now.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
5796886f465c1406cb6bf2148103ed13a630b244
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
7cbb4f99707069add786f17436ee4e829e4a4021
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
7cbb4f99707069add786f17436ee4e829e4a4021
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
7cbb4f99707069add786f17436ee4e829e4a4021
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer this PR is in the right direction but I am afraid simply fetching the workload labels for the purpose of fixing sidecar/envoyfilter is not sufficient. The proxyServiceInstances is deeply tied to various other parts of the pilot plugin system. RBAC/AuthN plugins rely on the service instance to obtain the appropriate configs for the workloads based on the service attached. We need to ensure that these plugins dont segfault if the service instance is nil.
For example in pilot/plugin/authn/authentication.go: OnInboundFilterChains
	port := in.ServiceInstance.Endpoint.ServicePort
	authnPolicy := model.GetConsolidateAuthenticationPolicy(in.Env.IstioConfigStore, in.ServiceInstance.Service, port)
	return setupFilterChains(authnPolicy, in.Env.Mesh.SdsUdsPath, in.Env.Mesh.EnableSdsTokenMount, in.Env.Mesh.SdsUseK8SSaJwt, in.Node.Metadata)

Even the sidecar code has some dependency on the service instances. It wont create ingress listeners if there are no service instances attached to the sidecar.
This is part of the thing that we want to fix and I am working on it.

/hold

@rshriram: I don't fully understand your point:

The proxyServiceInstances is deeply tied to various other parts of the pilot plugin system. RBAC/AuthN plugins rely on the service instance to obtain the appropriate configs for the workloads based on the service attached.

If I understand you correctly you are saying that Deployments that don't have a Service are generally not supported by Istio. I can hardly believe given that I am running this locally all the time.
The PR also doesn't change the way how Proxy.ServiceInstances is set.
Also a plugin method like OnInboundFilterChains() is called by ConfigGeneratorImpl.buildSidecarInboundListenerForPortOrUDS().
Now a consumer only application does not get an InboundListener, so it should also not fail here, right?
The test TestLDSWithSidecarForWorkloadWithoutService should already show that this actually does not segfault, it uses the plugins you mentioned and it uses a Proxy without ServiceInstance.
Or did I miss sth that you mentioned?

Even the sidecar code has some dependency on the service instances. It wont create ingress listeners if there are no service instances attached to the sidecar.

This is part of the thing that we want to fix and I am working on it.

But this is a general thing that is not working right now and unrelated to this PR, right?
I added 2 tests to make check that no listeners are created for proxies without services, both with and without Sidecars, these were missing.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
7cbb4f99707069add786f17436ee4e829e4a4021
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
62b576fb95f8064ea0f7f94a1d044f96bd1be39c
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
62b576fb95f8064ea0f7f94a1d044f96bd1be39c
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
62b576fb95f8064ea0f7f94a1d044f96bd1be39c
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram Thinking more about this I guess you could have meant that Pilot could Segfault if the Sidecar contains an Ingress specification.
Therefore I extended the test configuration to have an Ingress.
But I still could not get it to Segfault:
If the SidecarScope has an Ingress, but there is no Service, there's no Ingress Listener generated:
https://github.com/istio/istio/blob/497c85e7f4e0a0783991d015023a1393a60d4731/pilot/pkg/networking/core/v1alpha3/listener.go#L393-L401
So could you please explain what problems you see?
At the moment I don't see how the code you linked above could be called.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
62b576fb95f8064ea0f7f94a1d044f96bd1be39c
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
d2055a2c7666c6453840c2fe93adbb920792010b
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
d2055a2c7666c6453840c2fe93adbb920792010b
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
d2055a2c7666c6453840c2fe93adbb920792010b
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
d2055a2c7666c6453840c2fe93adbb920792010b
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
0deeb2105e9ac467a1f2cf9895efc69be4d71d86
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
0deeb2105e9ac467a1f2cf9895efc69be4d71d86
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
0deeb2105e9ac467a1f2cf9895efc69be4d71d86
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

The ci/circleci: e2e-pilot-noauth-v1alpha3-v2 failure looks like a flake/infra problem:
Failed
The connection to the server <IP_ADDRESS>:8443 was refused - did you specify the right host or port?
error: the path "/tmp/istio.e2e.577923725/yaml/istio.yaml" does not exist

Rerunning the circleci workflow to see if it occurs again.

/lgtm
Should have more eyes on it

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: howardjohn, hzxuzhonghu, robertpanzer
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: mandarjog, nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @mandarjog @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS
pilot/OWNERS [hzxuzhonghu]
tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

New changes are detected. LGTM label has been removed.

/hold cancel
/lgtm

@rshriram: changing LGTM is restricted to assignees, and assigning you to the PR failed.

In response to this:

/hold cancel
/lgtm

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, hzxuzhonghu, robertpanzer, rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS [rshriram]
pilot/OWNERS [hzxuzhonghu,rshriram]
tests/OWNERS [rshriram]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
0deeb2105e9ac467a1f2cf9895efc69be4d71d86
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
f0df9b55a074a80c10030381029a877333b84a2e
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@robertpanzer: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
f0df9b55a074a80c10030381029a877333b84a2e
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
f0df9b55a074a80c10030381029a877333b84a2e
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, hzxuzhonghu, robertpanzer, rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

mixer/OWNERS [rshriram]
pilot/OWNERS [hzxuzhonghu,rshriram]
tests/OWNERS [rshriram]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Will be in 1.1.3 which should be published on 4/15


create links for git and github tutorial

@debrajhyper
Can you please add hacktoberfest tag into this?
Thanks


Why is it impossible to use the React18 Suspense SSR feature with the nginx agent?

I use examples/cloudflare-workers-react-full to look the effect.

I don't know, I haven't tried with Ngnix.
I'm inclined to think it's a Ngnix misconfiguration. Closing, but keep us updated about your findings.


Archives

Perhaps you want to implement something already present in C++17, std::filesystem.
https://en.cppreference.com/w/cpp/filesystem

Not really, the purpose of archives are to provide a virtual file system which can handle compressed archives as well as default directories. What filesystem does is simply act as a wrapper to the OS file functions. std::filesystem does have it place in the StandardArchive implementation.

[ ] Basic Archive Implementation
[ ] Implement loader just for .qb files using Archive


Create a how-to guide to deploy Kubeflow using Terraform solution

Context
Write a how-to guide for deplying kubeflow using the Terraform solution
What needs to get done
Write a how-to guide
Definition of Done
There is a how-to guide

Guide https://charmed-kubeflow.io/docs/deploy-charmed-kubeflow-using-terraform


[feature request] keyboard support

arrow keys, enter, esc
see for example: http://www.material-ui.com/#/components/date-picker

good advice

Support escape for cancel and enter for confirm in v1.3.0.
Due to months restriction mechanism, support of arrow keys is not available yet.


Nan loss after few epochs

Hey,
I got the code a while ago and trying to run it. However, after setting everything up I used to get Nan loss after the first epoch. I tried so many things, from changing optimizers to checking for null values in inputs, to changing learning rates. Only after your latest change of hard setting the batches_per_epoch to 1000 things became better, however still starting from epoch 6 it gives back nan loss values. What seems to be the problem? also, I couldn't reproduce the paper precision and recall values the best I could get after which is only after 5 epochs are:
This is the restaurant dataset
precision-recall  f1-score   support
     Food      0.786     0.183     0.296       887
    Staff      0.527     0.281     0.367       352
 Ambience      0.327     0.131     0.188       251

Hi,
I am not sure why you got the NaN loss, since I didn't encounter this issue with the current code or the code before changing batches_per_epoch to 1000. So far, no other people has reported this issue. Maybe you use another machine to run?
As for evaluation, did you manually assign the cluster_map by yourself in evaluation.py? As the cluster_map I provided in the evaluation.py is only used for the uploaded trained model. If you train a model again, you need to manually assign the mapping first before evaluation.
You can try to evaluate the uploaded trained restaurant model by running evaluation.py directly. This will give you similar results as reported in paper. And you can have a look at how I assigned the aspect label to each cluster by looking at the cluster_map and the aspect.log in pre_trained_model/restaurant/.

Thanks I will look into the evaluation part, however, there still some correlation between the gradient explosion (causing the nan loss) and the number of batches per epoch (may be a certain training set out of the dataset) cause the problem. I will still look into it and update you when I get something new.
Regards
From: Ruidan He
Sent: Wednesday, 4 July 2018 3:06 PM
To: ruidan/Unsupervised-Aspect-Extraction
Cc: yassienshaalan; Author
Subject: Re: [ruidan/Unsupervised-Aspect-Extraction] Nan loss after few epochs(#9)
Hi,
I am not sure why you got the NaN loss, since I didn't encounter this issue with the current code or the code before changing batches_per_epoch to 1000. So far, no other people has reported this issue. Maybe you use another machine to run?
As for evaluation, did you manually assign the cluster_map by yourself in evaluation.py? As the cluster_map I provided in the evaluation.py is only used for the uploaded trained model. If you train a model again, you need to manually assign the mapping first before evaluation.
You can try to evaluate the uploaded trained restaurant model by running evaluation.py directly. This will give you similar results as reported in paper. And you can have a look at how I assigned the aspect label to each cluster by looking at the cluster_map and the aspect.log in pre_trained_model/restaurant/.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I I had the same problem. since I used a large data（3million). Have you solved the problem yet?

I am also getting nan values
Aspect 0:
welsh:nan staff:nan bitches:nan sick":nan stolen:nan christmas:nan edward:nan genius:nan selena:nan emily:nan socks:nan 21st:nan kings:nan roof:nan incredibly:nan walmart:nan bein:nan ga:nan luckily:nan gud:nan cricket:nan reunion:nan accidentally:nan kobe:nan steak:nan fridays:nan disneyland:nan snap:nan involved:nan carry:nan security:nan delivery:nan police:nan theatre:nan prince:nan iranelection:nan sounded:nan
Aspect 1:
welsh:nan staff:nan bitches:nan sick":nan stolen:nan christmas:nan edward:nan genius:nan selena:nan emily:nan socks:nan 21st:nan kings:nan roof:nan incredibly:nan walmart:nan bein:nan ga:nan luckily:nan gud:nan cricket:nan reunion:nan accidentally:nan kobe:nan steak:nan fridays:nan disneyland:nan snap:nan involved:nan carry:nan security:nan delivery:nan police:nan theatre:nan prince:nan iranelection:nan sounded:nan
and so on....

I encountered this problem for some data. Actually, vector representation of aspects is exactly the same. Thus, ortho_reg loss is nan.


Use underscore instead hyphen in path

Why are the changes needed?

Use underscore instead hyphen in path, friendly to sql.
How was this patch tested?

[X] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Codecov Report

Merging #1000 (8b3459a) into master (7141f2f) will decrease coverage by 0.18%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master    #1000      +/-   ##
============================================
- Coverage     80.14%   79.96%   -0.19%
  Complexity       11       11
============================================
  Files           151      151
  Lines          5514     5514
  Branches        643      643
============================================
- Hits           4419     4409      -10
- Misses          712      724      +12
+ Partials        383      381       -2

Impacted Files
Coverage Δ

...n/scala/org/apache/kyuubi/events/KyuubiEvent.scala
0.00% <0.00%> (ø)

.../org/apache/kyuubi/operation/KyuubiOperation.scala
38.18% <0.00%> (-16.37%)
:arrow_down:

.../scala/org/apache/kyuubi/server/KyuubiServer.scala
53.48% <0.00%> (-2.33%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7141f2f...8b3459a. Read the comment docs.

thanks, merging to master/branch-1.3

hi @hddong there is some conflict in branch-1.3, can you send a new PR for that branch ?

@ulysses-you: Sure, I will do it in a while.

@ulysses-you: I confirmed, it is not need. The code is not in branch-1.3.

@hddong  ah, I see. thank you for the confirm !


FIX: correct previous recon-all reporting

Closes https://github.com/poldracklab/fmriprep/issues/2049
Turns out we weren't passing in the sub- identifier, which in turn made ReconAll (rightfully) assume the subject wasn't run.
This also imports the modified ReconAll interface used in smriprep.

Issue introduced in https://github.com/nipreps/niworkflows/pull/299, released in niworkflows 0.7.0, and thus affecting fMRIPrep 1.3.0+.
Can you target maint/20.0.x, and then we can just merge this forward?

untangling some more webs:
In [1]: from smriprep.interfaces import freesurfer as fs
In [2]: fs.ReconAll
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-53ff371892c6> in <module>
----> 1 fs.ReconAll

AttributeError: module 'niworkflows.interfaces.freesurfer' has no attribute 'ReconAll'

In [3]: from smriprep.interfaces.freesurfer import ReconAll
In [4]: exit

this seems hazardous given smriprep.interfaces.freesurfer exists..

Ugh. That's a nasty one.

before:
https://13616-53175327-gh.circle-artifacts.com/0/tmp/ds005/derivatives/fmriprep/sub-01.html
after:
https://13789-53175327-gh.circle-artifacts.com/0/tmp/ds005/derivatives/fmriprep/sub-01.html#Summary


fix link

What's changed, or what was fixed?

fixes broken link to console api reference

Fixes: N/A
Target Live Date: 2019-03-01

[x] This has been reviewed and approved by @kaycebasques
[ ] I have run npm test locally and all tests pass.
[x] I have added the appropriate type-something label.
[ ] I've staged the site and manually verified that my content displays correctly.

CC: @petele

Whoops!
There were 5 warnings that will prevent this PR from being merged. Please take a look, and either fix, or provide a justification for why they can't be fixed.
WARNINGS
src/content/en/tools/chrome-devtools/console/utilities.md#L191 - Common typo found: 'javascript' Should it be 'JavaScript'?
src/content/en/tools/chrome-devtools/console/utilities.md#L161 - Common typo found: 'node' Should it be 'Node'?
src/content/en/tools/chrome-devtools/console/utilities.md#L276 - Common typo found: 'implicity' Should it be 'implicitly'?
src/content/en/tools/chrome-devtools/console/utilities.md#L5 - WF Tag 'wf_updated_on' must be within the last 7 days.
src/content/en/tools/chrome-devtools/console/utilities.md#L18 - Line must be at most 100 characters

:tada: This has been pushed live to https://developers.google.com/web/


Support for 59.94Hz

Can you add support for 59.94 fps to be consistent with other outputs that are running at video rates?

This is pending the latest PR. It will likely be in the next beta release.

Latest PR from @bud3699 includes this as an option. Release coming shortly.


Add some firebase testlab tests on lower API versions

For example, firebase testlab has some API 18 and 19 devices available.  We should add those to our test matrix at least as a smoke test that we still work on those versions.
/cc @blasten - I suspect this is an easy change but you might know better

https://flutter-review.googlesource.com/c/recipes/+/9620

https://flutter-review.googlesource.com/c/recipes/+/9620


Wrong keywords in forwardingList

I noticed that in the forwarding list there are keywords "server-name" and "service-name".
In the template from the ApplicationPattern I only see "server-name", so I guess this might be wrong as I would expect this should be "server-name" everywhere...
Example:
  - forwarding-name: ServiceRequestCausesLoggingRequest
    uuid: okm-2-0-1-op-fc-bm-002
    forwarding-type: InvariantProcessSnippet
    management-requests:
      operation-client-update:
        - server-name: /v1/redirect-service-request-information
          uuid: okm-2-0-1-op-s-bm-002
      fc-port-update:
        - server-name: /v1/redirect-service-request-information
          uuid: okm-2-0-1-op-s-bm-002
      fc-port-deletion:
      operation-client-deletion:
    initiating-requests:
...
      - server-name: /v1/inform-about-release-history
        uuid: okm-2-0-1-op-s-bs-003
      - server-name: /v1/inform-about-release-history-in-generic-representation
        uuid: okm-2-0-1-op-s-bs-004
      - service-name: /v1/regard-application
        uuid: okm-2-0-1-op-s-is-001
      - service-name: /v1/disregard-application
        uuid: okm-2-0-1-op-s-is-002
      - service-name: /v1/list-applications
        uuid: okm-2-0-1-op-s-is-003
      - service-name: /v1/regard-updated-link
        uuid: okm-2-0-1-op-s-is-004
    consequent-requests:
      - client-name: ExecutionAndTraceLog://v1/record-service-request
        uuid: okm-2-0-1-op-c-bs-eatl-1-0-0-000

Changed as proposed.


IndexedDB - SELECT + where EXISTS(SELECT ...) - Error

Hi, I'm using AlaSql with IndexedDB as selected engine, all my tables were succesfully created and stored. If I do a simple SELECT in any of those tables, everything works well...
BUT, when I try to use an EXISTS, then an error occurs.
My select query is as follow:
SELECT A.* FROM AppMenu A     WHERE EXISTS(        SELECT 1 FROM AppMenuPerm X             WHERE X.IdMenu = A.IdMenu             AND (X.IdConta = 166 OR X.IdGrupo in (5, 4))      )      ORDER BY A.Order, A.Description
this is the error that occurs:

Uncaught TypeError: Cannot read property 'data' of undefined
at T.Query.eval [as wherefn] (eval at X.Select.compileWhere (alasql.min.js?v=19.08-Q2_29-08-19_7180.16207:formatted:28022)

Someone can point me what is wrong or some fix to this?
EDIT:
Actually, at least for me, not even a very simple EXISTS works...
SELECT Desc FROM tab1 WHERE EXISTS(SELECT 1 FROM tab2 WHERE tab1.IdTab = tab2.IdTab)
AND, apparently, using localStorage instead of IndexedDB, the EXISTS seems to work, but I can't use localStorage because it's slow and doesn't have sufficient amount of storage space.
(By the way, in the Wiki page, the page about Where is saying to use EXIST instead of EXISTS, but then it trhows another error, I think that it is a typo on the Wiki page)
Details:
Version: AlaSQL v0.4.11
Engine: IndexedDB
This select is executed with: alasql.promise(sql, null).then(...

Actually, at least for me, not even a very simple EXISTS works...
 Thank you for reporting this.

Hi @mathiasrw I want to start contributing to this project . Can I work on this issue ?

@biddwan09 For sure!


Update app.py

Fixes # .
Changes proposed in this pull request:

#124

Coverage remained the same at 100.0% when pulling 0bfdd8f41b68bd67e2c1707315b9391e7ebd09a4 on vishnubhagwan:patch-1 into 10a0c15f6fb3208ae4de9b59b1ff82058af55b6c on zalando:master.


Padlock: SHA1/256/512 code is never used

It seems that SHA1 and SHA256 VIA Padlock functions were implemented in assembly and defined as C functions, but never exported in an engine. That's why these functions are never hardware-accelerated and assembly implementation is a dead code.
https://github.com/openssl/openssl/blob/38fc02a7084438e384e152effa84d4bf085783c9/engines/e_padlock.c#L221-L224
Moreover, the assembly contains implementation of sha512, which is not even mentioned in the .c file.
https://github.com/openssl/openssl/blob/38fc02a7084438e384e152effa84d4bf085783c9/engines/asm/e_padlock-x86.pl#L579
As seen with current master, both openssl speed -evp sha256 and openssl speed -evp sha256 -engine padlock produce the same unimpressive result:
$ openssl speed -evp sha256 -engine padlock
Engine "padlock" set.
Doing sha256 for 3s on 16 size blocks: 500166 sha256's in 3.00s
Doing sha256 for 3s on 64 size blocks: 317539 sha256's in 2.99s
Doing sha256 for 3s on 256 size blocks: 169713 sha256's in 3.00s
Doing sha256 for 3s on 1024 size blocks: 57075 sha256's in 3.00s
Doing sha256 for 3s on 8192 size blocks: 7926 sha256's in 3.00s
Doing sha256 for 3s on 16384 size blocks: 3997 sha256's in 3.00s
version: 3.0.7
built on: Thu Jan 19 20:31:42 2023 UTC
options: bn(64,32)
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/home/user/openssl/openssl-3.0.7=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2
CPUINFO: OPENSSL_ia32cap=0x4181a7c9bfff:0x0
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
sha256            2667.55k     6796.82k    14482.18k    19481.60k    21643.26k    21828.95k

IMO，the padlock code are all c and assembly language, don't need any external binary.it can be embedded to sha*-586.pl/sha*-x86_64.pl/aes-586.pl and aes-x86_64.pl.

IMO，the padlock code are all c and assembly language, don't need any external binary.it can be embedded to sha*-586.pl/sha*-x86_64.pl/aes-586.pl and aes-x86_64.pl.

Yeah, this is basically an RFE. And IMO the proper way would be to use the padlock routines in the existing assembly code based on the CPU detection and not require an engine. Also if there is a reason why these cannot be integrated but must be loadable separately, they should be implemented in a provider and not an engine because we aren't likely to take any RFEs in engines. They are deprecated.

I'm still wondering if the padlock hardware is still manufactured...  We have big maintenance issues with hardware that
is not widely used, and what happens when we want to fix anything in this code, but cannot test ourselves, and in a way
the padlock engine is a good example, as it was obviously broken all the time, and never used for years, before you
found some major issues in the implementation of that engine.

as https://github.com/openssl/openssl/pull/5145 mentioned，the zhaoxin company probably is still manufacturing hardware support padlock.

@bernd-edlinger, as @zzl360 mentioned, Zhaoxin CPUs, which are pretty recent and actively developed as far as I can say, support Padlock, including SM3/SM4 algorithms which were proposed in #8706.
I'm re-purposing old thin clients on VIA Eden Esther ULV, the CPU is slow and manages to get only 7-8MB/s of unaccelerated aes-128-gcm for example, while having a gigabit NIC, that's why hardware acceleration is very useful on this device. If one need access to this device, I can setup SSH and provide it indefinitely, or I can donate several devices to you.

it was obviously broken all the time, and never used for years

Could it be that the offending code with ifndef AES_ASM just wasn't activated by default until recently and everything eventually worked correctly? I can hardly believe such a bug hasn't been spotted by Zhaoxin developers.
https://github.com/openssl/openssl/blob/55ff8fb4ed4d48cb819ff5ae5d74cc08256e7ed1/engines/e_padlock.c#L649-L654
Current engine also does not support AES-GCM, which is one of the most widespread cipher on the internet. This could be easily hacked-up by reusing SPARK T4 code, but at least there's no code for this mode while there is for SHA, it's just not used.

@ValdikSS I'm not familar with engine in openssl.is it possible that you must manual enable padlock engine to enable hardware acceleration on VIA machine.but most user don't konw it.as https://github.com/openssl/openssl/issues/20167#issuecomment-1408253075 suggest,this will enable hardware acceleration by default on VIA machine.

@zzl360, yes, engine needs to be activated either manually or in the configuration file. Right now it works pretty good overall, but without SHA or AES-GCM support.

Could it be that the offending code with ifndef AES_ASM just wasn't activated by default until recently and everything eventually worked correctly? I can hardly believe such a bug hasn't been spotted by Zhaoxin developers.

That is a combination of several bugs.  Most importantly starting with 1.1.1 the AES_ASM macro is NEVER defined
when the engines are compiled, but only when the AES C files are compiled.  That broke the engine first.
Then we have 87bea6550ae0dda7c40937cff2e86cc2b0b09491 which was merged to 1.1.1 but not in master.
That almost fixed the engine, except the issue in the padlock_key_bswap, modulo the AES_CONST_TIME issue.

I searched the internet and found some information.
Let's start with the issues. REP XSHA1 / REP XSHA256 always finalize SHA hashing and is not suitable for common sha_init / sha_update / sha_update / … / sha_finish calls. This is what is implemented under padlock_sha1_oneshot / padlock_sha256_oneshot functions.
VIA Nano family has added partial class of SHA functions which do not perform finalizing. This is done by supplying 0xFFFFFFFF in EAX before calling the command, and is implemented under padlock_sha1_blocks / padlock_sha256_blocks in OpenSSL (and doesn't include CPUID checks by the way). I could not find the documentation, but it is implemented in PadlockSDK_3.1_Release_20090121.zip (full source available). My device does not support partial hashing.
However, it's possible to do partial hashing on unsupported hardware with a page fault hack, proposed by Andy Polyakov and implemented by Michal Ludvig, but that would be difficult to implement in multi-thread environment and IMO not worth it.

PHE saves its current state into a memory on every process switch and as well on any page fault that occurs during the run. This state includes number of bytes hashed and an intermediate result that could be used as an initial value for subsequent rounds. So far so good. The only remaining question is how to trigger a context switch or a page fault at the place we need. Solution: mmap(2) two or more pages, mprotect(2) the last one to deny all access (PROT_NONE). This creates an inaccessible piece of memory exactly at the place we need. Now we put all our input data just before this barrier and engage PHE. However we'll tell it to hash slightly more data than we put into the buffer. With these instructions PHE will crunch all our input and attempt to hash some more.

Now, about SHA512. I could not find any information about it and it is not present in the SDK. Wikipedia X86_instruction_listings article even mentions it in the list of undocumented instructions:

Supported by OpenSSL as part of its VIA PadLock support, but not documented by the VIA PadLock Programming Guide

Now the patches. There were several patches in the mailing list for 1.0.x and 0.9.8 which haven't been merged:

<EMAIL_ADDRESS><EMAIL_ADDRESS>

For the record, Zhaoxin maintains their own fork of OpenSSL.

@bernd-edlinger Concerning the hardware -- yes it is very very old but there are these iBase stand-alone firewall appliances that use them as well (and some ITX mainboards) and making them even more performant would safe us from a little bit of trash. I will be happy to provide some hardware.


Jammed

I have the lock setup in my HomeKit app. When trying to operate (lock/unlock) the status changes to Jammed. I was unable to figure out what the secret key is. The QR reader needs an image but what QR am I using?

QR code can be obtained from Sesame app.
This is official document https://ameblo.jp/candyhouse-inc/entry-12653318542.html.
Be sure to choose Owner key or Manager key in order to control over Wi-Fi.
Good luck :+1:

If issue still exists, please open new issue.


terminal quick fix contribution: list return types

Testing #166762
the provider returns TerminalQuickFix. I first didn't see the subtypes TerminalQuickFixCommandAction and TerminalQuickFixOpenerAction.

define TerminalQuickFix as type TerminalQuickFix = TerminalQuickFixCommandAction |  TerminalQuickFixOpenerAction
align the naming: TerminalQuickFix -> TerminalQuickFixAction

I was told that type= should be avoided in the API.

Fix and Action are synonymous, so we should get rid of Action


Allow editing of allow-axfr-from

Powerdns allows specifying allow-axfr per domain:
https://doc.powerdns.com/md/authoritative/domainmetadata/#allow-axfr-from
It would be nice to either be able to specify possible destinations or have a checkbox ( AUTO-NS yes / no).

Hello,
This request is stale for too long. Regarding our policy to cleanup this project, I'm closing it. Feel free to open a new issue later.
We have #1133 issue opened for future options arround domains metadata, not handled yet.
Regards
PowerDNS Admin Team
See https://github.com/PowerDNS-Admin/PowerDNS-Admin/issues/1153 for the policy


Clear individual session/local storage items instead of clearing all

The useClearSessionStorage and useClearLocalStorage hooks weren't a very good idea, because unrelated components might use these storages and inadvertently clear each others' storage. For example this made the <BrowserSupport.WarningPortal> component not work correctly, because the logout event handler in LocalStorageKeystore would clear session storage immediately after <BrowserSupport.WarningPortal> set its override flag.
This refactors the useState hooks to instead also return a clear function, which can be called with no arguments to reset the state of the hook and remove the storage entry. There's also a useClearStorages() hook which composes several such clear functions into one, so that all of a component's state can be cleared easily without accidentally clearing other components' state as well.
One potential issue with this is that the appToken and sessionState storage items are no longer reset when the user logs out in a different tab, because LocalStorageKeystore no longer resets them.

The issue with the <BrowserSupport.WarningPortal> component appears to have been resolved.
However, after some test, we noticed the recurrence of the following error if we 1) login/signup, 2) navigate to account settings and 3) logout. It's uncertain whether this is related to the previous issue or if it should be treated as a separate matter.
Uncaught runtime errors:
×
ERROR
Cannot destructure property 'webauthnCredentialCredentialId' of 'api.getSession(...)' as it is null.
TypeError: Cannot destructure property 'webauthnCredentialCredentialId' of 'api.getSession(...)' as it is null.
    at Home (http://localhost:3000/static/js/src_pages_AccountSettings_AccountSettings_tsx.chunk.js:716:37)
    at renderWithHooks (http://localhost:3000/static/js/bundle.js:47322:22)
    at updateFunctionComponent (http://localhost:3000/static/js/bundle.js:50204:24)
    at beginWork (http://localhost:3000/static/js/bundle.js:51916:20)
    at HTMLUnknownElement.callCallback (http://localhost:3000/static/js/bundle.js:36914:18)
    at Object.invokeGuardedCallbackDev (http://localhost:3000/static/js/bundle.js:36958:20)
    at invokeGuardedCallback (http://localhost:3000/static/js/bundle.js:37015:35)
    at beginWork$1 (http://localhost:3000/static/js/bundle.js:56889:11)
    at performUnitOfWork (http://localhost:3000/static/js/bundle.js:56136:16)
    at workLoopSync (http://localhost:3000/static/js/bundle.js:56059:9)

Thanks! This seems to be fixed now by wrapping the <AccountSettings> route with <PrivateRoute>. This overlaps with commit 4bdb7389 in PR #127, but the branches seem to merge without conflicts since the change is identical in both branches.


smoothScrollToStep & scrollToStep are not working in onCreate ?

Thanks for the awesome library.
I'm having 10 steps in my app. Current layout only shows 4 steps. I'm assigning currentcount via onCreate(). But I tried smoothScrollToStep to 6. But it is not scrolling to 6th step...
        StatusViewScroller mStatus = findViewById(R.id.status);
        List<String> statusArray = Arrays.asList(getResources().getStringArray(R.array.indiv));
        mStatus.getStatusView().setStatusList(statusArray);
        mStatus.getStatusView().setStepCount(statusArray.size());
        mStatus.getStatusView().setCurrentCount(6);
        mStatus.smoothScrollToStep(6);

It is a bug. I looked at the code and the problem is that it the scroll positions are using drawingData which only available after onLayout and it also resets after changing a property.
So it only works after onLayout has been called and then also until a property is modified.
(eg if you have a "Next" button then you need to do the scrolling first and then call setCurrentCount after that)

Can you do an example i have the same trouble! thanks!


compactor: Enable vertical compaction without replica labels.

We use vertical compaction now on prod and it works super smooth. We do that for offline dedup of exactly the sample samples replicas, but it actually saves a lot of troubles on overlaps. Compactor just easily deals with that.
I would propose to allow enabling vertical compaction without any replicas dedup (: Thoughts? @brancz @metalmatze @kakkoyun

Generally happy with this.
There was a bug iirc where blocks with no labels were rejected or something along those lines. If this has been resolved (iirc some fake label was added or was discussed to be added), then we should be ok with this.

Still valid.


Error when generating a PDF

Describe the bug
Unable to generate a PDF just get a spinning wheel
To Reproduce
Click print icon with A3 Portrait PDF 96 DPI,
Expected behavior
A PDF is generated.
Screenshots

Desktop (please complete the following information):

OS: Windows
Browser: Chrome
Version: 103

Additional context
We are generating many layers behind the scenes using various plugins and some generated layers ourselves.
Is this something we need to fix or something the plugin can handle?
Error text:
mapbox-gl.js:31 Error: layers[140].type: expected one of [fill, line, symbol, circle, heatmap, fill-extrusion, raster, hillshade, background], "sky" found
    at Object.Bn [as emitValidationErrors] (mapbox-gl.js:31:133534)
    at Ue (mapbox-gl.js:35:112388)
    at i._load (mapbox-gl.js:35:114931)
    at mapbox-gl.js:35:114750
    at timer (zone.js:2405:1)
    at _ZoneDelegate.invokeTask (zone.js:406:1)
    at Object.onInvokeTask (core.mjs:25595:1)
    at _ZoneDelegate.invokeTask (zone.js:405:1)
    at Zone.runTask (zone.js:178:1)
    at invokeTask (zone.js:487:1)```

Going to see if I can somehow find out which layers are missing the type.

Ah ok my bad 'sky' isn't a valid type.


While robot is moving and above ground plane ,It is detecting static objects as moving objects ,how can i solve this issue ?

Hi, I'm using this pkg with a simulated drone, When the drone is on the ground plane and in idle condition this pkg works perfectly, but after takeoff, due to the robot motion it is detecting every object as static, I have tried reducing the speed of the drone and still not working.is there is any way /suggestion to make it working ?

did you find any solution to this?

I apologize for the late response - I hope you already found a solution. Like you found out, this package was mainly designed for ground-based operation. Any contribution aiming at improving it also for flight-based operation is welcome.


Check for room when split-moving windows, fix some other related stuff

(Details are in the commit messages.)
On a somewhat related note, I noticed WinNewPre is fired even before win_split_ins checks for room or attempts to create any window (that is, WinNewPre may be fired, but not WinNew if E36 happens).
Is that intended? I guess it might be as it seems win_split_ins may resize windows before giving E36 actually it's only when failing to alloc a new window, and it's documented as being "before commands that modify window layout".

yeah, we may need to amend the documentation, that because of E36 WinNew may not be triggered.

Seems good :+1:
Glancing at the code, I can see a few other potential issues with how WinNewPre is currently handled (e.g: it's fired while curwin might be invalid and has no entry in the window list or frame, plus switching windows and tabpages in the autocmd could be problematic). I'll probably try to fix those things and the documentation in a different PR when I have the time.

thanks. Let me know when it's ready to review and I'll have a closer look.

Thanks , looks good


Doubts about the sample poison

I would be very grateful if you could guide me to run the example poison.I did not find deletion related operations in the poison folder. In other words, how should I get repaired_model.hdf5？My email is<EMAIL_ADDRESS>and I would be grateful for any guidance from you.

We are currently working on a few jupyter notebooks to show how the code should be used. Stay tuned!

OK，thanks a lot.


Add fields Project/Separator

Hello,
This PR adds 2 custom variables to plugin:

g:header_field_project
g:header_field_last_separator

And also minor a modification to b:field_timestamp content from "Date" -> "File Created"
Feel free to comment/merge and/or reject it.
KR

Sorry for the late reply and thank you for your pr!
Moving the "File Created" field above the author field looks a little odd at first but I think it will match the modifier and modified date fields order.
I am not sure about the "separator" field, content should be adjustable or can be merged with the "copyright" field in the future since both do similar jobs.

At the end I needed to revert the file created field name back to "Date" and the field orders. Because it can't update the existing headers and that will make some trouble for old users.
All other features you provided are kept.

Thank you!


Feature Request: Option to hide overview content for participants

This is no longer relevant as all sensitive overview content has been removed


source code is not complete

Hello ,
I am reading your source code,
awared of there is no code provided in app/actions and etc.
It is not finished yet?

hey @zhangk2000, this is a boilerplate, app/actions is empty because the actions depend on your project, the boilerplate just defines a folder for you to place them in a structured way


Custom prompt to RetrievalQAWithSourcesChain ?

How can I add custom prompt to:
qa_chain = load_qa_with_sources_chain(llm, chain_type="stuff",)
qa = RetrievalQAWithSourcesChain(combine_documents_chain=qa_chain, retriever=docsearch.as_retriever())

There is no prompt= for this class...

+1 Encountering the same issue here

@VladoPortos can you specify?
For me it works without any problems; there are 2 prompts ("prompt" for the FewShot prompt, "document_prompt" for the inserted documents. E.g. one example I use (and which works flawlessly):

GERMAN_QA_PROMPT = PromptTemplate(template=german_few_shot_doc_prompt, input_variables=["summaries", "question"])
GERMAN_DOC_PROMPT = PromptTemplate(
    template="Inhalt: {page_content}\nQuelle: {source}",
    input_variables=["page_content", "source"])

qa_chain = load_qa_with_sources_chain(llm, chain_type="stuff",
                                      prompt=GERMAN_QA_PROMPT,
                                      document_prompt=GERMAN_DOC_PROMPT)
chain = RetrievalQAWithSourcesChain(combine_documents_chain=qa_chain, retriever=retriever,
                                     reduce_k_below_max_tokens=True, max_tokens_limit=3375,
                                     return_source_documents=True)

(define german_few_shot_doc_prompt as you wish, see the original prompts here - https://github.com/hwchase17/langchain/blob/85dae78548ed0c11db06e9154c7eb4236a1ee246/langchain/chains/qa_with_sources/map_reduce_prompt.py#L13 ).
You probably missed the constructors because they are somewhat hidden in the loading.py file here:
https://github.com/hwchase17/langchain/blob/85dae78548ed0c11db06e9154c7eb4236a1ee246/langchain/chains/qa_with_sources/loading.py#L48

Is there a way to add memory to this?

Thanks @jpdus that seems to be the way, thanks again ! :)
Also, same question like @blazickjp is there a way to add chat memory to this ?
Currently, I was doing it in two steps, getting the answer from this chain and then chat chai with the answer and custom prompt + memory to provide the final reply. But that's two calls to API :-/

Also, same question like @blazickjp is there a way to add chat memory to this ?
Currently, I was doing it in two steps, getting the answer from this chain and then chat chai with the answer and custom prompt + memory to provide the final reply. But that's two calls to API :-/

Well, this is not that easy because the DocumentQA/DocumentCombine chains have customized prompts that imho don't really work great for a conversation chain.
In my opinion the correct solution would be to use a Chat Conversation Agent that utilizes the RetrievalQAWithSourcesChain as a tool.
Just tried it and works as expected:
from langchain.agents import Tool
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent
from langchain.agents import AgentType

tools = [
    Tool(
        name = "Search",
        func=chain,
        description="useful to answer factual questions"
    ),
]
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
llm=ChatOpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, verbose=True, memory=memory)
agent_chain.run(input="Some Question")
...
agent_chain.run(input="Some follow-up Question")

where chain is the RetrievalQAWithSourcesChain as defined above.
Hope that helps!

@jpdus Thank you! I'm still very confused on the design aspect of this; why an agent would be needed for something that feels very much like a chain.

@jpdus Thank you! I'm still very confused on the design aspect of this; why an agent would be needed for something that feels very much like a chain.

Well it is confusing at first but makes sense on a second thought - you basically have two chains (one for the conversation and one for the document QA) and the desired output is not strictly sequential (e.g. the model has to have access to both, the conversation history and the document information, but it´s not always required in the same order).
The agent can then decide on when and how to use the retrieval to best answer the user query. I can´t really imagine any scenario where a sequential/fixed chain would work better for that use case?

Thanks @jpdus I will give it a try tomorrow :)

Well, but I stand somehow corrected, in the meantime there is already another chain that probably does what you want without invoking an Agent (even if I'd probably prefer that because its way easier to extend); see:
https://python.langchain.com/en/latest/modules/chains/index_examples/chat_vector_db.html#conversationalretrievalchain-with-question-answering-with-sources

But is there a way to customize promt a little for this chain ? It's not mentioned there. I would like to add some custom personality or just minor adjust the prompt it is using.

Sure, you can customize the QA prompt the same way as written above an the "condense_prompt" for the "question generator" (see example in the documentation) can be changed as well (default one can be found here: https://github.com/hwchase17/langchain/blob/85dae78548ed0c11db06e9154c7eb4236a1ee246/langchain/chains/conversational_retrieval/prompts.py#L4 ).
However if you really want to add custom personality and leverage Chat System prompts, you should imho go with the agent for more flexiblity and better customization (see https://github.com/hwchase17/langchain/blob/85dae78548ed0c11db06e9154c7eb4236a1ee246/langchain/agents/conversational_chat/base.py#L59 ).

 qa_chain = load_qa_with_sources_chain(llm, chain_type="stuff",
                                      prompt=GERMAN_QA_PROMPT,
                                      document_prompt=GERMAN_DOC_PROMPT)
 chain = RetrievalQAWithSourcesChain(combine_documents_chain=qa_chain, retriever=retriever,
                                     reduce_k_below_max_tokens=True, max_tokens_limit=3375,
                                     return_source_documents=True)

from langchain.agents import Tool
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent
from langchain.agents import AgentType

tools = [
    Tool(
        name = "Search",
        func=chain,
        description="useful to answer factual questions"
    ),
]
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
llm=ChatOpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, verbose=True, memory=memory)
agent_chain.run(input="Some Question")
...
agent_chain.run(input="Some follow-up Question")

If I try your approach I will end up with the error "NotImplementedError: Saving not supported for this chain type."
The problem is here in "langchain/chains/base.py", line 45, in _chain_type, which throws, none of the chains like StuffDocumentsChain or RetrievalQAWithSourcesChain inherit and implement that property.
If I create derived classes from those two above with the property defined, the agent behaves quite strangely.
@jpdus does your approach still work, for the current version 0.0.157?

Hey @jpdus , not to derail the conversation here (which is great). But is there a way to use Structured Response Parser with load_qa_with_sources_chain?

 qa_chain = load_qa_with_sources_chain(llm, chain_type="stuff",
                                      prompt=GERMAN_QA_PROMPT,
                                      document_prompt=GERMAN_DOC_PROMPT)
 chain = RetrievalQAWithSourcesChain(combine_documents_chain=qa_chain, retriever=retriever,
                                     reduce_k_below_max_tokens=True, max_tokens_limit=3375,
                                     return_source_documents=True)

from langchain.agents import Tool
from langchain.memory import ConversationBufferMemory
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent
from langchain.agents import AgentType

tools = [
    Tool(
        name = "Search",
        func=chain,
        description="useful to answer factual questions"
    ),
]
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
llm=ChatOpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, verbose=True, memory=memory)
agent_chain.run(input="Some Question")
...
agent_chain.run(input="Some follow-up Question")

If I try your approach I will end up with the error "NotImplementedError: Saving not supported for this chain type."
The problem is here in "langchain/chains/base.py", line 45, in _chain_type, which throws, none of the chains like StuffDocumentsChain or RetrievalQAWithSourcesChain inherit and implement that property.
If I create derived classes from those two above with the property defined, the agent behaves quite strangely.
@jpdus does your approach still work, for the current version 0.0.157?

I am also facing this issue. I tried this on latest version: 0.0.162

Tried to run the chain like this instead:
`tools = [Tool(func=chain.run, description=db_desc, name='Internal DB')]
conversational_agent = initialize_agent(
agent='chat-conversational-react-description',
tools=tools,
llm=llm,
verbose=True,
max_iterations=4,
early_stopping_method="generate",
memory=memory,
)`
But I'm receiving a ValueError: run not supported when there is not exactly one output key. Got ['answer', 'sources'].
So, I'm getting back an answer and a source but the agent can't handle it.

@drahoslavzan @hetthummar @olaf-hoops you are right, the code doesn't work anymore in the current version (0.0.162 as of now).
There were multiple breaking changes and i can't exactly figure out which one broke the approach ( #3913 updated related docs - @vowelparrot maybe you can point out what was the root issue here?).
You have to change the tool definition to the following to get it working again:
tools = [
    Tool.from_function(
        name = "Search",
        func=chain.__call__, #chain doesn't work anymore because of some inspection issue and there is no call function
        coroutine=chain.acall, #if you want to use async
        description="useful to answer factual questions"
    ),
]

Hey @jpdus , not to derail the conversation here (which is great). But is there a way to use Structured Response Parser with load_qa_with_sources_chain?

There surely is some way - but I wouldn't recommend it because there is some custom parsing logic in the RetrievalQAWithSourcesChain, see
https://github.com/hwchase17/langchain/blob/2ceb807da24e3ad7f04ff79120842982f341cda8/langchain/chains/qa_with_sources/base.py#L131
If you want to parse/transform the answer, you should probably combine the RetrievalQAWithSourcesChain with some custom Transformation Chain ( see https://python.langchain.com/en/latest/modules/chains/generic/transformation.html ) and apply the output parser aftererwards.
But this depends on the exact use case.....

Hello @jpdus thanks for the examples above. Have you tried running this in async? For me it just hangs.
I'm trying to run it with:  results = await qa_chain.acall(inputs=data)
I also defined an async callback StreamingHandler to stream the results. Maybe that's blocking the execution somehow?:
'from uuid import UUID
from langchain.callbacks.base import AsyncCallbackHandler
from langchain.schema import LLMResult
from typing import Any, Dict, List, Optional
import socketio
from datetime import datetime
class StreamingHandler(AsyncCallbackHandler):
def init(self, sio: socketio.AsyncServer, session_id, namespace: str, color: Optional[str] = None) -> None:
"""Initialize callback handler."""
self.sio = sio
self.session_id = session_id
self.namespace = namespace
self.color = color
print("Initialized StreamingHandler with session_id:", self.session_id)
# This will stream the tokens to the client
async def on_llm_new_token(self, token: str, **kwargs) -> None:
    emit_time = datetime.now().strftime('%M:%S.%f')[:-3]
    print("Emitting new_token event at:", token, emit_time)
    await self.sio.emit('new_token', {"new_token": token}, namespace="/results", room=self.session_id)

# Signal to the client that generation is starting
async def on_llm_start(self, serialized: Dict[str, Any], prompts: List[str], *, run_id: UUID, parent_run_id: UUID | None = None, **kwargs: Any) -> Any:
    await self.sio.emit('first_token', namespace = "/results", room=self.session_id)

# Signal to the client that generation is ending
# async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> Any:
#     await self.sio.emit('llm_end', namespace = "/results", room=self.session_id)

async def catch_all(*args, **kwargs):
    pass

on_agent_action = on_agent_finish = on_chain_end = on_chain_error = on_chain_start = on_llm_end = on_llm_error = on_text = on_tool_end = on_tool_error = on_tool_start = catch_all'

@tomatefarcie123 Yes, I tried my example above in async and it works. Please note that async execution and streaming are two related but different things and that not all chains work with streaming tokens. I would advise to first try out a non-streaming async version and/or a basic streaming LLM with your custom streaming handler.

Thanks for your reply. That streaming worked in sync mode with the RetrievalsQAWithSourcesChain but as you suggest I might turn it off for now until I get the rest of the async code to work. Cheers!

The source data was being carried right into the last observation by the agent and then dropped in formulating the answer even though return-source-documents is True in initialize_agent and tyhe tool was built with VectorStoreQAWithSoucesTool. I kluged around the problem by setting return_intermediate_steps True in initialize_agent and then mining the souces from the intermediate results with:
    try:
        data=json.loads(result["intermediate_steps"][-1][-1])
        print(data["sources"])
    except:
        print("sources not available",result)

I was also curious about this. I want to generate two answers, one crisp and informative and the other a bit sweeter and polite version of  the above answer. How do you think I can accomplish this. I want the chain to also quote the sources at the end.

@dsantiago How would I control the final prompt based on that example?

qa_chain = create_qa_with_sources_chain(ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613"))

doc_prompt = PromptTemplate(
    template="Content: {page_content}\nSource: {source}",
    input_variables=["page_content", "source"],
)

final_qa_chain = StuffDocumentsChain(
    llm_chain=qa_chain,
    document_variable_name="context",
    document_prompt=doc_prompt,
)

retrieval_qa = RetrievalQA(
    retriever=vectorstore.as_retriever(), combine_documents_chain=final_qa_chain
)

print(retrieval_qa.run("What Ukrainian dishes do we have?"))

Answer
{
  "answer": "Some popular Ukrainian dishes include borscht with pork ribs, varenyky (dumplings), holubtsi (stuffed cabbage rolls), salo (cured pork fat), kovbasa (sausage), and deruny (potato pancakes).",
  "sources": ["/recipes/borscht-with-pork-ribs"]
}

The problem here is that the output is adding information that is not in the vector db. I should be able to control that with a prompt on the RetrieverQA, but doesn't seem like I can do that.

The source data was being carried right into the last observation by the agent and then dropped in formulating the answer even though return-source-documents is True in initialize_agent and tyhe tool was built with VectorStoreQAWithSoucesTool. I kluged around the problem by setting return_intermediate_steps True in initialize_agent and then mining the souces from the intermediate results with:
    try:
        data=json.loads(result["intermediate_steps"][-1][-1])
        print(data["sources"])
    except:
        print("sources not available",result)

Hey tevslin,
I am using the following to get the answers:
qa_chain = RetrievalQAWithSourcesChain.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=db.as_retriever(),
chain_type_kwargs=chain_type_kwargs,
return_source_documents=True,
verbose=True
)
how can I use it with initialize_agent to get the sources please?