[HEALTH] 4.15 Health Check please don't merge this

Description

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):
How Has This Been Tested?

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1297

@blueorangutan test matrix

@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✖debian. JID-1302

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1310

@blueorangutan test matrix

@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-1642)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38222 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1642-kvm-centos7.zip
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-1643)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41348 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1643-vmware-67u3.zip
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Trillian test result (tid-1641)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 53035 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1641-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_deploy_kubernetes_cluster
Error
3608.60
test_kubernetes_clusters.py

test_02_deploy_kubernetes_ha_cluster
Error
0.04
test_kubernetes_clusters.py

test_04_deploy_and_upgrade_kubernetes_cluster
Error
3612.75
test_kubernetes_clusters.py

test_05_deploy_and_upgrade_kubernetes_ha_cluster
Error
0.03
test_kubernetes_clusters.py

test_06_deploy_and_invalid_upgrade_kubernetes_cluster
Error
0.03
test_kubernetes_clusters.py

test_07_deploy_and_scale_kubernetes_cluster
Error
0.03
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
35.60
test_kubernetes_clusters.py

test_01_scale_vm
Failure
11.29
test_scale_vm.py

Trillian test result (tid-1650)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40702 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1650-kvm-centos7.zip
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1320

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1327

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-1664)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 59269 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1664-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 80 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_deploy_kubernetes_ha_cluster
Error
3617.91
test_kubernetes_clusters.py

test_04_deploy_and_upgrade_kubernetes_cluster
Error
0.03
test_kubernetes_clusters.py

test_05_deploy_and_upgrade_kubernetes_ha_cluster
Error
0.03
test_kubernetes_clusters.py

test_06_deploy_and_invalid_upgrade_kubernetes_cluster
Error
0.03
test_kubernetes_clusters.py

test_07_deploy_and_scale_kubernetes_cluster
Error
0.03
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
45.78
test_kubernetes_clusters.py

test_01_scale_vm
Failure
19.51
test_scale_vm.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
419.61
test_vpc_redundant.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1338

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✔debian. JID-1363

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✖debian. JID-1370

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1371

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-1715)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42285 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1715-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_scale_vm
Failure
18.52
test_scale_vm.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
412.86
test_vpc_redundant.py

Trillian test result (tid-1723)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41709 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1723-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-1724)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43365 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1724-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
318.70
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
321.86
test_vpc_redundant.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1373

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1375

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-1726)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43318 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1726-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan test matrix

@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-1731)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46540 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1731-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 79 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_reboot_ssvm
Failure
25.72
test_ssvm.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1413

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1423

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-1768)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46154 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1768-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_add_delete_kubernetes_supported_version
Error
1806.17
test_kubernetes_supported_versions.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
341.71
test_vpc_redundant.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1449

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1476

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1533

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-1990)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39495 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1990-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_scale_vm
Failure
12.32
test_scale_vm.py

test_05_rvpc_multi_tiers
Failure
366.74
test_vpc_redundant.py

Trillian test result (tid-1991)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40045 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t1991-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_domain_service_offerings.py
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1548

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1549

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1555

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1566

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2060)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39862 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2060-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_scale_vm
Failure
18.49
test_scale_vm.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
406.43
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Failure
397.28
test_vpc_redundant.py

Trillian test result (tid-2062)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44866 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2062-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_reboot_ssvm
Failure
22.38
test_ssvm.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1574

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-2118)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34212 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2118-kvm-centos7.zip
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✖debian. JID-1580

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian.  I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1620

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2219)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47180 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2219-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

ContextSuite context=Test01DeployVM>:setup
Error
0.00
test_vm_life_cycle.py

ContextSuite context=Test02VMLifeCycle>:setup
Error
0.00
test_vm_life_cycle.py

ContextSuite context=Test03SecuredVmMigration>:setup
Error
0.00
test_vm_life_cycle.py

Trillian test result (tid-2226)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 53011 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2226-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 78 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_reboot_ssvm
Failure
41.87
test_ssvm.py

ContextSuite context=Test01DeployVM>:setup
Error
0.00
test_vm_life_cycle.py

ContextSuite context=Test02VMLifeCycle>:setup
Error
0.00
test_vm_life_cycle.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
653.76
test_vpc_redundant.py

VM lifecycle regression was in test, fixed  in https://github.com/shapeblue/Trillian/commit/3d030dedd3a0ce8b5a6a280707edd9437c43a29c
@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1625

one of the test envs was running for 2d, destroyed for maintenance:
pr4131-t2218-xenserver-71

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✔debian. JID-1636

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1639

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian test result (tid-2260)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 53366 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2260-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
407.35
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Error
313.03
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3903.68
test_vpc_redundant.py

@blueorangutan package

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1663

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2275)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44277 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2275-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 78 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
502.13
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3827.00
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3827.03
test_vpc_redundant.py

test_05_rvpc_multi_tiers
Error
0.01
test_vpc_redundant.py

ContextSuite context=TestVPCRedundancy>:teardown
Error
0.01
test_vpc_redundant.py

Trillian test result (tid-2274)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44325 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2274-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_add_delete_kubernetes_supported_version
Error
1807.30
test_kubernetes_supported_versions.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Failure
350.91
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3846.82
test_vpc_redundant.py

@blueorangutan package

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian.  I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1677

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✖debian. JID-1679

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✔debian. JID-1682

re-kicked centos7 pkg manually

@blueorangutan test centos7 vmware-67u3

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian.  I'll keep you posted as I make progress.

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✔debian. JID-1746

Manually rekicked pkging for centos7

Pkging OK at backend for both centos7 and ubuntu, kicked test matrix at backend manually

Trillian test result (tid-2401)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 58290 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2401-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_deploy_kubernetes_ha_cluster
Error
97.64
test_kubernetes_clusters.py

test_04_deploy_and_upgrade_kubernetes_cluster
Error
86.86
test_kubernetes_clusters.py

test_05_deploy_and_upgrade_kubernetes_ha_cluster
Error
0.04
test_kubernetes_clusters.py

test_06_deploy_and_invalid_upgrade_kubernetes_cluster
Error
72.59
test_kubernetes_clusters.py

test_07_deploy_and_scale_kubernetes_cluster
Error
70.56
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
105.86
test_kubernetes_clusters.py

test_01_scale_vm
Failure
29.68
test_scale_vm.py

Trillian test result (tid-2413)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server u18
Total time taken: 42141 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2413-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 84 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_hostha_kvm_host_fencing
Error
236.96
test_hostha_kvm.py

Trillian test result (tid-2412)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49096 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2412-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_human_readable_logs.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 82 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_02_enableHumanReadableLogs
Failure
65.72
test_human_readable_logs.py

test_01_add_delete_kubernetes_supported_version
Error
1808.51
test_kubernetes_supported_versions.py

test_07_reboot_ssvm
Failure
25.69
test_ssvm.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔debian. JID-1755

Trillian test result (tid-2411)
Environment: xcpng76 (x2), Advanced Networking with Mgmt server 7
Total time taken: 59913 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2411-xcpng76.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 82 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_add_delete_kubernetes_supported_version
Error
1812.49
test_kubernetes_supported_versions.py

test_01_scale_vm
Failure
22.68
test_scale_vm.py

test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL
Error
1330.62
test_vpc_redundant.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Error
524.45
test_vpc_redundant.py

test_05_rvpc_multi_tiers
Error
4209.76
test_vpc_redundant.py

test_05_rvpc_multi_tiers
Error
4257.55
test_vpc_redundant.py

Trillian test result (tid-2446)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41809 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2446-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 83 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_add_delete_kubernetes_supported_version
Error
1807.35
test_kubernetes_supported_versions.py

test_07_reboot_ssvm
Failure
25.63
test_ssvm.py

@blueorangutan package

pkging job manually kicked at backend jenkins

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1777

@blueorangutan test

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan package

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian.  I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✖debian. JID-1820

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1823

@blueorangutan test matrix

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

Trillian test result (tid-2559)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55453 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2559-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_deploy_kubernetes_cluster
Error
0.11
test_kubernetes_clusters.py

test_02_deploy_kubernetes_ha_cluster
Error
0.06
test_kubernetes_clusters.py

test_04_deploy_and_upgrade_kubernetes_cluster
Error
0.05
test_kubernetes_clusters.py

test_05_deploy_and_upgrade_kubernetes_ha_cluster
Error
0.06
test_kubernetes_clusters.py

test_06_deploy_and_invalid_upgrade_kubernetes_cluster
Error
0.04
test_kubernetes_clusters.py

test_07_deploy_and_scale_kubernetes_cluster
Error
0.04
test_kubernetes_clusters.py

test_03_vpc_privategw_restart_vpc_cleanup
Failure
260.06
test_privategw_acl.py

test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers
Error
389.67
test_vpc_redundant.py

test_04_rvpc_network_garbage_collector_nics
Error
3894.66
test_vpc_redundant.py

test_hostha_kvm_host_fencing
Error
104.48
test_hostha_kvm.py

Trillian test result (tid-2558)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 57937 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2558-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 82 look OK, 3 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_add_delete_kubernetes_supported_version
Error
733.72
test_kubernetes_supported_versions.py

test_01_scale_vm
Failure
41.45
test_scale_vm.py

test_01_redundant_vpc_site2site_vpn
Failure
655.25
test_vpc_vpn.py

test_01_vpc_site2site_vpn
Failure
364.19
test_vpc_vpn.py

Trillian test result (tid-2560)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 58996 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4131-t2560-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 83 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_01_add_delete_kubernetes_supported_version
Error
1807.99
test_kubernetes_supported_versions.py

test_07_reboot_ssvm
Failure
51.34
test_ssvm.py

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian.  I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian.  I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✖centos8 ✖debian. JID-1868

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✖centos8 ✔debian.   I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1897

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✖centos7 ✖centos8 ✖debian. JID-1914


elasticsearch snapshot deployment/configuration issues/questions

I did a development deployment from main branch, and found the following:

elastic_snapshot_cronjob_enable used in docker-compose template, but not set
I changed this to elasticsearch_snapshot_cronjob_enable for consistancy and defaulted to "false" in deploy.sh for dev/staging
elastic-snapshot service starts fails and restarts
wrapped elastic-snapshot service definition in {% if elasticsearch_snapshot_cronjob_enable == 'true' %} to disable

So right now it's disabled for development and staging deploys,
and I think it will be re-run whenever a production deployment is done (typically after 00:00 GMT, right after when it runs via cronjob).
Issues/questions:

Would it be meaningful/useful to run/test elasticsearch snapshots  in dev deploys?
Ideally, it would be good to gave staging deploys run the code once (ie; with cron disabled).  If this is possible/reasonable, what additional configuration is needed?
Is all configuration in place to deploy elasticsearch snapshots to production?

We created two S3 buckets (snapshot-repositories) mediacloud-elasticsearch-snapshots and test-mediacloud-elasticsearch-snapshots for production and staging/dev.
I believe we can have three buckets for each of the stages (means adding a staging-mediacloud-elasticsearch-snapshots repo)

Ideally, it would be good to gave staging deploys run the code once (ie; with cron disabled). If this is possible/reasonable, what additional configuration is needed?

Wouldn't we want to take snapshots for staging periodically? could be longer than the current default (daily) timeframes?
Configuration wise: would involve add a bash script on the es-setup service to load the S3 settings into Elasticsearch's keystore.

Is all configuration in place to deploy elasticsearch snapshots to production?

Yes,

Regarding snapshotting the staging ES DB;
Since staging only runs a single batch (of 500K articles, with no cron restart), and only to test whether it's ready for production, I think the data in ES is disposable (ie; could be erased before every run) tho right now it has data from a week-long run (with an older schema?) that could be of use testing the news-search-api.
So, I think the primary value of snapshotting staging would be to test that the code to trigger snapshots does the right thing, and that snapshots indeed appear on S3 (perhaps via a command line option to the elastic-snapshot script that logs at ERROR or CRITICAL if the snapshot fails?)
Finally (thinking out loud here): Might it make sense to:

have indexer.scripts.elastic-snapshot make sure the snapshot is registered
make sure the S3 keys are loaded?


How can I restore the content with the extracted metadata

Hi, I tried this tool and it is awesome but I have a question here:
Say that I want to extract the main page content (article) and show the restyled articles (like the pocket text view). But unfluff just extract the whole text (with images and videos splitted) in the json object, How can I restore the content? I even don't know how many paragraphs it has and where is the image position in the article. Thanks!

You can't do that with this package, unfortunately. It's not really designed to do that. It's designed to just grab the plain text and discard the original document structure.
You could modify the code work differently, but that's not really what it's designed to do right now. Sorry :(

well, ok! Thanks @ageitgey


chore: release v0.3.15

:robot: I have created a release beep boop
0.3.15 (2024-08-24)
Bug Fixes

deps: update dependency tsx to v4.17.1 (#125) (44fe30c)
deps: update dependency tsx to v4.18.0 (#127) (f1eaafd)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v0.3.15
:sunflower:


What's the purpose of deleteInstallation

Description
Describe your issue here.
What type of issue is this? (place an x in one of the [ ])

[ ] bug
[ ] enhancement (feature request)
[x] question
[ ] documentation related
[ ] example code related
[ ] testing related
[ ] discussion

Requirements (place an x in each of the [ ])

[x ] I've read and understood the Contributing guidelines and have done my best effort to follow them.
[ x] I've read and agree to the Code of Conduct.
[x ] I've searched for any related issues and avoided creating a duplicate issue.

This might be a silly question, but what's the exact purpose of deleteInstallation? I mean, I know it's meant to remove the installation information, but when is it actually triggered?
When installing the app via the /slack/install route, storeInstallation is triggered to store the installation, so I was expecting deleteInstallation would be triggered when the app gets deleted (via Slack -> manage apps -> delete), but that's not the case apparently... So, if you can't track those uninstalls, why even bother cleaning up installations?

OK, it was indeed silly. By now, I found that you can actually subscribe to the app_uninstalled event.


User Story #938 - SEDOL data generation

Description
This PR is to fulfil the requirements of User Story #938, to enable users to generate SEDOL data.
Changes
We already had the capability to generate SEDOL codes included as part of the existing ISIN generation code, and some work had been done to expose this to profiles, although it was not in a functional state.
This PR fixes issues which users would have encountered had they tried to use the undocumented SEDOL-generation functionality, and also updates the documentation.
Additional notes
Cucumber testing has been carried out by duplicating the current ISIN tests for SEDOLs, on advice given by @d-withers
Issue
Resolves #938

what testing is planned for this feature?

@pdaulbyscottlogic I'm going to discuss that with the testers.

This might be too big a comment for this pr.
But do we instead want our isin and sedol restrictions as extensions/decorators of the textual restriction class?

This might be too big a comment for this pr.
But do we instead want our isin and sedol restrictions as extensions/decorators of the textual restriction class?

Potentially so, but firstly I do not think that that is a definite, and secondly, now is not the best time to jump into doing that when only a small amount of work was needed to do to implement this PR in a way that meets our current set of requirements.


Include organization_guid and space_guid in deprovision and bind/unbind requests

The organization_guid and space_guid should be included in the requests, just like the service_id and plan_id are.
This would allow the broker to retrieve instance metadata even when it isn't stored centrally, but per-tenant (per-space in OSB terminology).
Here's a more detailed description of why we need this:
The Messaging-as-a-Service broker allows provisioning of messaging addresses collocated in the same messaging infrastructure (the infrastructure is per-tenant). The broker currently doesn't use its own storage for storing instances - instead, it simply creates k8s resources directly. The resources themselves represent the instances. Being multi-tenant, it stores these resources across different k8s namespaces. To retrieve a service instance by uuid therefore currently requires scanning all those namespaces in order to find the namespace the instance is stored in.
I'd like to not have to use an additional storage layer just for storing what would essentially be duplicate data just for efficient retrieval, since the platform knows all the data I need (organization_guid and space_guid) and it could easily pass it on to the broker.
WDYT?

We are trying to move away from having org/space guids as top level properties: https://github.com/openservicebrokerapi/servicebroker/issues/115
You can see the current proposal from @duglin (https://github.com/duglin/servicebroker/tree/CFisms) that moves these fields into a context object (https://github.com/duglin/servicebroker/blob/CFisms/context-profiles.md).
Perhaps the context object could also be sent in deprovision and bind/unbind requests.

@avade  any idea when context object would be getting added to deprovision/bind/unbind requests as well ? Any specific reason it was left out and added only for PUT/PATCH methods.

@jagadish-kb The context object was added for provision service instance, update service instance and bind service instance. It was left out of unbind service instance and delete service instance as those DELETE requests don't support request bodies, and we didn't want to have to find a way to encode this information in a query parameter.
We're hoping that brokers can either use their own state or rely on other components to remember any contextual information they need in order to successfully unbind or deprovision.

@mattmcneeney Thanks for the response.
Our service broker is stateless and we don't persist any state at the moment. Reason: Actual state might get out of synch with CC & provisioning (BOSH) System (cases of orphaned instances/Orphaned Deployments) & if broker also starts maintaining this mapping then data reconciliation has to happen across CC, Broker & the provisioning system. So at-least to minimise this reconciliation effort, since broker as the name suggests is just brokering we go back to CC if we need instance info or the provisioning system (like BOSH) to figure out any deployment info & get the actual state.
We would want to avoid broker having to go back to Cloud Controller as we have requests from platforms other than CF, where they want to use our broker in their environments as well. So in this regard we want to be compliant with OSBAPI as much as possible. Consumers from other platforms  are ready to send few additional context params as part of Delete requests but since the spec does not have them in, wanted to check.
With regards to HTTP DELETE not allowing request bodies is debatable as the HTTP Spec does not call this out explicitly. Interesting conversation around this on stackoverflow .

@mattmcneeney  @avade
any update on the above issue?
We want to be compliant with OSBAPI however the above issue is forcing us to be stateful, which certainly requires a bigger effort in terms of implementation and also ensuring the state that is maintained in the broker is in synch with actual deployment state (corner cases related to orphan deployments/ service instances). Ideally it will help us a lot if even DELETE method carries this context information. I am sure, there could be at least a few other service broker implementation who are stateless (mostly those who provision BOSH based deployments) & this will be certainly good have.
Would appreciate if you can please provide a resolution on the above & let us know if the above would be considered to be done in future or not.

While I can certainly understand wanting to have org and spec flow on the requests, I don't think we should include a body on DELETE - I've heard too many times about httpd servers or proxies that don't support it that it seems risky to do so.
I'd like to understand the usecase a bit more... how does the org/space help you find the instances across the k8s namespaces?

I've heard too many times about httpd servers or proxies that don't support it that it seems risky to do so.

I would hope we would embrace the forward direction of specs and standards rather than anchor ourselves to today's broken implementations. But that said, a body on delete feels real weird.
In regards to stateless brokers, the direction we are taking seems to me to require some small amount of state to be maintained by the broker for the broker to be useful. Some state is definitely required to handle async requests. I think async requests are the future of brokers and sync requests are a mistake to support. Therefore I believe all brokers must have state and stateless brokers should be discouraged.

@n3wscott For handling Asynch requests, the spec already has a mechanism in place & one can handle asynch requests being completely stateless. The operation attrib that is returned back as part of asynch request's response is played back as part of the lastoperation polling. So whatever minimal state is required can be base64 encoded, sent as part of provision/deprovision/update response & the same gets played back. So for async request handling one need not maintain state per-se.
But limitations such as above certainly would make one to be stateful.
@duglin

I'd like to understand the usecase a bit more...

Our broker implementation is currently tied to CF, but we have a requirement for it to be CF agnostic & also handle requests coming in K8s/other homegrown consumers, allowing for cross platform service consumption.
Now org Id & space id are certainly not required for instance identification but instead required to identify the service instance backup/restore information in S3/Swift stores. The file paths to service instance backup/restore info have been prefixed with org/space id along with instance id.  This is done for easier listing/access at org/space level if required. Now backup/restore are not in the purview of the spec, but these are extension use cases which our broker implementation has built around. This info from S3/Swift stores must also be deleted post instance deletion. This is just one scenario, we have couple of other custom tasks which are platform specific ex., creation /deletion of security groups for service instance as in CF.
You might ask, in the erstwhile CF service broker API as well the org/space guid was not flowing as part of DELETE and how were we managing. - So in our current state, for such requests where this info is missing we query back CF & retrieve this information. If at-least platform info is passed, it certainly would help. This can be avoided, if this info flows automatically.
I know it certainly feels weird to have a payload on DELETE. Now i only suggested including context info in DELETE, as it will make this uniform across requests. However, if its felt wrong to do this then why not pass it as part of Query params? The spec can say context flows either in body or query params. Ass a consumer of this spec I certainly feel there is value in passing this info in all requests.
Obviously the above problem completely goes off, if one were to be stateful. If being stateful is the recommendation of the Spec it certainly helps to call this out explicitly in the documentation as then teams implementing the same don't have to debate / discuss & spend time on this design consideration.

The context object now includes the organization_guid and space_guid.
This only applies for provision, update and bind requests though, as we haven't found a solution for including context in DELETE requests yet (unbind and deprovision). That discussion can be tracked in #201.


Make Windows plugin APIs thread-safe

Currently many core plugin APIs in Windows are platform-thread only, as on other platforms. However, as discussed in https://github.com/flutter/engine/pull/31134#pullrequestreview-991213668 and https://github.com/flutter/flutter/issues/79213 this doesn't work well on Windows, both in terms of OS APIs to facilitate it (there aren't any, unlike other platforms), and convention (Win32 APIs generally don't care what thread you are on). To better match platform conventions, we should make APIs like sending channel messages internally thread safe (by having the embedding internally post them to the platform thread, which we already have a mechanism for).
This may will be straightforward for the C API, but we'll need to make the C++ wrapper that almost all code actually uses that safe as well, which may require some locking.

Not sure what you mean by Win32 APIs not caring about which thread you're on. HWNDs are bound to a thread, and often COM thread apartment. It's generally safe to send/post messages from other threads, but randomly calling WIN32 methods from other threads can be problematic, especially if there are COM calls underneath.

Having read the discussion in https://github.com/flutter/engine/pull/31134#pullrequestreview-991213668 I'm not sure I agree with the conclusion. Windows does have a strong concept of thread affinity (both for HWNDs and STA COM objects). And while there technically isn't a system designated main thread like on macOS and you can have separate thread for every HWND where you pump the messages, this seems rarely the case in practice and there almost always is a de facto main thread.
I'd prefer a simple way to schedule things on main thread (which is always bit of a PITA in windows) rather than assuming that API calls are thread safe. For the record I've seen crashes from something seemingly mundane as SetFocus called from wrong thread.

I'd prefer a simple way to schedule things on main thread (which is always bit of a PITA in windows) rather than assuming that API calls are thread safe.

It wouldn't be assumed, it would be explicitly documented.
Why would having clients deal with thread management rather than us doing it internally be preferable?

I'd prefer a simple way to schedule things on main thread (which is always bit of a PITA in windows) rather than assuming that API calls are thread safe.

It wouldn't be assumed, it would be explicitly documented.
Why would having clients deal with thread management rather than us doing it internally be preferable?

I meant Win32 API calls, referring to your original comment. The assumption being "Win32 APIs generally don't care what thread you are on" which I don't quite agree with. As for Flutter API, it's not thread safe on other platforms and I don't quite what makes Windows that much different. But there are also practical reasons:
What brought me here specifically was this PR - https://github.com/flutter/engine/pull/47924, which, in order to make the API thread safe while retaining the signature would schedule a block on main thread and block the background thread until complete. This seems like a pretty big can of worms.

which, in order to make the API thread safe while retaining the signature would schedule a block on main thread and block the background thread until complete. This seems like a pretty big can of worms.

We should definitely not do that; it would be much better to add new threadsafe versions of things that return values that do so via a callback.
I hadn't fully thought through the return value aspect of this. I guess the complexity (for us) of maintaining a callback system for those at the C level might be an argument for doing the single wrapper instead? I'll defer to @cbracken on which way he wants to go with this. By the time we get to the C++ level (which is the API I expect ~everyone would use) I would think callback lambdas will be easier for clients to manage, but we could also do a general util at the C level and then use that to implement async callbacks at the C++ level.

To summarize, there are the following options:

Drop the idea of making everything thread-safe and introduce a dispatch_async-like mechanism instead (like https://github.com/flutter/engine/pull/31134). Make FlutterDesktop* implementations assert that they're called on the platform thread.

pros: low effort, generic solution, consistent with other platforms (like dispatch_async on Darwin)
cons: calling requires slightly more boilerplate code

Do thread hopping at the C level which requires changing signatures of FlutterDesktop* functions or adding additional functions in order to accept callbacks

cons: will get messy quickly

Do the thread hopping at the C++ client wrapper level and add additional overloads/optional arguments accepting callbacks.

pros: less effort than doing thread hopping at the C level

Hi Flutter team, any update on this?

@jnschulze Thank you for your summary. As a short-term solution, we can add APIs to post tasks to the engine's platform thread. Here is a strawman API:
class FlutterEngine : public PluginRegistry {

  // ...

  // Schedule a callback to be called on the platform thread.
  //
  // This may be called on any thread. The callback is executed only
  // once on the platform thread.
  void PostPlatformThreadTask(std::function<void()> callback);
}

// Schedule a callback to be invoked on the engine's platform thread.
//
// This may be called on any thread. The callback is executed only
// once on the platform thread.
FLUTTER_EXPORT void FlutterDesktopEnginePostPlatformThreadTask(
    FlutterDesktopEngineRef engine,
    VoidCallback callback,
    void* user_data);

This wouldn't doesn't address this issue, but it does make it easier for plugin authors to use the existing APIs. In the future, we should consider moving forward with one of the options listed above.
/cc @cbracken @yaakovschectman @stuartmorgan for proposed API

Well I can't say I see any drawbacks to the API.

Is this a feature I need ? or should I need to open a new issue ?
scenario :
pure dart code to call SetWindowSubclass on windows platform using ffi. SetWindowSubclass can not used across threads.
I can't do that at the moment, because the thread that is used to call SetWindowSubclass via ffi is not the thread that was used to create the win32 window.

@l619534951 It sounds like you want https://github.com/flutter/flutter/issues/136314. This issue is is about calls to the Windows embedding APIs in native code.

I want to use Piegon to call a Flutter function in a new thread, but I get the error Platform channel messages must be sent on the platform thread, according to the documentation, on Android this can be done via
Handler(Looper. getMainLooper()).post {
  // Call the desired channel message here.
}

But I didn't find the usage for Windows platform in the documentation. This Issue seems to be related to problem, and I found the following function.
class FlutterEngine : public PluginRegistry {
 // Schedule a callback to be called after the next frame is drawn.
 //
 // This must be called from the platform thread. The callback is executed only
 // once on the platform thread.
 void SetNextFrameCallback(std::function<void()> callback);

How can I use this function? or would anybody like give me other solutions?

@qingzt There isn't an easy way to switch to the platform thread on Windows. Adding an API to make this easier is tracked by https://github.com/flutter/flutter/issues/79213.
For now, you can create a hidden window and post it messages to send work to the platform thread. This is what the Windows embedder does internally: https://github.com/flutter/engine/pull/24232

@loic-sharma are there any news on pushing short-term solution?


Implement Roon as a source

https://roon.app/en/
https://github.com/RoonLabs/node-roon-api

Hey @FoxxMD do you know if this is possible? I'm scribbling to last.fm through Roon directly but would love to switch to listenbrainz instead. :)

It seems possible but I'll need to get a trial account and (hopefully) have something implemented before it is over as I don't use Roon personally. I'll have more time in January to look into this.

I have sent an email to roon support inquiring about an open-source program so I can develop/test


Removed useless imports. Added GTM url and ID to .env.xxxx files. Aut…

Added:

Search bar automatically gets the cursor within the text area when conan/center/ or wherever the search bar is loaded.
WIP: conanfile.txt and conanfile.py tabs are available in "Use it" section.

Modified:

Removed some useless imports.
Removed SSRProvider. It seems it's not needed anymore for React >= 18: https://react-spectrum.adobe.com/react-aria/ssr.html#react--18
gtm ID and URL put as env variables.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix bug and simplify the structure

Apologies for my negligence on this one (2014!). We're not actively using this role. Changes look fine, although I would have liked to see continued support for RedHat based distros. Also, it should be updated for Ansible 2.0, and our formatting standards have moved on since this was created ... I'll create an issue for this anyway, it wouldn't take a lot of time to bring this up to date.


deno.cache command results in 'method not found' alert

Not sure if this is an issue with Nova or the Deno LSP honestly.

Deno Language Server[15:32:40.040000] Sending JSON-RPC request: 82 workspace/executeCommand
{
  "jsonrpc" : "2.0",
  "id" : 82,
  "method" : "workspace\/executeCommand",
  "params" : {
    "command" : "deno.cache",
    "arguments" : [
      [
        "https:\/\/deno.land\/std\/bytes\/bytes_list.ts"
      ]
    ]
  }
}

Deno Language Server[15:32:40.040000] Got a workspace/executeCommand request, but it is not implemented

Deno Language Server[15:32:40.087000] Received JSON-RPC response: 82 workspace/executeCommand
{
  "error" : {
    "message" : "Method not found",
    "code" : -32601
  },
  "id" : 82,
  "jsonrpc" : "2.0"
}

Deno Language Server[15:32:40.087000] -32601 Method not found```

I don't know if this information is novel, but I think I've confirmed that the Visual Studio Code extension registers a deno.cache command, which is somehow invoked when the code action is selected.

Registering a deno.cache command ourselves doesn't work, as described here: https://devforum.nova.app/t/lsp-executecommand-and-method-not-found-alert-whos-requesting-who/1650/4
Maybe this requires Nova to imitate VS Code's behavior, which is supposedly to execute extension-registered commands in response to certain LSP requests (?).

As of Nova 11.7 and Deno 1.38.5, this seems to have been fixed!!!


Invariant Violation: Tried to register two views with the same name RNSVGSvgView

Invariant Violation: Tried to register two views with the same name RNSVGSvgView

Please explain your senario. if you can repoduce the issue in snack it would be great or put a screenshot It would be great.

Can not Reproduce.


should.throws doesn't catch the error when it fails to match it

When using should.throws, if the error parameter you are matching against doesn't match the thrown error, instead of catching the error and logging actual vs expected (like most assertions), it doesn't catch the error at all.
Example:
'use strict';

const assert = require('assert');

const should = require('should');

describe.only('stub for missing method', function() {
  beforeEach(function() {
    this.testFn = function() {
      throw new Error('aaa');
    };
  });

  it('should pass when a matching error is thrown', function() {
    //assert.throws(() => this.test(), /aaa/);
    should.throws(() => this.testFn(), /aaa/);
  });

  it('should error with an expectation not "aaa"', function() {
    //assert.throws(() => this.test(), /bbb/);
    should.throws(() => this.testFn(), /bbb/);
  });
});

logs
1) stub for missing method should error with an expectation not "aaa":
     Error: aaa
      at Context.testFn (test.spec.js:10:13)
      at should.throws (test.spec.js:21:30)
      at Function._throws (node_modules/should/cjs/should.js:686:5)
      at Function.assert.throws (node_modules/should/cjs/should.js:719:11)
      at Context.<anonymous> (test.spec.js:21:12)

Is this intended behavior? It seems unintuitive and I can't think of a use for it besides nesting should.throws inside it's first argument or something to do with sequenced assertions and any (I can't find documentation for those concepts). At any rate, rejectedWith will show actual vs expected rather than letting the promise reject if the argument doesn't match which is inconsistent with throws.

Yes this is intendent behaviour. I did port nodejs 0.xx assert module in 2015, at that time it does not have such interface. I do not think it is worth my time to keep it up to date (i would remove it).
Also you mixed static should functions which is literally assert functions implemented using should.js methods, another thing you did wrong it is about any and rejectedWith first workin only with .properties and second working only with promises. (I highly recommend you first read api docs)

Here are the docs for any:
Any modifier - it affect on execution of sequenced assertion to do not check all, but check any of.
Not a lot to read up on. And I'm forced to mix static methods because that is the only way to test synchronous throwing, maybe that could be the change; but error matching for synchronous errors should using the same same matching you use for asynchronous error matching, which is currently impossible. And I don't think rejectedWith works with non promises given:
Assert given promise will be rejected with some sort of error. Arguments is the same for Assertion#throw.
Result of assertion is still .thenable and should be handled accordingly.

It says it works for only promises.

About docs for any i agree. Need to add more.
About sync and async throws you are completly wrong, :
function simpleFunction() {
  throw new Error('boom');
}

should(simpleFunction).throw()

It seems @tsclaus you do not have any more questions?


added transpose stub to match new capability in oops

Description
This pr adds a stub for the transpose function in oops, but does not implement it.
Issue(s) addressed
Resolves PR https://github.com/JCSDA-internal/oops/pull/2603
Dependencies
List the other PRs that this PR is dependent on:

[ ] https://github.com/JCSDA-internal/oops/pull/2603

Impact
Expected impact on downstream repositories:
Checklist

[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[x] I have run the unit tests before creating the PR

I think this can be merged before the OOPS PR (which is waiting on fv3).


Reveal doesn't show the hidden content

Bug Report
Reveal doesn't show the hidden content in google chrome, Version 86.0.4240.111 (Official Build) (x86_64), in other browsers seems that everything is ok.
Steps
Download  google chrome, Version 86.0.4240.111, and go to https://react.semantic-ui.com/elements/reveal/#variations-instant
Expected Result
Here is the result from Safari.

Actual Result

Version
2.0.1
Testcase
The bug from your official page.

It works for me on Chrome 105. One thing I don't understand, why did you show the snapshot example using Safari but then you use Chrome in the "steps"?


Control-s key binding doesn't overwrite the default save binding

🐛 Bug
Overwriting the StandardKeyBinding.Save with the viewer.bind_key('Control-s', <callback>, overwrite=True) doesn't work. My callback is not called. I think that the _save_selected_layers action is called instead.
This only occurs when there is a layer added to the viewer. See code to reproduce behavior.
Note - this may be the desired behavior because the Control-s command is special, but I can at least imagine one scenario (mine) in which a user may want to overwrite the default save behavior.
To Reproduce
This code block reproduces the failure to overwrite the 'Control-s' key binding.
import napari
import numpy as np

def new_save_method(viewer):
    print("this is the new save method")

viewer = napari.Viewer()
image = viewer.add_image(np.random.normal(0,1,(5,100,100)))

viewer.bind_key('Control-s', new_save_method, overwrite=True)

# Then, once the viewer is open, press 'Control-s'. It'll open a dialog to save selected layers.

If you remove the image layer, then the 'Control-s' key binding works.
import napari
import numpy as np

def new_save_method(viewer):
    print("this is the new save method")

viewer = napari.Viewer()
viewer.bind_key('Control-s', new_save_method, overwrite=True)

# Then, once the viewer is open, press 'Control-s'. It'll call the <new_save_method> callback.

Expected behavior
I think it would make sense to allow a key_binding to overwrite any key binding, not just some.
Environment
napari: 0.4.18
Platform: Windows-10-10.0.19045-SP0
Python: 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:15:42) [MSC v.1916 64 bit (AMD64)]
Qt: 5.15.2
PyQt5: 5.15.9
NumPy: 1.24.3
SciPy: 1.11.3
Dask: 2023.8.0
VisPy: 0.12.2
magicgui: 0.7.3
superqt: unknown
in-n-out: 0.1.8
app-model: 0.2.0
npe2: 0.7.2
OpenGL:

GL version: 4.6.0 - Build <IP_ADDRESS>1
MAX_TEXTURE_SIZE: 16384

Screens:

screen 1: resolution 1920x1080, scale 1.0
screen 2: resolution 1080x1920, scale 1.0

Settings path:

...\AppData\Local\napari\vrAnalysis_0354d897aaa42669b0619d6be1a33d26edd0a985\settings.yaml

Thanks for the issue.
This came up at the hackathon.
We largely agreed that in an end-user script it should be anything goes, with regards to keybinds.
I think the issue is how the menu item keybinds are hard-coded.

I see, thanks for your response. Glad to know others had the same thought. Was there any discussion of how to solve this? Or is it too complicated to be worth it?

There was a discussion in person at the napari meeting. There is no current such issue. I will try to create a summary and open an issue for this.
This definitely needs to be solved.

This is essentially the same issue as
https://github.com/napari/napari/issues/2799
and
https://github.com/napari/napari/issues/6440
viewer keybindings cannot be over-written.
If you can use a @layer.bind_key that should work and over-write existing ones in napari.

For anyone who needs it, I find this works:
`
from app_model.types import KeyCode, KeyMod
from napari.layers.labels.labels import Labels
from napari.layers.image.image import Image
@Image.bind_key(KeyCode.KeyZ, overwrite=True):
def func1()
print(" ")
@Labels.bind_key(KeyCode.KeyZ, overwrite=True):
def func2()
print(" ")
`
Got the hints from here:
https://github.com/napari/napari/blob/main/napari/layers/labels/_labels_key_bindings.py


Add the ability to customize the options passed to the Sentry SDK

Add an event triggered when defining the Sentry SDK configuration to add the ability to add custom options.
Example:
\yii\base\Event::on(
  \statikbe\sentry\services\SentryService::class,
  \statikbe\sentry\services\SentryService::EVENT_DEFINE_SENTRY_SDK_CONFIGURATION,
  function(DefineSentrySdkConfigurationEvent $event) {
    $event->options['context_lines'] = 5;
  }
);

This PR also changes the default Sentry SDK options to honor the Craft httpProxy general setting, so that the plugin works even behind a proxy if Craft is properly configured..

Hey @nstCactus, thanks for this! And sorry about the long wait! i'll try to get this merged & released this week.

This is now available in 1.1.0. Thanks again for the PR!


Add sveltekit:prefetch

closing due to not resolving conflicts, feel free to pull updates and pr again


Recycler could cache infinite resources

In Netty.4.0.24.final.
In class io.netty.util.Recycler, there's a DEFAULT_MAX_CAPACITY variable to constrain the size of the stack in Recycler. From INITIAL_CAPACITY, whenever the stack is full, it's size doubled until hitting the DEFAULT_MAX_CAPACITY limit(Line 341 in Netty.4.0.24.final).

As you can see in the image above, stack size is constrained by maxCapacity. And if the size > maxCapacity, this check will fail forever, beacuse the check use "if (size == maxCapacity)" rather than "if (size > maxCapacity)".  And there's a way to set size bigger than maxCapacity.
In class WeakOrderQueue which is an inner class in Recycler, there's a method called "transfer" which can also double the stack size in Recycler, but there's no check to ensure the stack size not exceed DEFAULT_MAX_CAPACITY limit(Line 218 in Netty.4.0.24.final) like the one mentioned before.

In the image above, variable "to" is a stack in Recycler and count is resources's quantity recycled by other thread rather than the Recycler's owner. If there's no enough space for those resources to be recycled, the stack's size will be doubled. I think there should have a check for the stack size here. And whenever the stack size greater than maxCapacity, it can grow infinitely.

@trustin do you think there is still something related in Recycler to be fixed since you guys are bumping the milestone?

@ninja- I believe we have addressed this issue in #3203 and a9fda3c8e0b20a1a3da0353c033cb57be7ad49b2. I'm not sure this is the problem you are experiencing, though, because I didn't follow the discussion between you and @normanmaurer. Maybe @normanmaurer can answer better than I.
@tea-dragon Do you still experience this issue?

@trustin personally I am fine with the current state of the recycler in .30, the only thing I am wondering about is what did increase the need for buffers at the same time growing the recycler in .30. could it be because I recently added tcp_nodelay? (not considering it a bug, but I am tweaking down the maximum recycler's amount so it would stay clean when the proxy is empty)

@ninja- I'm not sure updating netty will fix this since it's a netty implementation issue on BungeeCord's side. Along with the changes I made above, I also updated to the latest netty. I don't remember that fixing the issue on its own.

@Iceee yes, your patch didn't fix the problem by itself. It's fixed in netty .25+ while upstream bungee keeps using .24

@ninja- I'm not sure updating netty alone will fix this since it's a netty implementation issue on BungeeCord's side. Along with the changes I made above, I also updated to the latest netty. I don't remember that fixing the issue on its own.
There may be a netty bug in regards to recycling entries on closed connections, but this issue specifically stems from BungeeCord not properly implementing netty, and therefore netty blindly writing to buffers that are growing to unlimited sizes.

@Iceee there wasn't any error in bungee's implementation. your patch in fact did nothing and didn't stop the infinity bug from eating majority of ram when randomly triggered so there was no difference

@ninja- Actually, it fixed everyone's problem as you can read in the issue!

@Iceee @ninja- - There should be some bound on data that is written when the channel is not writable (as Iceee/BungeeCord@1a997bf appears to do). However it looks like your are dropping data in this case (not sure if this is a problem for you), and also leaking buffers (write will call release on the buffer, but since you are not writing it release will not be called).  So it seems like you need both the "not writing too much" concept and Recycler bug (fixed in 4.0.25).

@Iceee well I remember @md-5 comment about the patch that it only reduced leak's severity and not fixed it...sure it's a good thing to have this check along with Recycler fix and that's what I personally did.
@Scottmitch it's interesting what you say about release() but I am pretty sure it couldn't leak a buffer via refcnt in any case. have you seen HandlerBoss#channelRead?
Regarding data dropping it shouldn't be a problem because if connections were to lag that much they wouldn't be a working connection anyway so it doesn't matter(I would rather close such connections just like the timeout check does..)

have you seen HandlerBoss#channelRead?

@ninja- - No. My leak assessment is based upon the write method modified by Iceee/BungeeCord@1a997bf in isolation. The contract for the write(Object o) method appears to assume ownership / release responsibility of the parameter. This method provides no return status or indication if the write was actually done so I'm not sure how a caller of this method could reliably know if a write was done or not, and thus release the buffer after this method call if needed.

(I would rather close such connections just like the timeout check does

If you use case permits this behavior that is great...nice and simple. However keep in mind these conditions may fluctuate (according to the transport you use, networking conditions, and other environmental factors).

@Scottmitch I would assume that write() would release the object even if it denied the channel write (that would be a good api design for this case, checking futures for every write would be an overhead). But there shouldn't be currently such a case given isWritable() check, right?

It seems there is some confusion, let me clarify:
Channel.write (the interface and Netty implementations) will ensure release() is called on any ReferenceCounted object after it is completely written.
ChannelWrapper.write(..) and Iceee/BungeeCord@1a997bf both may have a leak if the Object packet parameter is (or contains) a ReferenceCounted, because there is a condition for which they don't call Channel.write(..), and are also not calling packet.release().  I'm not sure of the usage pattern (if/when this conditional clause will evaluate to false) but if/when the conditional evaluates to false does there will be a leak.

release the object even if it denied the channel write

What do you mean by denied?  Hopefully my first statement clarified any confusion regarding the Channel.write(..) interface?

But there shouldn't be currently such a case given isWritable() check, right?

You are more likely to leak buffers by adding the isWritable() check because the conditional is more likely to evaluate to false, and thus you aren't calling Channel.write().

@Scottmitch you are right about this part, sorry. let me see if this will fix the leaks on production?

but are you sure you are evaluating Iceee's code in context of the HandlerBoss...? these write() calls always happen during channelRead and the buffers are always .released() at the end @ HandlerBoss

Based upon a quick glance calls to ChannelWrapper.write(..) from HandlerBoss.channelRead(..) should not leak based upon the following:
When ChannelWrapper.write(..) encounters a PacketWrapper it marks it as released before passing the buffer to write.  For the PacketWrapper type this will ensure it is released 1 time either in ChannelWrapper.write(..) or HandlerBoss.channelRead(..) because PacketWrapper.trySingleRelease() checks if it is "released" before calling buf.release().
A few followup questions:

if handler != null is false more analysis is needed to see if there is a leak.
Are all calls to ChannelWrapper.write(..)? always from HandlerBoss.channelRead(..)?
How does ChannelWrapper.write(..) coordinate release of objects which are not of type PacketWrapper when a ch.write does not occur?

An an unrelated note....I think you also mentioned no copy operations were being done before. Not sure if this code is run in production but there is a copy done in MinecraftDecoder

@Scottmitch in my own version I removed all .copy() but the problem but the $Entry problem also appears there.
Regarding your questsions about releasing the buffer, I have added
inally
            {
                ByteBuf b = packet.getBuffer();
                packet.trySingleRelease();

                if (b != null && b.refCnt() > 0) {
                    BungeeCord.getInstance().getLogger().log(Level.SEVERE, "Packet leaked while hadling for " + handler + ": " + packet + ", " + b.refCnt() + " refs!!!");

                }
            }

it's always 0 + netty leak detector is silent, isn't that enough proof that the .release() mechanism works correctly here?

it's always 0 + netty leak detector is silent, isn't that enough proof that the .release() mechanism works correctly here?

@ninja- - I gave a pretty detailed analysis in https://github.com/netty/netty/issues/3166#issuecomment-135304391 and came to the conclusion for what appears to be the "common" case there should not be any buffer leaks (with some followup questions and caveats).
Can you try to use the UnpooledByteBufAllocator?  I did a quick check and I don't see any Recycler being used.

@Scottmitch great idea I'll try the allocator.

How does ChannelWrapper.write(..) coordinate release of objects which are not of type PacketWrapper when a ch.write does not occur?

they aren't released because they aren't detained so the balance stays the same (0)

1:        597079       42989688  io.netty.buffer.UnpooledUnsafeDirectByteBuf
6:        597080       19106560  java.nio.DirectByteBuffer$Deallocator
4:        597081       23883240  sun.misc.Cleaner
3:        597088       38213632  java.nio.DirectByteBuffer
2:        597089       38213696  io.netty.channel.ChannelOutboundBuffer$Entry
5:        597378       19116096  io.netty.util.Recycler$DefaultHandle
3 minutes with unpooled allocator...

they aren't released because they aren't detained so the balance stays the same (0)

Can you clarify? Do mean to say the are never of type ReferenceCounted in this case?

3 minutes with unpooled allocator...

Sry...just found a few more RECYCLERs being used. Have you done a heap dump analysis? Can you tell me what the "strong" GC root references to the UnpooledUnsafeDirectByteBuf are (just a few samples)

They output buffers are normally detained/released by other handler such as decoders etc. and the app only needs to keep it's balance to zero.

yeah the only strong ref to DirectByteBuf is Recycler$DefaultHandle via OutboundBuffer$Entry

weird, when looking at the heap dump the DirectByteBufs are gone.
just the io.netty.channel.ChannelOutboundBuffer$Entry leaking - 3 million of them.

@Scottmitch I think that I know what's wrong with netty's recycler. but first, I can't get netty debug to print. how do I turn it on from the start?

@ninja- you mean enable debug logging from the start ?

yeah
static {
InternalLoggerFactory.setDefaultFactory(new JdkLoggerFactory());
Logger.getGlobal().setLevel(Level.ALL);
Logger.getLogger("io.netty").setLevel(Level.ALL);
}
doesnt do much

@ninja- you have a logging.properties file that tells where to log ?

@normanmaurer no, is it for the java logger?

@ninja- yes... that said I never used jdk logger. Anyway here seems to be an example:
http://stackoverflow.com/questions/960099/how-to-set-up-java-logging-using-a-properties-file-java-util-logging

@normanmaurer  :\ I was looking at wrong property because I was checking out on master and they differ. so it ignored my property...

So that was my problem...the property to change recycler settings differs in master and netty 4.0. I would guess it should work correctly now.

@ninja- Can you clarify? Are you saying you are no longer believe there is a leak in Recycler, and the behavior you are seeing was just the maxCapacity was configured to be to high?

@Scottmitch yup.
it changed from
 int maxCapacity = SystemPropertyUtil.getInt("io.netty.recycler.maxCapacity.default", 0);

to
 int maxCapacity = SystemPropertyUtil.getInt("io.netty.recycler.maxCapacity", 0);

and I never expected this

@ninja- - Yes that is unfortunate :( 9fe9710315bb89b015a3de0fbf4c1019a10e5c85
@tea-dragon - Would you mind providing a re-producer? We may want to move the discussion to another issue too...

Maybe a more sane Recycler default capacity could be implemented? Why in the world is it so high anyways?
Pool reasonably low then let people tweak higher if they really believe itll help their application.

@aikar you don't want this trust me...don't deploy it on prod

@ninja- what do you mean?
Are you saying limiting the recycler is bad? I really hope that something isn't depending on a super large object pool.
Surely 5k instances of something should be enough to handle per thread...

@aikar I spent few months fixing this...take patches from my branch and it will be down to ~300

can we move discussion to https://github.com/netty/netty/issues/4146?
@ninja- - I thought the issues you were observing related to this issue were because of setting the wrong system property https://github.com/netty/netty/issues/3166#issuecomment-135485992? Lets open a new issue if there are other issues...becomes too tough to track otherwise.

@Scottmitch there's no issues right now really as I am trying to explain this to @aikar...

I understand that, I just noticed discussion on the maxCapacity setting that was used to I'm assuming to minimize impact that this issue was rising, and I was stating a desire for a more sane limit. As regardless of the underlying issues being fixed or not, it still makes sense for a sane limit for future cases that could cause the recycler to grow large for a temporary spike.
But I see now there is another issue open about better limits, so all is good and can discuss there.

the current 266k*threads limit is sane...unless you want to allocate 200mb to java and have 1000% cpu usage

I think we added a lot of things to be able to specify max limits etc. So we can close this now

Thanks!I have solved the problem of large size in oldgen by the issue


fix: [MSSQL] default fog server_pairs not read from config

Although we were exposing a config to allow passing server_pairs
to csb-azure-mssql-fog-run-failover service, it was never used.
CSB complained whenever we tried to create an instance without
specifying server_pairs explicitly.
The name of the property in the config file is very intriguing:
service.csb-azure-mssql-fog-run-failover.provision.defaults.server_pairs
because it seems to suggest that CSB would read some "magic" paths
to determine default values, etc.
But I don't know if this feature was removed or has never worked.
Checklist:

[ ] Have you added Release Notes in the docs repositories?
[x] Have you followed the Conventional Commits specification?

Definitely not necessary right now, I agree. I suggest that we keep the PR open and discuss it in the future.

Closing this PR in favour of a more in-depth team discussion about the excess of variable resolution mechanisms.


get a better landmask

Replace landmask file with a better one (mainly fixing Central America).
make sure there are observations in the dirac test that are near Panama so that the network check on the land mask can be properly tested.

@travissluka  I uploaded an improved landmask.nc then the test of test_umdsst_staticbinit failed due to
"Level   1: 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60%!!! ABORT in mpl_nc_dim_define_or_get on task #0002: dimension nc1b has a different size in file".
Those files that have "nc1b" are bump/bump_sst_nicas_000002-000001.nc and bump/bump_sst_nicas_000002-000002.nc, and the nc1b values in these two files are 1049 and 841 respectively.
If I use the original landmask, all of the tests can pass. What might be the problem?

@travissluka The improved landmask is at the feature/betterLandmask branch.

oops, I had said the bump files should be deleted before staticbinit is run, but it turns out i forgot the rm -f $f line after https://github.com/UMD-AOSC/UMD-SST/blob/89c45319844aaa3e05c47732832816232d9441dd/test/test_wrapper.sh#L13, so the files are not actually being deleted. If you fix that it might fix the problem.
I'm away from home this week, but I can take a closer look when I get back tuesday.

closed by https://github.com/UMD-AOSC/UMD-SST/pull/63


[feature] Fix email_read Core Scope [No Ticket]

Purpose

Use the correct string when checking request scopes... h/t @cwisecarver @icereval

Ticket

None

This is another way to do it. https://stackoverflow.com/questions/19128793/per-field-permission-in-django-rest-framework?answertab=votes#tab-top
class UserSerializer(stuff):
    # all the stuff

class ReadEmailUserSerializer(UserSerializer):
   email = Email

Went with @cwisecarver's proposed solution, and also removed the users_email/user_email/read_email magic string in favor of a constant (also per @cwisecarver...).


php is running in daemon mode... make it stop

need to add:
sed -i 's/;daemonize = yes/daemonize = no/g' /etc/php/7.0/fpm/php-fpm.conf
to php.sh

testing with commit: a5138aa8e3e139f9300182fc2d41f7e6dd995326

this solution did not correct the issue. Nor did restarting nginx and php services. moving on to other issues, will return to this issue when I am able to test if it has any impact on actually processing php files.


Number of Advanced features in UI went from 37 to 1.

In the front end the advanced setting checktab has 37 on my last use. I went into ComfuUI and loaded a workflow for hi - res rendering, used the work flow and upon going back to the frontend, it now shows just one. How do I get the other 36 back without having to re-install it? In the screen shot it just says 1 in the advanced settings. I hate to install it over and over because the installer always loads the stability default checkpoint of 6 gigs!--

no need to reinstall generally
the installer only downloads the model if you leave that checked, you can just uncheck it
other settings aren't gone forever they're just hidden if irrelevant
if you've enabled a comfy workflow you can disable it at any time:

There are no workflows set. I cannot use anything if I don't have access. I tried different models, different lora's --  models. If they are hidden as you say -- however -- I need them for new prompts.

Sooomething's bugged there. You have parameters hidden but not for any clear reason. Try force-refreshing (CTRL+F5) or try Quick Tools (top right corner) -> Reset Params To Default

those options did not work, however i tried another option. I have a workflow with like 80+ nodes, I imported to generation tab once it was loaded -- went back to generation tab, refreshed-- nothing -- I went back to the comfy workflow tab removed it from there. then went back to generation tab and the 37 advanced numbers showed back. kinda like the remove workflow from the main screen is broke. Swarm now is my main driver for generation but still gotta go to auto1111 for the 3 items i mention last week, Conrolnet, ipadapter and animated diff. just loving it this thing called swarm

ControlNet is fully natively supported - though you'll want https://github.com/Fannovel16/comfyui_controlnet_aux to have more preprocessor options.
IP Adapter is supported in the main UI if you install https://github.com/cubiq/ComfyUI_IPAdapter_plus
AnimateDiff is currently unavailable outside of comfy workflows, though SVD is supported natively


Page is not loading through Docker Compose

I copied the Docker-compose.yml file to local machine (Debian 11.7).
Docker Version 23.0.0
Docker compose version 2.15.1
Showing no error either in logs but page is not loading images and showing error logs in console.

Docker logs:

I had this problem with version 1.3.4 too, so I'm using version 1.3.5.


SPM checkpoint - Storage qs runs w/o Auth - Auth builds for mac

Storage quickstart now builds and runs with Auth.  Only remaining qs workaround is the Firebase module.

Now the Firebase module works and the FirebaseStorage quickstart runs without alterations.


Add option '--skip-tx-preflight' to disable preflight checks

closes #58

@jordy25519 thanks a bunch, that was fast! I filed a feature request on Friday, and now I just pulled your fresh commits and it runs.
:pray: :clap: :clap: :clap:


Performance benchmarks: Better configuration

Feature description
Currently, we're using bash scripts to configure a Docker container (in which Conduit will be running) and also to create test pipelines. There are two implications of this:

The configuration for a single test is split into two files (run-workload.sh which runs a Docker container, and workload scripts which create pipelines)
Workload scripts are pretty verbose (cURL calls)

Once we're done with pipeline config files, we can easily configure a whole performance test with a single YML file (one part of it would configure the test instance, another would the pipeline YML config).

This issue might be a requirement before we do automated benchmarks. At this point, it is not important.


Disable indentation guides in diffview

Hello,
I'm using this plugin for a better diff mode experience in nvim. The plugins has several hooks in order to change some buffer local settings when inside a diff view. I use the diff_buf_win_enter hook to disable line-wrap, listchars, relative numbers and I would also like to disable indentation guides.
require'diffview'.setup({
    hooks = {
       diff_buf_win_enter = function()

            -- Turn off line wrapping and list chars and relative numbers
            vim.opt_local.wrap = false
            vim.opt_local.list = false
            vim.opt_local.relativenumber = false

            -- Disable indentation guides
            require("indent_blankline.commands").disable()
        end,
    }
})

However, disabling the indentation guides is not working. The hook itself works and the disable command is executed. The plugin thinks that it's in state "disabled" but the indentation guides are still shown. I can verify this with the following: I have to execute:IndentBlanklineToggle twice in the local buffer in order to turn off the indent guides (first toggle to turn them on again, second to turn them off, but this time lines are actually hidden).
Could some explain me why this is not working?
Thanks x Tummetott

This should work, and it works for me when I run :lua require("indent_blankline.commands").disable() directly.
Not sure if diffview does anything special here, but I can't think of a reason why it wouldn't work.

yes it works when I run the command like you at the cmd prompt after opening a diffview. But it has no effekt when the command is executed within the hook ... I don't get it. I'll open a issue at the diffview repo

What sindrets wrote makes sense. Can you try this PR and see if it fixes the problem?
https://github.com/lukas-reineke/indent-blankline.nvim/pull/415

Yep that solves the issue. Amazing!

:+1: I'll release it tomorrow

Sorry for the delay, I forgot about this :sweat_smile:
It's released now


Update plugin-build-metrics.yml

I would like to be able to release my plugin

Reference: https://github.com/jenkinsci/build-metrics-plugin/commits/master


Remove unneeded type declarations

Purpose
Remove unneeded/unused type declarations.
Summary of Changes

rm types/assertions.d.ts - these were for ember-cli-custom-assertions, which we are no longer using
rm types/qunit.d.ts - this was an override of qunit's types for custom negative assertions

Side Effects / Testing Notes
N/A
Ticket
N/A
Reviewer Checklist

[ ] meets requirements
[ ] easy to understand
[ ] DRY
[ ] testable and includes test(s)
[ ] changes described in CHANGELOG.md

Pull Request Test Coverage Report for Build 1380

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 61.574%

Totals

Change from base Build 1379:
0.0%

Covered Lines:
2230

Relevant Lines:
3432

💛  - Coveralls


Сan I expect update the plugin for Redmine 3.0.1?

Hello
Are there any plans to update the plugin?

Hi dima21m,
I have this plugin on 3.0.1.
I will update this repo later this week.
Thanks.


BSEC <IP_ADDRESS>

I followed all the steps explained in the readme of the repository but I encountered the fatal error
BSEC<IP_ADDRESS>_Generic_Release_23012023/algo/normal_version/bin/Raspberry/PiThree_ArmV6/bsec_interface.h: No such file or directory
What should I do to overcome this problem?
Thanks,
Fabio

i am having same issue.. any solution to this ?

The BSEC release from BOSCH Sensortec is a 32 bit library for both the V6 and V8 releases.
Rasbian 64 bit OS will not run 32 bit code out of the box.
One way forward might be to install the 32 bit libraries:
dpkg --add-architecture armhf
I have not tried this, so let us know how you get on.
There is a good article in the Debian Wiki on running multiple architectures in a single system, here


custom access log format won't break line

Use default access log format will result in logs without break lines. I have to attach a \n in format string.
"format": "[%START_TIME%] "%REQ(:METHOD)%" "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%" %PROTOCOL% %RESPONSE_CODE% %FAILURE_REASON% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%" "%REQ(X-FORWARDED-FOR?X-ENVOY-EXTERNAL-ADDRESS)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%""
Tested on ubuntu 14.04 with envoy 1.0.

This is a doc bug, the default format string compiled into the code does include a '\n'. We will fix the docs and make the lack of new line more clear. We don't want to assume a new line because different deployments might want different separators for some reason.

thanks for you clarification.

Reopening just to track doc fix

Put more details into access log docs: https://github.com/lyft/envoy/pull/93


Surelog doesn't import wires declared in packages

TEST:
package my_package1;

parameter  p1 = 1;
localparam p2 = 2;

typedef logic [1:0] word;

word v;

endpackage

module test();

import my_package1::*;

initial begin
  v = p1 + p2;
end

endmodule

Surelog fails with message:
dut.sv:17:3: Illegal lhs of type wire "v".

but it should pass.

Fixed by #2562


[Question] How to use Relax attention op in mlc-llm workflow

❓ General Questions
Recently flash attn v2 is supported by tvm and I am trying to use it to optimize performance of mlc-llm on GPU.
Currently in mlc_llm/relax_model/llama.py, the attention computation is done using many basic operations instead of Relax's attention. I tried to use relax attention op (see this code)  but it didn't quite work out.
The performance of attention is very low and seems it's not even running on GPU:
Time elapsed: encoding 4.145921468734741 seconds, decoding 0.03513669967651367 secs
Profiling...
======================= Encoding Profiling =======================
Name                                              Time (ms)   Count   Total time (ms)   Pct (%)   Memory (MB)     Bandwidth (GB/s)  Shape
attention                                         126.0235    32      4032.7533         97.21     100.38          0.7779            (1, 1001, 32, 128), (1, 1001, 32, 128), (1, 1001, 32, 128), (1, 1001, 32, 128), (32, 1001, 1001), (32, 1001), (32, 1001), (32, 1001, 128)
fused_fused_decode4_NT_matmul2                    1.3956      32      44.6596           1.08      98.23           68.7348           (22016, 512), (22016, 128), (1, 1001, 4096), (1, 1001, 22016)
fused_fused_decode5_fused_NT_matmul3_add          0.8974      32      28.7167           0.69      60.85           66.2129           (4096, 1376), (4096, 344), (1, 1001, 11008), (1, 1001, 4096), (1, 1001, 4096)
fused_fused_decode2_NT_matmul                     0.8103      32      25.9304           0.63      58.28           70.2376           (12288, 512), (12288, 128), (1, 1001, 4096), (1, 1001, 12288)
fused_fused_decode3_fused_NT_matmul1_add          0.3381      32      10.8188           0.26      32.46           93.7632           (4096, 512), (4096, 128), (1, 1001, 4096), (1, 1001, 4096), (1, 1001, 4096)
fused_split1_silu_multiply                        0.0685      32      2.1920            0.05      63.05           898.8994          (1, 1001, 22016), (1, 1001, 11008)
split                                             0.0622      32      1.9911            0.05      46.92           736.4342          (1, 1001, 12288), (1, 1001, 4096), (1, 1001, 4096), (1, 1001, 4096)
rms_norm                                          0.0203      65      1.3225            0.03      15.65           751.0578          (1, 1001, 4096), (4096,), (1, 1001, 4096)
fused_fused_decode1_fused_NT_matmul4_cast         0.0673      1       0.0673            0.00      70.44           1021.9943         (32000, 512), (32000, 128), (1, 1, 4096), (1, 1, 32000)
fused_fused_decode1_take                          0.0149      1       0.0149            0.00      78.14           5117.9188         (32000, 512), (32000, 128), (1001,), (1001, 4096)
slice                                             0.0027      1       0.0027            0.00      7.83            2878.7277         (1, 1001, 4096), (1, 1, 4096)
Total time: 4148.4693 ms

======================= Decoding Profiling =======================
Name                                              Time (ms)   Count   Total time (ms)   Pct (%)   Memory (MB)     Bandwidth (GB/s)  Shape
attention1                                        0.8677      32      27.7673           85.70     15.74           17.7153           (1, 1, 32, 128), (1, 1002, 32, 128), (1, 1002, 32, 128), (1, 1, 32, 128), (32, 1, 1002), (32, 1), (32, 1), (32, 1, 128)
fused_fused_decode4_NT_matmul7                    0.0481      32      1.5406            4.76      48.42           982.2368          (22016, 512), (22016, 128), (1, 1, 4096), (1, 1, 22016)
fused_fused_decode5_fused_NT_matmul8_add1         0.0340      32      1.0884            3.36      24.22           695.4920          (4096, 1376), (4096, 344), (1, 1, 11008), (1, 1, 4096), (1, 1, 4096)
fused_fused_decode2_NT_matmul5                    0.0288      32      0.9223            2.85      27.03           915.8815          (12288, 512), (12288, 128), (1, 1, 4096), (1, 1, 12288)
rms_norm1                                         0.0080      65      0.5225            1.61      0.02            2.8474            (1, 1, 4096), (4096,), (1, 1, 4096)
fused_fused_decode3_fused_NT_matmul6_add1         0.0124      32      0.3965            1.22      9.02            711.1918          (4096, 512), (4096, 128), (1, 1, 4096), (1, 1, 4096), (1, 1, 4096)
fused_split2_silu1_multiply1                      0.0029      32      0.0921            0.28      0.06            21.3774           (1, 1, 22016), (1, 1, 11008)
fused_fused_decode1_fused_NT_matmul4_cast         0.0674      1       0.0674            0.21      70.44           1020.3387         (32000, 512), (32000, 128), (1, 1, 4096), (1, 1, 32000)
fused_fused_decode1_take1                         0.0028      1       0.0028            0.01      70.32           24505.3005        (32000, 512), (32000, 128), (1,), (1, 4096)
Total time: 32.4000 ms

There's no proper documentations about Relax to refer to. Could anyone tell me what's the proper usage of R.nn.attention and why mlc-llm is not using it to implement the model?

See rewrite_attention function in https://github.com/mlc-ai/mlc-llm/pull/651. Flash v2 actually is not fast for a single-query workload (https://github.com/Dao-AILab/flash-attention/issues/427#issuecomment-1668257877), so for the decoder we use the xformer kernel.

thx a lot


CapRover one-click app for quickly setting up Vikunja

Incase anyone is interested and uses CapRover, I have a one-click app for Vikunja with the docker image. CapRover seems good for "we" less technical folk to use Docker with ease.
CapRover does not have Vikunja in it's one-click-apps repo.
https://github.com/CurbSoftwareInc/caprover-curb-click-apps
Also wanted to let the Vikunja team know to make sure they are cool with it - I referenced throughout my repo.

Nice!