|                                      100th percentile latency |                      index |   5919.36        |   6134.57        |    215.214   |     ms |   +3.64% |
|                                  50th percentile service time |                      index |   1409.85        |   1432.16        |     22.3136  |     ms |   +1.58% |
|                                  90th percentile service time |                      index |   2702.79        |   2656.41        |    -46.3816  |     ms |   -1.72% |
|                                  99th percentile service time |                      index |   4017.51        |   3992.06        |    -25.4543  |     ms |   -0.63% |
|                                99.9th percentile service time |                      index |   4965.88        |   4709.89        |   -255.992   |     ms |   -5.16% |
|                               99.99th percentile service time |                      index |   5472.78        |   5683.12        |    210.34    |     ms |   +3.84% |
|                                 100th percentile service time |                      index |   5919.36        |   6134.57        |    215.214   |     ms |   +3.64% |
|                                                    error rate |                      index |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput |  block-source-index-writes |     27.8368      |     30.7275      |      2.89074 |  ops/s |  +10.38% |
|                                               Mean Throughput |  block-source-index-writes |     27.8368      |     30.7275      |      2.89074 |  ops/s |  +10.38% |
|                                             Median Throughput |  block-source-index-writes |     27.8368      |     30.7275      |      2.89074 |  ops/s |  +10.38% |
|                                                Max Throughput |  block-source-index-writes |     27.8368      |     30.7275      |      2.89074 |  ops/s |  +10.38% |
|                                      100th percentile latency |  block-source-index-writes |     35.5021      |     32.0829      |     -3.41925 |     ms |   -9.63% |
|                                 100th percentile service time |  block-source-index-writes |     35.5021      |     32.0829      |     -3.41925 |     ms |   -9.63% |
|                                                    error rate |  block-source-index-writes |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput |              downsample-1m |      0.000615223 |      0.000620579 |      1e-05   |  ops/s |   +0.87% |
|                                               Mean Throughput |              downsample-1m |      0.000615223 |      0.000620579 |      1e-05   |  ops/s |   +0.87% |
|                                             Median Throughput |              downsample-1m |      0.000615223 |      0.000620579 |      1e-05   |  ops/s |   +0.87% |
|                                                Max Throughput |              downsample-1m |      0.000615223 |      0.000620579 |      1e-05   |  ops/s |   +0.87% |
|                                      100th percentile latency |              downsample-1m |      1.62543e+06 |      1.61139e+06 | -14034       |     ms |   -0.86% |
|                                 100th percentile service time |              downsample-1m |      1.62543e+06 |      1.61139e+06 | -14034       |     ms |   -0.86% |
|                                                    error rate |              downsample-1m |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput |              downsample-1h |      0.00194355  |      0.0019134   |     -3e-05   |  ops/s |   -1.55% |
|                                               Mean Throughput |              downsample-1h |      0.00194355  |      0.0019134   |     -3e-05   |  ops/s |   -1.55% |
|                                             Median Throughput |              downsample-1h |      0.00194355  |      0.0019134   |     -3e-05   |  ops/s |   -1.55% |
|                                                Max Throughput |              downsample-1h |      0.00194355  |      0.0019134   |     -3e-05   |  ops/s |   -1.55% |
|                                      100th percentile latency |              downsample-1h | 514522           | 522629           |   8106.34    |     ms |   +1.58% |
|                                 100th percentile service time |              downsample-1h | 514522           | 522629           |   8106.34    |     ms |   +1.58% |
|                                                    error rate |              downsample-1h |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput |              downsample-1d |      0.00202059  |      0.00201068  |     -1e-05   |  ops/s |   -0.49% |
|                                               Mean Throughput |              downsample-1d |      0.00202059  |      0.00201068  |     -1e-05   |  ops/s |   -0.49% |
|                                             Median Throughput |              downsample-1d |      0.00202059  |      0.00201068  |     -1e-05   |  ops/s |   -0.49% |
|                                                Max Throughput |              downsample-1d |      0.00202059  |      0.00201068  |     -1e-05   |  ops/s |   -0.49% |
|                                      100th percentile latency |              downsample-1d | 494905           | 497341           |   2435.47    |     ms |   +0.49% |
|                                 100th percentile service time |              downsample-1d | 494905           | 497341           |   2435.47    |     ms |   +0.49% |
|                                                    error rate |              downsample-1d |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput | date-histo-entire-range-1m |      2.16746     |      2.16091     |     -0.00655 |  ops/s |   -0.30% |
|                                               Mean Throughput | date-histo-entire-range-1m |      2.17702     |      2.17448     |     -0.00253 |  ops/s |   -0.12% |
|                                             Median Throughput | date-histo-entire-range-1m |      2.17812     |      2.17543     |     -0.00268 |  ops/s |   -0.12% |
|                                                Max Throughput | date-histo-entire-range-1m |      2.18219     |      2.18089     |     -0.0013  |  ops/s |   -0.06% |
|                                       50th percentile latency | date-histo-entire-range-1m |    454.869       |    454.361       |     -0.50874 |     ms |   -0.11% |
|                                       90th percentile latency | date-histo-entire-range-1m |    459.526       |    456.201       |     -3.32465 |     ms |   -0.72% |
|                                       99th percentile latency | date-histo-entire-range-1m |    470.396       |    469.468       |     -0.92839 |     ms |   -0.20% |
|                                      100th percentile latency | date-histo-entire-range-1m |    472.24        |    470.772       |     -1.46835 |     ms |   -0.31% |
|                                  50th percentile service time | date-histo-entire-range-1m |    454.869       |    454.361       |     -0.50874 |     ms |   -0.11% |
|                                  90th percentile service time | date-histo-entire-range-1m |    459.526       |    456.201       |     -3.32465 |     ms |   -0.72% |
|                                  99th percentile service time | date-histo-entire-range-1m |    470.396       |    469.468       |     -0.92839 |     ms |   -0.20% |
|                                 100th percentile service time | date-histo-entire-range-1m |    472.24        |    470.772       |     -1.46835 |     ms |   -0.31% |
|                                                    error rate | date-histo-entire-range-1m |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput | date-histo-entire-range-1h |     75.1258      |     70.4492      |     -4.67665 |  ops/s |   -6.23% |
|                                               Mean Throughput | date-histo-entire-range-1h |     75.1258      |     72.1158      |     -3.01003 |  ops/s |   -4.01% |
|                                             Median Throughput | date-histo-entire-range-1h |     75.1258      |     72.1158      |     -3.01003 |  ops/s |   -4.01% |
|                                                Max Throughput | date-histo-entire-range-1h |     75.1258      |     73.7824      |     -1.3434  |  ops/s |   -1.79% |
|                                       50th percentile latency | date-histo-entire-range-1h |     12.2315      |     12.2237      |     -0.00779 |     ms |   -0.06% |
|                                       90th percentile latency | date-histo-entire-range-1h |     12.635       |     12.5816      |     -0.05339 |     ms |   -0.42% |
|                                       99th percentile latency | date-histo-entire-range-1h |     17.6784      |     13.5774      |     -4.10091 |     ms |  -23.20% |
|                                      100th percentile latency | date-histo-entire-range-1h |     18.9449      |     13.9872      |     -4.95765 |     ms |  -26.17% |
|                                  50th percentile service time | date-histo-entire-range-1h |     12.2315      |     12.2237      |     -0.00779 |     ms |   -0.06% |
|                                  90th percentile service time | date-histo-entire-range-1h |     12.635       |     12.5816      |     -0.05339 |     ms |   -0.42% |
|                                  99th percentile service time | date-histo-entire-range-1h |     17.6784      |     13.5774      |     -4.10091 |     ms |  -23.20% |
|                                 100th percentile service time | date-histo-entire-range-1h |     18.9449      |     13.9872      |     -4.95765 |     ms |  -26.17% |
|                                                    error rate | date-histo-entire-range-1h |      0           |      0           |      0       |      % |    0.00% |
|                                                Min Throughput | date-histo-entire-range-1d |    169.101       |    153.939       |    -15.1618  |  ops/s |   -8.97% |
|                                               Mean Throughput | date-histo-entire-range-1d |    169.101       |    153.939       |    -15.1618  |  ops/s |   -8.97% |
|                                             Median Throughput | date-histo-entire-range-1d |    169.101       |    153.939       |    -15.1618  |  ops/s |   -8.97% |
|                                                Max Throughput | date-histo-entire-range-1d |    169.101       |    153.939       |    -15.1618  |  ops/s |   -8.97% |
|                                       50th percentile latency | date-histo-entire-range-1d |      5.44292     |      5.67548     |      0.23256 |     ms |   +4.27% |
|                                       90th percentile latency | date-histo-entire-range-1d |      5.66774     |      5.84934     |      0.1816  |     ms |   +3.20% |
|                                       99th percentile latency | date-histo-entire-range-1d |      7.50534     |      6.20047     |     -1.30488 |     ms |  -17.39% |
|                                      100th percentile latency | date-histo-entire-range-1d |      8.95539     |      6.22027     |     -2.73512 |     ms |  -30.54% |
|                                  50th percentile service time | date-histo-entire-range-1d |      5.44292     |      5.67548     |      0.23256 |     ms |   +4.27% |
|                                  90th percentile service time | date-histo-entire-range-1d |      5.66774     |      5.84934     |      0.1816  |     ms |   +3.20% |
|                                  99th percentile service time | date-histo-entire-range-1d |      7.50534     |      6.20047     |     -1.30488 |     ms |  -17.39% |
|                                 100th percentile service time | date-histo-entire-range-1d |      8.95539     |      6.22027     |     -2.73512 |     ms |  -30.54% |
|                                                    error rate | date-histo-entire-range-1d |      0           |      0           |      0       |      % |    0.00% |

-------------------------------
[INFO] SUCCESS (took 0 seconds)
-------------------------------

I ran the first elastic/logs comparison, again similar with some (what looks like) run-to-run variation. I'm going to re-run this again and plot the metrics side-by-side, as there are a bunch of queries with lower latency, whilst indexing is slower by ~3%.

 elastic/logs
baseline: e64ddaa3-31b5-4ca7-862f-ba2171789e32
contender: 61ee5e93-ba06-400e-87d1-e4ac985ae21d

$ esrally compare --baseline e64ddaa3-31b5-4ca7-862f-ba2171789e32 --contender 61ee5e93-ba06-400e-87d1-e4ac985ae21d

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

Comparing baseline
  Race ID: e64ddaa3-31b5-4ca7-862f-ba2171789e32
  Race timestamp: 2022-12-25 20:00:00
  Challenge: logging-querying
  Car: external
  User tags: benchmark-name=elastic-logs, benchmark-type=rally-es-client-upgrade, created-by=esbench, division=engineering, env-id=ad4930b9-160f-4f45-949e-d55afe50b4fc, git-username=Brad Deam, host-username=bradleydeam, org=elasticsearch, project=esbench-nightly, setup=esbench, team=performance

with contender
  Race ID: 61ee5e93-ba06-400e-87d1-e4ac985ae21d
  Race timestamp: 2022-12-25 20:00:00
  Challenge: logging-querying
  Car: external
  User tags: benchmark-name=elastic-logs, benchmark-type=rally-es-client-upgrade, created-by=esbench, division=engineering, env-id=01a0e4c2-867c-4816-9baf-244f446c11bf, git-username=Brad Deam, host-username=bradleydeam, org=elasticsearch, project=esbench-nightly, setup=esbench, team=performance

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                        Metric |                                   Task |        Baseline |       Contender |            Diff |   Unit |   Diff % |
|--------------------------------------------------------------:|---------------------------------------:|----------------:|----------------:|----------------:|-------:|---------:|
|                    Cumulative indexing time of primary shards |                                        |  1022.41        |  1042.33        |    19.9194      |    min |   +1.95% |
|             Min cumulative indexing time across primary shard |                                        |     3.052       |     2.92385     |    -0.12815     |    min |   -4.20% |
|          Median cumulative indexing time across primary shard |                                        |    12.3355      |    12.5978      |     0.26236     |    min |   +2.13% |
|             Max cumulative indexing time across primary shard |                                        |   163.016       |   170.748       |     7.7313      |    min |   +4.74% |
|           Cumulative indexing throttle time of primary shards |                                        |     0           |     0           |     0           |    min |    0.00% |
|    Min cumulative indexing throttle time across primary shard |                                        |     0           |     0           |     0           |    min |    0.00% |
| Median cumulative indexing throttle time across primary shard |                                        |     0           |     0           |     0           |    min |    0.00% |
|    Max cumulative indexing throttle time across primary shard |                                        |     0           |     0           |     0           |    min |    0.00% |
|                       Cumulative merge time of primary shards |                                        |   912.403       |   921.727       |     9.32352     |    min |   +1.02% |
|                      Cumulative merge count of primary shards |                                        |  4381           |  4407           |    26           |        |   +0.59% |
|                Min cumulative merge time across primary shard |                                        |     0.0486667   |     0.0809333   |     0.03227     |    min |  +66.30% |
|             Median cumulative merge time across primary shard |                                        |     1.19991     |     1.09475     |    -0.10516     |    min |   -8.76% |
|                Max cumulative merge time across primary shard |                                        |   287.551       |   276.565       |   -10.9863      |    min |   -3.82% |
|              Cumulative merge throttle time of primary shards |                                        |   499.716       |   499.867       |     0.15163     |    min |   +0.03% |
|       Min cumulative merge throttle time across primary shard |                                        |     0           |     0           |     0           |    min |    0.00% |
|    Median cumulative merge throttle time across primary shard |                                        |     0.241108    |     0.23115     |    -0.00996     |    min |   -4.13% |
|       Max cumulative merge throttle time across primary shard |                                        |   161.998       |   155.37        |    -6.62868     |    min |   -4.09% |
|                     Cumulative refresh time of primary shards |                                        |    68.3403      |    69.6101      |     1.26982     |    min |   +1.86% |
|                    Cumulative refresh count of primary shards |                                        | 14241           | 14397           |   156           |        |   +1.10% |
|              Min cumulative refresh time across primary shard |                                        |     0.0214333   |     0.0180667   |    -0.00337     |    min |  -15.71% |
|           Median cumulative refresh time across primary shard |                                        |     0.236492    |     0.264617    |     0.02813     |    min |  +11.89% |
|              Max cumulative refresh time across primary shard |                                        |    17.2647      |    16.9812      |    -0.2835      |    min |   -1.64% |
|                       Cumulative flush time of primary shards |                                        |    49.3541      |    50.3453      |     0.99125     |    min |   +2.01% |
|                      Cumulative flush count of primary shards |                                        |  1820           |  1844           |    24           |        |   +1.32% |
|                Min cumulative flush time across primary shard |                                        |     0.187833    |     0.1742      |    -0.01363     |    min |   -7.26% |
|             Median cumulative flush time across primary shard |                                        |     1.05712     |     1.09666     |     0.03954     |    min |   +3.74% |
|                Max cumulative flush time across primary shard |                                        |     2.98335     |     3.44165     |     0.4583      |    min |  +15.36% |
|                                       Total Young Gen GC time |                                        |   219.325       |   221.089       |     1.764       |      s |   +0.80% |
|                                      Total Young Gen GC count |                                        | 14574           | 14500           |   -74           |        |   -0.51% |
|                                         Total Old Gen GC time |                                        |     0           |     0           |     0           |      s |    0.00% |
|                                        Total Old Gen GC count |                                        |     0           |     0           |     0           |        |    0.00% |
|                                                    Store size |                                        |   186.542       |   186.423       |    -0.11838     |     GB |   -0.06% |
|                                                 Translog size |                                        |     4.30271e-06 |     4.30271e-06 |     0           |     GB |    0.00% |
|                                        Heap used for segments |                                        |     0           |     0           |     0           |     MB |    0.00% |
|                                      Heap used for doc values |                                        |     0           |     0           |     0           |     MB |    0.00% |
|                                           Heap used for terms |                                        |     0           |     0           |     0           |     MB |    0.00% |
|                                           Heap used for norms |                                        |     0           |     0           |     0           |     MB |    0.00% |
|                                          Heap used for points |                                        |     0           |     0           |     0           |     MB |    0.00% |
|                                   Heap used for stored fields |                                        |     0           |     0           |     0           |     MB |    0.00% |
|                                                 Segment count |                                        |   831           |   861           |    30           |        |   +3.61% |
|                                   Total Ingest Pipeline count |                                        |     6.68026e+08 |     6.68026e+08 |     0           |        |    0.00% |
|                                    Total Ingest Pipeline time |                                        |     4.49945e+07 |     4.62659e+07 |     1.27138e+06 |     ms |   +2.83% |
|                                  Total Ingest Pipeline failed |                                        |     0           |     0           |     0           |        |    0.00% |
|                                                Min Throughput |                       insert-pipelines |     6.58154     |     6.25561     |    -0.32594     |  ops/s |   -4.95% |
|                                               Mean Throughput |                       insert-pipelines |     6.58154     |     6.25561     |    -0.32594     |  ops/s |   -4.95% |
|                                             Median Throughput |                       insert-pipelines |     6.58154     |     6.25561     |    -0.32594     |  ops/s |   -4.95% |
|                                                Max Throughput |                       insert-pipelines |     6.58154     |     6.25561     |    -0.32594     |  ops/s |   -4.95% |
|                                      100th percentile latency |                       insert-pipelines |  2275.77        |  2375.78        |   100.004       |     ms |   +4.39% |
|                                 100th percentile service time |                       insert-pipelines |  2275.77        |  2375.78        |   100.004       |     ms |   +4.39% |
|                                                    error rate |                       insert-pipelines |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                             insert-ilm |    22.7674      |    23.1781      |     0.41066     |  ops/s |   +1.80% |
|                                               Mean Throughput |                             insert-ilm |    22.7674      |    23.1781      |     0.41066     |  ops/s |   +1.80% |
|                                             Median Throughput |                             insert-ilm |    22.7674      |    23.1781      |     0.41066     |  ops/s |   +1.80% |
|                                                Max Throughput |                             insert-ilm |    22.7674      |    23.1781      |     0.41066     |  ops/s |   +1.80% |
|                                      100th percentile latency |                             insert-ilm |    42.361       |    41.7791      |    -0.58197     |     ms |   -1.37% |
|                                 100th percentile service time |                             insert-ilm |    42.361       |    41.7791      |    -0.58197     |     ms |   -1.37% |
|                                                    error rate |                             insert-ilm |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput | validate-package-template-installation |    62.4097      |    55.9017      |    -6.50797     |  ops/s |  -10.43% |
|                                               Mean Throughput | validate-package-template-installation |    62.4097      |    55.9017      |    -6.50797     |  ops/s |  -10.43% |
|                                             Median Throughput | validate-package-template-installation |    62.4097      |    55.9017      |    -6.50797     |  ops/s |  -10.43% |
|                                                Max Throughput | validate-package-template-installation |    62.4097      |    55.9017      |    -6.50797     |  ops/s |  -10.43% |
|                                      100th percentile latency | validate-package-template-installation |    15.498       |    17.4284      |     1.9304      |     ms |  +12.46% |
|                                 100th percentile service time | validate-package-template-installation |    15.498       |    17.4284      |     1.9304      |     ms |  +12.46% |
|                                                    error rate | validate-package-template-installation |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |        update-custom-package-templates |    26.1971      |    23.5839      |    -2.61318     |  ops/s |   -9.98% |
|                                               Mean Throughput |        update-custom-package-templates |    26.1971      |    23.5839      |    -2.61318     |  ops/s |   -9.98% |
|                                             Median Throughput |        update-custom-package-templates |    26.1971      |    23.5839      |    -2.61318     |  ops/s |   -9.98% |
|                                                Max Throughput |        update-custom-package-templates |    26.1971      |    23.5839      |    -2.61318     |  ops/s |   -9.98% |
|                                      100th percentile latency |        update-custom-package-templates |   457.548       |   508.357       |    50.809       |     ms |  +11.10% |
|                                 100th percentile service time |        update-custom-package-templates |   457.548       |   508.357       |    50.809       |     ms |  +11.10% |
|                                                    error rate |        update-custom-package-templates |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                             bulk-index |   709.029       |   684.917       |   -24.1118      | docs/s |   -3.40% |
|                                               Mean Throughput |                             bulk-index | 23523           | 22832.4         |  -690.631       | docs/s |   -2.94% |
|                                             Median Throughput |                             bulk-index | 23201.6         | 22497           |  -704.572       | docs/s |   -3.04% |
|                                                Max Throughput |                             bulk-index | 26451.9         | 26229.4         |  -222.527       | docs/s |   -0.84% |
|                                       50th percentile latency |                             bulk-index |   430.432       |   437.91        |     7.47786     |     ms |   +1.74% |
|                                       90th percentile latency |                             bulk-index |   872.946       |   885.136       |    12.1898      |     ms |   +1.40% |
|                                       99th percentile latency |                             bulk-index |  1481.41        |  1498.31        |    16.8984      |     ms |   +1.14% |
|                                     99.9th percentile latency |                             bulk-index |  2178.17        |  2219.83        |    41.6538      |     ms |   +1.91% |
|                                    99.99th percentile latency |                             bulk-index |  4422.29        |  4440.53        |    18.2402      |     ms |   +0.41% |
|                                      100th percentile latency |                             bulk-index |  7014.32        |  8246.53        |  1232.21        |     ms |  +17.57% |
|                                  50th percentile service time |                             bulk-index |   430.541       |   438.026       |     7.48527     |     ms |   +1.74% |
|                                  90th percentile service time |                             bulk-index |   872.473       |   885.129       |    12.6563      |     ms |   +1.45% |
|                                  99th percentile service time |                             bulk-index |  1481.36        |  1498.42        |    17.058       |     ms |   +1.15% |
|                                99.9th percentile service time |                             bulk-index |  2178.02        |  2219.45        |    41.4345      |     ms |   +1.90% |
|                               99.99th percentile service time |                             bulk-index |  4422.29        |  4440.53        |    18.2402      |     ms |   +0.41% |
|                                 100th percentile service time |                             bulk-index |  7014.32        |  8246.53        |  1232.21        |     ms |  +17.57% |
|                                                    error rate |                             bulk-index |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                        discover/search |     0.0011454   |     0.0011455   |     0           |  ops/s |    0.01% |
|                                               Mean Throughput |                        discover/search |     0.00259723  |     0.00265597  |     6e-05       |  ops/s |   +2.26% |
|                                             Median Throughput |                        discover/search |     0.0027322   |     0.00291103  |     0.00018     |  ops/s |   +6.55% |
|                                                Max Throughput |                        discover/search |     0.00384711  |     0.00384729  |     0           |  ops/s |    0.00% |
|                                       50th percentile latency |                        discover/search |   521.415       |   467.002       |   -54.4125      |     ms |  -10.44% |
|                                       90th percentile latency |                        discover/search |  2446.15        |  2426.18        |   -19.9706      |     ms |   -0.82% |
|                                      100th percentile latency |                        discover/search |  2782.69        |  2667.41        |  -115.284       |     ms |   -4.14% |
|                                  50th percentile service time |                        discover/search |   468.879       |   436.658       |   -32.221       |     ms |   -6.87% |
|                                  90th percentile service time |                        discover/search |  1087.06        |  1082.15        |    -4.91472     |     ms |   -0.45% |
|                                 100th percentile service time |                        discover/search |  2299.63        |  2321.17        |    21.5439      |     ms |   +0.94% |
|                                                    error rate |                        discover/search |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                     discover/visualize |     0.00115754  |     0.00115754  |     0           |  ops/s |    0.00% |
|                                               Mean Throughput |                     discover/visualize |     0.0042437   |     0.00424385  |     0           |  ops/s |    0.00% |
|                                             Median Throughput |                     discover/visualize |     0.00415885  |     0.00415883  |    -0           |  ops/s |   -0.00% |
|                                                Max Throughput |                     discover/visualize |     0.00703735  |     0.00703735  |     0           |  ops/s |    0.00% |
|                                       50th percentile latency |                     discover/visualize |   440.027       |   453.018       |    12.991       |     ms |   +2.95% |
|                                       90th percentile latency |                     discover/visualize |  7378.09        |  6416.89        |  -961.207       |     ms |  -13.03% |
|                                      100th percentile latency |                     discover/visualize | 10599           |  9638.81        |  -960.166       |     ms |   -9.06% |
|                                  50th percentile service time |                     discover/visualize |   437.753       |   450.806       |    13.0526      |     ms |   +2.98% |
|                                  90th percentile service time |                     discover/visualize |   984.497       |   891.715       |   -92.7817      |     ms |   -9.42% |
|                                 100th percentile service time |                     discover/visualize | 10595.9         |  9635.66        |  -960.239       |     ms |   -9.06% |
|                                                    error rate |                     discover/visualize |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                                  kafka |     0.00250883  |     0.00250884  |     0           |  ops/s |    0.00% |
|                                               Mean Throughput |                                  kafka |     0.00529489  |     0.00523735  |    -6e-05       |  ops/s |   -1.09% |
|                                             Median Throughput |                                  kafka |     0.00514966  |     0.00510487  |    -4e-05       |  ops/s |   -0.87% |
|                                                Max Throughput |                                  kafka |     0.0091963   |     0.00919634  |     0           |  ops/s |    0.00% |
|                                       50th percentile latency |                                  kafka |   460.908       |   450.167       |   -10.741       |     ms |   -2.33% |
|                                       90th percentile latency |                                  kafka |   733.533       |   706.297       |   -27.2365      |     ms |   -3.71% |
|                                      100th percentile latency |                                  kafka |  5654.53        |  5177.3         |  -477.234       |     ms |   -8.44% |
|                                  50th percentile service time |                                  kafka |   459.248       |   448.449       |   -10.7987      |     ms |   -2.35% |
|                                  90th percentile service time |                                  kafka |   732.566       |   704.463       |   -28.1023      |     ms |   -3.84% |
|                                 100th percentile service time |                                  kafka |  5652.76        |  5176.79        |  -475.963       |     ms |   -8.42% |
|                                                    error rate |                                  kafka |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                                  nginx |     0.00322612  |     0.00322612  |     0           |  ops/s |    0.00% |
|                                               Mean Throughput |                                  nginx |     0.0113405   |     0.0113406   |     0           |  ops/s |    0.00% |
|                                             Median Throughput |                                  nginx |     0.0113286   |     0.0113286   |     0           |  ops/s |    0.00% |
|                                                Max Throughput |                                  nginx |     0.0169596   |     0.0169596   |     0           |  ops/s |    0.00% |
|                                       50th percentile latency |                                  nginx |  1733.79        |  1730.05        |    -3.73206     |     ms |   -0.22% |
|                                       90th percentile latency |                                  nginx |  2991.66        |  2979.85        |   -11.8091      |     ms |   -0.39% |
|                                      100th percentile latency |                                  nginx |  4748.47        |  5288.53        |   540.057       |     ms |  +11.37% |
|                                  50th percentile service time |                                  nginx |  1716.7         |  1715.16        |    -1.54541     |     ms |   -0.09% |
|                                  90th percentile service time |                                  nginx |  2434.06        |  2205.79        |  -228.272       |     ms |   -9.38% |
|                                 100th percentile service time |                                  nginx |  4747.4         |  5287.25        |   539.858       |     ms |  +11.37% |
|                                                    error rate |                                  nginx |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                                 apache |     0.0113867   |     0.011386    |    -0           |  ops/s |   -0.01% |
|                                               Mean Throughput |                                 apache |     0.0353251   |     0.0353108   |    -1e-05       |  ops/s |   -0.04% |
|                                             Median Throughput |                                 apache |     0.039014    |     0.0389979   |    -2e-05       |  ops/s |   -0.04% |
|                                                Max Throughput |                                 apache |     0.0671222   |     0.0670454   |    -8e-05       |  ops/s |   -0.11% |
|                                       50th percentile latency |                                 apache |   462.079       |   431.859       |   -30.2204      |     ms |   -6.54% |
|                                       90th percentile latency |                                 apache |   845.805       |   632.908       |  -212.897       |     ms |  -25.17% |
|                                      100th percentile latency |                                 apache |  2155.19        |  1507.31        |  -647.887       |     ms |  -30.06% |
|                                  50th percentile service time |                                 apache |   455.295       |   387.825       |   -67.47        |     ms |  -14.82% |
|                                  90th percentile service time |                                 apache |   658.437       |   630.928       |   -27.5095      |     ms |   -4.18% |
|                                 100th percentile service time |                                 apache |  2154.12        |  1506.68        |  -647.431       |     ms |  -30.06% |
|                                                    error rate |                                 apache |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                            system/auth |     0.027945    |     0.027945    |     0           |  ops/s |    0.00% |
|                                               Mean Throughput |                            system/auth |     0.047061    |     0.0470608   |    -0           |  ops/s |   -0.00% |
|                                             Median Throughput |                            system/auth |     0.0490516   |     0.0490516   |     0           |  ops/s |    0.00% |
|                                                Max Throughput |                            system/auth |     0.0668275   |     0.0668277   |     0           |  ops/s |    0.00% |
|                                       50th percentile latency |                            system/auth |   360.863       |   359.239       |    -1.62357     |     ms |   -0.45% |
|                                       90th percentile latency |                            system/auth |   601.751       |   552.583       |   -49.1674      |     ms |   -8.17% |
|                                       99th percentile latency |                            system/auth |  2303.95        |  2312.41        |     8.45625     |     ms |   +0.37% |
|                                      100th percentile latency |                            system/auth |  2361.13        |  2434.03        |    72.8955      |     ms |   +3.09% |
|                                  50th percentile service time |                            system/auth |   357.412       |   355.205       |    -2.2077      |     ms |   -0.62% |
|                                  90th percentile service time |                            system/auth |   600.131       |   511.53        |   -88.601       |     ms |  -14.76% |
|                                  99th percentile service time |                            system/auth |  2302.36        |  2311.03        |     8.67571     |     ms |   +0.38% |
|                                 100th percentile service time |                            system/auth |  2359.64        |  2432.88        |    73.2449      |     ms |   +3.10% |
|                                                    error rate |                            system/auth |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                system/syslog/dashboard |     0.0120582   |     0.0120585   |     0           |  ops/s |    0.00% |
|                                               Mean Throughput |                system/syslog/dashboard |     0.0257416   |     0.0257438   |     0           |  ops/s |    0.01% |
|                                             Median Throughput |                system/syslog/dashboard |     0.0252089   |     0.0252101   |     0           |  ops/s |    0.00% |
|                                                Max Throughput |                system/syslog/dashboard |     0.0333809   |     0.0333836   |     0           |  ops/s |    0.01% |
|                                       50th percentile latency |                system/syslog/dashboard |   389.411       |   392.09        |     2.67874     |     ms |   +0.69% |
|                                       90th percentile latency |                system/syslog/dashboard |   544.788       |   517.994       |   -26.7945      |     ms |   -4.92% |
|                                      100th percentile latency |                system/syslog/dashboard |  1246.3         |  1057.68        |  -188.624       |     ms |  -15.13% |
|                                  50th percentile service time |                system/syslog/dashboard |   382.237       |   386.055       |     3.81789     |     ms |   +1.00% |
|                                  90th percentile service time |                system/syslog/dashboard |   530.408       |   501.468       |   -28.9405      |     ms |   -5.46% |
|                                 100th percentile service time |                system/syslog/dashboard |  1243.93        |  1055.84        |  -188.09        |     ms |  -15.12% |
|                                                    error rate |                system/syslog/dashboard |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                     system/syslog/lens |     0.00622729  |     0.00622727  |    -0           |  ops/s |   -0.00% |
|                                               Mean Throughput |                     system/syslog/lens |     0.0107395   |     0.0107395   |    -0           |  ops/s |   -0.00% |
|                                             Median Throughput |                     system/syslog/lens |     0.0108108   |     0.0108108   |    -0           |  ops/s |   -0.00% |
|                                                Max Throughput |                     system/syslog/lens |     0.0144265   |     0.0144264   |    -0           |  ops/s |   -0.00% |
|                                       50th percentile latency |                     system/syslog/lens |   858.045       |   857.893       |    -0.15125     |     ms |   -0.02% |
|                                       90th percentile latency |                     system/syslog/lens |  1241.13        |  1100.52        |  -140.608       |     ms |  -11.33% |
|                                      100th percentile latency |                     system/syslog/lens |  1966.16        |  1609.52        |  -356.638       |     ms |  -18.14% |
|                                  50th percentile service time |                     system/syslog/lens |   791.453       |   756.912       |   -34.5406      |     ms |   -4.36% |
|                                  90th percentile service time |                     system/syslog/lens |  1122.73        |  1053.57        |   -69.1552      |     ms |   -6.16% |
|                                 100th percentile service time |                     system/syslog/lens |  1965.35        |  1608.35        |  -356.999       |     ms |  -18.16% |
|                                                    error rate |                     system/syslog/lens |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                        mysql/dashboard |     0.00112503  |     0.001125    |    -0           |  ops/s |   -0.00% |
|                                               Mean Throughput |                        mysql/dashboard |     0.00505707  |     0.005057    |    -0           |  ops/s |   -0.00% |
|                                             Median Throughput |                        mysql/dashboard |     0.00525097  |     0.00525092  |    -0           |  ops/s |   -0.00% |
|                                                Max Throughput |                        mysql/dashboard |     0.00741984  |     0.0074197   |    -0           |  ops/s |   -0.00% |
|                                       50th percentile latency |                        mysql/dashboard |   331.51        |   329.357       |    -2.15297     |     ms |   -0.65% |
|                                       90th percentile latency |                        mysql/dashboard |   463.387       |   467.433       |     4.04657     |     ms |   +0.87% |
|                                      100th percentile latency |                        mysql/dashboard |   593.428       |   599.147       |     5.71881     |     ms |   +0.96% |
|                                  50th percentile service time |                        mysql/dashboard |   329.84        |   328.162       |    -1.67801     |     ms |   -0.51% |
|                                  90th percentile service time |                        mysql/dashboard |   456.319       |   448.486       |    -7.83364     |     ms |   -1.72% |
|                                 100th percentile service time |                        mysql/dashboard |   491.589       |   498.405       |     6.81619     |     ms |   +1.39% |
|                                                    error rate |                        mysql/dashboard |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                                  redis |     0.004778    |     0.00477841  |     0           |  ops/s |    0.01% |
|                                               Mean Throughput |                                  redis |     0.0144478   |     0.0144484   |     0           |  ops/s |    0.00% |
|                                             Median Throughput |                                  redis |     0.0103472   |     0.0103473   |     0           |  ops/s |    0.00% |
|                                                Max Throughput |                                  redis |     0.0338673   |     0.0338693   |     0           |  ops/s |    0.01% |
|                                       50th percentile latency |                                  redis |   544.124       |   535.442       |    -8.68192     |     ms |   -1.60% |
|                                       90th percentile latency |                                  redis |  2749.03        |  2611.25        |  -137.788       |     ms |   -5.01% |
|                                      100th percentile latency |                                  redis |  4625.62        |  4589.29        |   -36.3223      |     ms |   -0.79% |
|                                  50th percentile service time |                                  redis |   480.825       |   484.627       |     3.80234     |     ms |   +0.79% |
|                                  90th percentile service time |                                  redis |  1471.74        |  1341.8         |  -129.94        |     ms |   -8.83% |
|                                 100th percentile service time |                                  redis |  4623.79        |  4587.53        |   -36.2544      |     ms |   -0.78% |
|                                                    error rate |                                  redis |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                             mysql/lens |     0.00337621  |     0.00337618  |    -0           |  ops/s |   -0.00% |
|                                               Mean Throughput |                             mysql/lens |     0.0136494   |     0.0136486   |    -0           |  ops/s |   -0.01% |
|                                             Median Throughput |                             mysql/lens |     0.0140874   |     0.0140867   |    -0           |  ops/s |   -0.00% |
|                                                Max Throughput |                             mysql/lens |     0.0223932   |     0.0223905   |    -0           |  ops/s |   -0.01% |
|                                       50th percentile latency |                             mysql/lens |  1008.97        |   976.339       |   -32.6282      |     ms |   -3.23% |
|                                       90th percentile latency |                             mysql/lens |  4446.84        |  4052.06        |  -394.777       |     ms |   -8.88% |
|                                      100th percentile latency |                             mysql/lens |  4894.51        |  4306.24        |  -588.269       |     ms |  -12.02% |
|                                  50th percentile service time |                             mysql/lens |   979.891       |   974.966       |    -4.92413     |     ms |   -0.50% |
|                                  90th percentile service time |                             mysql/lens |  4445.52        |  4050.27        |  -395.252       |     ms |   -8.89% |
|                                 100th percentile service time |                             mysql/lens |  4892.59        |  4304.41        |  -588.188       |     ms |  -12.02% |
|                                                    error rate |                             mysql/lens |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                    postgresql/overview |     0.00256505  |     0.00256513  |     0           |  ops/s |    0.00% |
|                                               Mean Throughput |                    postgresql/overview |     0.0277639   |     0.0277643   |     0           |  ops/s |    0.00% |
|                                             Median Throughput |                    postgresql/overview |     0.0294589   |     0.0294595   |     0           |  ops/s |    0.00% |
|                                                Max Throughput |                    postgresql/overview |     0.0405386   |     0.0405392   |     0           |  ops/s |    0.00% |
|                                       50th percentile latency |                    postgresql/overview |  1023.96        |  1022.52        |    -1.43839     |     ms |   -0.14% |
|                                       90th percentile latency |                    postgresql/overview |  1156.5         |  1151.44        |    -5.06165     |     ms |   -0.44% |
|                                       99th percentile latency |                    postgresql/overview |  1678.71        |  1678.72        |     0.00578     |     ms |    0.00% |
|                                      100th percentile latency |                    postgresql/overview |  1681.08        |  1680.86        |    -0.21753     |     ms |   -0.01% |
|                                  50th percentile service time |                    postgresql/overview |  1021.39        |  1021.12        |    -0.27118     |     ms |   -0.03% |
|                                  90th percentile service time |                    postgresql/overview |  1107.32        |  1094.65        |   -12.6724      |     ms |   -1.14% |
|                                  99th percentile service time |                    postgresql/overview |  1677.48        |  1676.79        |    -0.69524     |     ms |   -0.04% |
|                                 100th percentile service time |                    postgresql/overview |  1679.29        |  1678.32        |    -0.97009     |     ms |   -0.06% |
|                                                    error rate |                    postgresql/overview |     0           |     0           |     0           |      % |    0.00% |
|                                                Min Throughput |                    postgresql/duration |     0.0110205   |     0.01102     |    -0           |  ops/s |   -0.00% |
|                                               Mean Throughput |                    postgresql/duration |     0.0196583   |     0.0197319   |     7e-05       |  ops/s |   +0.37% |
|                                             Median Throughput |                    postgresql/duration |     0.0153411   |     0.0153561   |     2e-05       |  ops/s |   +0.10% |
|                                                Max Throughput |                    postgresql/duration |     0.0375056   |     0.0374991   |    -1e-05       |  ops/s |   -0.02% |
|                                       50th percentile latency |                    postgresql/duration |   351.961       |   345.079       |    -6.88199     |     ms |   -1.96% |
|                                       90th percentile latency |                    postgresql/duration |  1991.16        |  1817.18        |  -173.98        |     ms |   -8.74% |
|                                      100th percentile latency |                    postgresql/duration |  9300.07        |  7879.88        | -1420.2         |     ms |  -15.27% |
|                                  50th percentile service time |                    postgresql/duration |   338.743       |   335.812       |    -2.93097     |     ms |   -0.87% |
|                                  90th percentile service time |                    postgresql/duration |  1743.66        |  1620.82        |  -122.846       |     ms |   -7.05% |
|                                 100th percentile service time |                    postgresql/duration |  9298.7         |  7877.71        | -1420.99        |     ms |  -15.28% |
|                                                    error rate |                    postgresql/duration |     0           |     0           |     0           |      % |    0.00% |

-------------------------------
[INFO] SUCCESS (took 0 seconds)
-------------------------------

I also need to revisit some of the rally-tracks PRs around named arguments and backport them.

I re-ran the elastic/logs comparison, and I think I'm comfortable with the difference being run-to-run variation. This time the different in median throughput is < 1%:
-Median Throughput |                             bulk-index | 22988.9         | 22772.2         |    -216.708   | docs/s |   -0.94% |

To further confirm I compared the bulk-index throughput to the nightly benchmarks over the last 7 weeks:

This is really great work @b-deam.
One question/suggestion: Do you think it's worth running a CCS benchmark with this PR? A priori, I don't think there would be any issues there, but instantiating clients for multiple clusters is a code path that isn't otherwise exercised by the testing you've done, so it may be worth a sanity check.

Good suggestion. I tested the CCS challenge locally, but for completeness' sake I ran a comparison using the nightly configuration. The results look OK to me.

 ccs
baseline: 218b1a1e-e73e-4aba-a895-bdfe6ac5d47c
contender: 51ed0fb6-5e2a-476f-9e86-00bcbdd0ec19
$ esrally compare --baseline 218b1a1e-e73e-4aba-a895-bdfe6ac5d47c --contender 51ed0fb6-5e2a-476f-9e86-00bcbdd0ec19

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

Comparing baseline
  Race ID: 218b1a1e-e73e-4aba-a895-bdfe6ac5d47c
  Race timestamp: 2022-12-25 20:00:00
  Challenge: cross-clusters-search-and-snapshot
  Car: external
  User tags: benchmark-name=ccs, benchmark-type=rally-es-client-upgrade, created-by=esbench, division=engineering, env-id=27e4c9c7-e516-4572-8d56-9cba273853a7, git-username=Brad Deam, host-username=bradleydeam, org=elasticsearch, project=esbench-nightly, setup=esbench, team=performance

with contender
  Race ID: 51ed0fb6-5e2a-476f-9e86-00bcbdd0ec19
  Race timestamp: 2022-12-25 20:00:00
  Challenge: cross-clusters-search-and-snapshot
  Car: external
  User tags: benchmark-name=ccs, benchmark-type=rally-es-client-upgrade, created-by=esbench, division=engineering, env-id=e7ec9d18-e84f-46a3-9ea4-5d113ec9e579, git-username=Brad Deam, host-username=bradleydeam, org=elasticsearch, project=esbench-nightly, setup=esbench, team=performance

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                        Metric |                                                                  Task |         Baseline |        Contender |          Diff |   Unit |   Diff % |
|--------------------------------------------------------------:|----------------------------------------------------------------------:|-----------------:|-----------------:|--------------:|-------:|---------:|
|                    Cumulative indexing time of primary shards |                                                                       |    121.781       |    120.467       |      -1.3146  |    min |   -1.08% |
|             Min cumulative indexing time across primary shard |                                                                       |      0.00378333  |      0.00381667  |       3e-05   |    min |   +0.88% |
|          Median cumulative indexing time across primary shard |                                                                       |      0.0186333   |      0.0182333   |      -0.0004  |    min |   -2.15% |
|             Max cumulative indexing time across primary shard |                                                                       |      0.25585     |      0.259267    |       0.00342 |    min |   +1.34% |
|           Cumulative indexing throttle time of primary shards |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|    Min cumulative indexing throttle time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
| Median cumulative indexing throttle time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|    Max cumulative indexing throttle time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|                       Cumulative merge time of primary shards |                                                                       |    121.869       |    125.728       |       3.85937 |    min |   +3.17% |
|                      Cumulative merge count of primary shards |                                                                       |  25085           |  25279           |     194       |        |   +0.77% |
|                Min cumulative merge time across primary shard |                                                                       |      0.000183333 |      0.0002      |       2e-05   |    min |   +9.09% |
|             Median cumulative merge time across primary shard |                                                                       |      0.0025      |      0.0027      |       0.0002  |    min |   +8.00% |
|                Max cumulative merge time across primary shard |                                                                       |      0.48085     |      0.528917    |       0.04807 |    min |  +10.00% |
|              Cumulative merge throttle time of primary shards |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|       Min cumulative merge throttle time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|    Median cumulative merge throttle time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|       Max cumulative merge throttle time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|                     Cumulative refresh time of primary shards |                                                                       |    109.815       |    110.781       |       0.96625 |    min |   +0.88% |
|                    Cumulative refresh count of primary shards |                                                                       | 201570           | 202507           |     937       |        |   +0.46% |
|              Min cumulative refresh time across primary shard |                                                                       |      0.00025     |      0.000283333 |       3e-05   |    min |  +13.33% |
|           Median cumulative refresh time across primary shard |                                                                       |      0.00705833  |      0.00736667  |       0.00031 |    min |   +4.37% |
|              Max cumulative refresh time across primary shard |                                                                       |      0.301733    |      0.308333    |       0.0066  |    min |   +2.19% |
|                       Cumulative flush time of primary shards |                                                                       |      2.23607     |      2.39573     |       0.15967 |    min |   +7.14% |
|                      Cumulative flush count of primary shards |                                                                       |   4756           |   4750           |      -6       |        |   -0.13% |
|                Min cumulative flush time across primary shard |                                                                       |      0           |      0           |       0       |    min |    0.00% |
|             Median cumulative flush time across primary shard |                                                                       |      0.000333333 |      0.0003      |      -3e-05   |    min |  -10.00% |
|                Max cumulative flush time across primary shard |                                                                       |      0.0120167   |      0.0116      |      -0.00042 |    min |   -3.47% |
|                                       Total Young Gen GC time |                                                                       |    151.959       |    151.062       |      -0.897   |      s |   -0.59% |
|                                      Total Young Gen GC count |                                                                       |   1555           |   1534           |     -21       |        |   -1.35% |
|                                         Total Old Gen GC time |                                                                       |      0           |      0           |       0       |      s |    0.00% |
|                                        Total Old Gen GC count |                                                                       |      0           |      0           |       0       |        |    0.00% |
|                                                    Store size |                                                                       |     10.6716      |     10.6759      |       0.00434 |     GB |   +0.04% |
|                                                 Translog size |                                                                       |      0.000166474 |      0.000166474 |       0       |     GB |    0.00% |
|                                        Heap used for segments |                                                                       |      0           |      0           |       0       |     MB |    0.00% |
|                                      Heap used for doc values |                                                                       |      0           |      0           |       0       |     MB |    0.00% |
|                                           Heap used for terms |                                                                       |      0           |      0           |       0       |     MB |    0.00% |
|                                           Heap used for norms |                                                                       |      0           |      0           |       0       |     MB |    0.00% |
|                                          Heap used for points |                                                                       |      0           |      0           |       0       |     MB |    0.00% |
|                                   Heap used for stored fields |                                                                       |      0           |      0           |       0       |     MB |    0.00% |
|                                                 Segment count |                                                                       |   3250           |   3250           |       0       |        |    0.00% |
|                                   Total Ingest Pipeline count |                                                                       |      9.83069e+07 |      9.83069e+07 |       0       |        |    0.00% |
|                                    Total Ingest Pipeline time |                                                                       |      1.19726e+07 |      1.14634e+07 | -509234       |     ms |   -4.25% |
|                                  Total Ingest Pipeline failed |                                                                       |      0           |      0           |       0       |        |    0.00% |
|                                                Min Throughput |                           delete-existing-datastreams-on-all-clusters |      6.79743     |      6.64217     |      -0.15526 |  ops/s |   -2.28% |
|                                               Mean Throughput |                           delete-existing-datastreams-on-all-clusters |      6.79743     |      6.64217     |      -0.15526 |  ops/s |   -2.28% |
|                                             Median Throughput |                           delete-existing-datastreams-on-all-clusters |      6.79743     |      6.64217     |      -0.15526 |  ops/s |   -2.28% |
|                                                Max Throughput |                           delete-existing-datastreams-on-all-clusters |      6.79743     |      6.64217     |      -0.15526 |  ops/s |   -2.28% |
|                                      100th percentile latency |                           delete-existing-datastreams-on-all-clusters |    292.499       |    299.258       |       6.75897 |     ms |   +2.31% |
|                                 100th percentile service time |                           delete-existing-datastreams-on-all-clusters |    292.499       |    299.258       |       6.75897 |     ms |   +2.31% |
|                                                    error rate |                           delete-existing-datastreams-on-all-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                tune-recovery-settings-on-all-clusters |      3.33942     |      3.37792     |       0.03849 |  ops/s |   +1.15% |
|                                               Mean Throughput |                                tune-recovery-settings-on-all-clusters |      3.33942     |      3.37792     |       0.03849 |  ops/s |   +1.15% |
|                                             Median Throughput |                                tune-recovery-settings-on-all-clusters |      3.33942     |      3.37792     |       0.03849 |  ops/s |   +1.15% |
|                                                Max Throughput |                                tune-recovery-settings-on-all-clusters |      3.33942     |      3.37792     |       0.03849 |  ops/s |   +1.15% |
|                                      100th percentile latency |                                tune-recovery-settings-on-all-clusters |    298.628       |    294.712       |      -3.91669 |     ms |   -1.31% |
|                                 100th percentile service time |                                tune-recovery-settings-on-all-clusters |    298.628       |    294.712       |      -3.91669 |     ms |   -1.31% |
|                                                    error rate |                                tune-recovery-settings-on-all-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                      insert-pipelines |     12.7278      |     11.5084      |      -1.2194  |  ops/s |   -9.58% |
|                                               Mean Throughput |                                                      insert-pipelines |     12.7278      |     11.5084      |      -1.2194  |  ops/s |   -9.58% |
|                                             Median Throughput |                                                      insert-pipelines |     12.7278      |     11.5084      |      -1.2194  |  ops/s |   -9.58% |
|                                                Max Throughput |                                                      insert-pipelines |     12.7278      |     11.5084      |      -1.2194  |  ops/s |   -9.58% |
|                                      100th percentile latency |                                                      insert-pipelines |   1177.02        |   1294           |     116.98    |     ms |   +9.94% |
|                                 100th percentile service time |                                                      insert-pipelines |   1177.02        |   1294           |     116.98    |     ms |   +9.94% |
|                                                    error rate |                                                      insert-pipelines |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                            insert-ilm |     27.9609      |     27.291       |      -0.66988 |  ops/s |   -2.40% |
|                                               Mean Throughput |                                                            insert-ilm |     27.9609      |     27.291       |      -0.66988 |  ops/s |   -2.40% |
|                                             Median Throughput |                                                            insert-ilm |     27.9609      |     27.291       |      -0.66988 |  ops/s |   -2.40% |
|                                                Max Throughput |                                                            insert-ilm |     27.9609      |     27.291       |      -0.66988 |  ops/s |   -2.40% |
|                                      100th percentile latency |                                                            insert-ilm |     34.6168      |     34.5742      |      -0.04253 |     ms |   -0.12% |
|                                 100th percentile service time |                                                            insert-ilm |     34.6168      |     34.5742      |      -0.04253 |     ms |   -0.12% |
|                                                    error rate |                                                            insert-ilm |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                validate-package-template-installation |     45.8875      |     50.0293      |       4.14174 |  ops/s |   +9.03% |
|                                               Mean Throughput |                                validate-package-template-installation |     45.8875      |     50.0293      |       4.14174 |  ops/s |   +9.03% |
|                                             Median Throughput |                                validate-package-template-installation |     45.8875      |     50.0293      |       4.14174 |  ops/s |   +9.03% |
|                                                Max Throughput |                                validate-package-template-installation |     45.8875      |     50.0293      |       4.14174 |  ops/s |   +9.03% |
|                                      100th percentile latency |                                validate-package-template-installation |     21.3813      |     19.5102      |      -1.87109 |     ms |   -8.75% |
|                                 100th percentile service time |                                validate-package-template-installation |     21.3813      |     19.5102      |      -1.87109 |     ms |   -8.75% |
|                                                    error rate |                                validate-package-template-installation |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                       update-custom-package-templates |     25.8823      |     24.4688      |      -1.41351 |  ops/s |   -5.46% |
|                                               Mean Throughput |                                       update-custom-package-templates |     25.8823      |     24.4688      |      -1.41351 |  ops/s |   -5.46% |
|                                             Median Throughput |                                       update-custom-package-templates |     25.8823      |     24.4688      |      -1.41351 |  ops/s |   -5.46% |
|                                                Max Throughput |                                       update-custom-package-templates |     25.8823      |     24.4688      |      -1.41351 |  ops/s |   -5.46% |
|                                      100th percentile latency |                                       update-custom-package-templates |    463.235       |    489.939       |      26.7045  |     ms |   +5.76% |
|                                 100th percentile service time |                                       update-custom-package-templates |    463.235       |    489.939       |      26.7045  |     ms |   +5.76% |
|                                                    error rate |                                       update-custom-package-templates |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                            bulk-index |   5974.36        |   1619.33        |   -4355.03    | docs/s |  -72.90% |
|                                               Mean Throughput |                                                            bulk-index |   9446.91        |   9512.2         |      65.2976  | docs/s |   +0.69% |
|                                             Median Throughput |                                                            bulk-index |   9326.87        |   9388.87        |      61.9944  | docs/s |   +0.66% |
|                                                Max Throughput |                                                            bulk-index |  10526.6         |  16442.7         |    5916.02    | docs/s |  +56.20% |
|                                       50th percentile latency |                                                            bulk-index |   3070.81        |   3040.4         |     -30.4023  |     ms |   -0.99% |
|                                       90th percentile latency |                                                            bulk-index |   4271.13        |   4281.41        |      10.2838  |     ms |   +0.24% |
|                                       99th percentile latency |                                                            bulk-index |   6073.46        |   6110.36        |      36.9023  |     ms |   +0.61% |
|                                     99.9th percentile latency |                                                            bulk-index |   7507.86        |   7603.37        |      95.5115  |     ms |   +1.27% |
|                                    99.99th percentile latency |                                                            bulk-index |   9021.62        |   8695.52        |    -326.096   |     ms |   -3.61% |
|                                      100th percentile latency |                                                            bulk-index |   9807.61        |   8939.99        |    -867.621   |     ms |   -8.85% |
|                                  50th percentile service time |                                                            bulk-index |   3070.75        |   3040.16        |     -30.5882  |     ms |   -1.00% |
|                                  90th percentile service time |                                                            bulk-index |   4270.78        |   4282.5         |      11.715   |     ms |   +0.27% |
|                                  99th percentile service time |                                                            bulk-index |   6076.07        |   6112.52        |      36.4469  |     ms |   +0.60% |
|                                99.9th percentile service time |                                                            bulk-index |   7507.86        |   7603.37        |      95.5115  |     ms |   +1.27% |
|                               99.99th percentile service time |                                                            bulk-index |   9021.62        |   8695.52        |    -326.096   |     ms |   -3.61% |
|                                 100th percentile service time |                                                            bulk-index |   9807.61        |   8939.99        |    -867.621   |     ms |   -8.85% |
|                                                    error rate |                                                            bulk-index |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                       register-snapshot-repository-on-remote-clusters |      0.627022    |      0.616449    |      -0.01057 |  ops/s |   -1.69% |
|                                               Mean Throughput |                       register-snapshot-repository-on-remote-clusters |      0.627022    |      0.616449    |      -0.01057 |  ops/s |   -1.69% |
|                                             Median Throughput |                       register-snapshot-repository-on-remote-clusters |      0.627022    |      0.616449    |      -0.01057 |  ops/s |   -1.69% |
|                                                Max Throughput |                       register-snapshot-repository-on-remote-clusters |      0.627022    |      0.616449    |      -0.01057 |  ops/s |   -1.69% |
|                                      100th percentile latency |                       register-snapshot-repository-on-remote-clusters |   1594.15        |   1619.12        |      24.9664  |     ms |   +1.57% |
|                                 100th percentile service time |                       register-snapshot-repository-on-remote-clusters |   1594.15        |   1619.12        |      24.9664  |     ms |   +1.57% |
|                                                    error rate |                       register-snapshot-repository-on-remote-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                            create-snapshot-many-clusters-ccs-snapshot |     15.395       |     12.2879      |      -3.10718 |  ops/s |  -20.18% |
|                                               Mean Throughput |                            create-snapshot-many-clusters-ccs-snapshot |     15.395       |     12.2879      |      -3.10718 |  ops/s |  -20.18% |
|                                             Median Throughput |                            create-snapshot-many-clusters-ccs-snapshot |     15.395       |     12.2879      |      -3.10718 |  ops/s |  -20.18% |
|                                                Max Throughput |                            create-snapshot-many-clusters-ccs-snapshot |     15.395       |     12.2879      |      -3.10718 |  ops/s |  -20.18% |
|                                      100th percentile latency |                            create-snapshot-many-clusters-ccs-snapshot |     64.5192      |     79.2015      |      14.6823  |     ms |  +22.76% |
|                                 100th percentile service time |                            create-snapshot-many-clusters-ccs-snapshot |     64.5192      |     79.2015      |      14.6823  |     ms |  +22.76% |
|                                                    error rate |                            create-snapshot-many-clusters-ccs-snapshot |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                          wait-for-snapshot-many-clusters-ccs-snapshot |      3.13262e+07 |      3.16819e+07 |  355640       | byte/s |   +1.14% |
|                                               Mean Throughput |                          wait-for-snapshot-many-clusters-ccs-snapshot |      3.13262e+07 |      3.16819e+07 |  355640       | byte/s |   +1.14% |
|                                             Median Throughput |                          wait-for-snapshot-many-clusters-ccs-snapshot |      3.13262e+07 |      3.16819e+07 |  355640       | byte/s |   +1.14% |
|                                                Max Throughput |                          wait-for-snapshot-many-clusters-ccs-snapshot |      3.13262e+07 |      3.16819e+07 |  355640       | byte/s |   +1.14% |
|                                      100th percentile latency |                          wait-for-snapshot-many-clusters-ccs-snapshot | 518136           | 509989           |   -8146.97    |     ms |   -1.57% |
|                                 100th percentile service time |                          wait-for-snapshot-many-clusters-ccs-snapshot | 518136           | 509989           |   -8146.97    |     ms |   -1.57% |
|                                                    error rate |                          wait-for-snapshot-many-clusters-ccs-snapshot |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |      0.170329    |      0.171223    |       0.00089 |  ops/s |   +0.52% |
|                                               Mean Throughput |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |      0.170329    |      0.171223    |       0.00089 |  ops/s |   +0.52% |
|                                             Median Throughput |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |      0.170329    |      0.171223    |       0.00089 |  ops/s |   +0.52% |
|                                                Max Throughput |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |      0.170329    |      0.171223    |       0.00089 |  ops/s |   +0.52% |
|                                      100th percentile latency |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |   5869.71        |   5837.79        |     -31.9185  |     ms |   -0.54% |
|                                 100th percentile service time |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |   5869.71        |   5837.79        |     -31.9185  |     ms |   -0.54% |
|                                                    error rate |        recover-snapshot-many-clusters-ccs-snapshot-on-remote-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      0.000949647 |      0.000988958 |       4e-05   |  ops/s |   +4.14% |
|                                               Mean Throughput | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      0.000949647 |      0.000988958 |       4e-05   |  ops/s |   +4.14% |
|                                             Median Throughput | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      0.000949647 |      0.000988958 |       4e-05   |  ops/s |   +4.14% |
|                                                Max Throughput | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      0.000949647 |      0.000988958 |       4e-05   |  ops/s |   +4.14% |
|                                      100th percentile latency | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      1.05302e+06 |      1.01116e+06 |  -41858.2     |     ms |   -3.98% |
|                                 100th percentile service time | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      1.05302e+06 |      1.01116e+06 |  -41858.2     |     ms |   -3.98% |
|                                                    error rate | wait-for-snapshot-recovery-many-clusters-ccs-snapshot-on-all-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                     check-cluster-health-all-clusters |      3.8425      |      3.75721     |      -0.08529 |  ops/s |   -2.22% |
|                                               Mean Throughput |                                     check-cluster-health-all-clusters |      3.8425      |      3.75721     |      -0.08529 |  ops/s |   -2.22% |
|                                             Median Throughput |                                     check-cluster-health-all-clusters |      3.8425      |      3.75721     |      -0.08529 |  ops/s |   -2.22% |
|                                                Max Throughput |                                     check-cluster-health-all-clusters |      3.8425      |      3.75721     |      -0.08529 |  ops/s |   -2.22% |
|                                      100th percentile latency |                                     check-cluster-health-all-clusters |    259.281       |    263.498       |       4.21741 |     ms |   +1.63% |
|                                 100th percentile service time |                                     check-cluster-health-all-clusters |    259.281       |    263.498       |       4.21741 |     ms |   +1.63% |
|                                                    error rate |                                     check-cluster-health-all-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                  setup-bi-directional-remote-clusters |      0.435378    |      0.432788    |      -0.00259 |  ops/s |   -0.59% |
|                                               Mean Throughput |                                  setup-bi-directional-remote-clusters |      0.435378    |      0.432788    |      -0.00259 |  ops/s |   -0.59% |
|                                             Median Throughput |                                  setup-bi-directional-remote-clusters |      0.435378    |      0.432788    |      -0.00259 |  ops/s |   -0.59% |
|                                                Max Throughput |                                  setup-bi-directional-remote-clusters |      0.435378    |      0.432788    |      -0.00259 |  ops/s |   -0.59% |
|                                      100th percentile latency |                                  setup-bi-directional-remote-clusters |   2296.34        |   2310.01        |      13.6631  |     ms |   +0.59% |
|                                 100th percentile service time |                                  setup-bi-directional-remote-clusters |   2296.34        |   2310.01        |      13.6631  |     ms |   +0.59% |
|                                                    error rate |                                  setup-bi-directional-remote-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                               reset-recovery-settings-on-all-clusters |      3.37303     |      3.31428     |      -0.05876 |  ops/s |   -1.74% |
|                                               Mean Throughput |                               reset-recovery-settings-on-all-clusters |      3.37303     |      3.31428     |      -0.05876 |  ops/s |   -1.74% |
|                                             Median Throughput |                               reset-recovery-settings-on-all-clusters |      3.37303     |      3.31428     |      -0.05876 |  ops/s |   -1.74% |
|                                                Max Throughput |                               reset-recovery-settings-on-all-clusters |      3.37303     |      3.31428     |      -0.05876 |  ops/s |   -1.74% |
|                                      100th percentile latency |                               reset-recovery-settings-on-all-clusters |    295.802       |    300.963       |       5.16156 |     ms |   +1.74% |
|                                 100th percentile service time |                               reset-recovery-settings-on-all-clusters |    295.802       |    300.963       |       5.16156 |     ms |   +1.74% |
|                                                    error rate |                               reset-recovery-settings-on-all-clusters |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                       discover/search |      0.00228331  |      0.00228458  |       0       |  ops/s |   +0.06% |
|                                               Mean Throughput |                                                       discover/search |      0.00347744  |      0.00347908  |       0       |  ops/s |   +0.05% |
|                                             Median Throughput |                                                       discover/search |      0.00353725  |      0.00353854  |       0       |  ops/s |   +0.04% |
|                                                Max Throughput |                                                       discover/search |      0.00455195  |      0.00455467  |       0       |  ops/s |   +0.06% |
|                                       50th percentile latency |                                                       discover/search |   3118.47        |   2992.23        |    -126.241   |     ms |   -4.05% |
|                                      100th percentile latency |                                                       discover/search |   3787.64        |   3878.36        |      90.7241  |     ms |   +2.40% |
|                                  50th percentile service time |                                                       discover/search |   1099.33        |   1390.95        |     291.622   |     ms |  +26.53% |
|                                 100th percentile service time |                                                       discover/search |   1774.22        |   1696.47        |     -77.7512  |     ms |   -4.38% |
|                                                    error rate |                                                       discover/search |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                               discover/search/no_minimized_roundtrips |      0.00205669  |      0.00205608  |      -0       |  ops/s |   -0.03% |
|                                               Mean Throughput |                               discover/search/no_minimized_roundtrips |      0.00249479  |      0.00249377  |      -0       |  ops/s |   -0.04% |
|                                             Median Throughput |                               discover/search/no_minimized_roundtrips |      0.00249479  |      0.00249377  |      -0       |  ops/s |   -0.04% |
|                                                Max Throughput |                               discover/search/no_minimized_roundtrips |      0.00293289  |      0.00293145  |      -0       |  ops/s |   -0.05% |
|                                       50th percentile latency |                               discover/search/no_minimized_roundtrips | 124688           | 125130           |     442.34    |     ms |   +0.35% |
|                                      100th percentile latency |                               discover/search/no_minimized_roundtrips | 149497           | 150018           |     521.141   |     ms |   +0.35% |
|                                  50th percentile service time |                               discover/search/no_minimized_roundtrips |  49845.1         |  50033.6         |     188.475   |     ms |   +0.38% |
|                                 100th percentile service time |                               discover/search/no_minimized_roundtrips |  50236.7         |  50453.5         |     216.754   |     ms |   +0.43% |
|                                                    error rate |                               discover/search/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                    discover/visualize |      0.00231261  |      0.00231337  |       0       |  ops/s |   +0.03% |
|                                               Mean Throughput |                                                    discover/visualize |      0.00485273  |      0.00485428  |       0       |  ops/s |   +0.03% |
|                                             Median Throughput |                                                    discover/visualize |      0.00517466  |      0.00517652  |       0       |  ops/s |   +0.04% |
|                                                Max Throughput |                                                    discover/visualize |      0.00683491  |      0.00683712  |       0       |  ops/s |   +0.03% |
|                                       50th percentile latency |                                                    discover/visualize |   1265.31        |   1221.07        |     -44.24    |     ms |   -3.50% |
|                                      100th percentile latency |                                                    discover/visualize |   2226.01        |   2160.46        |     -65.5498  |     ms |   -2.94% |
|                                  50th percentile service time |                                                    discover/visualize |   1104           |   1090.64        |     -13.3573  |     ms |   -1.21% |
|                                 100th percentile service time |                                                    discover/visualize |   2225.53        |   2141.38        |     -84.145   |     ms |   -3.78% |
|                                                    error rate |                                                    discover/visualize |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                            discover/visualize/no_minimized_roundtrips |      0.00218627  |      0.00218443  |      -0       |  ops/s |   -0.08% |
|                                               Mean Throughput |                            discover/visualize/no_minimized_roundtrips |      0.00307764  |      0.00307429  |      -0       |  ops/s |   -0.11% |
|                                             Median Throughput |                            discover/visualize/no_minimized_roundtrips |      0.003108    |      0.00310475  |      -0       |  ops/s |   -0.10% |
|                                                Max Throughput |                            discover/visualize/no_minimized_roundtrips |      0.00393865  |      0.0039337   |      -0       |  ops/s |   -0.13% |
|                                       50th percentile latency |                            discover/visualize/no_minimized_roundtrips | 123427           | 124351           |     923.664   |     ms |   +0.75% |
|                                      100th percentile latency |                            discover/visualize/no_minimized_roundtrips | 171022           | 172008           |     986.719   |     ms |   +0.58% |
|                                  50th percentile service time |                            discover/visualize/no_minimized_roundtrips |  50138.8         |  50366.7         |     227.906   |     ms |   +0.45% |
|                                 100th percentile service time |                            discover/visualize/no_minimized_roundtrips |  50323.3         |  50592.4         |     269.02    |     ms |   +0.53% |
|                                                    error rate |                            discover/visualize/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                                 kafka |      0.00501314  |      0.00501331  |       0       |  ops/s |    0.00% |
|                                               Mean Throughput |                                                                 kafka |      0.011064    |      0.0110643   |       0       |  ops/s |    0.00% |
|                                             Median Throughput |                                                                 kafka |      0.0108078   |      0.010808    |       0       |  ops/s |    0.00% |
|                                                Max Throughput |                                                                 kafka |      0.0183774   |      0.018378    |       0       |  ops/s |    0.00% |
|                                       50th percentile latency |                                                                 kafka |    579.42        |    570.366       |      -9.0545  |     ms |   -1.56% |
|                                      100th percentile latency |                                                                 kafka |    862.63        |    905.347       |      42.7164  |     ms |   +4.95% |
|                                  50th percentile service time |                                                                 kafka |    576.671       |    567.924       |      -8.74707 |     ms |   -1.52% |
|                                 100th percentile service time |                                                                 kafka |    788.854       |    681.839       |    -107.015   |     ms |  -13.57% |
|                                                    error rate |                                                                 kafka |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                         kafka/no_minimized_roundtrips |      0.00493846  |      0.00493647  |      -0       |  ops/s |   -0.04% |
|                                               Mean Throughput |                                         kafka/no_minimized_roundtrips |      0.0106933   |      0.0106898   |      -0       |  ops/s |   -0.03% |
|                                             Median Throughput |                                         kafka/no_minimized_roundtrips |      0.0105239   |      0.0105217   |      -0       |  ops/s |   -0.02% |
|                                                Max Throughput |                                         kafka/no_minimized_roundtrips |      0.0174484   |      0.017433    |      -2e-05   |  ops/s |   -0.09% |
|                                       50th percentile latency |                                         kafka/no_minimized_roundtrips |  15283.7         |  15296.3         |      12.6743  |     ms |   +0.08% |
|                                      100th percentile latency |                                         kafka/no_minimized_roundtrips |  27475.4         |  27779.1         |     303.639   |     ms |   +1.11% |
|                                  50th percentile service time |                                         kafka/no_minimized_roundtrips |   8917.96        |   8943.3         |      25.3374  |     ms |   +0.28% |
|                                 100th percentile service time |                                         kafka/no_minimized_roundtrips |   9144.62        |   9174.73        |      30.1025  |     ms |   +0.33% |
|                                                    error rate |                                         kafka/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                                 nginx |      0.00639487  |      0.00640231  |       1e-05   |  ops/s |   +0.12% |
|                                               Mean Throughput |                                                                 nginx |      0.0228754   |      0.0228882   |       1e-05   |  ops/s |   +0.06% |
|                                             Median Throughput |                                                                 nginx |      0.023571    |      0.0235786   |       1e-05   |  ops/s |   +0.03% |
|                                                Max Throughput |                                                                 nginx |      0.0338225   |      0.0338351   |       1e-05   |  ops/s |   +0.04% |
|                                       50th percentile latency |                                                                 nginx |   2734.4         |   2523.41        |    -210.982   |     ms |   -7.72% |
|                                       90th percentile latency |                                                                 nginx |   4648.48        |   4600.98        |     -47.4985  |     ms |   -1.02% |
|                                      100th percentile latency |                                                                 nginx |   5985.47        |   6053.42        |      67.9507  |     ms |   +1.14% |
|                                  50th percentile service time |                                                                 nginx |   2274.54        |   2298.54        |      24.0051  |     ms |   +1.06% |
|                                  90th percentile service time |                                                                 nginx |   3118.58        |   3059.32        |     -59.2573  |     ms |   -1.90% |
|                                 100th percentile service time |                                                                 nginx |   4316.53        |   3810.15        |    -506.381   |     ms |  -11.73% |
|                                                    error rate |                                                                 nginx |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                         nginx/no_minimized_roundtrips |      0.00554318  |      0.00554135  |      -0       |  ops/s |   -0.03% |
|                                               Mean Throughput |                                         nginx/no_minimized_roundtrips |      0.0133854   |      0.0133757   |      -1e-05   |  ops/s |   -0.07% |
|                                             Median Throughput |                                         nginx/no_minimized_roundtrips |      0.0147864   |      0.0147863   |      -0       |  ops/s |   -0.00% |
|                                                Max Throughput |                                         nginx/no_minimized_roundtrips |      0.0173465   |      0.0173397   |      -1e-05   |  ops/s |   -0.04% |
|                                       50th percentile latency |                                         nginx/no_minimized_roundtrips | 154031           | 154352           |     321       |     ms |   +0.21% |
|                                       90th percentile latency |                                         nginx/no_minimized_roundtrips | 297485           | 298397           |     912.525   |     ms |   +0.31% |
|                                      100th percentile latency |                                         nginx/no_minimized_roundtrips | 330457           | 331573           |    1116.44    |     ms |   +0.34% |
|                                  50th percentile service time |                                         nginx/no_minimized_roundtrips |  35692.6         |  35742.4         |      49.875   |     ms |   +0.14% |
|                                  90th percentile service time |                                         nginx/no_minimized_roundtrips |  52641.6         |  52898.8         |     257.241   |     ms |   +0.49% |
|                                 100th percentile service time |                                         nginx/no_minimized_roundtrips |  53357.6         |  53220.5         |    -137.156   |     ms |   -0.26% |
|                                                    error rate |                                         nginx/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                                apache |      0.0430125   |      0.0430118   |      -0       |  ops/s |   -0.00% |
|                                               Mean Throughput |                                                                apache |      0.0854283   |      0.0854202   |      -1e-05   |  ops/s |   -0.01% |
|                                             Median Throughput |                                                                apache |      0.0881506   |      0.0881434   |      -1e-05   |  ops/s |   -0.01% |
|                                                Max Throughput |                                                                apache |      0.135594    |      0.135573    |      -2e-05   |  ops/s |   -0.02% |
|                                       50th percentile latency |                                                                apache |    679.21        |    683.014       |       3.80383 |     ms |   +0.56% |
|                                       90th percentile latency |                                                                apache |    975.597       |    997.167       |      21.5702  |     ms |   +2.21% |
|                                      100th percentile latency |                                                                apache |   1319.49        |   1471.65        |     152.159   |     ms |  +11.53% |
|                                  50th percentile service time |                                                                apache |    643.467       |    625.495       |     -17.9714  |     ms |   -2.79% |
|                                  90th percentile service time |                                                                apache |    917.67        |    993.921       |      76.251   |     ms |   +8.31% |
|                                 100th percentile service time |                                                                apache |   1297.47        |   1469.62        |     172.153   |     ms |  +13.27% |
|                                                    error rate |                                                                apache |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                        apache/no_minimized_roundtrips |      0.0469826   |      0.0469767   |      -1e-05   |  ops/s |   -0.01% |
|                                               Mean Throughput |                                        apache/no_minimized_roundtrips |      0.0644584   |      0.0642278   |      -0.00023 |  ops/s |   -0.36% |
|                                             Median Throughput |                                        apache/no_minimized_roundtrips |      0.0676609   |      0.0673621   |      -0.0003  |  ops/s |   -0.44% |
|                                                Max Throughput |                                        apache/no_minimized_roundtrips |      0.0736745   |      0.0733709   |      -0.0003  |  ops/s |   -0.41% |
|                                       50th percentile latency |                                        apache/no_minimized_roundtrips | 105515           | 107784           |    2268.93    |     ms |   +2.15% |
|                                       90th percentile latency |                                        apache/no_minimized_roundtrips | 135162           | 136280           |    1118.7     |     ms |   +0.83% |
|                                      100th percentile latency |                                        apache/no_minimized_roundtrips | 150284           | 151696           |    1411.94    |     ms |   +0.94% |
|                                  50th percentile service time |                                        apache/no_minimized_roundtrips |   8864.11        |   8887.94        |      23.8359  |     ms |   +0.27% |
|                                  90th percentile service time |                                        apache/no_minimized_roundtrips |  13589.8         |  13802.8         |     213.025   |     ms |   +1.57% |
|                                 100th percentile service time |                                        apache/no_minimized_roundtrips |  53689.9         |  54078.8         |     388.863   |     ms |   +0.72% |
|                                                    error rate |                                        apache/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                           system/auth |      0.0618102   |      0.0618097   |      -0       |  ops/s |   -0.00% |
|                                               Mean Throughput |                                                           system/auth |      0.105605    |      0.105588    |      -2e-05   |  ops/s |   -0.02% |
|                                             Median Throughput |                                                           system/auth |      0.105841    |      0.105839    |      -0       |  ops/s |   -0.00% |
|                                                Max Throughput |                                                           system/auth |      0.135613    |      0.135519    |      -9e-05   |  ops/s |   -0.07% |
|                                       50th percentile latency |                                                           system/auth |    769.523       |    804.585       |      35.0623  |     ms |   +4.56% |
|                                       90th percentile latency |                                                           system/auth |   1451.1         |   1597.07        |     145.968   |     ms |  +10.06% |
|                                      100th percentile latency |                                                           system/auth |   2153.97        |   2854.41        |     700.449   |     ms |  +32.52% |
|                                  50th percentile service time |                                                           system/auth |    713.603       |    743.489       |      29.8864  |     ms |   +4.19% |
|                                  90th percentile service time |                                                           system/auth |   1418.39        |   1446.22        |      27.8297  |     ms |   +1.96% |
|                                 100th percentile service time |                                                           system/auth |   1714.06        |   1990.35        |     276.289   |     ms |  +16.12% |
|                                                    error rate |                                                           system/auth |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                   system/auth/no_minimized_roundtrips |      0.0310842   |      0.0309496   |      -0.00013 |  ops/s |   -0.43% |
|                                               Mean Throughput |                                   system/auth/no_minimized_roundtrips |      0.0430451   |      0.0429004   |      -0.00014 |  ops/s |   -0.34% |
|                                             Median Throughput |                                   system/auth/no_minimized_roundtrips |      0.0434394   |      0.0432975   |      -0.00014 |  ops/s |   -0.33% |
|                                                Max Throughput |                                   system/auth/no_minimized_roundtrips |      0.0519486   |      0.051781    |      -0.00017 |  ops/s |   -0.32% |
|                                       50th percentile latency |                                   system/auth/no_minimized_roundtrips | 399015           | 400762           |    1747.42    |     ms |   +0.44% |
|                                       90th percentile latency |                                   system/auth/no_minimized_roundtrips | 577365           | 580284           |    2918.41    |     ms |   +0.51% |
|                                      100th percentile latency |                                   system/auth/no_minimized_roundtrips | 590603           | 593199           |    2596.31    |     ms |   +0.44% |
|                                  50th percentile service time |                                   system/auth/no_minimized_roundtrips |   4974.38        |   4998.37        |      23.9854  |     ms |   +0.48% |
|                                  90th percentile service time |                                   system/auth/no_minimized_roundtrips |  49413.7         |  49469.8         |      56.1875  |     ms |   +0.11% |
|                                 100th percentile service time |                                   system/auth/no_minimized_roundtrips |  50250.8         |  50244           |      -6.84375 |     ms |   -0.01% |
|                                                    error rate |                                   system/auth/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                               system/syslog/dashboard |      0.0396284   |      0.0396286   |       0       |  ops/s |    0.00% |
|                                               Mean Throughput |                                               system/syslog/dashboard |      0.0542198   |      0.0545029   |       0.00028 |  ops/s |   +0.52% |
|                                             Median Throughput |                                               system/syslog/dashboard |      0.0544903   |      0.0545031   |       1e-05   |  ops/s |   +0.02% |
|                                                Max Throughput |                                               system/syslog/dashboard |      0.0667024   |      0.0667026   |       0       |  ops/s |    0.00% |
|                                       50th percentile latency |                                               system/syslog/dashboard |    576.769       |    588.487       |      11.7179  |     ms |   +2.03% |
|                                       90th percentile latency |                                               system/syslog/dashboard |    960.935       |    960.733       |      -0.20248 |     ms |   -0.02% |
|                                      100th percentile latency |                                               system/syslog/dashboard |   1280.93        |   1581.61        |     300.683   |     ms |  +23.47% |
|                                  50th percentile service time |                                               system/syslog/dashboard |    573.981       |    586.893       |      12.9116  |     ms |   +2.25% |
|                                  90th percentile service time |                                               system/syslog/dashboard |    642.932       |    652.93        |       9.99825 |     ms |   +1.56% |
|                                 100th percentile service time |                                               system/syslog/dashboard |    682.784       |    839.746       |     156.962   |     ms |  +22.99% |
|                                                    error rate |                                               system/syslog/dashboard |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                       system/syslog/dashboard/no_minimized_roundtrips |      0.0368127   |      0.0368037   |      -1e-05   |  ops/s |   -0.02% |
|                                               Mean Throughput |                       system/syslog/dashboard/no_minimized_roundtrips |      0.0527674   |      0.052687    |      -8e-05   |  ops/s |   -0.15% |
|                                             Median Throughput |                       system/syslog/dashboard/no_minimized_roundtrips |      0.0521717   |      0.0521559   |      -2e-05   |  ops/s |   -0.03% |
|                                                Max Throughput |                       system/syslog/dashboard/no_minimized_roundtrips |      0.0732733   |      0.0729109   |      -0.00036 |  ops/s |   -0.49% |
|                                       50th percentile latency |                       system/syslog/dashboard/no_minimized_roundtrips |  50852.7         |  51315.1         |     462.463   |     ms |   +0.91% |
|                                       90th percentile latency |                       system/syslog/dashboard/no_minimized_roundtrips |  83260.5         |  83913.6         |     653.087   |     ms |   +0.78% |
|                                      100th percentile latency |                       system/syslog/dashboard/no_minimized_roundtrips |  91742.3         |  91902.8         |     160.492   |     ms |   +0.17% |
|                                  50th percentile service time |                       system/syslog/dashboard/no_minimized_roundtrips |   8859.56        |   8885.84        |      26.272   |     ms |   +0.30% |
|                                  90th percentile service time |                       system/syslog/dashboard/no_minimized_roundtrips |   9029.46        |   9023.75        |      -5.70811 |     ms |   -0.06% |
|                                 100th percentile service time |                       system/syslog/dashboard/no_minimized_roundtrips |   9125.72        |   9285.02        |     159.297   |     ms |   +1.75% |
|                                                    error rate |                       system/syslog/dashboard/no_minimized_roundtrips |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                                                    system/syslog/lens |      0.0112183   |      0.0112195   |       0       |  ops/s |   +0.01% |
|                                               Mean Throughput |                                                    system/syslog/lens |      0.0186166   |      0.0186209   |       0       |  ops/s |   +0.02% |
|                                             Median Throughput |                                                    system/syslog/lens |      0.0157819   |      0.0157835   |       0       |  ops/s |   +0.01% |
|                                                Max Throughput |                                                    system/syslog/lens |      0.0478242   |      0.0478512   |       3e-05   |  ops/s |   +0.06% |
|                                       50th percentile latency |                                                    system/syslog/lens |    575.009       |    582.879       |       7.87042 |     ms |   +1.37% |
|                                       90th percentile latency |                                                    system/syslog/lens |    994.776       |   1026.55        |      31.7781  |     ms |   +3.19% |
|                                      100th percentile latency |                                                    system/syslog/lens |   1001.32        |   1079.7         |      78.3796  |     ms |   +7.83% |
|                                  50th percentile service time |                                                    system/syslog/lens |    573.311       |    581.091       |       7.78015 |     ms |   +1.36% |
|                                  90th percentile service time |                                                    system/syslog/lens |    992.783       |   1024.32        |      31.5378  |     ms |   +3.18% |
|                                 100th percentile service time |                                                    system/syslog/lens |    998.99        |   1077.58        |      78.5936  |     ms |   +7.87% |
|                                                    error rate |                                                    system/syslog/lens |      0           |      0           |       0       |      % |    0.00% |
|                                                Min Throughput |                            system/syslog/lens/no_minimized_roundtrips |      0.0111025   |      0.0111029   |       0       |  ops/s |    0.00% |
|                                               Mean Throughput |                            system/syslog/lens/no_minimized_roundtrips |      0.0208552   |      0.0208548   |      -0       |  ops/s |   -0.00% |
|                                             Median Throughput |                            system/syslog/lens/no_minimized_roundtrips |      0.0161463   |      0.0161468   |       0       |  ops/s |    0.00% |
|                                                Max Throughput |                            system/syslog/lens/no_minimized_roundtrips |      0.0488851   |      0.0488727   |      -1e-05   |  ops/s |   -0.03% |
|                                       50th percentile latency |                            system/syslog/lens/no_minimized_roundtrips |  13759.3         |  13795.5         |      36.1846  |     ms |   +0.26% |
|                                       90th percentile latency |                            system/syslog/lens/no_minimized_roundtrips |  17927.8         |  18090.7         |     162.849   |     ms |   +0.91% |