[WIP] Top k as generic histogram tests

This PR improves the cardinality estimation of SQL queries in hyrise and delivers a tool to do more precise analyses of such estimations.
This was achieved by
(1) Implementing a new cardinality estimation class - a Top K histogram modeled as a generic histogram (TopKGenericHistogram)
(2) Fixing a bug in the selectivity calculation negatively impacting the cardinality estimation of all tables with NULL values
(3) Implementing a cardinality estimation analyses tool
In the following, we will explain all three parts in detail:
(1) Top K As Generic Histogram Implementation
Cardinality Estimation in hyrise is implemented via histograms. (The cardinality estimator actually expects to be able to access a histogram interface.) In short, a histogram saves estimations about the occurrence count of each distinct value in a table column and can be updated via applying operations of an SQL query to it. It then stores estimations about how often a value is expected to remain in the table after the operation was executed.
The newly implemented type of histogram is different from the already existing histograms because it stores the exact occurrence count of the most common values (the Top K values). This allows us to have exact and/or better estimations when estimating operations, including frequently occurring values. As the most common values often already cover a huge share of the total number of values in the table, storing their exact values provides a nice trade-off between limited additional storage and operation costs needed and a notable improvement of cardinality estimations.
Our implementation is built using the already existing generic histogram (GenericHistogram) class and constructing the bins for this histogram in a specific way. Each Top K value is modeled as one bin with a height of its stored count. For all Non-Top K values between two Top K bins, one bin is created using a uniform distribution assumption.
In more detail:
By implementing the from_column() method, a Top K histogram can be created for each column in a table as a statistic_object. When called, the Top K values and their occurrence count are saved into vectors sorted lexicographically by value. Furthermore, they are removed from the value distribution.
With that in place, the generic histogram is built in the following way:
Each top k value gets its own bin with its value count as the height. Because it is the only value in the bin, the cardinality estimator can now use an exact estimation of its occurrence count. Every Non-Top K value is distributed in the bins between two top k values. This is done with an equal distribution assumption, meaning that we assume all Non-Top K values to have the same occurrence count.  If there are no Non-Top K values between two Top K values, no Non-Top K value bin is created. In the end, a potential last Non-Top K value bin is added, and the generic histogram is returned.
Through that, the returned histogram has a single bin for every Top K value and, if necessary, a Non-Top K value bin between them. Adding a potential last bucket, the overall maximum number of buckets will be 2 * K + 1.
Performance
To measure our performance impact we tested different Top K values: {10, 50, 100, 300, 1000, 5000}. The best performance increase was created with K = 300, yielding -25% decrease on latency and +31% increase on throughput. It was interesting that while very small K (10, 50) already did show a nearly as highly decreasing impact as with K = 300, very high K values (5000) even lead to a decreasing performance. We assume this is due to a to high fragmentation of our created histogram, which doesn't work well for the used cardinality estimation techniques.
We also noted that the SemiJoinReductionRule does not work well with the new cardinality estimations, severly negatively impacting the performance. As the SemiJoinReductionRule itself is currently under construction we chose to benchmark without it.
We assume that the nearly equally good estimations with different lower K values are caused by the fact that even the low K values often already cover a quite high share of the total occurrences (one our example table 27% of all rows stemed from the Top 100 values, in comparison top 1000 covered only 3 % more with 30%). With increasing K count adding more K values naturally has less impact, because the values with the highest occurrences count shares have already been added.
As higher K values also lead to longer preprocessing times for creating each histogram, we chose K=100 as a standard value for our K, as a balanced trade-off between preprocessing time and nearly equal performance improvement.

JOB K 10 vs JOB K 100
 +Configuration Overview---------------------------------------------------+-------------------------------------------------------+
 | Parameter        | benchmark_as_generic_wo_semijoin_reduction_k_10.json | benchmark_as_generic_wo_semijoin_reduction_k_100.json |
 +------------------+------------------------------------------------------+-------------------------------------------------------+
 |  GIT-HASH        | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty       | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty        |
 |  benchmark_mode  | Ordered                                              | Ordered                                               |
 |  build_type      | release                                              | release                                               |
 |  chunk_size      | 65535                                                | 65535                                                 |
 |  clients         | 1                                                    | 1                                                     |
 |  compiler        | gcc 9.3                                              | gcc 9.3                                               |
 |  cores           | 0                                                    | 0                                                     |
 |  date            | 2021-07-31 20:14:47                                  | 2021-07-31 22:48:51                                   |
 |  encoding        | {'default': {'encoding': 'Dictionary'}}              | {'default': {'encoding': 'Dictionary'}}               |
 |  indexes         | False                                                | False                                                 |
 |  max_duration    |<PHONE_NUMBER>0                                          |<PHONE_NUMBER>0                                           |
 |  max_runs        | 100                                                  | 100                                                   |
 |  time_unit       | ns                                                   | ns                                                    |
 |  using_scheduler | False                                                | False                                                 |
 |  verify          | False                                                | False                                                 |
 |  warmup_duration | 0                                                    | 0                                                     |
 +------------------+------------------------------------------------------+-------------------------------------------------------+

 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Item    || Latency (ms/iter)   | Change || Throughput (iter/s) | Change |              p-value |
 |         ||      old |      new |        ||      old |      new |        |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | 10a     ||    519.4 |    511.4 |   -2%˄ ||     1.93 |     1.96 |   +2%˄ | (run time too short) |
+| 10b     ||    427.4 |    404.8 |   -5%˄ ||     2.34 |     2.47 |   +6%˄ | (run time too short) |
 | 10c     ||    478.9 |    465.8 |   -3%˄ ||     2.09 |     2.15 |   +3%˄ | (run time too short) |
 | 11a     ||    175.3 |    173.6 |   -1%˄ ||     5.70 |     5.76 |   +1%˄ | (run time too short) |
 | 11b     ||    169.8 |    169.1 |   -0%˄ ||     5.89 |     5.91 |   +0%˄ | (run time too short) |
-| 11c     ||    252.3 |    267.2 |   +6%˄ ||     3.96 |     3.74 |   -6%˄ | (run time too short) |
 | 11d     ||    201.6 |    199.5 |   -1%˄ ||     4.96 |     5.01 |   +1%˄ | (run time too short) |
+| 12a     ||    149.1 |    135.0 |   -9%˄ ||     6.70 |     7.41 |  +10%˄ | (run time too short) |
 | 12b     ||    234.7 |    230.8 |   -2%˄ ||     4.26 |     4.33 |   +2%˄ | (run time too short) |
+| 12c     ||    313.0 |    295.6 |   -6%˄ ||     3.19 |     3.38 |   +6%˄ | (run time too short) |
+| 13a     ||    302.6 |    284.3 |   -6%˄ ||     3.30 |     3.52 |   +6%˄ | (run time too short) |
 | 13b     ||    265.5 |    257.3 |   -3%˄ ||     3.77 |     3.89 |   +3%˄ | (run time too short) |
+| 13c     ||    183.4 |    169.1 |   -8%˄ ||     5.45 |     5.92 |   +9%˄ | (run time too short) |
 | 13d     ||    643.5 |    616.9 |   -4%  ||     1.55 |     1.62 |   +4%  |               0.0000 |
 | 14a     ||    311.9 |    310.7 |   -0%˄ ||     3.21 |     3.22 |   +0%˄ | (run time too short) |
 | 14b     ||    437.6 |    441.5 |   +1%˄ ||     2.29 |     2.27 |   -1%˄ | (run time too short) |
 | 14c     ||    375.8 |    376.1 |   +0%˄ ||     2.66 |     2.66 |   -0%˄ | (run time too short) |
 | 15a     ||    301.2 |    306.1 |   +2%˄ ||     3.32 |     3.27 |   -2%˄ | (run time too short) |
 | 15b     ||    269.3 |    267.9 |   -1%˄ ||     3.71 |     3.73 |   +1%˄ | (run time too short) |
 | 15c     ||    283.8 |    282.0 |   -1%˄ ||     3.52 |     3.55 |   +1%˄ | (run time too short) |
+| 15d     ||    363.8 |    339.4 |   -7%˄ ||     2.75 |     2.95 |   +7%˄ | (run time too short) |
-| 16a     ||    281.5 |   1726.9 | +514%˄ ||     3.55 |     0.58 |  -84%˄ | (run time too short) |
 | 16b     ||   3551.0 |   3618.5 |   +2%  ||     0.28 |     0.28 |   -2%  |               0.4337 |
-| 16c     ||    557.3 |   1989.9 | +257%˄ ||     1.79 |     0.50 |  -72%˄ | (run time too short) |
-| 16d     ||    496.8 |   1958.3 | +294%˄ ||     2.01 |     0.51 |  -75%˄ | (run time too short) |
 | 17a     ||    604.3 |    585.0 |   -3%˄ ||     1.65 |     1.71 |   +3%˄ | (run time too short) |
 | 17b     ||    531.1 |    523.9 |   -1%˄ ||     1.88 |     1.91 |   +1%˄ | (run time too short) |
 | 17c     ||    499.4 |    492.7 |   -1%˄ ||     2.00 |     2.03 |   +1%˄ | (run time too short) |
 | 17d     ||    614.1 |    605.1 |   -1%˄ ||     1.63 |     1.65 |   +1%˄ |               0.1348 |
 | 17e     ||   2900.2 |   2892.1 |   -0%  ||     0.34 |     0.35 |   +0%  |               0.8137 |
 | 17f     ||   1313.2 |   1305.4 |   -1%  ||     0.76 |     0.77 |   +1%  |               0.5469 |
 | 18a     ||    268.6 |    267.0 |   -1%˄ ||     3.72 |     3.75 |   +1%˄ | (run time too short) |
 | 18b     ||   2666.0 |   2677.8 |   +0%  ||     0.38 |     0.37 |   -0%  |               0.6230 |
+| 18c     ||   5593.4 |   2964.3 |  -47%  ||     0.18 |     0.34 |  +89%  |               0.0000 |
 | 19a     ||   5001.3 |   5042.5 |   +1%  ||     0.20 |     0.20 |   -1%  |               0.1213 |
 | 19b     ||   2273.1 |   2322.6 |   +2%  ||     0.44 |     0.43 |   -2%  |               0.0008 |
 | 19c     ||   5037.6 |   5121.6 |   +2%  ||     0.20 |     0.20 |   -2%  |               0.0269 |
 | 19d     ||   4245.3 |   4219.5 |   -1%  ||     0.24 |     0.24 |   +1%  |               0.2859 |
 | 1a      ||     46.0 |     46.2 |   +0%˄ ||    21.72 |    21.66 |   -0%˄ | (run time too short) |
 | 1b      ||     54.2 |     54.8 |   +1%˄ ||    18.45 |    18.25 |   -1%˄ | (run time too short) |
-| 1c      ||     26.0 |     28.0 |   +8%˄ ||    38.39 |    35.65 |   -7%˄ | (run time too short) |
 | 1d      ||     61.2 |     61.2 |   -0%˄ ||    16.34 |    16.34 |   +0%˄ | (run time too short) |
 | 20a     ||    744.3 |    726.7 |   -2%  ||     1.34 |     1.38 |   +2%  |               0.0000 |
 | 20b     ||    809.3 |    792.6 |   -2%  ||     1.24 |     1.26 |   +2%  |               0.0000 |
 | 20c     ||    644.0 |    644.7 |   +0%  ||     1.55 |     1.55 |   -0%  |               0.8202 |
 | 21a     ||   3082.8 |   3218.7 |   +4%  ||     0.32 |     0.31 |   -4%  |               0.0001 |
 | 21b     ||    181.8 |    185.1 |   +2%˄ ||     5.50 |     5.40 |   -2%˄ | (run time too short) |
 | 21c     ||   3230.5 |   3340.4 |   +3%  ||     0.31 |     0.30 |   -3%  |               0.0002 |
 | 22a     ||    308.8 |    308.1 |   -0%˄ ||     3.24 |     3.25 |   +0%˄ | (run time too short) |
 | 22b     ||    299.3 |    294.6 |   -2%˄ ||     3.34 |     3.39 |   +2%˄ | (run time too short) |
 | 22c     ||    427.1 |    421.6 |   -1%˄ ||     2.34 |     2.37 |   +1%˄ | (run time too short) |
 | 22d     ||    496.7 |    494.7 |   -0%˄ ||     2.01 |     2.02 |   +0%˄ | (run time too short) |
 | 23a     ||    285.8 |    291.7 |   +2%˄ ||     3.50 |     3.43 |   -2%˄ | (run time too short) |
 | 23b     ||    252.1 |    257.2 |   +2%˄ ||     3.97 |     3.89 |   -2%˄ | (run time too short) |
-| 23c     ||    288.3 |    314.3 |   +9%˄ ||     3.47 |     3.18 |   -8%˄ | (run time too short) |
 | 24a     ||   4689.6 |   4753.9 |   +1%  ||     0.21 |     0.21 |   -1%  |               0.0308 |
 | 24b     ||   4682.2 |   4688.1 |   +0%  ||     0.21 |     0.21 |   -0%  |               0.8720 |
 | 25a     ||   2685.8 |   2705.7 |   +1%  ||     0.37 |     0.37 |   -1%  |               0.0552 |
 | 25b     ||   2629.0 |   2613.2 |   -1%  ||     0.38 |     0.38 |   +1%  |               0.3276 |
+| 25c     ||   5693.6 |   2983.5 |  -48%  ||     0.18 |     0.34 |  +91%  |               0.0000 |
 | 26a     ||    631.0 |    629.8 |   -0%  ||     1.58 |     1.59 |   +0%  |               0.6889 |
 | 26b     ||   4236.3 |   4125.9 |   -3%  ||     0.24 |     0.24 |   +3%  |               0.1607 |
 | 26c     ||   4352.3 |   4291.4 |   -1%  ||     0.23 |     0.23 |   +1%  |               0.3588 |
 | 27a     ||   2846.0 |   2836.9 |   -0%  ||     0.35 |     0.35 |   +0%  |               0.7251 |
 | 27b     ||   2781.3 |   2825.1 |   +2%  ||     0.36 |     0.35 |   -2%  |               0.0268 |
 | 27c     ||   3384.1 |   3338.8 |   -1%  ||     0.30 |     0.30 |   +1%  |               0.2688 |
 | 28a     ||    421.4 |    414.4 |   -2%˄ ||     2.37 |     2.41 |   +2%˄ | (run time too short) |
 | 28b     ||   2844.1 |   2878.2 |   +1%  ||     0.35 |     0.35 |   -1%  |               0.3080 |
 | 28c     ||    423.7 |    427.1 |   +1%˄ ||     2.36 |     2.34 |   -1%˄ | (run time too short) |
-| 29a     ||   2315.6 |   2429.8 |   +5%  ||     0.43 |     0.41 |   -5%  |               0.0065 |
 | 29b     ||   1012.9 |   1033.8 |   +2%  ||     0.99 |     0.97 |   -2%  |               0.2503 |
 | 29c     ||   4839.8 |   4863.5 |   +0%  ||     0.21 |     0.21 |   -0%  |               0.6965 |
 | 2a      ||     47.0 |     47.7 |   +2%˄ ||    21.28 |    20.95 |   -2%˄ | (run time too short) |
 | 2b      ||     40.6 |     40.9 |   +1%˄ ||    24.64 |    24.47 |   -1%˄ | (run time too short) |
 | 2c      ||     30.8 |     31.0 |   +1%˄ ||    32.48 |    32.20 |   -1%˄ | (run time too short) |
 | 2d      ||    115.1 |    115.8 |   +1%˄ ||     8.69 |     8.64 |   -1%˄ | (run time too short) |
 | 30a     ||   2675.8 |   2671.5 |   -0%  ||     0.37 |     0.37 |   +0%  |               0.8000 |
 | 30b     ||   2769.2 |   2833.7 |   +2%  ||     0.36 |     0.35 |   -2%  |               0.0044 |
+| 30c     ||   5543.6 |   2986.9 |  -46%  ||     0.18 |     0.33 |  +86%  |               0.0000 |
 | 31a     ||   2680.2 |   2681.2 |   +0%  ||     0.37 |     0.37 |   -0%  |               0.9676 |
 | 31b     ||   2775.6 |   2800.6 |   +1%  ||     0.36 |     0.36 |   -1%  |               0.3788 |
+| 31c     ||   5491.5 |   2899.9 |  -47%  ||     0.18 |     0.34 |  +89%  |               0.0000 |
 | 32a     ||     27.6 |     27.1 |   -2%˄ ||    36.17 |    36.89 |   +2%˄ | (run time too short) |
 | 32b     ||     55.9 |     57.7 |   +3%˄ ||    17.87 |    17.34 |   -3%˄ | (run time too short) |
 | 33a     ||     38.7 |     40.1 |   +4%˄ ||    25.83 |    24.95 |   -3%˄ | (run time too short) |
+| 33b     ||     46.5 |     42.6 |   -8%˄ ||    21.48 |    23.46 |   +9%˄ | (run time too short) |
 | 33c     ||    102.3 |    102.0 |   -0%˄ ||     9.78 |     9.80 |   +0%˄ | (run time too short) |
 | 3a      ||   2981.1 |   2979.3 |   -0%  ||     0.34 |     0.34 |   +0%  |               0.7782 |
 | 3b      ||     32.9 |     32.8 |   -0%˄ ||    30.37 |    30.49 |   +0%˄ | (run time too short) |
 | 3c      ||   3243.4 |   3189.3 |   -2%  ||     0.31 |     0.31 |   +2%  |               0.0079 |
 | 4a      ||     56.8 |     57.4 |   +1%˄ ||    17.61 |    17.42 |   -1%˄ | (run time too short) |
 | 4b      ||     31.6 |     31.3 |   -1%˄ ||    31.67 |    31.95 |   +1%˄ | (run time too short) |
 | 4c      ||     64.2 |     64.6 |   +1%˄ ||    15.57 |    15.47 |   -1%˄ | (run time too short) |
+| 5a      ||   3217.2 |   2973.0 |   -8%  ||     0.31 |     0.34 |   +8%  |               0.0000 |
 | 5b      ||     62.1 |     59.9 |   -3%˄ ||    16.11 |    16.69 |   +4%˄ | (run time too short) |
+| 5c      ||   3404.2 |   3216.0 |   -6%  ||     0.29 |     0.31 |   +6%  |               0.0000 |
 | 6a      ||    291.6 |    296.1 |   +2%˄ ||     3.43 |     3.38 |   -1%˄ | (run time too short) |
 | 6b      ||    320.2 |    323.5 |   +1%˄ ||     3.12 |     3.09 |   -1%˄ | (run time too short) |
 | 6c      ||    278.3 |    286.7 |   +3%˄ ||     3.59 |     3.49 |   -3%˄ | (run time too short) |
 | 6d      ||    534.9 |    524.1 |   -2%˄ ||     1.87 |     1.91 |   +2%˄ | (run time too short) |
 | 6e      ||    301.8 |    308.5 |   +2%˄ ||     3.31 |     3.24 |   -2%˄ | (run time too short) |
 | 6f      ||   1134.0 |   1155.7 |   +2%  ||     0.88 |     0.87 |   -2%  |               0.0031 |
 | 7a      ||    176.1 |    183.5 |   +4%˄ ||     5.68 |     5.45 |   -4%˄ | (run time too short) |
 | 7b      ||    132.7 |    138.6 |   +4%˄ ||     7.53 |     7.22 |   -4%˄ | (run time too short) |
 | 7c      ||    720.8 |    699.7 |   -3%  ||     1.39 |     1.43 |   +3%  |               0.1943 |
 | 8a      ||    252.0 |    255.5 |   +1%˄ ||     3.97 |     3.91 |   -1%˄ | (run time too short) |
 | 8b      ||    305.0 |    309.7 |   +2%˄ ||     3.28 |     3.23 |   -2%˄ | (run time too short) |
+| 8c      ||   3037.3 |   2568.7 |  -15%  ||     0.33 |     0.39 |  +18%  |               0.0000 |
 | 8d      ||    418.9 |    426.5 |   +2%˄ ||     2.39 |     2.34 |   -2%˄ | (run time too short) |
 | 9a      ||   2976.0 |   2972.8 |   -0%  ||     0.34 |     0.34 |   +0%  |               0.9151 |
 | 9b      ||    270.1 |    264.2 |   -2%˄ ||     3.70 |     3.79 |   +2%˄ | (run time too short) |
 | 9c      ||   2895.3 |   2859.8 |   -1%  ||     0.35 |     0.35 |   +1%  |               0.1945 |
 | 9d      ||   3096.0 |   3096.8 |   +0%  ||     0.32 |     0.32 |   -0%  |               0.9759 |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Sum     || 157475.9 | 150789.4 |   -4%  ||          |          |        |                      |
 | Geomean ||          |          |        ||          |          |   -1%  |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 |   Notes || ˄ Execution stopped due to max runs reached                                         |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+

JOB K 50 vs JOB K 100
 +Configuration Overview---------------------------------------------------+-------------------------------------------------------+
 | Parameter        | benchmark_as_generic_wo_semijoin_reduction_k_50.json | benchmark_as_generic_wo_semijoin_reduction_k_100.json |
 +------------------+------------------------------------------------------+-------------------------------------------------------+
 |  GIT-HASH        | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty       | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty        |
 |  benchmark_mode  | Ordered                                              | Ordered                                               |
 |  build_type      | release                                              | release                                               |
 |  chunk_size      | 65535                                                | 65535                                                 |
 |  clients         | 1                                                    | 1                                                     |
 |  compiler        | gcc 9.3                                              | gcc 9.3                                               |
 |  cores           | 0                                                    | 0                                                     |
 |  date            | 2021-07-31 21:31:22                                  | 2021-07-31 22:48:51                                   |
 |  encoding        | {'default': {'encoding': 'Dictionary'}}              | {'default': {'encoding': 'Dictionary'}}               |
 |  indexes         | False                                                | False                                                 |
 |  max_duration    |<PHONE_NUMBER>0                                          |<PHONE_NUMBER>0                                           |
 |  max_runs        | 100                                                  | 100                                                   |
 |  time_unit       | ns                                                   | ns                                                    |
 |  using_scheduler | False                                                | False                                                 |
 |  verify          | False                                                | False                                                 |
 |  warmup_duration | 0                                                    | 0                                                     |
 +------------------+------------------------------------------------------+-------------------------------------------------------+

 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Item    || Latency (ms/iter)   | Change || Throughput (iter/s) | Change |              p-value |
 |         ||      old |      new |        ||      old |      new |        |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
+| 10a     ||    536.9 |    511.4 |   -5%˄ ||     1.86 |     1.96 |   +5%˄ | (run time too short) |
+| 10b     ||    439.1 |    404.8 |   -8%˄ ||     2.28 |     2.47 |   +8%˄ | (run time too short) |
+| 10c     ||    495.0 |    465.8 |   -6%˄ ||     2.02 |     2.15 |   +6%˄ | (run time too short) |
+| 11a     ||    181.5 |    173.6 |   -4%˄ ||     5.51 |     5.76 |   +5%˄ | (run time too short) |
 | 11b     ||    169.8 |    169.1 |   -0%˄ ||     5.89 |     5.91 |   +0%˄ | (run time too short) |
-| 11c     ||    253.0 |    267.2 |   +6%˄ ||     3.95 |     3.74 |   -5%˄ | (run time too short) |
 | 11d     ||    198.8 |    199.5 |   +0%˄ ||     5.03 |     5.01 |   -0%˄ | (run time too short) |
 | 12a     ||    135.2 |    135.0 |   -0%˄ ||     7.39 |     7.41 |   +0%˄ | (run time too short) |
 | 12b     ||    228.6 |    230.8 |   +1%˄ ||     4.37 |     4.33 |   -1%˄ | (run time too short) |
 | 12c     ||    288.9 |    295.6 |   +2%˄ ||     3.46 |     3.38 |   -2%˄ | (run time too short) |
 | 13a     ||    284.4 |    284.3 |   -0%˄ ||     3.52 |     3.52 |   +0%˄ | (run time too short) |
 | 13b     ||    249.0 |    257.3 |   +3%˄ ||     4.02 |     3.89 |   -3%˄ | (run time too short) |
 | 13c     ||    170.5 |    169.1 |   -1%˄ ||     5.86 |     5.92 |   +1%˄ | (run time too short) |
 | 13d     ||    609.0 |    616.9 |   +1%  ||     1.64 |     1.62 |   -1%  |               0.0000 |
-| 14a     ||    288.7 |    310.7 |   +8%˄ ||     3.46 |     3.22 |   -7%˄ | (run time too short) |
-| 14b     ||    409.7 |    441.5 |   +8%˄ ||     2.44 |     2.27 |   -7%˄ | (run time too short) |
-| 14c     ||    351.3 |    376.1 |   +7%˄ ||     2.85 |     2.66 |   -7%˄ | (run time too short) |
 | 15a     ||    297.9 |    306.1 |   +3%˄ ||     3.36 |     3.27 |   -3%˄ | (run time too short) |
 | 15b     ||    267.2 |    267.9 |   +0%˄ ||     3.74 |     3.73 |   -0%˄ | (run time too short) |
 | 15c     ||    281.0 |    282.0 |   +0%˄ ||     3.56 |     3.55 |   -0%˄ | (run time too short) |
 | 15d     ||    336.2 |    339.4 |   +1%˄ ||     2.97 |     2.95 |   -1%˄ | (run time too short) |
+| 16a     ||   2047.7 |   1726.9 |  -16%  ||     0.49 |     0.58 |  +19%  |               0.0000 |
+| 16b     ||   4119.0 |   3618.5 |  -12%  ||     0.24 |     0.28 |  +14%  |               0.0000 |
+| 16c     ||   2362.0 |   1989.9 |  -16%  ||     0.42 |     0.50 |  +19%  |               0.0000 |
+| 16d     ||   2258.0 |   1958.3 |  -13%  ||     0.44 |     0.51 |  +15%  |               0.0000 |
 | 17a     ||    579.2 |    585.0 |   +1%˄ ||     1.73 |     1.71 |   -1%˄ | (run time too short) |
 | 17b     ||    527.0 |    523.9 |   -1%˄ ||     1.90 |     1.91 |   +1%˄ | (run time too short) |
+| 17c     ||    519.4 |    492.7 |   -5%˄ ||     1.93 |     2.03 |   +5%˄ | (run time too short) |
+| 17d     ||    643.8 |    605.1 |   -6%˄ ||     1.55 |     1.65 |   +6%˄ |               0.0000 |
+| 17e     ||   3222.8 |   2892.1 |  -10%  ||     0.31 |     0.35 |  +11%  |               0.0000 |
 | 17f     ||   1340.9 |   1305.4 |   -3%  ||     0.75 |     0.77 |   +3%  |               0.0182 |
 | 18a     ||    273.1 |    267.0 |   -2%˄ ||     3.66 |     3.75 |   +2%˄ | (run time too short) |
 | 18b     ||   2580.4 |   2677.8 |   +4%  ||     0.39 |     0.37 |   -4%  |               0.0012 |
 | 18c     ||   2938.0 |   2964.3 |   +1%  ||     0.34 |     0.34 |   -1%  |               0.3961 |
 | 19a     ||   5034.3 |   5042.5 |   +0%  ||     0.20 |     0.20 |   -0%  |               0.7683 |
 | 19b     ||   2313.5 |   2322.6 |   +0%  ||     0.43 |     0.43 |   -0%  |               0.5786 |
 | 19c     ||   5062.3 |   5121.6 |   +1%  ||     0.20 |     0.20 |   -1%  |               0.1130 |
 | 19d     ||   4247.7 |   4219.5 |   -1%  ||     0.24 |     0.24 |   +1%  |               0.2256 |
 | 1a      ||     46.5 |     46.2 |   -1%˄ ||    21.52 |    21.66 |   +1%˄ | (run time too short) |
 | 1b      ||     54.1 |     54.8 |   +1%˄ ||    18.47 |    18.25 |   -1%˄ | (run time too short) |
-| 1c      ||     26.7 |     28.0 |   +5%˄ ||    37.43 |    35.65 |   -5%˄ | (run time too short) |
 | 1d      ||     61.0 |     61.2 |   +0%˄ ||    16.38 |    16.34 |   -0%˄ | (run time too short) |
 | 20a     ||    745.2 |    726.7 |   -2%  ||     1.34 |     1.38 |   +3%  |               0.0000 |
 | 20b     ||    826.0 |    792.6 |   -4%  ||     1.21 |     1.26 |   +4%  |               0.0000 |
 | 20c     ||    624.9 |    644.7 |   +3%  ||     1.60 |     1.55 |   -3%  |               0.0000 |
 | 21a     ||   3098.9 |   3218.7 |   +4%  ||     0.32 |     0.31 |   -4%  |               0.0008 |
 | 21b     ||    179.1 |    185.1 |   +3%˄ ||     5.58 |     5.40 |   -3%˄ | (run time too short) |
 | 21c     ||   3393.8 |   3340.4 |   -2%  ||     0.29 |     0.30 |   +2%  |               0.0901 |
 | 22a     ||    309.6 |    308.1 |   -0%˄ ||     3.23 |     3.25 |   +0%˄ | (run time too short) |
 | 22b     ||    296.3 |    294.6 |   -1%˄ ||     3.38 |     3.39 |   +1%˄ | (run time too short) |
 | 22c     ||    424.6 |    421.6 |   -1%˄ ||     2.36 |     2.37 |   +1%˄ | (run time too short) |
 | 22d     ||    486.0 |    494.7 |   +2%˄ ||     2.06 |     2.02 |   -2%˄ | (run time too short) |
 | 23a     ||    282.8 |    291.7 |   +3%˄ ||     3.54 |     3.43 |   -3%˄ | (run time too short) |
-| 23b     ||    244.9 |    257.2 |   +5%˄ ||     4.08 |     3.89 |   -5%˄ | (run time too short) |
-| 23c     ||    289.5 |    314.3 |   +9%˄ ||     3.45 |     3.18 |   -8%˄ | (run time too short) |
 | 24a     ||   4748.6 |   4753.9 |   +0%  ||     0.21 |     0.21 |   -0%  |               0.8613 |
 | 24b     ||   4688.6 |   4688.1 |   -0%  ||     0.21 |     0.21 |   +0%  |               0.9906 |
 | 25a     ||   2677.5 |   2705.7 |   +1%  ||     0.37 |     0.37 |   -1%  |               0.0103 |
 | 25b     ||   2605.7 |   2613.2 |   +0%  ||     0.38 |     0.38 |   -0%  |               0.6381 |
 | 25c     ||   2993.5 |   2983.5 |   -0%  ||     0.33 |     0.34 |   +0%  |               0.5160 |
 | 26a     ||    632.1 |    629.8 |   -0%  ||     1.58 |     1.59 |   +0%  |               0.5020 |
 | 26b     ||   4194.8 |   4125.9 |   -2%  ||     0.24 |     0.24 |   +2%  |               0.3643 |
 | 26c     ||   4316.7 |   4291.4 |   -1%  ||     0.23 |     0.23 |   +1%  |               0.6915 |
 | 27a     ||   2807.4 |   2836.9 |   +1%  ||     0.36 |     0.35 |   -1%  |               0.1955 |
 | 27b     ||   2804.5 |   2825.1 |   +1%  ||     0.36 |     0.35 |   -1%  |               0.2886 |
 | 27c     ||   3290.6 |   3338.8 |   +1%  ||     0.30 |     0.30 |   -1%  |               0.2528 |
 | 28a     ||    416.7 |    414.4 |   -1%˄ ||     2.40 |     2.41 |   +1%˄ | (run time too short) |
 | 28b     ||   2814.1 |   2878.2 |   +2%  ||     0.36 |     0.35 |   -2%  |               0.0502 |
 | 28c     ||    410.7 |    427.1 |   +4%˄ ||     2.43 |     2.34 |   -4%˄ | (run time too short) |
-| 29a     ||   2317.9 |   2429.8 |   +5%  ||     0.43 |     0.41 |   -5%  |               0.0119 |
 | 29b     ||   1012.4 |   1033.8 |   +2%  ||     0.99 |     0.97 |   -2%  |               0.2922 |
 | 29c     ||   4822.4 |   4863.5 |   +1%  ||     0.21 |     0.21 |   -1%  |               0.5474 |
 | 2a      ||     47.7 |     47.7 |   +0%˄ ||    20.98 |    20.95 |   -0%˄ | (run time too short) |
 | 2b      ||     41.7 |     40.9 |   -2%˄ ||    23.99 |    24.47 |   +2%˄ | (run time too short) |
+| 2c      ||     34.2 |     31.0 |   -9%˄ ||    29.25 |    32.20 |  +10%˄ | (run time too short) |
+| 2d      ||    122.6 |    115.8 |   -6%˄ ||     8.15 |     8.64 |   +6%˄ | (run time too short) |
 | 30a     ||   2681.8 |   2671.5 |   -0%  ||     0.37 |     0.37 |   +0%  |               0.6111 |
 | 30b     ||   2790.6 |   2833.7 |   +2%  ||     0.36 |     0.35 |   -2%  |               0.0604 |
 | 30c     ||   2962.1 |   2986.9 |   +1%  ||     0.34 |     0.33 |   -1%  |               0.3669 |
 | 31a     ||   2690.0 |   2681.2 |   -0%  ||     0.37 |     0.37 |   +0%  |               0.7427 |
 | 31b     ||   2800.3 |   2800.6 |   +0%  ||     0.36 |     0.36 |   -0%  |               0.9913 |
 | 31c     ||   2874.8 |   2899.9 |   +1%  ||     0.35 |     0.34 |   -1%  |               0.4371 |
 | 32a     ||     26.8 |     27.1 |   +1%˄ ||    37.25 |    36.89 |   -1%˄ | (run time too short) |
 | 32b     ||     57.0 |     57.7 |   +1%˄ ||    17.54 |    17.34 |   -1%˄ | (run time too short) |
 | 33a     ||     39.1 |     40.1 |   +3%˄ ||    25.59 |    24.95 |   -2%˄ | (run time too short) |
 | 33b     ||     41.7 |     42.6 |   +2%˄ ||    23.96 |    23.46 |   -2%˄ | (run time too short) |
 | 33c     ||    100.7 |    102.0 |   +1%˄ ||     9.93 |     9.80 |   -1%˄ | (run time too short) |
 | 3a      ||   2960.5 |   2979.3 |   +1%  ||     0.34 |     0.34 |   -1%  |               0.0107 |
+| 3b      ||     34.8 |     32.8 |   -6%˄ ||    28.75 |    30.49 |   +6%˄ | (run time too short) |
 | 3c      ||   3231.3 |   3189.3 |   -1%  ||     0.31 |     0.31 |   +1%  |               0.0331 |
 | 4a      ||     57.7 |     57.4 |   -1%˄ ||    17.32 |    17.42 |   +1%˄ | (run time too short) |
 | 4b      ||     31.0 |     31.3 |   +1%˄ ||    32.25 |    31.95 |   -1%˄ | (run time too short) |
 | 4c      ||     65.4 |     64.6 |   -1%˄ ||    15.28 |    15.47 |   +1%˄ | (run time too short) |
 | 5a      ||   2996.0 |   2973.0 |   -1%  ||     0.33 |     0.34 |   +1%  |               0.1855 |
 | 5b      ||     59.9 |     59.9 |   -0%˄ ||    16.69 |    16.69 |   +0%˄ | (run time too short) |
 | 5c      ||   3323.9 |   3216.0 |   -3%  ||     0.30 |     0.31 |   +3%  |               0.0039 |
 | 6a      ||    295.5 |    296.1 |   +0%˄ ||     3.38 |     3.38 |   -0%˄ | (run time too short) |
 | 6b      ||    323.4 |    323.5 |   +0%˄ ||     3.09 |     3.09 |   -0%˄ | (run time too short) |
 | 6c      ||    294.3 |    286.7 |   -3%˄ ||     3.40 |     3.49 |   +3%˄ | (run time too short) |
 | 6d      ||    513.6 |    524.1 |   +2%˄ ||     1.95 |     1.91 |   -2%˄ | (run time too short) |
 | 6e      ||    305.8 |    308.5 |   +1%˄ ||     3.27 |     3.24 |   -1%˄ | (run time too short) |
 | 6f      ||   1149.3 |   1155.7 |   +1%  ||     0.87 |     0.87 |   -1%  |               0.1692 |
 | 7a      ||    178.8 |    183.5 |   +3%˄ ||     5.59 |     5.45 |   -3%˄ | (run time too short) |
 | 7b      ||    133.9 |    138.6 |   +4%˄ ||     7.47 |     7.22 |   -3%˄ | (run time too short) |
+| 7c      ||    857.3 |    699.7 |  -18%  ||     1.17 |     1.43 |  +23%  |               0.0000 |
 | 8a      ||    250.1 |    255.5 |   +2%˄ ||     4.00 |     3.91 |   -2%˄ | (run time too short) |
 | 8b      ||    303.3 |    309.7 |   +2%˄ ||     3.30 |     3.23 |   -2%˄ | (run time too short) |
 | 8c      ||   2568.7 |   2568.7 |   +0%  ||     0.39 |     0.39 |   -0%  |               0.9992 |
 | 8d      ||    421.6 |    426.5 |   +1%˄ ||     2.37 |     2.34 |   -1%˄ | (run time too short) |
 | 9a      ||   2980.1 |   2972.8 |   -0%  ||     0.34 |     0.34 |   +0%  |               0.8025 |
 | 9b      ||    269.2 |    264.2 |   -2%˄ ||     3.71 |     3.79 |   +2%˄ | (run time too short) |
 | 9c      ||   2878.5 |   2859.8 |   -1%  ||     0.35 |     0.35 |   +1%  |               0.5288 |
 | 9d      ||   3100.2 |   3096.8 |   -0%  ||     0.32 |     0.32 |   +0%  |               0.9130 |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Sum     || 152380.1 | 150789.4 |   -1%  ||          |          |        |                      |
 | Geomean ||          |          |        ||          |          |   +1%  |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 |   Notes || ˄ Execution stopped due to max runs reached                                         |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+

JOB K 1000 vs JOB K 100
 +Configuration Overview-----------------------------------------------------+-------------------------------------------------------+
 | Parameter        | benchmark_as_generic_wo_semijoin_reduction_k_1000.json | benchmark_as_generic_wo_semijoin_reduction_k_100.json |
 +------------------+--------------------------------------------------------+-------------------------------------------------------+
 |  GIT-HASH        | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty         | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty        |
 |  benchmark_mode  | Ordered                                                | Ordered                                               |
 |  build_type      | release                                                | release                                               |
 |  chunk_size      | 65535                                                  | 65535                                                 |
 |  clients         | 1                                                      | 1                                                     |
 |  compiler        | gcc 9.3                                                | gcc 9.3                                               |
 |  cores           | 0                                                      | 0                                                     |
 |  date            | 2021-08-01 00:06:39                                    | 2021-07-31 22:48:51                                   |
 |  encoding        | {'default': {'encoding': 'Dictionary'}}                | {'default': {'encoding': 'Dictionary'}}               |
 |  indexes         | False                                                  | False                                                 |
 |  max_duration    |<PHONE_NUMBER>0                                            |<PHONE_NUMBER>0                                           |
 |  max_runs        | 100                                                    | 100                                                   |
 |  time_unit       | ns                                                     | ns                                                    |
 |  using_scheduler | False                                                  | False                                                 |
 |  verify          | False                                                  | False                                                 |
 |  warmup_duration | 0                                                      | 0                                                     |
 +------------------+--------------------------------------------------------+-------------------------------------------------------+

 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Item    || Latency (ms/iter)   | Change || Throughput (iter/s) | Change |              p-value |
 |         ||      old |      new |        ||      old |      new |        |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | 10a     ||    496.0 |    511.4 |   +3%˄ ||     2.02 |     1.96 |   -3%˄ | (run time too short) |
 | 10b     ||    404.2 |    404.8 |   +0%˄ ||     2.47 |     2.47 |   -0%˄ | (run time too short) |
 | 10c     ||    462.0 |    465.8 |   +1%˄ ||     2.16 |     2.15 |   -1%˄ | (run time too short) |
+| 11a     ||    222.3 |    173.6 |  -22%˄ ||     4.50 |     5.76 |  +28%˄ | (run time too short) |
+| 11b     ||    217.0 |    169.1 |  -22%˄ ||     4.61 |     5.91 |  +28%˄ | (run time too short) |
+| 11c     ||    294.4 |    267.2 |   -9%˄ ||     3.40 |     3.74 |  +10%˄ | (run time too short) |
+| 11d     ||    244.8 |    199.5 |  -19%˄ ||     4.08 |     5.01 |  +23%˄ | (run time too short) |
+| 12a     ||    202.1 |    135.0 |  -33%˄ ||     4.95 |     7.41 |  +50%˄ | (run time too short) |
+| 12b     ||    299.4 |    230.8 |  -23%˄ ||     3.34 |     4.33 |  +30%˄ | (run time too short) |
+| 12c     ||    362.4 |    295.6 |  -18%˄ ||     2.76 |     3.38 |  +23%˄ | (run time too short) |
 | 13a     ||    281.6 |    284.3 |   +1%˄ ||     3.55 |     3.52 |   -1%˄ | (run time too short) |
 | 13b     ||    254.3 |    257.3 |   +1%˄ ||     3.93 |     3.89 |   -1%˄ | (run time too short) |
 | 13c     ||    173.2 |    169.1 |   -2%˄ ||     5.77 |     5.92 |   +2%˄ | (run time too short) |
 | 13d     ||    635.5 |    616.9 |   -3%  ||     1.57 |     1.62 |   +3%  |               0.0004 |
+| 14a     ||    379.1 |    310.7 |  -18%˄ ||     2.64 |     3.22 |  +22%˄ | (run time too short) |
+| 14b     ||    504.9 |    441.5 |  -13%˄ ||     1.98 |     2.27 |  +14%˄ | (run time too short) |
+| 14c     ||    437.6 |    376.1 |  -14%˄ ||     2.29 |     2.66 |  +16%˄ | (run time too short) |
 | 15a     ||    315.1 |    306.1 |   -3%˄ ||     3.17 |     3.27 |   +3%˄ | (run time too short) |
+| 15b     ||    284.9 |    267.9 |   -6%˄ ||     3.51 |     3.73 |   +6%˄ | (run time too short) |
+| 15c     ||    295.7 |    282.0 |   -5%˄ ||     3.38 |     3.55 |   +5%˄ | (run time too short) |
+| 15d     ||    359.4 |    339.4 |   -6%˄ ||     2.78 |     2.95 |   +6%˄ | (run time too short) |
+| 16a     ||   2051.7 |   1726.9 |  -16%  ||     0.49 |     0.58 |  +19%  |               0.3322 |
+| 16b     ||   4103.4 |   3618.5 |  -12%  ||     0.24 |     0.28 |  +13%  |               0.4732 |
+| 16c     ||   2425.5 |   1989.9 |  -18%  ||     0.41 |     0.50 |  +22%  |               0.2653 |
+| 16d     ||   2343.9 |   1958.3 |  -16%  ||     0.43 |     0.51 |  +20%  |               0.3034 |
+| 17a     ||    629.0 |    585.0 |   -7%˄ ||     1.59 |     1.71 |   +8%˄ | (run time too short) |
+| 17b     ||    560.9 |    523.9 |   -7%˄ ||     1.78 |     1.91 |   +7%˄ | (run time too short) |
+| 17c     ||    533.1 |    492.7 |   -8%˄ ||     1.88 |     2.03 |   +8%˄ | (run time too short) |
+| 17d     ||    647.5 |    605.1 |   -7%˄ ||     1.54 |     1.65 |   +7%˄ |               0.3451 |
-| 17e     ||   2242.1 |   2892.1 |  +29%  ||     0.45 |     0.35 |  -22%  |               0.0009 |
+| 17f     ||   1383.9 |   1305.4 |   -6%  ||     0.72 |     0.77 |   +6%  |               0.4198 |
+| 18a     ||    301.0 |    267.0 |  -11%˄ ||     3.32 |     3.75 |  +13%˄ | (run time too short) |
+| 18b     ||   2845.7 |   2677.8 |   -6%  ||     0.35 |     0.37 |   +6%  |               0.3338 |
+| 18c     ||   3183.3 |   2964.3 |   -7%  ||     0.31 |     0.34 |   +7%  |               0.3182 |
 | 19a     ||   5053.8 |   5042.5 |   -0%  ||     0.20 |     0.20 |   +0%  |               0.9055 |
 | 19b     ||   2307.9 |   2322.6 |   +1%  ||     0.43 |     0.43 |   -1%  |               0.7331 |
 | 19c     ||   5271.4 |   5121.6 |   -3%  ||     0.19 |     0.20 |   +3%  |               0.1300 |
+| 19d     ||   4473.4 |   4219.5 |   -6%  ||     0.22 |     0.24 |   +6%  |               0.0026 |
+| 1a      ||     48.3 |     46.2 |   -4%˄ ||    20.69 |    21.66 |   +5%˄ | (run time too short) |
 | 1b      ||     55.8 |     54.8 |   -2%˄ ||    17.91 |    18.25 |   +2%˄ | (run time too short) |
 | 1c      ||     28.2 |     28.0 |   -0%˄ ||    35.47 |    35.65 |   +0%˄ | (run time too short) |
 | 1d      ||     62.9 |     61.2 |   -3%˄ ||    15.89 |    16.34 |   +3%˄ | (run time too short) |
 | 20a     ||    732.8 |    726.7 |   -1%  ||     1.36 |     1.38 |   +1%  |               0.4671 |
 | 20b     ||    801.2 |    792.6 |   -1%  ||     1.25 |     1.26 |   +1%  |               0.3734 |
 | 20c     ||    628.2 |    644.7 |   +3%  ||     1.59 |     1.55 |   -3%  |               0.0352 |
 | 21a     ||   3170.5 |   3218.7 |   +2%  ||     0.32 |     0.31 |   -1%  |               0.4709 |
 | 21b     ||    189.3 |    185.1 |   -2%˄ ||     5.28 |     5.40 |   +2%˄ | (run time too short) |
 | 21c     ||   3286.4 |   3340.4 |   +2%  ||     0.30 |     0.30 |   -2%  |               0.3914 |
+| 22a     ||    325.5 |    308.1 |   -5%˄ ||     3.07 |     3.25 |   +6%˄ | (run time too short) |
+| 22b     ||    312.0 |    294.6 |   -6%˄ ||     3.21 |     3.39 |   +6%˄ | (run time too short) |
+| 22c     ||    443.3 |    421.6 |   -5%˄ ||     2.26 |     2.37 |   +5%˄ | (run time too short) |
 | 22d     ||    510.3 |    494.7 |   -3%˄ ||     1.96 |     2.02 |   +3%˄ | (run time too short) |
 | 23a     ||    301.6 |    291.7 |   -3%˄ ||     3.32 |     3.43 |   +3%˄ | (run time too short) |
 | 23b     ||    260.9 |    257.2 |   -1%˄ ||     3.83 |     3.89 |   +1%˄ | (run time too short) |
 | 23c     ||    309.0 |    314.3 |   +2%˄ ||     3.24 |     3.18 |   -2%˄ | (run time too short) |
 | 24a     ||   4941.7 |   4753.9 |   -4%  ||     0.20 |     0.21 |   +4%  |               0.3645 |
+| 24b     ||   5025.5 |   4688.1 |   -7%  ||     0.20 |     0.21 |   +7%  |               0.2448 |
 | 25a     ||   2761.4 |   2705.7 |   -2%  ||     0.36 |     0.37 |   +2%  |               0.3766 |
 | 25b     ||   2693.9 |   2613.2 |   -3%  ||     0.37 |     0.38 |   +3%  |               0.3363 |
 | 25c     ||   3110.1 |   2983.5 |   -4%  ||     0.32 |     0.34 |   +4%  |               0.0933 |
+| 26a     ||    673.4 |    629.8 |   -6%  ||     1.49 |     1.59 |   +7%  |               0.0123 |
+| 26b     ||   4567.9 |   4125.9 |  -10%  ||     0.22 |     0.24 |  +11%  |               0.0137 |
+| 26c     ||   4796.0 |   4291.4 |  -11%  ||     0.21 |     0.23 |  +12%  |               0.0001 |
+| 27a     ||   3096.0 |   2836.9 |   -8%  ||     0.32 |     0.35 |   +9%  |               0.0129 |
+| 27b     ||   2960.4 |   2825.1 |   -5%  ||     0.34 |     0.35 |   +5%  |               0.1818 |
 | 27c     ||   3391.8 |   3338.8 |   -2%  ||     0.29 |     0.30 |   +2%  |               0.6849 |
+| 28a     ||    458.9 |    414.4 |  -10%˄ ||     2.18 |     2.41 |  +11%˄ | (run time too short) |
-| 28b     ||    387.9 |   2878.2 | +642%˄ ||     2.58 |     0.35 |  -87%˄ | (run time too short) |
+| 28c     ||    460.2 |    427.1 |   -7%˄ ||     2.17 |     2.34 |   +8%˄ | (run time too short) |
+| 29a     ||   2645.4 |   2429.8 |   -8%  ||     0.38 |     0.41 |   +9%  |               0.4921 |
+| 29b     ||   1161.8 |   1033.8 |  -11%  ||     0.86 |     0.97 |  +12%  |               0.3633 |
+| 29c     ||   5245.3 |   4863.5 |   -7%  ||     0.19 |     0.21 |   +8%  |               0.3883 |
 | 2a      ||     49.1 |     47.7 |   -3%˄ ||    20.36 |    20.95 |   +3%˄ | (run time too short) |
+| 2b      ||     43.6 |     40.9 |   -6%˄ ||    22.94 |    24.47 |   +7%˄ | (run time too short) |
+| 2c      ||     33.3 |     31.0 |   -7%˄ ||    30.02 |    32.20 |   +7%˄ | (run time too short) |
 | 2d      ||    118.1 |    115.8 |   -2%˄ ||     8.47 |     8.64 |   +2%˄ | (run time too short) |
+| 30a     ||   2870.7 |   2671.5 |   -7%  ||     0.35 |     0.37 |   +7%  |               0.2657 |
 | 30b     ||   2950.6 |   2833.7 |   -4%  ||     0.34 |     0.35 |   +4%  |               0.5008 |
 | 30c     ||   3103.6 |   2986.9 |   -4%  ||     0.32 |     0.33 |   +4%  |               0.5263 |
+| 31a     ||   2870.1 |   2681.2 |   -7%  ||     0.35 |     0.37 |   +7%  |               0.3131 |
+| 31b     ||   3012.6 |   2800.6 |   -7%  ||     0.33 |     0.36 |   +8%  |               0.3169 |
+| 31c     ||   3064.6 |   2899.9 |   -5%  ||     0.33 |     0.34 |   +6%  |               0.4247 |
+| 32a     ||     30.5 |     27.1 |  -11%˄ ||    32.75 |    36.89 |  +13%˄ | (run time too short) |
 | 32b     ||     58.8 |     57.7 |   -2%˄ ||    17.02 |    17.34 |   +2%˄ | (run time too short) |
+| 33a     ||     52.6 |     40.1 |  -24%˄ ||    19.01 |    24.95 |  +31%˄ | (run time too short) |
+| 33b     ||     53.5 |     42.6 |  -20%˄ ||    18.69 |    23.46 |  +25%˄ | (run time too short) |
+| 33c     ||    109.2 |    102.0 |   -7%˄ ||     9.16 |     9.80 |   +7%˄ | (run time too short) |
 | 3a      ||   2956.6 |   2979.3 |   +1%  ||     0.34 |     0.34 |   -1%  |               0.0350 |
 | 3b      ||     33.5 |     32.8 |   -2%˄ ||    29.89 |    30.49 |   +2%˄ | (run time too short) |
 | 3c      ||   3192.9 |   3189.3 |   -0%  ||     0.31 |     0.31 |   +0%  |               0.7410 |
 | 4a      ||     58.8 |     57.4 |   -2%˄ ||    17.00 |    17.42 |   +2%˄ | (run time too short) |
+| 4b      ||     32.7 |     31.3 |   -4%˄ ||    30.55 |    31.95 |   +5%˄ | (run time too short) |
 | 4c      ||     66.6 |     64.6 |   -3%˄ ||    15.00 |    15.47 |   +3%˄ | (run time too short) |
 | 5a      ||   2973.7 |   2973.0 |   -0%  ||     0.34 |     0.34 |   +0%  |               0.9484 |
 | 5b      ||     61.1 |     59.9 |   -2%˄ ||    16.38 |    16.69 |   +2%˄ | (run time too short) |
 | 5c      ||   3190.0 |   3216.0 |   +1%  ||     0.31 |     0.31 |   -1%  |               0.0665 |
 | 6a      ||    292.7 |    296.1 |   +1%˄ ||     3.42 |     3.38 |   -1%˄ | (run time too short) |
 | 6b      ||    318.5 |    323.5 |   +2%˄ ||     3.14 |     3.09 |   -2%˄ | (run time too short) |
 | 6c      ||    280.4 |    286.7 |   +2%˄ ||     3.57 |     3.49 |   -2%˄ | (run time too short) |
 | 6d      ||    514.2 |    524.1 |   +2%˄ ||     1.94 |     1.91 |   -2%˄ | (run time too short) |
 | 6e      ||    306.8 |    308.5 |   +1%˄ ||     3.26 |     3.24 |   -1%˄ | (run time too short) |
 | 6f      ||   1132.8 |   1155.7 |   +2%  ||     0.88 |     0.87 |   -2%  |               0.0011 |
+| 7a      ||    271.3 |    183.5 |  -32%˄ ||     3.69 |     5.45 |  +48%˄ | (run time too short) |
+| 7b      ||    219.8 |    138.6 |  -37%˄ ||     4.55 |     7.22 |  +59%˄ | (run time too short) |
+| 7c      ||    920.2 |    699.7 |  -24%  ||     1.09 |     1.43 |  +32%  |               0.0862 |
 | 8a      ||    265.0 |    255.5 |   -4%˄ ||     3.77 |     3.91 |   +4%˄ | (run time too short) |
 | 8b      ||    320.9 |    309.7 |   -3%˄ ||     3.12 |     3.23 |   +4%˄ | (run time too short) |
 | 8c      ||   2616.4 |   2568.7 |   -2%  ||     0.38 |     0.39 |   +2%  |               0.5905 |
+| 8d      ||    467.6 |    426.5 |   -9%˄ ||     2.14 |     2.34 |  +10%˄ | (run time too short) |
 | 9a      ||   3102.6 |   2972.8 |   -4%  ||     0.32 |     0.34 |   +4%  |               0.4495 |
+| 9b      ||    297.9 |    264.2 |  -11%˄ ||     3.36 |     3.79 |  +13%˄ | (run time too short) |
 | 9c      ||   2928.6 |   2859.8 |   -2%  ||     0.34 |     0.35 |   +2%  |               0.7115 |
+| 9d      ||   3296.8 |   3096.8 |   -6%  ||     0.30 |     0.32 |   +6%  |               0.3178 |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Sum     || 155866.4 | 150789.4 |   -3%  ||          |          |        |                      |
+| Geomean ||          |          |        ||          |          |   +5%  |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 |   Notes || ˄ Execution stopped due to max runs reached                                         |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+

 JOB K 5000 vs JOB K 100

 +Configuration Overview-----------------------------------------------------+-------------------------------------------------------+
 | Parameter        | benchmark_as_generic_wo_semijoin_reduction_k_5000.json | benchmark_as_generic_wo_semijoin_reduction_k_100.json |
 +------------------+--------------------------------------------------------+-------------------------------------------------------+
 |  GIT-HASH        | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty         | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty        |
 |  benchmark_mode  | Ordered                                                | Ordered                                               |
 |  build_type      | release                                                | release                                               |
 |  chunk_size      | 65535                                                  | 65535                                                 |
 |  clients         | 1                                                      | 1                                                     |
 |  compiler        | gcc 9.3                                                | gcc 9.3                                               |
 |  cores           | 0                                                      | 0                                                     |
 |  date            | 2021-08-01 01:31:24                                    | 2021-07-31 22:48:51                                   |
 |  encoding        | {'default': {'encoding': 'Dictionary'}}                | {'default': {'encoding': 'Dictionary'}}               |
 |  indexes         | False                                                  | False                                                 |
 |  max_duration    |<PHONE_NUMBER>0                                            |<PHONE_NUMBER>0                                           |
 |  max_runs        | 100                                                    | 100                                                   |
 |  time_unit       | ns                                                     | ns                                                    |
 |  using_scheduler | False                                                  | False                                                 |
 |  verify          | False                                                  | False                                                 |
 |  warmup_duration | 0                                                      | 0                                                     |
 +------------------+--------------------------------------------------------+-------------------------------------------------------+

 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Item    || Latency (ms/iter)   | Change || Throughput (iter/s) | Change |              p-value |
 |         ||      old |      new |        ||      old |      new |        |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | 10a     ||    528.6 |    511.4 |   -3%˄ ||     1.89 |     1.96 |   +3%˄ | (run time too short) |
+| 10b     ||    443.8 |    404.8 |   -9%˄ ||     2.25 |     2.47 |  +10%˄ | (run time too short) |
-| 10c     ||    247.3 |    465.8 |  +88%˄ ||     4.04 |     2.15 |  -47%˄ | (run time too short) |
+| 11a     ||    406.4 |    173.6 |  -57%˄ ||     2.46 |     5.76 | +134%˄ | (run time too short) |
+| 11b     ||    404.9 |    169.1 |  -58%˄ ||     2.47 |     5.91 | +139%˄ | (run time too short) |
+| 11c     ||    497.1 |    267.2 |  -46%˄ ||     2.01 |     3.74 |  +86%˄ | (run time too short) |
+| 11d     ||    430.3 |    199.5 |  -54%˄ ||     2.32 |     5.01 | +116%˄ | (run time too short) |
+| 12a     ||    487.8 |    135.0 |  -72%˄ ||     2.05 |     7.41 | +261%˄ | (run time too short) |
+| 12b     ||    625.5 |    230.8 |  -63%˄ ||     1.60 |     4.33 | +171%˄ | (run time too short) |
+| 12c     ||    732.8 |    295.6 |  -60%˄ ||     1.36 |     3.38 | +148%˄ | (run time too short) |
+| 13a     ||    301.2 |    284.3 |   -6%˄ ||     3.32 |     3.52 |   +6%˄ | (run time too short) |
+| 13b     ||    278.5 |    257.3 |   -8%˄ ||     3.59 |     3.89 |   +8%˄ | (run time too short) |
+| 13c     ||    196.7 |    169.1 |  -14%˄ ||     5.08 |     5.92 |  +16%˄ | (run time too short) |
+| 13d     ||    664.9 |    616.9 |   -7%  ||     1.50 |     1.62 |   +8%  |               0.0513 |
+| 14a     ||    717.0 |    310.7 |  -57%˄ ||     1.39 |     3.22 | +131%˄ | (run time too short) |
+| 14b     ||    984.3 |    441.5 |  -55%˄ ||     1.02 |     2.27 | +123%˄ | (run time too short) |
+| 14c     ||    859.6 |    376.1 |  -56%˄ ||     1.16 |     2.66 | +129%˄ | (run time too short) |
+| 15a     ||    390.9 |    306.1 |  -22%˄ ||     2.56 |     3.27 |  +28%˄ | (run time too short) |
+| 15b     ||    365.4 |    267.9 |  -27%˄ ||     2.74 |     3.73 |  +36%˄ | (run time too short) |
+| 15c     ||    367.8 |    282.0 |  -23%˄ ||     2.72 |     3.55 |  +30%˄ | (run time too short) |
+| 15d     ||    425.0 |    339.4 |  -20%˄ ||     2.35 |     2.95 |  +25%˄ | (run time too short) |
+| 16a     ||   1826.1 |   1726.9 |   -5%  ||     0.55 |     0.58 |   +6%  |               0.9473 |
+| 16b     ||  20688.0 |   3618.5 |  -83%  ||     0.05 |     0.28 | +472%  |                    ˅ |
+| 16c     ||   3770.5 |   1989.9 |  -47%  ||     0.27 |     0.50 |  +89%  |               0.4322 |
+| 16d     ||   3338.6 |   1958.3 |  -41%  ||     0.30 |     0.51 |  +70%  |               0.5144 |
+| 17a     ||    912.2 |    585.0 |  -36%˄ ||     1.10 |     1.71 |  +56%˄ | (run time too short) |
+| 17b     ||    802.5 |    523.9 |  -35%˄ ||     1.25 |     1.91 |  +53%˄ | (run time too short) |
+| 17c     ||    765.1 |    492.7 |  -36%˄ ||     1.31 |     2.03 |  +55%˄ | (run time too short) |
+| 17d     ||    958.1 |    605.1 |  -37%˄ ||     1.04 |     1.65 |  +58%˄ |               0.1919 |
+| 17e     ||   3582.7 |   2892.1 |  -19%  ||     0.28 |     0.35 |  +24%  |               0.5429 |
+| 17f     ||   2085.4 |   1305.4 |  -37%  ||     0.48 |     0.77 |  +60%  |               0.1881 |
+| 18a     ||    462.3 |    267.0 |  -42%˄ ||     2.16 |     3.75 |  +73%˄ | (run time too short) |
+| 18b     ||   3846.6 |   2677.8 |  -30%  ||     0.26 |     0.37 |  +44%  |               0.2371 |
+| 18c     ||   4377.0 |   2964.3 |  -32%  ||     0.23 |     0.34 |  +48%  |               0.2302 |
+| 19a     ||   5486.2 |   5042.5 |   -8%  ||     0.18 |     0.20 |   +9%  |               0.3415 |
+| 19b     ||   2455.0 |   2322.6 |   -5%  ||     0.41 |     0.43 |   +6%  |               0.5013 |
+| 19c     ||   5518.0 |   5121.6 |   -7%  ||     0.18 |     0.20 |   +8%  |               0.3766 |
+| 19d     ||   4651.5 |   4219.5 |   -9%  ||     0.21 |     0.24 |  +10%  |               0.2333 |
+| 1a      ||     53.2 |     46.2 |  -13%˄ ||    18.80 |    21.66 |  +15%˄ | (run time too short) |
+| 1b      ||     61.1 |     54.8 |  -10%˄ ||    16.36 |    18.25 |  +12%˄ | (run time too short) |
+| 1c      ||     33.7 |     28.0 |  -17%˄ ||    29.68 |    35.65 |  +20%˄ | (run time too short) |
+| 1d      ||     68.2 |     61.2 |  -10%˄ ||    14.66 |    16.34 |  +12%˄ | (run time too short) |
+| 20a     ||    771.6 |    726.7 |   -6%  ||     1.30 |     1.38 |   +6%  |               0.2644 |
+| 20b     ||    844.3 |    792.6 |   -6%  ||     1.18 |     1.26 |   +7%  |               0.2248 |
 | 20c     ||    660.2 |    644.7 |   -2%  ||     1.51 |     1.55 |   +2%  |               0.6361 |
+| 21a     ||   3407.1 |   3218.7 |   -6%  ||     0.29 |     0.31 |   +6%  |               0.4959 |
+| 21b     ||    222.5 |    185.1 |  -17%˄ ||     4.49 |     5.40 |  +20%˄ | (run time too short) |
+| 21c     ||   3554.5 |   3340.4 |   -6%  ||     0.28 |     0.30 |   +6%  |               0.5012 |
+| 22a     ||    394.0 |    308.1 |  -22%˄ ||     2.54 |     3.25 |  +28%˄ | (run time too short) |
+| 22b     ||    383.7 |    294.6 |  -23%˄ ||     2.61 |     3.39 |  +30%˄ | (run time too short) |
+| 22c     ||    525.2 |    421.6 |  -20%˄ ||     1.90 |     2.37 |  +25%˄ | (run time too short) |
+| 22d     ||    581.7 |    494.7 |  -15%˄ ||     1.72 |     2.02 |  +18%˄ | (run time too short) |
+| 23a     ||    381.2 |    291.7 |  -23%˄ ||     2.62 |     3.43 |  +31%˄ | (run time too short) |
+| 23b     ||    318.3 |    257.2 |  -19%˄ ||     3.14 |     3.89 |  +24%˄ | (run time too short) |
+| 23c     ||    381.0 |    314.3 |  -17%˄ ||     2.62 |     3.18 |  +21%˄ | (run time too short) |
+| 24a     ||   6207.7 |   4753.9 |  -23%  ||     0.16 |     0.21 |  +31%  |               0.2475 |
+| 24b     ||   6368.3 |   4688.1 |  -26%  ||     0.16 |     0.21 |  +36%  |               0.2460 |
+| 25a     ||   3086.6 |   2705.7 |  -12%  ||     0.32 |     0.37 |  +14%  |               0.2410 |
+| 25b     ||   3072.9 |   2613.2 |  -15%  ||     0.33 |     0.38 |  +18%  |               0.2668 |
+| 25c     ||   3453.7 |   2983.5 |  -14%  ||     0.29 |     0.34 |  +16%  |               0.2762 |
+| 26a     ||    708.5 |    629.8 |  -11%  ||     1.41 |     1.59 |  +12%  |               0.3303 |
+| 26b     ||   4537.0 |   4125.9 |   -9%  ||     0.22 |     0.24 |  +10%  |               0.4226 |
+| 26c     ||   4788.8 |   4291.4 |  -10%  ||     0.21 |     0.23 |  +12%  |               0.3438 |
+| 27a     ||   3257.2 |   2836.9 |  -13%  ||     0.31 |     0.35 |  +15%  |               0.2834 |
+| 27b     ||   3262.5 |   2825.1 |  -13%  ||     0.31 |     0.35 |  +15%  |               0.2819 |
+| 27c     ||   3831.1 |   3338.8 |  -13%  ||     0.26 |     0.30 |  +15%  |               0.3901 |
+| 28a     ||    600.2 |    414.4 |  -31%˄ ||     1.67 |     2.41 |  +45%˄ | (run time too short) |
-| 28b     ||    514.2 |   2878.2 | +460%˄ ||     1.94 |     0.35 |  -82%˄ | (run time too short) |
+| 28c     ||    608.0 |    427.1 |  -30%˄ ||     1.64 |     2.34 |  +42%˄ | (run time too short) |
+| 29a     ||   6024.7 |   2429.8 |  -60%  ||     0.17 |     0.41 | +148%  |               0.1233 |
+| 29b     ||   2405.0 |   1033.8 |  -57%  ||     0.42 |     0.97 | +133%  |               0.1240 |
+| 29c     ||   8732.3 |   4863.5 |  -44%  ||     0.11 |     0.21 |  +80%  |                    ˅ |
+| 2a      ||     58.4 |     47.7 |  -18%˄ ||    17.13 |    20.95 |  +22%˄ | (run time too short) |
+| 2b      ||     51.9 |     40.9 |  -21%˄ ||    19.26 |    24.47 |  +27%˄ | (run time too short) |
+| 2c      ||     41.2 |     31.0 |  -25%˄ ||    24.25 |    32.20 |  +33%˄ | (run time too short) |
+| 2d      ||    126.4 |    115.8 |   -8%˄ ||     7.91 |     8.64 |   +9%˄ | (run time too short) |
+| 30a     ||   3585.3 |   2671.5 |  -25%  ||     0.28 |     0.37 |  +34%  |               0.2373 |
+| 30b     ||   3786.8 |   2833.7 |  -25%  ||     0.26 |     0.35 |  +34%  |               0.2676 |
+| 30c     ||   4043.7 |   2986.9 |  -26%  ||     0.25 |     0.33 |  +35%  |               0.2688 |
+| 31a     ||   3667.2 |   2681.2 |  -27%  ||     0.27 |     0.37 |  +37%  |               0.2671 |
+| 31b     ||   4002.0 |   2800.6 |  -30%  ||     0.25 |     0.36 |  +43%  |               0.2331 |
+| 31c     ||   4078.0 |   2899.9 |  -29%  ||     0.25 |     0.34 |  +41%  |               0.2529 |
+| 32a     ||     38.8 |     27.1 |  -30%˄ ||    25.75 |    36.89 |  +43%˄ | (run time too short) |
+| 32b     ||     68.0 |     57.7 |  -15%˄ ||    14.70 |    17.34 |  +18%˄ | (run time too short) |
+| 33a     ||    103.9 |     40.1 |  -61%˄ ||     9.63 |    24.95 | +159%˄ | (run time too short) |
+| 33b     ||     97.6 |     42.6 |  -56%˄ ||    10.24 |    23.46 | +129%˄ | (run time too short) |
+| 33c     ||    154.4 |    102.0 |  -34%˄ ||     6.48 |     9.80 |  +51%˄ | (run time too short) |
 | 3a      ||   3012.3 |   2979.3 |   -1%  ||     0.33 |     0.34 |   +1%  |               0.2873 |
+| 3b      ||     37.2 |     32.8 |  -12%˄ ||    26.86 |    30.49 |  +13%˄ | (run time too short) |
 | 3c      ||   3279.5 |   3189.3 |   -3%  ||     0.30 |     0.31 |   +3%  |               0.0172 |
+| 4a      ||     65.4 |     57.4 |  -12%˄ ||    15.28 |    17.42 |  +14%˄ | (run time too short) |
+| 4b      ||     38.7 |     31.3 |  -19%˄ ||    25.81 |    31.95 |  +24%˄ | (run time too short) |
+| 4c      ||     73.5 |     64.6 |  -12%˄ ||    13.61 |    15.47 |  +14%˄ | (run time too short) |
 | 5a      ||   3079.3 |   2973.0 |   -3%  ||     0.32 |     0.34 |   +4%  |               0.0335 |
+| 5b      ||     70.6 |     59.9 |  -15%˄ ||    14.16 |    16.69 |  +18%˄ | (run time too short) |
 | 5c      ||   3327.5 |   3216.0 |   -3%  ||     0.30 |     0.31 |   +3%  |               0.0351 |
 | 6a      ||    307.0 |    296.1 |   -4%˄ ||     3.26 |     3.38 |   +4%˄ | (run time too short) |
 | 6b      ||    332.7 |    323.5 |   -3%˄ ||     3.01 |     3.09 |   +3%˄ | (run time too short) |
 | 6c      ||    294.8 |    286.7 |   -3%˄ ||     3.39 |     3.49 |   +3%˄ | (run time too short) |
 | 6d      ||    537.8 |    524.1 |   -3%˄ ||     1.86 |     1.91 |   +3%˄ | (run time too short) |
 | 6e      ||    317.3 |    308.5 |   -3%˄ ||     3.15 |     3.24 |   +3%˄ | (run time too short) |
 | 6f      ||   1196.6 |   1155.7 |   -3%  ||     0.84 |     0.87 |   +4%  |               0.0753 |
+| 7a      ||   1073.1 |    183.5 |  -83%˄ ||     0.93 |     5.45 | +485%˄ | (run time too short) |
+| 7b      ||    667.8 |    138.6 |  -79%˄ ||     1.50 |     7.22 | +382%˄ | (run time too short) |
+| 7c      ||   2870.2 |    699.7 |  -76%  ||     0.35 |     1.43 | +310%  |               0.0553 |
+| 8a      ||    328.4 |    255.5 |  -22%˄ ||     3.05 |     3.91 |  +29%˄ | (run time too short) |
+| 8b      ||    391.6 |    309.7 |  -21%˄ ||     2.55 |     3.23 |  +26%˄ | (run time too short) |
+| 8c      ||   3009.6 |   2568.7 |  -15%  ||     0.33 |     0.39 |  +17%  |               0.2772 |
+| 8d      ||    534.2 |    426.5 |  -20%˄ ||     1.87 |     2.34 |  +25%˄ | (run time too short) |
+| 9a      ||   4151.5 |   2972.8 |  -28%  ||     0.24 |     0.34 |  +40%  |               0.2483 |
+| 9b      ||    438.9 |    264.2 |  -40%˄ ||     2.28 |     3.79 |  +66%˄ | (run time too short) |
+| 9c      ||   4009.1 |   2859.8 |  -29%  ||     0.25 |     0.35 |  +40%  |               0.2680 |
+| 9d      ||   4462.5 |   3096.8 |  -31%  ||     0.22 |     0.32 |  +44%  |               0.2168 |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum     || 211793.0 | 150789.4 |  -29%  ||          |          |        |                      |
+| Geomean ||          |          |        ||          |          |  +38%  |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 |   Notes || ˄ Execution stopped due to max runs reached                                         |
 |         || ˅ Insufficient number of runs for p-value calculation                               |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+

JOB Baseline with SemiReductionRule vs JOB K 100
 +Configuration Overview----------------------------------------------------------------+-----------------------------------------------------------------------------+
 | Parameter        | ./benchmark_hyrise_baseline_new_data.JSON | ./benchmark_as_generic_wo_semijoin_reduction_k_100.json |
 +------------------+-------------------------------------------------------------------+-----------------------------------------------------------------------------+
 |  GIT-HASH        | 3bae40c078b6e1d87b2b1190da050f3b8b30a1c0-dirty                    | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty                              |
 |  benchmark_mode  | Ordered                                                           | Ordered                                                                     |
 |  build_type      | release                                                           | release                                                                     |
 |  chunk_size      | 65535                                                             | 65535                                                                       |
 |  clients         | 1                                                                 | 1                                                                           |
 |  compiler        | gcc 9.3                                                           | gcc 9.3                                                                     |
 |  cores           | 0                                                                 | 0                                                                           |
 |  date            | 2021-07-20 16:57:32                                               | 2021-07-31 22:48:51                                                         |
 |  encoding        | {'default': {'encoding': 'Dictionary'}}                           | {'default': {'encoding': 'Dictionary'}}                                     |
 |  indexes         | False                                                             | False                                                                       |
 |  max_duration    |<PHONE_NUMBER>0                                                       |<PHONE_NUMBER>0                                                                 |
 |  max_runs        | 100                                                               | 100                                                                         |
 |  time_unit       | ns                                                                | ns                                                                          |
 |  using_scheduler | False                                                             | False                                                                       |
 |  verify          | False                                                             | False                                                                       |
 |  warmup_duration | 0                                                                 | 0                                                                           |
 +------------------+-------------------------------------------------------------------+-----------------------------------------------------------------------------+

 +---------++----------+----------+---------++----------+----------+---------+----------------------+
 | Item    || Latency (ms/iter)   |  Change || Throughput (iter/s) |  Change |              p-value |
 |         ||      old |      new |         ||      old |      new |         |                      |
 +---------++----------+----------+---------++----------+----------+---------+----------------------+
-| 10a     ||    345.5 |    511.4 |   +48%˄ ||     2.89 |     1.96 |   -32%˄ | (run time too short) |
-| 10b     ||    264.7 |    404.8 |   +53%˄ ||     3.78 |     2.47 |   -35%˄ | (run time too short) |
+| 10c     ||    592.3 |    465.8 |   -21%˄ ||     1.69 |     2.15 |   +27%˄ | (run time too short) |
-| 11a     ||    120.9 |    173.6 |   +44%˄ ||     8.27 |     5.76 |   -30%˄ | (run time too short) |
-| 11b     ||    120.1 |    169.1 |   +41%˄ ||     8.33 |     5.91 |   -29%˄ | (run time too short) |
-| 11c     ||    114.3 |    267.2 |  +134%˄ ||     8.75 |     3.74 |   -57%˄ | (run time too short) |
-| 11d     ||     56.2 |    199.5 |  +255%˄ ||    17.80 |     5.01 |   -72%˄ | (run time too short) |
-| 12a     ||     99.8 |    135.0 |   +35%˄ ||    10.02 |     7.41 |   -26%˄ | (run time too short) |
-| 12b     ||    166.1 |    230.8 |   +39%˄ ||     6.02 |     4.33 |   -28%˄ | (run time too short) |
+| 12c     ||   3103.4 |    295.6 |   -90%˄ ||     0.32 |     3.38 |  +950%˄ | (run time too short) |
+| 13a     ||    620.9 |    284.3 |   -54%˄ ||     1.61 |     3.52 |  +118%˄ | (run time too short) |
-| 13b     ||    212.3 |    257.3 |   +21%˄ ||     4.71 |     3.89 |   -17%˄ | (run time too short) |
-| 13c     ||    111.8 |    169.1 |   +51%˄ ||     8.95 |     5.92 |   -34%˄ | (run time too short) |
+| 13d     ||   1272.2 |    616.9 |   -52%  ||     0.79 |     1.62 |  +106%  |               0.0000 |
+| 14a     ||   4527.1 |    310.7 |   -93%˄ ||     0.22 |     3.22 | +1357%˄ | (run time too short) |
+| 14b     ||   4223.4 |    441.5 |   -90%˄ ||     0.24 |     2.27 |  +857%˄ | (run time too short) |
+| 14c     ||   4695.0 |    376.1 |   -92%˄ ||     0.21 |     2.66 | +1148%˄ | (run time too short) |
-| 15a     ||    164.0 |    306.1 |   +87%˄ ||     6.10 |     3.27 |   -46%˄ | (run time too short) |
-| 15b     ||    160.7 |    267.9 |   +67%˄ ||     6.22 |     3.73 |   -40%˄ | (run time too short) |
-| 15c     ||     95.0 |    282.0 |  +197%˄ ||    10.53 |     3.55 |   -66%˄ | (run time too short) |
-| 15d     ||    163.0 |    339.4 |  +108%˄ ||     6.14 |     2.95 |   -52%˄ | (run time too short) |
+| 16a     ||   4870.9 |   1726.9 |   -65%  ||     0.21 |     0.58 |  +182%  |               0.0000 |
+| 16b     ||   7196.4 |   3618.5 |   -50%  ||     0.14 |     0.28 |   +99%  |                    ˅ |
+| 16c     ||   5096.3 |   1989.9 |   -61%  ||     0.20 |     0.50 |  +156%  |               0.0000 |
+| 16d     ||   5075.6 |   1958.3 |   -61%  ||     0.20 |     0.51 |  +159%  |               0.0000 |
+| 17a     ||   1081.3 |    585.0 |   -46%˄ ||     0.92 |     1.71 |   +85%˄ | (run time too short) |
+| 17b     ||    844.3 |    523.9 |   -38%˄ ||     1.18 |     1.91 |   +61%˄ | (run time too short) |
+| 17c     ||    785.6 |    492.7 |   -37%˄ ||     1.27 |     2.03 |   +59%˄ | (run time too short) |
+| 17d     ||    925.7 |    605.1 |   -35%˄ ||     1.08 |     1.65 |   +53%˄ |               0.0000 |
+| 17e     ||   3373.0 |   2892.1 |   -14%  ||     0.30 |     0.35 |   +17%  |               0.0000 |
+| 17f     ||   1854.0 |   1305.4 |   -30%  ||     0.54 |     0.77 |   +42%  |               0.0000 |
+| 18a     ||   3216.8 |    267.0 |   -92%˄ ||     0.31 |     3.75 | +1105%˄ | (run time too short) |
-| 18b     ||    273.3 |   2677.8 |  +880%˄ ||     3.66 |     0.37 |   -90%˄ | (run time too short) |
+| 18c     ||   4276.3 |   2964.3 |   -31%  ||     0.23 |     0.34 |   +44%  |               0.0000 |
+| 19a     ||   5760.1 |   5042.5 |   -12%  ||     0.17 |     0.20 |   +14%  |               0.0000 |
+| 19b     ||   2650.3 |   2322.6 |   -12%  ||     0.38 |     0.43 |   +14%  |               0.0000 |
+| 19c     ||   5582.4 |   5121.6 |    -8%  ||     0.18 |     0.20 |    +9%  |               0.0000 |
 | 19d     ||   4272.5 |   4219.5 |    -1%  ||     0.23 |     0.24 |    +1%  |               0.0495 |
+| 1a      ||     51.0 |     46.2 |    -9%˄ ||    19.62 |    21.66 |   +10%˄ | (run time too short) |
-| 1b      ||     21.0 |     54.8 |  +161%˄ ||    47.59 |    18.25 |   -62%˄ | (run time too short) |
-| 1c      ||     24.0 |     28.0 |   +17%˄ ||    41.58 |    35.65 |   -14%˄ | (run time too short) |
-| 1d      ||     21.3 |     61.2 |  +188%˄ ||    47.03 |    16.34 |   -65%˄ | (run time too short) |
+| 20a     ||   1279.0 |    726.7 |   -43%  ||     0.78 |     1.38 |   +76%  |               0.0000 |
+| 20b     ||   1056.4 |    792.6 |   -25%  ||     0.95 |     1.26 |   +33%  |               0.0000 |
+| 20c     ||    756.6 |    644.7 |   -15%  ||     1.32 |     1.55 |   +17%  |               0.0000 |
-| 21a     ||    154.0 |   3218.7 | +1991%˄ ||     6.50 |     0.31 |   -95%˄ | (run time too short) |
-| 21b     ||    124.1 |    185.1 |   +49%˄ ||     8.06 |     5.40 |   -33%˄ | (run time too short) |
-| 21c     ||    156.1 |   3340.4 | +2040%˄ ||     6.41 |     0.30 |   -95%˄ | (run time too short) |
+| 22a     ||   3422.4 |    308.1 |   -91%˄ ||     0.29 |     3.25 | +1011%˄ | (run time too short) |
+| 22b     ||   3225.9 |    294.6 |   -91%˄ ||     0.31 |     3.39 |  +995%˄ | (run time too short) |
+| 22c     ||   5177.7 |    421.6 |   -92%˄ ||     0.19 |     2.37 | +1128%˄ | (run time too short) |
+| 22d     ||   5272.5 |    494.7 |   -91%˄ ||     0.19 |     2.02 |  +966%˄ | (run time too short) |
-| 23a     ||    103.7 |    291.7 |  +181%˄ ||     9.64 |     3.43 |   -64%˄ | (run time too short) |
-| 23b     ||    103.9 |    257.2 |  +148%˄ ||     9.63 |     3.89 |   -60%˄ | (run time too short) |
-| 23c     ||    114.2 |    314.3 |  +175%˄ ||     8.75 |     3.18 |   -64%˄ | (run time too short) |
-| 24a     ||   2718.3 |   4753.9 |   +75%  ||     0.37 |     0.21 |   -43%  |               0.0000 |
-| 24b     ||   2661.6 |   4688.1 |   +76%  ||     0.38 |     0.21 |   -43%  |               0.0000 |
-| 25a     ||    536.0 |   2705.7 |  +405%˄ ||     1.87 |     0.37 |   -80%˄ | (run time too short) |
-| 25b     ||    257.5 |   2613.2 |  +915%˄ ||     3.88 |     0.38 |   -90%˄ | (run time too short) |
+| 25c     ||   5007.1 |   2983.5 |   -40%  ||     0.20 |     0.34 |   +68%  |               0.0000 |
+| 26a     ||    682.3 |    629.8 |    -8%  ||     1.47 |     1.59 |    +8%  |               0.0000 |
-| 26b     ||    625.2 |   4125.9 |  +560%  ||     1.60 |     0.24 |   -85%  |               0.0000 |
-| 26c     ||    787.3 |   4291.4 |  +445%  ||     1.27 |     0.23 |   -82%  |               0.0000 |
+| 27a     ||   3073.8 |   2836.9 |    -8%  ||     0.33 |     0.35 |    +8%  |               0.0000 |
+| 27b     ||   3070.0 |   2825.1 |    -8%  ||     0.33 |     0.35 |    +9%  |               0.0000 |
-| 27c     ||    161.1 |   3338.8 | +1973%˄ ||     6.21 |     0.30 |   -95%˄ | (run time too short) |
+| 28a     ||   4429.1 |    414.4 |   -91%˄ ||     0.23 |     2.41 |  +969%˄ | (run time too short) |
+| 28b     ||   3097.2 |   2878.2 |    -7%  ||     0.32 |     0.35 |    +8%  |               0.0173 |
+| 28c     ||   4568.2 |    427.1 |   -91%˄ ||     0.22 |     2.34 |  +969%˄ | (run time too short) |
+| 29a     ||   2807.2 |   2429.8 |   -13%  ||     0.36 |     0.41 |   +16%  |               0.0875 |
-| 29b     ||    185.4 |   1033.8 |  +458%˄ ||     5.39 |     0.97 |   -82%˄ | (run time too short) |
-| 29c     ||   2832.7 |   4863.5 |   +72%  ||     0.35 |     0.21 |   -42%  |               0.0000 |
+| 2a      ||     90.1 |     47.7 |   -47%˄ ||    11.10 |    20.95 |   +89%˄ | (run time too short) |
+| 2b      ||     74.8 |     40.9 |   -45%˄ ||    13.37 |    24.47 |   +83%˄ | (run time too short) |
+| 2c      ||     60.2 |     31.0 |   -48%˄ ||    16.62 |    32.20 |   +94%˄ | (run time too short) |
+| 2d      ||    121.6 |    115.8 |    -5%˄ ||     8.22 |     8.64 |    +5%˄ | (run time too short) |
-| 30a     ||    319.9 |   2671.5 |  +735%˄ ||     3.13 |     0.37 |   -88%˄ | (run time too short) |
-| 30b     ||    416.0 |   2833.7 |  +581%˄ ||     2.40 |     0.35 |   -85%˄ | (run time too short) |
+| 30c     ||   3927.8 |   2986.9 |   -24%  ||     0.25 |     0.33 |   +31%  |               0.0000 |
-| 31a     ||    328.7 |   2681.2 |  +716%˄ ||     3.04 |     0.37 |   -88%˄ | (run time too short) |
-| 31b     ||    449.1 |   2800.6 |  +524%˄ ||     2.23 |     0.36 |   -84%˄ | (run time too short) |
-| 31c     ||    375.2 |   2899.9 |  +673%˄ ||     2.67 |     0.34 |   -87%˄ | (run time too short) |
+| 32a     ||     32.2 |     27.1 |   -16%˄ ||    31.04 |    36.89 |   +19%˄ | (run time too short) |
+| 32b     ||     70.2 |     57.7 |   -18%˄ ||    14.25 |    17.34 |   +22%˄ | (run time too short) |
+| 33a     ||     49.0 |     40.1 |   -18%˄ ||    20.40 |    24.95 |   +22%˄ | (run time too short) |
+| 33b     ||     47.8 |     42.6 |   -11%˄ ||    20.94 |    23.46 |   +12%˄ | (run time too short) |
-| 33c     ||     57.4 |    102.0 |   +78%˄ ||    17.43 |     9.80 |   -44%˄ | (run time too short) |
+| 3a      ||   3524.0 |   2979.3 |   -15%  ||     0.28 |     0.34 |   +18%  |               0.0000 |
-| 3b      ||     30.6 |     32.8 |    +7%˄ ||    32.69 |    30.49 |    -7%˄ | (run time too short) |
+| 3c      ||   4503.7 |   3189.3 |   -29%  ||     0.22 |     0.31 |   +41%  |               0.0000 |
+| 4a      ||    465.7 |     57.4 |   -88%˄ ||     2.15 |    17.42 |  +711%˄ | (run time too short) |
-| 4b      ||     28.1 |     31.3 |   +12%˄ ||    35.62 |    31.95 |   -10%˄ | (run time too short) |
+| 4c      ||    529.5 |     64.6 |   -88%˄ ||     1.89 |    15.47 |  +719%˄ | (run time too short) |
+| 5a      ||   3384.1 |   2973.0 |   -12%  ||     0.30 |     0.34 |   +14%  |               0.0000 |
+| 5b      ||    226.6 |     59.9 |   -74%˄ ||     4.41 |    16.69 |  +278%˄ | (run time too short) |
+| 5c      ||   3927.4 |   3216.0 |   -18%  ||     0.25 |     0.31 |   +22%  |               0.0000 |
-| 6a      ||    236.8 |    296.1 |   +25%˄ ||     4.22 |     3.38 |   -20%˄ | (run time too short) |
-| 6b      ||    266.6 |    323.5 |   +21%˄ ||     3.75 |     3.09 |   -18%˄ | (run time too short) |
-| 6c      ||    226.6 |    286.7 |   +27%˄ ||     4.41 |     3.49 |   -21%˄ | (run time too short) |
+| 6d      ||    941.6 |    524.1 |   -44%˄ ||     1.06 |     1.91 |   +80%˄ | (run time too short) |
-| 6e      ||    236.8 |    308.5 |   +30%˄ ||     4.22 |     3.24 |   -23%˄ | (run time too short) |
+| 6f      ||   1381.6 |   1155.7 |   -16%  ||     0.72 |     0.87 |   +20%  |               0.0000 |
-| 7a      ||    159.1 |    183.5 |   +15%˄ ||     6.29 |     5.45 |   -13%˄ | (run time too short) |
-| 7b      ||    114.6 |    138.6 |   +21%˄ ||     8.73 |     7.22 |   -17%˄ | (run time too short) |
+| 7c      ||   1229.5 |    699.7 |   -43%  ||     0.81 |     1.43 |   +76%  |               0.0000 |
-| 8a      ||     90.6 |    255.5 |  +182%˄ ||    11.04 |     3.91 |   -65%˄ | (run time too short) |
-| 8b      ||    159.2 |    309.7 |   +95%˄ ||     6.28 |     3.23 |   -49%˄ | (run time too short) |
+| 8c      ||   2836.9 |   2568.7 |    -9%  ||     0.35 |     0.39 |   +10%  |               0.0000 |
+| 8d      ||    791.2 |    426.5 |   -46%˄ ||     1.26 |     2.34 |   +86%˄ | (run time too short) |
+| 9a      ||   3387.4 |   2972.8 |   -12%  ||     0.30 |     0.34 |   +14%  |               0.0000 |
-| 9b      ||    226.4 |    264.2 |   +17%˄ ||     4.42 |     3.79 |   -14%˄ | (run time too short) |
+| 9c      ||   3375.8 |   2859.8 |   -15%  ||     0.30 |     0.35 |   +18%  |               0.0000 |
+| 9d      ||   3773.2 |   3096.8 |   -18%  ||     0.27 |     0.32 |   +22%  |               0.0000 |
 +---------++----------+----------+---------++----------+----------+---------+----------------------+
+| Sum     || 179400.4 | 150789.4 |   -16%  ||          |          |         |                      |
 | Geomean ||          |          |         ||          |          |    -0%  |                      |
 +---------++----------+----------+---------++----------+----------+---------+----------------------+
 |   Notes || ˄ Execution stopped due to max runs reached                                           |
 |         || ˅ Insufficient number of runs for p-value calculation                                 |
 +---------++----------+----------+---------++----------+----------+---------+----------------------+

JOB Baseline w/o SemiJoinReductionRule vs JOB K 100
 +Configuration Overview-------------------------------------------------------------------------------+---------------------------------------------------------------------------+
 | Parameter        | Theresa.Hradilak/hyrise/benchmark_hyrise_baseline_w_o_semijoinreductionrule.json | Marc.Rosenau/hyrise/benchmark_as_generic_wo_semijoin_reduction_k_100.json |
 +------------------+----------------------------------------------------------------------------------+---------------------------------------------------------------------------+
 |  GIT-HASH        | d50c5db23f1f53dfa5745706ea95cbc3b4f7a0a3-dirty                                   | 1c8841abff0f79a3398662ddf7bc514d46b92f3f-dirty                            |
 |  benchmark_mode  | Ordered                                                                          | Ordered                                                                   |
 |  build_type      | release                                                                          | release                                                                   |
 |  chunk_size      | 65535                                                                            | 65535                                                                     |
 |  clients         | 1                                                                                | 1                                                                         |
 |  compiler        | gcc 9.3                                                                          | gcc 9.3                                                                   |
 |  cores           | 0                                                                                | 0                                                                         |
 |  date            | 2021-08-01 10:29:52                                                              | 2021-07-31 22:48:51                                                       |
 |  encoding        | {'default': {'encoding': 'Dictionary'}}                                          | {'default': {'encoding': 'Dictionary'}}                                   |
 |  indexes         | False                                                                            | False                                                                     |
 |  max_duration    |<PHONE_NUMBER>0                                                                      |<PHONE_NUMBER>0                                                               |
 |  max_runs        | 100                                                                              | 100                                                                       |
 |  time_unit       | ns                                                                               | ns                                                                        |
 |  using_scheduler | False                                                                            | False                                                                     |
 |  verify          | False                                                                            | False                                                                     |
 |  warmup_duration | 0                                                                                | 0                                                                         |
 +------------------+----------------------------------------------------------------------------------+---------------------------------------------------------------------------+

 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 | Item    || Latency (ms/iter)   | Change || Throughput (iter/s) | Change |              p-value |
 |         ||      old |      new |        ||      old |      new |        |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
-| 10a     ||    453.4 |    511.4 |  +13%˄ ||     2.21 |     1.96 |  -11%˄ | (run time too short) |
-| 10b     ||    367.9 |    404.8 |  +10%˄ ||     2.72 |     2.47 |   -9%˄ | (run time too short) |
+| 10c     ||    553.3 |    465.8 |  -16%˄ ||     1.81 |     2.15 |  +19%˄ | (run time too short) |
 | 11a     ||    166.0 |    173.6 |   +5%˄ ||     6.02 |     5.76 |   -4%˄ | (run time too short) |
-| 11b     ||    159.0 |    169.1 |   +6%˄ ||     6.29 |     5.91 |   -6%˄ | (run time too short) |
-| 11c     ||    251.7 |    267.2 |   +6%˄ ||     3.97 |     3.74 |   -6%˄ | (run time too short) |
+| 11d     ||    216.6 |    199.5 |   -8%˄ ||     4.62 |     5.01 |   +9%˄ | (run time too short) |
-| 12a     ||     91.0 |    135.0 |  +48%˄ ||    10.99 |     7.41 |  -33%˄ | (run time too short) |
+| 12b     ||    297.6 |    230.8 |  -22%˄ ||     3.36 |     4.33 |  +29%˄ | (run time too short) |
+| 12c     ||   2857.9 |    295.6 |  -90%˄ ||     0.35 |     3.38 | +867%˄ | (run time too short) |
 | 13a     ||    294.7 |    284.3 |   -4%˄ ||     3.39 |     3.52 |   +4%˄ | (run time too short) |
+| 13b     ||    345.8 |    257.3 |  -26%˄ ||     2.89 |     3.89 |  +34%˄ | (run time too short) |
+| 13c     ||    254.0 |    169.1 |  -33%˄ ||     3.94 |     5.92 |  +50%˄ | (run time too short) |
+| 13d     ||    645.8 |    616.9 |   -4%  ||     1.55 |     1.62 |   +5%  |               0.0000 |
+| 14a     ||   3295.2 |    310.7 |  -91%˄ ||     0.30 |     3.22 | +961%˄ | (run time too short) |
+| 14b     ||   3524.0 |    441.5 |  -87%˄ ||     0.28 |     2.27 | +698%˄ | (run time too short) |
+| 14c     ||   3358.7 |    376.1 |  -89%˄ ||     0.30 |     2.66 | +793%˄ | (run time too short) |
+| 15a     ||    353.1 |    306.1 |  -13%˄ ||     2.83 |     3.27 |  +15%˄ | (run time too short) |
+| 15b     ||    296.7 |    267.9 |  -10%˄ ||     3.37 |     3.73 |  +11%˄ | (run time too short) |
+| 15c     ||    323.0 |    282.0 |  -13%˄ ||     3.10 |     3.55 |  +15%˄ | (run time too short) |
+| 15d     ||    377.1 |    339.4 |  -10%˄ ||     2.65 |     2.95 |  +11%˄ | (run time too short) |
+| 16a     ||   2096.0 |   1726.9 |  -18%  ||     0.48 |     0.58 |  +21%  |               0.0000 |
+| 16b     ||   4307.3 |   3618.5 |  -16%  ||     0.23 |     0.28 |  +19%  |               0.0000 |
+| 16c     ||   2422.6 |   1989.9 |  -18%  ||     0.41 |     0.50 |  +22%  |               0.0000 |
+| 16d     ||   2467.7 |   1958.3 |  -21%  ||     0.41 |     0.51 |  +26%  |               0.0000 |
+| 17a     ||    622.2 |    585.0 |   -6%˄ ||     1.61 |     1.71 |   +6%˄ | (run time too short) |
 | 17b     ||    534.4 |    523.9 |   -2%˄ ||     1.87 |     1.91 |   +2%˄ | (run time too short) |
+| 17c     ||    537.6 |    492.7 |   -8%˄ ||     1.86 |     2.03 |   +9%˄ | (run time too short) |
+| 17d     ||    667.6 |    605.1 |   -9%˄ ||     1.50 |     1.65 |  +10%˄ |               0.0000 |
+| 17e     ||   3144.7 |   2892.1 |   -8%  ||     0.32 |     0.35 |   +9%  |               0.0000 |
+| 17f     ||   1429.0 |   1305.4 |   -9%  ||     0.70 |     0.77 |   +9%  |               0.0000 |
-| 18a     ||    211.2 |    267.0 |  +26%˄ ||     4.73 |     3.75 |  -21%˄ | (run time too short) |
 | 18b     ||   2780.4 |   2677.8 |   -4%  ||     0.36 |     0.37 |   +4%  |               0.0006 |
+| 18c     ||   5738.7 |   2964.3 |  -48%  ||     0.17 |     0.34 |  +94%  |               0.0000 |
+| 19a     ||   5700.3 |   5042.5 |  -12%  ||     0.18 |     0.20 |  +13%  |               0.0000 |
+| 19b     ||   2996.3 |   2322.6 |  -22%  ||     0.33 |     0.43 |  +29%  |               0.0000 |
+| 19c     ||   5568.2 |   5121.6 |   -8%  ||     0.18 |     0.20 |   +9%  |               0.0000 |
-| 19d     ||   3777.6 |   4219.5 |  +12%  ||     0.26 |     0.24 |  -10%  |               0.0000 |
+| 1a      ||     69.8 |     46.2 |  -34%˄ ||    14.33 |    21.66 |  +51%˄ | (run time too short) |
 | 1b      ||     55.7 |     54.8 |   -2%˄ ||    17.94 |    18.25 |   +2%˄ | (run time too short) |
+| 1c      ||     36.6 |     28.0 |  -23%˄ ||    27.30 |    35.65 |  +31%˄ | (run time too short) |
 | 1d      ||     62.9 |     61.2 |   -3%˄ ||    15.89 |    16.34 |   +3%˄ | (run time too short) |
+| 20a     ||    830.8 |    726.7 |  -13%  ||     1.20 |     1.38 |  +14%  |               0.0000 |
+| 20b     ||    908.1 |    792.6 |  -13%  ||     1.10 |     1.26 |  +15%  |               0.0000 |
+| 20c     ||    718.6 |    644.7 |  -10%  ||     1.39 |     1.55 |  +11%  |               0.0000 |
 | 21a     ||   3143.7 |   3218.7 |   +2%  ||     0.32 |     0.31 |   -2%  |               0.0696 |
-| 21b     ||    174.2 |    185.1 |   +6%˄ ||     5.74 |     5.40 |   -6%˄ | (run time too short) |
 | 21c     ||   3216.7 |   3340.4 |   +4%  ||     0.31 |     0.30 |   -4%  |               0.0000 |
+| 22a     ||   2887.4 |    308.1 |  -89%˄ ||     0.35 |     3.25 | +837%˄ | (run time too short) |
+| 22b     ||   2799.0 |    294.6 |  -89%˄ ||     0.36 |     3.39 | +850%˄ | (run time too short) |
+| 22c     ||   3429.4 |    421.6 |  -88%˄ ||     0.29 |     2.37 | +713%˄ | (run time too short) |
+| 22d     ||   3541.3 |    494.7 |  -86%˄ ||     0.28 |     2.02 | +616%˄ | (run time too short) |
+| 23a     ||    315.3 |    291.7 |   -7%˄ ||     3.17 |     3.43 |   +8%˄ | (run time too short) |
 | 23b     ||    251.3 |    257.2 |   +2%˄ ||     3.98 |     3.89 |   -2%˄ | (run time too short) |
 | 23c     ||    319.7 |    314.3 |   -2%˄ ||     3.13 |     3.18 |   +2%˄ | (run time too short) |
+| 24a     ||   5400.6 |   4753.9 |  -12%  ||     0.19 |     0.21 |  +14%  |               0.0000 |
+| 24b     ||   5464.7 |   4688.1 |  -14%  ||     0.18 |     0.21 |  +17%  |               0.0000 |
 | 25a     ||   2731.3 |   2705.7 |   -1%  ||     0.37 |     0.37 |   +1%  |               0.0440 |
 | 25b     ||   2700.1 |   2613.2 |   -3%  ||     0.37 |     0.38 |   +3%  |               0.0000 |
+| 25c     ||   5874.5 |   2983.5 |  -49%  ||     0.17 |     0.34 |  +97%  |               0.0000 |
+| 26a     ||    704.0 |    629.8 |  -11%  ||     1.42 |     1.59 |  +12%  |               0.0000 |
+| 26b     ||   4348.3 |   4125.9 |   -5%  ||     0.23 |     0.24 |   +5%  |               0.0127 |
 | 26c     ||   4410.7 |   4291.4 |   -3%  ||     0.23 |     0.23 |   +3%  |               0.0487 |
 | 27a     ||   2764.8 |   2836.9 |   +3%  ||     0.36 |     0.35 |   -3%  |               0.0022 |
 | 27b     ||   2748.0 |   2825.1 |   +3%  ||     0.36 |     0.35 |   -3%  |               0.0002 |
 | 27c     ||   3220.4 |   3338.8 |   +4%  ||     0.31 |     0.30 |   -4%  |               0.0015 |
+| 28a     ||   3478.8 |    414.4 |  -88%˄ ||     0.29 |     2.41 | +740%˄ | (run time too short) |
 | 28b     ||   2884.5 |   2878.2 |   -0%  ||     0.35 |     0.35 |   +0%  |               0.8674 |
+| 28c     ||   3568.5 |    427.1 |  -88%˄ ||     0.28 |     2.34 | +735%˄ | (run time too short) |
+| 29a     ||   2961.6 |   2429.8 |  -18%  ||     0.34 |     0.41 |  +22%  |               0.0000 |
+| 29b     ||   1167.5 |   1033.8 |  -11%  ||     0.86 |     0.97 |  +13%  |               0.0000 |
+| 29c     ||   5527.4 |   4863.5 |  -12%  ||     0.18 |     0.21 |  +14%  |               0.0000 |
 | 2a      ||     47.5 |     47.7 |   +0%˄ ||    21.03 |    20.95 |   -0%˄ | (run time too short) |
 | 2b      ||     40.9 |     40.9 |   -0%˄ ||    24.46 |    24.47 |   +0%˄ | (run time too short) |
 | 2c      ||     30.7 |     31.0 |   +1%˄ ||    32.59 |    32.20 |   -1%˄ | (run time too short) |
+| 2d      ||    124.6 |    115.8 |   -7%˄ ||     8.03 |     8.64 |   +8%˄ | (run time too short) |
 | 30a     ||   2780.0 |   2671.5 |   -4%  ||     0.36 |     0.37 |   +4%  |               0.0000 |
 | 30b     ||   2960.9 |   2833.7 |   -4%  ||     0.34 |     0.35 |   +4%  |               0.0000 |
+| 30c     ||   5565.0 |   2986.9 |  -46%  ||     0.18 |     0.33 |  +86%  |               0.0000 |
 | 31a     ||   2739.8 |   2681.2 |   -2%  ||     0.36 |     0.37 |   +2%  |               0.0424 |
+| 31b     ||   2964.6 |   2800.6 |   -6%  ||     0.34 |     0.36 |   +6%  |               0.0000 |
+| 31c     ||   5671.4 |   2899.9 |  -49%  ||     0.18 |     0.34 |  +96%  |               0.0000 |
+| 32a     ||     29.0 |     27.1 |   -7%˄ ||    34.46 |    36.89 |   +7%˄ | (run time too short) |
 | 32b     ||     58.2 |     57.7 |   -1%˄ ||    17.17 |    17.34 |   +1%˄ | (run time too short) |
 | 33a     ||     40.2 |     40.1 |   -0%˄ ||    24.87 |    24.95 |   +0%˄ | (run time too short) |
 | 33b     ||     42.3 |     42.6 |   +1%˄ ||    23.62 |    23.46 |   -1%˄ | (run time too short) |
 | 33c     ||    103.1 |    102.0 |   -1%˄ ||     9.70 |     9.80 |   +1%˄ | (run time too short) |
 | 3a      ||   3020.2 |   2979.3 |   -1%  ||     0.33 |     0.34 |   +1%  |               0.0058 |
+| 3b      ||     36.1 |     32.8 |   -9%˄ ||    27.73 |    30.49 |  +10%˄ | (run time too short) |
 | 3c      ||   3199.8 |   3189.3 |   -0%  ||     0.31 |     0.31 |   +0%  |               0.4980 |
+| 4a      ||     61.2 |     57.4 |   -6%˄ ||    16.33 |    17.42 |   +7%˄ | (run time too short) |
+| 4b      ||     34.6 |     31.3 |  -10%˄ ||    28.86 |    31.95 |  +11%˄ | (run time too short) |
 | 4c      ||     66.6 |     64.6 |   -3%˄ ||    15.01 |    15.47 |   +3%˄ | (run time too short) |
 | 5a      ||   2966.6 |   2973.0 |   +0%  ||     0.34 |     0.34 |   -0%  |               0.6401 |
+| 5b      ||     74.7 |     59.9 |  -20%˄ ||    13.39 |    16.69 |  +25%˄ | (run time too short) |
 | 5c      ||   3163.0 |   3216.0 |   +2%  ||     0.32 |     0.31 |   -2%  |               0.0002 |
+| 6a      ||    313.9 |    296.1 |   -6%˄ ||     3.19 |     3.38 |   +6%˄ | (run time too short) |
+| 6b      ||    342.7 |    323.5 |   -6%˄ ||     2.92 |     3.09 |   +6%˄ | (run time too short) |
+| 6c      ||    301.5 |    286.7 |   -5%˄ ||     3.32 |     3.49 |   +5%˄ | (run time too short) |
 | 6d      ||    535.5 |    524.1 |   -2%˄ ||     1.87 |     1.91 |   +2%˄ | (run time too short) |
+| 6e      ||    324.1 |    308.5 |   -5%˄ ||     3.09 |     3.24 |   +5%˄ | (run time too short) |
 | 6f      ||   1177.7 |   1155.7 |   -2%  ||     0.85 |     0.87 |   +2%  |               0.0031 |
 | 7a      ||    187.7 |    183.5 |   -2%˄ ||     5.33 |     5.45 |   +2%˄ | (run time too short) |
 | 7b      ||    142.7 |    138.6 |   -3%˄ ||     7.01 |     7.22 |   +3%˄ | (run time too short) |
+| 7c      ||    884.7 |    699.7 |  -21%  ||     1.13 |     1.43 |  +26%  |               0.0000 |
+| 8a      ||    270.3 |    255.5 |   -5%˄ ||     3.70 |     3.91 |   +6%˄ | (run time too short) |
+| 8b      ||    337.9 |    309.7 |   -8%˄ ||     2.96 |     3.23 |   +9%˄ | (run time too short) |
-| 8c      ||   2336.9 |   2568.7 |  +10%  ||     0.43 |     0.39 |   -9%  |               0.0000 |
+| 8d      ||    659.1 |    426.5 |  -35%˄ ||     1.52 |     2.34 |  +55%˄ | (run time too short) |
 | 9a      ||   3033.8 |   2972.8 |   -2%  ||     0.33 |     0.34 |   +2%  |               0.0719 |
 | 9b      ||    274.3 |    264.2 |   -4%˄ ||     3.65 |     3.79 |   +4%˄ | (run time too short) |
 | 9c      ||   2984.4 |   2859.8 |   -4%  ||     0.34 |     0.35 |   +4%  |               0.0006 |
+| 9d      ||   3278.2 |   3096.8 |   -6%  ||     0.31 |     0.32 |   +6%  |               0.0000 |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum     || 199406.7 | 150789.4 |  -24%  ||          |          |        |                      |
+| Geomean ||          |          |        ||          |          |  +31%  |                      |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+
 |   Notes || ˄ Execution stopped due to max runs reached                                         |
 +---------++----------+----------+--------++----------+----------+--------+----------------------+

Comparing the K=100 benchmark against the best baseline benchmark, we see a latency improvement of 16%. If the SemiJoinReductionRule is also commented out, like in out implementation, we even see an improvement of 24%. Due to the fact, that the SemiJoinReductionRule is going to be reworked, it is fair to run both benchmarks without the SemiJoinReductionRule.
(2) Selectivity Calculation Bug Fix
We also added a small bugfix that caused the selectivity calculation of an operator on a table to go wrong:
To estimate the cardinality of a result table after a Scan Predicate Operator comparing a Column vs. a Value was applied, the following steps happen:

If the predicate_condition is not one of {IS (NOT) NULL, (NOT) LIKE, BETWEEN ... AND .. } (these are handled in a separate case), the previously calculated histogram of values in the Column is sliced according to the predicate. The sliced histogram then only holds the values of the column that fulfill the predicate.

As a next step, the selectivity of the predicate on the column is calculated as:

751  selectivity = sliced_histogram->total_count() / scan_statistics_object->total_count();

Here scan_statistics_object holds the previous, not-sliced histogram. This means that the selectivity is calculated based on the total_count of values in the histogram.

Finally, the other columns AttributeStatistics are scaled according to the calculated selectivity, and the new row_count estimation for the result table is calculated as:

774 const auto row_count = Cardinality{input_table_statistics->row_count * selectivity};

So the selectivity is applied on the row_count of the input_table.
The estimation error is caused by the fact that histograms do not hold NULL values. Therefore the row_count of the input_table_statistics object can be different from the total_count of the histograms for the table's columns.
As the selectivity is applied on the input_table, it should be calculated based on the row_count of the input_table and not on the differing values count in the histogram.
In estimations for other Operator Types (e.g. inner_equi_join and semi_join), the selectivity is actually already being calculated like this.
912 const auto left_selectivity = Selectivity{
        left_input_table_statistics.row_count > 0 ? cardinality / left_input_table_statistics.row_count : 0.0f};

We introduced the same selectivity calculation in the calculation of Scan Predicate Operator comparing a Column vs. a Value estimation and updated the tests accordingly.
(3) Cardinality Estimation Tool
To further improve the future analysis of cardinality estimations, we implemented a tool to calculate the differences estimations and actual cardinalities per operation in a query plan.
For this we implemented:

a cardinality_writer that hooks into the query plan visualization workflow and produces a benchmark_cardinality_estimation.csv with cardinality information for each operation in each item of a benchmark.  As the evaluation depends on the assumption that lqp and pqp will usually have the same order of nodes, additional debug information is printed to identify and manually correct cases where this is not the case.
a shell script to run all benchmarks with a cardinality estimation analysis and store the created files in a fitting folder structure for later analysis

Usage:

Printing cardinality estimation is currently automatically enabled when setting the --visualize flag on a benchmark run. (The tool is intended to be stored on an extra branch and accessed when needed.) It produces the following output:

A benchmark_cardinality_estimation.csv with the following structure:
Benchmark Item ID ; LQP Node ; PQP Node ; LQP Row Count ; PQP Row Count
(The csv uses semicolons as seperators to be able to work with database values including commas.)

        The csv can then be used for further analysis e.g. analysing the distribution of differences between estimation and actual cardinality.

a benchmark_cardinality_estimation_debug_info.csv with the following structure:
Benchmark Item ID ; Number of LQP Nodes; Number of PQP Nodes

The cardinality_estimations_all.sh shell script can be run from the root folder via:
./scripts/cardinality_estimations_all.sh  ./path/to/build_folder ./path/to/cardinality_information_folder
It runs the following benchmarks:
The produced folder output structure is:

├── <cardinality_information_folder>
│   ├──  hyriseBenchmarkJoinOrder                    # Test files (alternatively `spec` or `tests`)
│   │   ├── hyriseBenchmarkJoinOrder_benchmark_cardinality_estimation.csv
│   │   ├── hyriseBenchmarkJoinOrder_benchmark_cardinality_estimation_debug_info.csv
│   │   └── visualized_plans
│   ├── hyriseBenchmarkTPCC
...

Now contained in #7