perf(common): Optimize null_bitmap

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
This Pr uses Set64 to optimize null bitmap to avoid overhead of vector.
Breaking change: Hard limit of 64 group keys now.
#8802
Checklist For Contributors

[x] I have written necessary rustdoc comments
[x] I have added necessary unit tests and integration tests
[x] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[x] I have demonstrated that backward compatibility is not broken by breaking changes and created issues to track deprecated features to be removed in the future. (Please refer to the issue)
[x] All checks passed in ./risedev check (or alias, ./risedev c)

Checklist For Reviewers

[ ] I have requested macro/micro-benchmarks as this PR can affect performance substantially, and the results are shown.

Documentation

[ ] My PR DOES NOT contain user-facing changes.

Click here for Documentation
Types of user-facing changes
Please keep the types that apply to your changes, and remove the others.

Installation and deployment
Connector (sources & sinks)
SQL commands, functions, and operators
RisingWave cluster configuration changes
Other (please specify in the release note below)

Release note

TLDR:

SerializedKey not much improvement.
FixedKey has improvement.
Mixed has regression (confuses me).

TODO:

Add more benchmarks with > 1024 keys, maybe like 10,000 or even 100,000.

Benchmark results:
vec ser Serial Key64, 128 rows, Pr[null]=0
                        time:   [1.5095 µs 1.5129 µs 1.5164 µs]
                        change: [-68.458% -68.372% -68.279%] (p = 0.00 < 0.05)
                        Performance has improved.

vec ser Serial Key64, 1024 rows, Pr[null]=0
                        time:   [10.143 µs 10.149 µs 10.156 µs]
                        change: [-73.913% -73.764% -73.605%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe

vec ser Serial Key64, 128 rows, Pr[null]=0.01
                        time:   [1.5059 µs 1.5079 µs 1.5099 µs]
                        change: [-68.315% -68.225% -68.140%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

vec ser Serial Key64, 1024 rows, Pr[null]=0.01
                        time:   [10.549 µs 10.585 µs 10.618 µs]
                        change: [-73.869% -73.721% -73.569%] (p = 0.00 < 0.05)
                        Performance has improved.

vec ser Serial Key64, 128 rows, Pr[null]=0.1
                        time:   [1.5218 µs 1.5278 µs 1.5341 µs]
                        change: [-67.935% -67.761% -67.591%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

vec ser Serial Key64, 1024 rows, Pr[null]=0.1
                        time:   [10.197 µs 10.679 µs 11.677 µs]
                        change: [-74.272% -73.342% -71.458%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe

vec ser int32 Key32, 128 rows, Pr[null]=0
                        time:   [1.0794 µs 1.1523 µs 1.3108 µs]
                        change: [-75.150% -74.488% -73.237%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe

vec ser int32 Key32, 1024 rows, Pr[null]=0
                        time:   [7.1355 µs 7.5216 µs 8.3347 µs]
                        change: [-80.444% -79.268% -77.059%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 17 outliers among 100 measurements (17.00%)
  7 (7.00%) high mild
  10 (10.00%) high severe

vec ser int32 Key32, 128 rows, Pr[null]=0.01
                        time:   [1.0656 µs 1.1357 µs 1.2875 µs]
                        change: [-75.121% -74.336% -72.847%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

vec ser int32 Key32, 1024 rows, Pr[null]=0.01
                        time:   [7.2263 µs 7.3904 µs 7.7369 µs]
                        change: [-79.289% -77.887% -74.470%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
  7 (7.00%) high mild
  8 (8.00%) high severe

vec ser int32 Key32, 128 rows, Pr[null]=0.1
                        time:   [1.1629 µs 1.2205 µs 1.3475 µs]
                        change: [-73.331% -72.395% -69.887%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

vec ser int32 Key32, 1024 rows, Pr[null]=0.1
                        time:   [7.8519 µs 7.8703 µs 7.8919 µs]
                        change: [-78.073% -77.889% -77.695%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe

vec ser int64 Key64, 128 rows, Pr[null]=0
                        time:   [1.5132 µs 1.6316 µs 1.8956 µs]
                        change: [-67.822% -66.449% -63.811%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low mild
  5 (5.00%) high mild
  4 (4.00%) high severe

vec ser int64 Key64, 1024 rows, Pr[null]=0
                        time:   [10.165 µs 10.861 µs 12.272 µs]
                        change: [-74.651% -74.025% -72.796%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe

vec ser int64 Key64, 128 rows, Pr[null]=0.01
                        time:   [1.5226 µs 1.6057 µs 1.7805 µs]
                        change: [-67.516% -66.117% -63.397%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe

vec ser int64 Key64, 1024 rows, Pr[null]=0.01
                        time:   [10.175 µs 10.911 µs 12.423 µs]
                        change: [-74.694% -73.797% -72.163%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  3 (3.00%) high mild
  13 (13.00%) high severe

vec ser int64 Key64, 128 rows, Pr[null]=0.1
                        time:   [1.5080 µs 1.5788 µs 1.7342 µs]
                        change: [-67.732% -65.433% -61.183%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe

vec ser int64 Key64, 1024 rows, Pr[null]=0.1
                        time:   [10.179 µs 10.806 µs 12.089 µs]
                        change: [-73.592% -72.385% -69.781%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  3 (3.00%) high mild
  11 (11.00%) high severe

vec ser varchar KeySerialized, 128 rows, Pr[null]=0
                        time:   [21.693 µs 23.383 µs 26.867 µs]
                        change: [-2.9484% +0.6584% +7.1774%] (p = 0.85 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

vec ser varchar KeySerialized, 1024 rows, Pr[null]=0
                        time:   [176.53 µs 193.08 µs 226.66 µs]
                        change: [-0.0791% +3.8552% +10.617%] (p = 0.26 > 0.05)
                        No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

vec ser varchar KeySerialized, 128 rows, Pr[null]=0.01
                        time:   [21.612 µs 21.656 µs 21.711 µs]
                        change: [-2.9821% -2.7857% -2.5892%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

vec ser varchar KeySerialized, 1024 rows, Pr[null]=0.01
                        time:   [160.32 µs 160.85 µs 161.48 µs]
                        change: [+0.9400% +1.2527% +1.6057%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  2 (2.00%) low mild
  8 (8.00%) high mild
  4 (4.00%) high severe

vec ser varchar KeySerialized, 128 rows, Pr[null]=0.1
                        time:   [16.784 µs 16.807 µs 16.834 µs]
                        change: [-5.1299% -4.8386% -4.5296%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

vec ser varchar KeySerialized, 1024 rows, Pr[null]=0.1
                        time:   [137.93 µs 138.82 µs 140.02 µs]
                        change: [-3.9034% -3.3056% -2.6082%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

vec ser composite fixed Key128, 128 rows, Pr[null]=0
                        time:   [2.4326 µs 2.4364 µs 2.4399 µs]
                        change: [-58.744% -58.431% -58.194%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

vec ser composite fixed Key128, 1024 rows, Pr[null]=0
                        time:   [17.209 µs 17.222 µs 17.240 µs]
                        change: [-62.932% -62.528% -62.135%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

vec ser composite fixed Key128, 128 rows, Pr[null]=0.01
                        time:   [2.4271 µs 2.4357 µs 2.4455 µs]
                        change: [-57.473% -57.338% -57.202%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

vec ser composite fixed Key128, 1024 rows, Pr[null]=0.01
                        time:   [17.696 µs 17.792 µs 17.888 µs]
                        change: [-61.900% -61.499% -61.120%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe

vec ser composite fixed Key128, 128 rows, Pr[null]=0.1
                        time:   [2.7466 µs 2.7579 µs 2.7701 µs]
                        change: [-52.663% -52.466% -52.270%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

vec ser composite fixed Key128, 1024 rows, Pr[null]=0.1
                        time:   [19.931 µs 19.995 µs 20.062 µs]
                        change: [-56.833% -56.698% -56.542%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

vec ser composite fixed Key128, 128 rows, Pr[null]=0 #2
                        time:   [2.8710 µs 2.8751 µs 2.8802 µs]
                        change: [-52.801% -52.610% -52.434%] (p = 0.00 < 0.05)
                        Performance has improved.

vec ser composite fixed Key128, 1024 rows, Pr[null]=0 #2
                        time:   [20.544 µs 20.561 µs 20.583 µs]
                        change: [-56.527% -56.438% -56.348%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe

vec ser composite fixed Key128, 128 rows, Pr[null]=0.01 #2
                        time:   [2.8540 µs 2.8574 µs 2.8620 µs]
                        change: [-52.659% -52.490% -52.330%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  9 (9.00%) high mild
  2 (2.00%) high severe

vec ser composite fixed Key128, 1024 rows, Pr[null]=0.01 #2
                        time:   [20.745 µs 20.765 µs 20.789 µs]
                        change: [-60.776% -60.283% -59.766%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

vec ser composite fixed Key128, 128 rows, Pr[null]=0.1 #2
                        time:   [3.0091 µs 3.0189 µs 3.0297 µs]
                        change: [-51.389% -51.282% -51.181%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe

vec ser composite fixed Key128, 1024 rows, Pr[null]=0.1 #2
                        time:   [22.080 µs 22.093 µs 22.108 µs]
                        change: [-55.695% -55.386% -55.102%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

vec ser mix fixed and not fixed, case 1 KeySerialized, 128 rows, Pr[null]=0
                        time:   [24.302 µs 24.418 µs 24.527 µs]
                        change: [+9.7524% +10.240% +10.741%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

vec ser mix fixed and not fixed, case 1 KeySerialized, 1024 rows, Pr[null]=0
                        time:   [215.94 µs 217.34 µs 218.72 µs]
                        change: [+15.361% +15.917% +16.534%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

vec ser mix fixed and not fixed, case 1 KeySerialized, 128 rows, Pr[null]=0.01
                        time:   [23.997 µs 24.047 µs 24.097 µs]
                        change: [+9.4781% +9.7969% +10.106%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

vec ser mix fixed and not fixed, case 1 KeySerialized, 1024 rows, Pr[null]=0.01
                        time:   [190.01 µs 190.56 µs 191.12 µs]
                        change: [+12.642% +13.035% +13.429%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

vec ser mix fixed and not fixed, case 1 KeySerialized, 128 rows, Pr[null]=0.1
                        time:   [20.279 µs 20.297 µs 20.318 µs]
                        change: [+7.6431% +7.8289% +8.0251%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe

vec ser mix fixed and not fixed, case 1 KeySerialized, 1024 rows, Pr[null]=0.1
                        time:   [174.34 µs 175.20 µs 176.04 µs]
                        change: [+13.815% +14.334% +14.791%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) low mild

vec ser mix fixed and not fixed, case 2 KeySerialized, 128 rows, Pr[null]=0
                        time:   [24.479 µs 24.556 µs 24.637 µs]
                        change: [+9.3228% +9.6905% +10.057%] (p = 0.00 < 0.05)
                        Performance has regressed.

vec ser mix fixed and not fixed, case 2 KeySerialized, 1024 rows, Pr[null]=0
                        time:   [220.69 µs 221.74 µs 222.88 µs]
                        change: [+15.986% +19.632% +24.189%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe

vec ser mix fixed and not fixed, case 2 KeySerialized, 128 rows, Pr[null]=0.01
                        time:   [24.316 µs 24.397 µs 24.478 µs]
                        change: [+9.4101% +9.7661% +10.155%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild

vec ser mix fixed and not fixed, case 2 KeySerialized, 1024 rows, Pr[null]=0.01
                        time:   [201.19 µs 202.53 µs 203.87 µs]
                        change: [+15.114% +15.691% +16.258%] (p = 0.00 < 0.05)
                        Performance has regressed.

vec ser mix fixed and not fixed, case 2 KeySerialized, 128 rows, Pr[null]=0.1
                        time:   [20.992 µs 21.059 µs 21.128 µs]
                        change: [+8.4631% +8.7552% +9.0771%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

vec ser mix fixed and not fixed, case 2 KeySerialized, 1024 rows, Pr[null]=0.1
                        time:   [180.23 µs 181.57 µs 182.75 µs]
                        change: [+13.875% +14.506% +15.232%] (p = 0.00 < 0.05)
                        Performance has regressed.

Performance regression for new bench case I added:
vec ser large mixed KeySerialized, 1024 rows, Pr[null]=0.1
                        time:   [477.53 µs 497.68 µs 517.94 µs]
                        change: [+26.370% +30.346% +34.587%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)

vec ser large fixed KeySerialized, 1024 rows, Pr[null]=0.1
                        time:   [519.17 µs 527.12 µs 535.16 µs]
                        change: [+70.830% +73.801% +76.936%] (p = 0.00 < 0.05)
                        Performance has regressed.

Cases:
        HashKeyBenchCaseBuilder {
            data_types: vec![DataType::Int64; 16],
            describe: "large fixed".to_string(),
        },
        HashKeyBenchCaseBuilder {
            data_types: { let mut v = vec![DataType::Int64; 16]; v[15] = DataType::Varchar; v },
            describe: "large mixed".to_string(),
        },

Latest bench has performance across the board. I guess I was benchmarking on wrong commit...
Here's the breakdown:

Mixed: ~15%
KeySerialized: ~15%
Fixed size key: ~70%.

I think it's good to go. Future PR can optimize further. The Mixed case seems to be forced to use KeySerialized. Perhaps we can come up with a better key encoding for it.
Additionally
Full
   Compiling risingwave_common v1.0.0-alpha (/Users/noelkwan/projects/risingwave/src/common)
    Finished bench [optimized] target(s) in 24.10s
     Running benches/bench_hash_key_encoding.rs (target/release/deps/bench_hash_key_encoding-fbf1b9e7c3ee200e)
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0: Collecting 100 samples in estimated 5.0494 s (490k iterations)
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0: Analyzing
vec ser 1024 rows, Serial Key64, Pr[null]=0
                        time:   [10.442 µs 10.492 µs 10.545 µs]
                        change: [-71.624% -71.453% -71.289%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.01
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.01: Collecting 100 samples in estimated 5.0214 s (485k iterations)
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.01: Analyzing
vec ser 1024 rows, Serial Key64, Pr[null]=0.01
                        time:   [10.284 µs 10.310 µs 10.338 µs]
                        change: [-72.985% -72.823% -72.654%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.1
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.1: Collecting 100 samples in estimated 5.0065 s (485k iterations)
Benchmarking vec ser 1024 rows, Serial Key64, Pr[null]=0.1: Analyzing
vec ser 1024 rows, Serial Key64, Pr[null]=0.1
                        time:   [10.257 µs 10.283 µs 10.312 µs]
                        change: [-73.100% -72.915% -72.726%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0: Collecting 100 samples in estimated 5.0186 s (722k iterations)
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0: Analyzing
vec ser 1024 rows, int32 Key32, Pr[null]=0
                        time:   [6.9406 µs 6.9467 µs 6.9530 µs]
                        change: [-80.854% -80.794% -80.723%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.01
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.01: Collecting 100 samples in estimated 5.0002 s (712k iterations)
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.01: Analyzing
vec ser 1024 rows, int32 Key32, Pr[null]=0.01
                        time:   [7.0223 µs 7.0288 µs 7.0356 µs]
                        change: [-80.121% -80.004% -79.881%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.1
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.1: Collecting 100 samples in estimated 5.0104 s (656k iterations)
Benchmarking vec ser 1024 rows, int32 Key32, Pr[null]=0.1: Analyzing
vec ser 1024 rows, int32 Key32, Pr[null]=0.1
                        time:   [7.6363 µs 7.6419 µs 7.6476 µs]
                        change: [-78.889% -78.722% -78.546%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0: Collecting 100 samples in estimated 5.0358 s (490k iterations)
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0: Analyzing
vec ser 1024 rows, int64 Key64, Pr[null]=0
                        time:   [10.245 µs 10.276 µs 10.309 µs]
                        change: [-73.574% -73.401% -73.215%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.01
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.01: Collecting 100 samples in estimated 5.0318 s (490k iterations)
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.01: Analyzing
vec ser 1024 rows, int64 Key64, Pr[null]=0.01
                        time:   [10.260 µs 10.287 µs 10.316 µs]
                        change: [-74.072% -73.993% -73.913%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.1
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.1: Collecting 100 samples in estimated 5.0117 s (485k iterations)
Benchmarking vec ser 1024 rows, int64 Key64, Pr[null]=0.1: Analyzing
vec ser 1024 rows, int64 Key64, Pr[null]=0.1
                        time:   [10.274 µs 10.298 µs 10.324 µs]
                        change: [-74.242% -74.182% -74.117%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0: Collecting 100 samples in estimated 5.4220 s (35k iterations)
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0: Analyzing
vec ser 1024 rows, varchar KeySerialized, Pr[null]=0
                        time:   [152.58 µs 152.85 µs 153.13 µs]
                        change: [-14.529% -14.352% -14.183%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.01
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.01: Collecting 100 samples in estimated 5.4057 s (40k iterations)
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.01: Analyzing
vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.01
                        time:   [133.66 µs 134.03 µs 134.39 µs]
                        change: [-16.864% -16.659% -16.430%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) low mild

Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.3180 s (45k iterations)
Benchmarking vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, varchar KeySerialized, Pr[null]=0.1
                        time:   [116.45 µs 116.81 µs 117.14 µs]
                        change: [-18.481% -18.189% -17.894%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) low mild
  4 (4.00%) high mild

Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0: Collecting 100 samples in estimated 5.3424 s (20k iterations)
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0: Analyzing
vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0
                        time:   [264.28 µs 264.74 µs 265.23 µs]
                        change: [-8.2031% -7.9638% -7.7300%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.01
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.01: Collecting 100 samples in estimated 5.7194 s (25k iterations)
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.01: Analyzing
vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.01
                        time:   [225.86 µs 226.90 µs 228.17 µs]
                        change: [-10.960% -10.589% -10.184%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.8698 s (30k iterations)
Benchmarking vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, composite varchar KeySerialized, Pr[null]=0.1
                        time:   [192.87 µs 193.57 µs 194.25 µs]
                        change: [-12.012% -11.734% -11.451%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0: Collecting 100 samples in estimated 5.0597 s (293k iterations)
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0: Analyzing
vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0
                        time:   [17.230 µs 17.276 µs 17.324 µs]

Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.01
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.01: Collecting 100 samples in estimated 5.0536 s (288k iterations)
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.01: Analyzing
vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.01
                        time:   [17.490 µs 17.533 µs 17.577 µs]

Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.1
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.1: Collecting 100 samples in estimated 5.0743 s (263k iterations)
Benchmarking vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.1: Analyzing
vec ser 1024 rows, composite fixed, case 1 Key128, Pr[null]=0.1
                        time:   [19.260 µs 19.303 µs 19.346 µs]

Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0: Collecting 100 samples in estimated 5.1023 s (247k iterations)
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0: Analyzing
vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0
                        time:   [20.579 µs 20.622 µs 20.667 µs]

Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.01
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.01: Collecting 100 samples in estimated 5.0442 s (242k iterations)
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.01: Analyzing
vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.01
                        time:   [20.750 µs 20.803 µs 20.857 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.1
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.1: Collecting 100 samples in estimated 5.0882 s (232k iterations)
Benchmarking vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.1: Analyzing
vec ser 1024 rows, composite fixed, case 2 Key128, Pr[null]=0.1
                        time:   [21.849 µs 21.892 µs 21.939 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0: Collecting 100 samples in estimated 5.7579 s (35k iterations)
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0: Analyzing
vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0
                        time:   [162.24 µs 162.67 µs 163.14 µs]
                        change: [-13.911% -13.622% -13.330%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.01
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.01: Collecting 100 samples in estimated 5.0910 s (35k iterations)
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.01: Analyzing
vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.01
                        time:   [145.74 µs 146.06 µs 146.38 µs]
                        change: [-15.497% -15.273% -15.060%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.2166 s (40k iterations)
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, mix fixed and not fixed, case 1 KeySerialized, Pr[null]=0.1
                        time:   [129.12 µs 129.43 µs 129.76 µs]
                        change: [-16.494% -16.316% -16.128%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0: Collecting 100 samples in estimated 5.0679 s (30k iterations)
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0: Analyzing
vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0
                        time:   [167.37 µs 167.72 µs 168.09 µs]
                        change: [-14.991% -13.822% -13.102%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.01
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.01: Collecting 100 samples in estimated 5.1593 s (35k iterations)
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.01: Analyzing
vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.01
                        time:   [146.33 µs 146.71 µs 147.12 µs]
                        change: [-15.601% -15.323% -15.053%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild

Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.2517 s (40k iterations)
Benchmarking vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, mix fixed and not fixed, case 2 KeySerialized, Pr[null]=0.1
                        time:   [129.20 µs 129.78 µs 130.35 µs]
                        change: [-17.176% -16.848% -16.527%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0: Collecting 100 samples in estimated 5.6422 s (40k iterations)
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0: Analyzing
vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0
                        time:   [139.40 µs 139.63 µs 139.87 µs]
                        change: [-17.659% -17.142% -16.522%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.01
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.01: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.01: Collecting 100 samples in estimated 5.6248 s (40k iterations)
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.01: Analyzing
vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.01
                        time:   [139.09 µs 139.26 µs 139.43 µs]
                        change: [-18.261% -18.110% -17.953%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.0468 s (35k iterations)
Benchmarking vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, medium fixed KeySerialized, Pr[null]=0.1
                        time:   [142.35 µs 142.55 µs 142.77 µs]
                        change: [-18.120% -17.943% -17.751%] (p = 0.00 < 0.05)
                        Performance has improved.

Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

Benchmarking vec ser 1024 rows, medium mixed KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, medium mixed KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, medium mixed KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.4841 s (25k iterations)
Benchmarking vec ser 1024 rows, medium mixed KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, medium mixed KeySerialized, Pr[null]=0.1
                        time:   [217.24 µs 217.59 µs 217.94 µs]
                        change: [-12.735% -12.554% -12.374%] (p = 0.00 < 0.05)
                        Performance has improved.

Found 9 outliers among 100 measurements (9.00%)
  8 (8.00%) high mild
  1 (1.00%) high severe

Benchmarking vec ser 1024 rows, large fixed KeySerialized, Pr[null]=0.1
Benchmarking vec ser 1024 rows, large fixed KeySerialized, Pr[null]=0.1: Warming up for 3.0000 s
Benchmarking vec ser 1024 rows, large fixed KeySerialized, Pr[null]=0.1: Collecting 100 samples in estimated 5.4026 s (20k iterations)
Benchmarking vec ser 1024 rows, large fixed KeySerialized, Pr[null]=0.1: Analyzing
vec ser 1024 rows, large fixed KeySerialized, Pr[null]=0.1
                        time:   [268.49 µs 269.90 µs 271.36 µs]
                        change: [-10.595% -10.265% -9.9564%] (p = 0.00 < 0.05)
                        Performance has improved.

Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

To summarize here are the follow-ups required after this PR.

[ ] Use stack based data structure for serialized key / reserve memory for serialized key to avoid multiple allocation calls.
[ ] The Mixed case seems to be forced to use KeySerialized. Perhaps we can come up with a better key encoding for it.
[ ] Additionally I have no idea why medium case, which has 8 bigint keys, uses KeySerialized rather than Key64. It has less than MAX_FIXED_SIZE_KEY_ELEMENTS (equals 8). Will figure that out separately too.

Will add some checks on the frontend regarding number of group keys

My only concern is whether a maximum number of 64 is acceptable for group keys.

Is it possible to use a workaround for those extreme cases, i.e. putting several columns/keys into a struct first and then group by this struct?

~Use stack based data structure for serialized key /~ reserve memory for serialized key to avoid multiple allocation calls.

Have we tried smallvec for serialized key? I guess it will be a seamless replacement and improvement.

~Use stack based data structure for serialized key /~ reserve memory for serialized key to avoid multiple allocation calls.

Have we tried smallvec for serialized key? I guess it will be a seamless replacement and improvement.

Will try it, thanks for the suggestion.

My only concern is whether a maximum number of 64 is acceptable for group keys.

Is it possible to use a workaround for those extreme cases, i.e. putting several columns/keys into a struct first and then group by this struct?

I think there's another solution:
If exceeds 64, we revert to heap based data structure, e.g. FixedBitSet. Then we just store pointer to it, which is usize, usually same size as u64.

My only concern is whether a maximum number of 64 is acceptable for group keys.

Is it possible to use a workaround for those extreme cases, i.e. putting several columns/keys into a struct first and then group by this struct?

Do you mean doing this by the optimizer or the compute node itself? As for the former, I guess we need a unified interface for hooking all group key derivations first, which might be hard. 🥵

My only concern is whether a maximum number of 64 is acceptable for group keys.

Is it possible to use a workaround for those extreme cases, i.e. putting several columns/keys into a struct first and then group by this struct?

I think there's another solution: If exceeds 64, we revert to heap based data structure, e.g. FixedBitSet. Then we just store pointer to it, which is usize, usually same size as u64.

Will add into this PR, since it will also resolve https://github.com/risingwavelabs/risingwave/pull/8941#discussion_r1159273588 above.

Then we just store pointer to it, which is usize, usually same size as u64.

Makes sense. Although there's an extra dereference with a pointer, I think it's tolerable since it's a rare case.
BTW, using an enum with pointer here may still increase the struct's stack size as the compiler may fail to find a position for the tag of the enum.

Then we just store pointer to it, which is usize, usually same size as u64.

Makes sense. Although there's an extra dereference with a pointer, I think it's tolerable since it's a rare case.
BTW, using an enum with pointer here may still increase the struct's stack size as the compiler may fail to find a position for the tag of the enum.

Here's the approach I used to avoid overhead from enum: https://github.com/risingwavelabs/risingwave/pull/9026/files.
Basically just parameterize HashKeys by the Bitmap type. However it's blocked due to clippy panic. Still investigating if there's a way around it.
I guess it is fine to merge this first, since group key > 64 seems unlikely in most cases.
Thoughts? @lmatz @BugenZhao @st1page


Initial support for areas

This PR adds initial support for Areas in UXsim. I approached it in the following way:

Introduced a new area attribute in the Node class to define the area a node belongs to.
Modified the __init__ method of the Node class to accept an optional area parameter.
Added W.NODES_AREA_DICT to store nodes grouped by their respective areas.
Implemented update_nodes_area_dict() to populate the area dictionary with nodes.
Added a method get_random_node_in_area() to retrieve a random node within a specified area.
Updated finalize_scenario() to call update_nodes_area_dict() ensuring the area dictionary is up to date.

In my model, I use it like this:
When adding the nodes to the model, I define the area keyword
# Create Nodes in UXsim from OSMnx graph nodes
for node_id, data in road_network.nodes(data=True):
    mrdh65 = data1.pc4_to_mrdh65[int(data['postcode'])]
    world.addNode(name=str(node_id), x=data['x'], y=data['y'], area=mrdh65)

When planning trips, I ask to get a random node from an area, which I use as starting point.
starting_node = self.model.uxsim_world.get_random_node_in_area(self.mrdh65)
target_node = self.model.uxsim_world.get_random_node_in_area(destination)

# Add the car trip to the UXsim world
self.model.uxsim_world.addVehicle(orig=starting_node, dest=target_node, departure_time=self.model.simulator.time)

More features could be added. But before I develop this further, I would like to know @toruseo if this is a desired development direction.
I could also understand that you want to approach this subject differently, by making Areas objects or by tracking areas in a different way.
Part of https://github.com/toruseo/UXsim/issues/102.

Thank you very much for your effort.
But I think "area" variable is better to be defined more generally and should not depends on the simulator too much. This is because "area" will be useful as a input and output for the simulation but will not be utilized by the simulation model itself.
For example, a node may belong to multiple areas in general (e.g., an intersection belong to a city and a prefecture). But for some particular purposes, areas need to be mutually exclusive. Therefore, I think it is better to leave this kind of setting to the users rather than implementing it within the simulator
I think I can implement them today.

Thanks for getting back! I agree adding only one single area per node might be a bit limiting.
I will open a separate PR to add get_random_node_in_area as util.

For reference, this is what I needed to do to work with the latest uxsim:

https://github.com/EwoutH/urban-self-driving-effects/commit/432ef66594b4302c6a806389349de04dab12ccfc

So I think you made the right call: The attribute is useful, and I didn't add too much code on my part.

Closing this PR, as current functionality is either enclosed in current UXsim or can easily be added like shown above.


Fix inferred argument validation with type reference types

Purpose
$title.
Fixes https://github.com/ballerina-platform/ballerina-lang/issues/39555
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[x] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

@shafreenAnfar @chamil321 With this PR http module is breaking. Looks like we used a bug as a feature. Shall we fix it?

@shafreenAnfar @chamil321 With this PR http module is breaking. Looks like we used a bug as a feature. Shall we fix it?

The error is for https://github.com/ballerina-platform/module-ballerina-http/blob/master/ballerina/http_request_context.bal#L88. The inferred part of the union and the remaining members have to be disjoint in order to be able to figure out the part of the type corresponding to inference. This particular line fails because value:Cloneable includes error and therefore isn't disjoint with ListenerError.
Similar code
import ballerina/lang.value;

public type ReqCtxMember value:Cloneable|isolated object {};

public type ReqCtxMemberType typedesc<ReqCtxMember>;

function f1(string key, ReqCtxMemberType targetType = <>) returns targetType|error = external; // now an error

function f2(string key, typedesc<value:Cloneable|isolated object {}> targetType = <>) returns targetType|error = external; // already an error

@shafreenAnfar @chamil321 With this PR http module is breaking. Looks like we used a bug as a feature. Shall we fix it?

The error is for https://github.com/ballerina-platform/module-ballerina-http/blob/master/ballerina/http_request_context.bal#L88. The inferred part of the union and the remaining members have to be disjoint in order to be able to figure out the part of the type corresponding to inference. This particular line fails because value:Cloneable includes error and therefore isn't disjoint with ListenerError.
Similar code
import ballerina/lang.value;

public type ReqCtxMember value:Cloneable|isolated object {};

public type ReqCtxMemberType typedesc<ReqCtxMember>;

function f1(string key, ReqCtxMemberType targetType = <>) returns targetType|error = external; // now an error

function f2(string key, typedesc<value:Cloneable|isolated object {}> targetType = <>) returns targetType|error = external; // already an error

@TharmiganK Please note

Ack, will fix this 👍


Add test for invalid include parameter for accounts PATCH.

Note that this test currently fails.
hoodiehq/camp#71

@gr2m Good catch! This is the thing that was obviously wrong! I've fixed it now.


Java packman

Updated packman task to support multiple languages
Updated java gprc task to use packman
Updated tests

Current coverage is 64.13%

Merging #5 into master will decrease coverage by -2.67% as of 23a46f6

@@            master      #5   diff @@
======================================
  Files           23      23
  Stmts          714     711     -3
  Branches         0       0
  Methods          0       0
======================================
- Hit            477     456    -21
  Partial          0       0
- Missed         237     255    +18

Review entire Coverage Diff as of 23a46f6

Powered by Codecov. Updated on successful CI builds.


headline with four * trigger bold syntax

This causes a lot of weird behaviour, especially when
org_hide_emphasis_markers is true.
To fix it, change the org_bold syntax to not allow bold statements that start with two *.

Thanks! Not using org_hide_emphasis_markers myself so I missed it.


Osmosis Rewards Epoch processor missing rewards from Osmosis for dates before distribution event was added

Our Epoch processor gets Osmosis reward distributions by analyzing Epoch event emissions and finding the distribution event type.
The distribution event type, included in the incentives module, is emitted when Epoch rewards are distributed.
However, this event emission was not added until this commit in August of 2021: https://github.com/osmosis-labs/osmosis/commit/af6cb98774ea86f5a9dd7f76d5bb62ce6fbf1a15
What this means is that, since we rely on the distribution event, all rewards before this event emission were added are invisible to the indexer in its current state.
One angle we could try for reward indexing before this date is to look for transfer events as a fallback. This could work because:

incentives calls SendCoinsFromModuleToAccount in the bank keeper
SendCoinsFromModuleToAccount gets the module address (for this one it would be the incentives module address) and sets that as the sender
SendCoinsFromModuleToAccount calls bank send-keeper SendCoins
SendCoins emits a transfer event (in this case setting the sender address as the incentives module address)

We would need to be careful with this one. If any other module sends coins through the bank keeper at Epochs, we would need to filter out transfer events that are not coming from the incentives module.

moving this to the backlog


Address not found

I send email to<EMAIL_ADDRESS>Address not found
--
Your message wasn't delivered to<EMAIL_ADDRESS>because the address couldn't be found, or is unable to receive mail.

@Pomazan-Bogdan that doesn't appear please send a report to<EMAIL_ADDRESS>with details on the specific stream you're trying to email and why you think that's it's email address.  That address does not appear to be in the correct format.

This adress correct.
Name channel "Cyrillic".
Support cyrillic name for chanel and mail stream add to 2.x


Seek Improvements

add convenience methods to AbstractConsumerSeekAware

compile tests with Java 11

24-Aug-2020 17:29:57 | > Task :api
-- | --
24-Aug-2020 17:29:57 | 1 error
24-Aug-2020 17:29:57 | src/api/overview.html:4: error: self-closing element not allowed
24-Aug-2020 17:29:57 | <hr/>
24-Aug-2020 17:29:57 | ^
24-Aug-2020 17:29:57 |
24-Aug-2020 17:29:57 | > Task :api FAILED

Hm. I should've gave you a couple minutes to check it before merging upstream.
Anyway let try switch it to Java 14 on CI!


test: Add scale test for nodepools with minValues in requirement.

Description
This PR adds the scale test for nodepools with minValues in the requirement. This is necessary since minValues adds more computation to the scheduler(to enforce the scheduler to consider the minimum requirements to create the NodeClaim) and hence evaluates if the nodes/pods can be scaled as expected.
How was this change tested?
make presubmit
Ran the test manually several times to check for any flakiness due to minValues being added. The reason for flakiness could be attributed to fewer number of instances available to meet the minimum requirements.
Does this change impact docs?

[ ] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[x] No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 82.443%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
5339

Relevant Lines:
6476

💛  - Coveralls


Transform all keys to JSON format for RKE config values

Currently, the CLI does not honor most values in a passed in rke-config file. The CLI silently ignored it, and new clusters used default values.
This fixes it, so the CLI respects and applies the settings in rancher_kubernetes_engine_config.
The same is now true for top-level keys above it, like docker_root_dir.
Unfortunately, this cannot be a perfect solution. But it's better than what we have now.
The problem is that most values in RancherKubernetesEngineConfig are defined with struct tags for both JSON and YAML in camelCase.
https://github.com/rancher/rancher/blob/release/v2.6/pkg/client/generated/management/v3/zz_generated_rancher_kubernetes_engine_config.go
Changing the tags will be a breaking change. For proper deserialization, we must convert all keys to camelCase.
Note that we ignore kebab-case keys. Users themselves should ensure any relevant keys
(especially top-level keys in services, like kube-api or kube-controller) are camelCase or snake-case in cluster config.
Otherwise, they will be ignored on deserialization, and defaults settings will be used. And some settings in kube-api, for example, are expected to be in kebab-case.
Our RKE docs, unfortunately, use kebab-case for some examples.
This should also fix the import flag. It would not do anything previously and create a default cluster. Now it just defers to the import command handler.
I would love to write an end-to-end test for this workflow: supply an RKE config file, compare resultant cluster configuration.
But we currently have no testing framework in place for such things in the CLI repository, since every such workflow makes a real API request to a live cluster.
https://github.com/rancher/rancher/issues/25416

This one is ready for review, now that v2.6.3 is out, @superseb, @rawmind0. Thanks!


snowflake_procedure execute_as=CALLER update does not work on existing resource

Terraform CLI and Provider Versions
Terraform: 1.7.2
Provider: 0.87
Terraform Configuration
resource "snowflake_procedure" "test_procedure" {
  database = "TEST"
  schema   = "TEST_SCHEMA
  name     = "TEST_PROCEDURE"
  arguments {
    name = "TEST_DATE"
    type = "DATE"
  }
  arguments {
    name = "NUMBER_OF_DAYS"
    type = "NUMBER"
  }
  language    = "SQL"
  comment     = "sample procedure"
  return_type = "TABLE (ROWS_UNLOADED NUMBER, INPUT_BYTES NUMBER, OUTPUT_BYTES NUMBER)"
  execute_as = "CALLER"
  statement = templatefile("${path.module}/test.sql", {
    bucket_name              = "test-bucket",
    storage_integration_name = "TEST_STORAGE_INTEGRATION,
  })
}

Expected Behavior
I have a snowflake procedure created with execute_as=OWNER through terraform. Modifying the resource to update execute_as=CALLER should just allow the procedure to be updated. Tried modifying the procedure in the snowflake console and it works.
Actual Behavior
terraform apply fails with below error
│ Error: 001003 (42000): SQL compilation error:
│ syntax error line 1 at position 93 unexpected 'CALLER'.
Steps to Reproduce

Runterraform apply with execute_as=OWNER
Procedure is created
Update the resource with execute_as=CALLER to modify the procedure
Run terraform applyand it fails

How much impact is this issue causing?
High
Logs
No response
Additional Information
We had to destroy the procedure and then recreate it using execute_as=CALLER as a workaround

Hey @krishreddy56. Thanks for reaching out to us.
Please run the terraform with TF_LOG=DEBUG flag enabled. You should be able to see the ALTER statement run for execute_as update (it should be close above to the error you have shared).

Re ran the terraform apply by enabling the debug log. I see below in the logs
2024-03-21T14:30:07.082-0600 [INFO]  Starting apply for module.sql_compliance.snowflake_procedure.test_procedure
2024-03-21T14:30:07.084-0600 [DEBUG] module.sql_compliance.snowflake_procedure.test_procedure: applying the planned Update change
2024-03-21T14:30:07.353-0600 [DEBUG] provider.terraform-provider-snowflake_v0.87.2: 2024/03/21 14:30:07 [DEBUG] sql-conn-exec: [query ALTER PROCEDURE "TEST"."TEST_SCHEMA"."TEST_PROCEDURE" (DATE, NUMBER) CALLER err 001003 (42000): SQL compilation error:
2024-03-21T14:30:07.353-0600 [DEBUG] provider.terraform-provider-snowflake_v0.87.2: syntax error line 1 at position 93 unexpected 'CALLER'. duration 267.033583ms args {}] (SENDGRID)
2024-03-21T14:30:07.353-0600 [DEBUG] provider.terraform-provider-snowflake_v0.87.2: 2024/03/21 14:30:07 [DEBUG] err: 001003 (42000): SQL compilation error:
2024-03-21T14:30:07.353-0600 [DEBUG] provider.terraform-provider-snowflake_v0.87.2: syntax error line 1 at position 93 unexpected 'CALLER'.
2024-03-21T14:30:07.353-0600 [ERROR] provider.terraform-provider-snowflake_v0.87.2: Response contains error diagnostic: @caller=github.com/hashicorp/terraform-plugin-go@v0.20.0/tfprotov6/internal/diag/diagnostics.go:62 @module=sdk.proto diagnostic_severity=ERROR tf_proto_version=6.4 tf_provider_addr=registry.terraform.io/Snowflake-Labs/snowflake tf_rpc=ApplyResourceChange diagnostic_detail="" tf_req_id=5a6a0c29-41b6-fac2-678b-e19ced386ce6 tf_resource_type=snowflake_procedure
  diagnostic_summary=
  | 001003 (42000): SQL compilation error:
  | syntax error line 1 at position 93 unexpected 'CALLER'.
   timestamp=2024-03-21T14:30:07.353-0600

Thanks, I see where the problem is. It should be an easy fix, we should be able to have it with the next week's release.

Thank you for looking into this, appreciate it

Hey @krishreddy56/ Fix released in v0.88.0.
Please check and close the issue if the problem is solved for you.

Closing as fixed in v.0.88.0. Please open a new issue if the problem persists in the newest provider version.


[bug] After compiling prod build can't find index.html when using nextjs

Describe the bug
When running npm run tauri build, the app compiles and whenever you launch it on any platform, tauri will not be able to find index.html and writes out the message below:
Could not get requested file.
The app works fine under tauri dev.
Reproduction

Set up tauri using nextjs template
run npm run tauri build
run the complied executable

Expected behavior
I'd expect tauri to navigate nextjs directories compiled inside ./out dir, and run the frontend from there.
Platform and versions
[✔] Environment
    - OS: Fedora 38.0.0 X64
    ✔ webkit2gtk-4.0: 2.40.2
    ✔ rsvg2: 2.56.1
    ✔ rustc: 1.70.0-nightly (1459b3128 2023-03-23)
    ✔ Cargo: 1.70.0-nightly (15d090969 2023-03-21)
    ✔ rustup: 1.25.2 (17db695f1 2023-02-01)
    ✔ Rust toolchain: nightly-x86_64-unknown-linux-gnu (default)
    - node: 18.16.0
    - pnpm: 7.30.3
    - yarn: 1.22.19
    - npm: 9.5.1

[-] Packages
    - tauri [RUST]: 1.3.0
    - tauri-build [RUST]: 1.3.0
    - wry [RUST]: 0.24.3
    - tao [RUST]: 0.16.1
    - @tauri-apps/api [NPM]: 1.4.0
    - @tauri-apps/cli [NPM]: 1.4.0

[-] App
    - build-type: bundle
    - CSP: unset
    - distDir: ../out
    - devPath: http://localhost:4200/
    - framework: React (Next.js)
    - bundler: Webpack

Stack trace
`Asset `index.html` not found; fallback to index.html.html`

Additional context
No response

could you upgrade tauri and tauri-build in src-tauri/Cargo.toml to latest version 1.4 and try again?

Just ran cargo update, and the issue persists.

I have used https://github.com/kvnxiao/tauri-nextjs-template to test out next.js with latest tauri and it worked just fine, maybe you have misconfigured your next.js config or tauri config. Please checkout out that repo and see if your configs match theirs, if not, please reach out to us on Discord where we can help you triage the issue.


[rules] Detect right-to-left character in commands or files

to: @mime-frame @ryandeivert
cc: @airbnb/streamalert-maintainers
size: small
Background
Malicious files can be disguised by using a encoding trick that uses the unicode character U+202E, also known as right-to-left-override (RLO). The trick hides a potentially malicious extension and makes them appear harmless.
https://krebsonsecurity.com/2011/09/right-to-left-override-aids-email-attacks
Changes

Adding osquery logs to kinesis placeholder.
Adding carbonblack logs to s3 placeholder.
Adding rule right_to_left_character using data normalization and tests.

Testing
Verification of schemas:
$ python manage.py validate-schemas

Testing of rule:
python manage.py lambda test --processor rule --test-file right_to_left_character

Coverage remained the same at 95.075% when pulling a9ed2c7a166b5e6122b2f8f1a07ca8910ed0ccd4 on javier-streamalert-rule-rtlo into 6114f665ae83d02d505c47cd19de12752cab3cfb on master.

Coverage remained the same at 95.075% when pulling f9d0a771364c1787cf18b3922f439bef491de526 on javier-streamalert-rule-rtlo into 16d19ad25c3a618104e820472ef914e59940c1b9 on master.

Coverage remained the same at 95.075% when pulling 1549df6dbdb89986abf43e00b32746262556ff4f on javier-streamalert-rule-rtlo into cf003de84b4f7da7addf75eafae71680826e0d2f on master.


CSS and themes do not apply

When go to install the CSS tweaks and Theme, nothing changes

Did you click Patch CSS in advanced options?

that worked, thanks for the help


Added aggregate java examples

Aggregate java examples. Part of the https://github.com/nemtech/symbol-docs/issues/101 task.
There is a TODO in the source code. I'm going to improve the SDK so it easier for users to add signed cosignatres to aggregate transactions.

Moved to this PR
https://github.com/nemtech/symbol-docs/pull/503


3.8.1 CDN delay?

I have an automated script that checks npm for newest raven-js version which I then use to update configurations of my app to use latest version CDN urls.
Now I noticed that even though 3.8.1 is released in npm and under GitHub releases, it is not in CDN.
Why the delay with updating official CDN?
$ npm info raven-js version
3.8.1
$ curl -I https://cdn.ravenjs.com/3.8.1/raven.min.js
HTTP/1.1 404 Not Found
$ curl -I https://cdn.ravenjs.com/3.8.0/raven.min.js
HTTP/1.1 200 OK

It's up now – bleh. Thanks for pointing this out.


Add a network pytest mark for tests that use the network

Sometimes it's useful to have the tests that use the network marked so they can be skipped easily when we know the network is not available.
This is useful for example on SUSE and openSUSE's build servers. When building our packages the network is disabled so we can assure reproducible builds (among other benefits). With this mark, it's easier to skip tests that can not succeed.
Patch suggesting the solution:
---
 pytest.ini            |    3 +++
 tests/test_serving.py |    5 +++++
 2 files changed, 8 insertions(+)

--- /dev/null
+++ b/pytest.ini
@@ -0,0 +1,3 @@
+[pytest]
+markers =
+    network: tests requiring network connection
--- a/tests/test_serving.py
+++ b/tests/test_serving.py
@@ -118,6 +118,7 @@ def test_windows_get_args_for_reloading(
     assert rv == argv

<EMAIL_ADDRESS> @pytest.mark.parametrize("find", [_find_stat_paths, _find_watchdog_paths])
 def test_exclude_patterns(find):
     # Imported paths under sys.prefix will be included by default.
@@ -157,6 +158,7 @@ def test_port_is_int():
         run_simple("<IP_ADDRESS>", "5000", None)

<EMAIL_ADDRESS> @pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
 @pytest.mark.parametrize("send_length", [False, True])
 def test_chunked_request(monkeypatch, dev_server, send_length):
@@ -241,6 +243,7 @@ def test_multiline_header_folding(standa
     assert data["HTTP_XYZ"] == "first\tsecond\tthird"

<EMAIL_ADDRESS> @pytest.mark.parametrize("endpoint", ["", "crash"])
 def test_streaming_close_response(dev_server, endpoint):
     """When using HTTP/1.0, chunked encoding is not supported. Fall
@@ -252,6 +255,7 @@ def test_streaming_close_response(dev_se
     assert r.data == "".join(str(x) + "\n" for x in range(5)).encode()

<EMAIL_ADDRESS> def test_streaming_chunked_response(dev_server):
     """When using HTTP/1.1, use Transfer-Encoding: chunked for streamed
     responses, since it can distinguish the end of the response without
@@ -264,6 +268,7 @@ def test_streaming_chunked_response(dev_
     assert r.data == "".join(str(x) + "\n" for x in range(5)).encode()

<EMAIL_ADDRESS> @pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
 def test_streaming_chunked_truncation(dev_server):
     """When using HTTP/1.1, chunked encoding allows the client to detect

Complete build logs with all packages used and steps taken to reproduce.

The "network" tests don't access an external network that could affect reproducibility, they only connect to localhost to test the dev server.

How can I disable networking to confirm that all necessary tests are marked? It would be helpful if you could make a PR.

I am sorry, I am a co-maintainer of over 2000 Python-related packages in openSUSE. I just cannot maintain checkouts of all those packages.

Sorry, I don't have that expertise. Since these tests don't require external connections, and haven't been demonstrated to affect reproducibility, and I can't test that the mark will be accurate, I'm going to close this at this time. Happy to review a PR if you or someone else on your team can make one.

I don't see how I would tell Docker to not allow localhost, and I don't have anything set up to run the tests in Docker. Since these tests don't require external connections, it would help if you could demonstrate how they affect reproducibility.

I'm about to release 2.1.2, so if you want these marks in those tests, I'm going to need a resolution in the next few hours.

It could also be that you're using a non-standard build process, that seems to be the issue with a lot of issues that OS packagers report. We expect to build with python setup.py sdist or preferably python -m build, and to be installed with pip install. setup.py install is not supported and may cause issues.

Any update on this? It sound like this should be a bug reported to SUSE about their Python build process or network isolation.

I'll add this mark because it's convenient to skip the tests that use the dev_server fixture since they're slow.
However, I would strongly encourage you to investigate why the tests are failing in your environment and address that there. While no one using Werkzeug from an operating system package should be using the development server, it still doesn't seem good to skip these tests in that context.

Sorry, I will get to this eventually, but I am currently swamped with other stuff (from our paying customers).

What's the point of pointing out "paying customers". I mean, it draws attention to the fact that SUSE isn't paying us, but is asking us to debug their build system, so now I'm sort of inclined to just close this?

Speaking of paying customers, I noticed that SUSE is listed on Wikipedia as having $450 million in revenue, but as far as I'm aware has never donated to Pallets. Presumably, some fraction of your paying customers are deploying Flask, Jinja, or Click applications, so it would be really appreciated if SUSE made a donation to the Pallets organization.

I am just a poor engineer, nobody deciding anything about the money.
We usually don’t give up money. Our contribution to the upstream open source world is in a zillion manhours spent on creating patches, supporting our users, etc.

I don't think I've seen a patch in a Pallets project from a SUSE employee, and you said you don't have time to make patches anyway.

How can I disable networking to confirm that all necessary tests are marked? It would be helpful if you could make a PR.

FWIW I coincidentally looked into this just a few days ago for one of my FOSS projects. There's a reasonably simple way to disable internet connectivity when running tests via tox: https://github.com/mitmproxy/mitmproxy/blob/9d1821e920d6225578a0b65881d203a537186298/.github/workflows/main.yml#L88-L96
(I don't want to add any heat to the discussion, so I'll leave it at that. Thanks @davidism!)

OK, dev_server works for me, I have just discovered that I have mistakenly included in my patch also tests/test_serving.py::test_exclude_patterns[_find_stat_paths], which has different problems. Let me file another ticket.


Contact field in the YAML

Problem description
the contact field (and value) must be removed according to: https://github.com/camaraproject/CallForwardingSignal/issues/72#issuecomment-2252366626
Expected behavior
update the YAML consequently

Fixed with: https://github.com/camaraproject/CallForwardingSignal/pull/68

Merged


Missing word

test_does_not_detect_a_word_as_its_own_anagram should contain "banana" inside outputs instead of being empty.

Thank you.
"banana" is not an anagram of "banana" -- for it to be an anagram, two criteria must be fulfilled:

same letters
in a different order to produce a new word

From wikipedia:

An anagram is a type of word play, the result of rearranging the letters of a word or phrase to produce a new word or phrase, using all the original letters exactly once

Oh, I didn't notice that, my bad.

No worries! Thanks for jumping in :)


Fix: Dangerfile could not find /fastlane/Constants/Constants

Issue
Danger could not find the file require './fastlane/Constants/Constants'  and would crash on CICD. './fastlane/Constants/Constants' was removed and change to './fastlane/Constants/Constants.swift'
Make sure Danger has all its variables available when using make.sh without './fastlane/Constants/Constants'.
Expected
Danger can run on CICD with success.
Steps to reproduce

Use the template on a new repo.
Run make.sh.
Make sure the project can run test locally.
Create PR on the new repo.
Danger should run and comments its output.

@blyscuit this issue is duplicated with #399? 🤔

Duped and fixed with #399


Dead kernel

Hi,
I followed the instructions, using iPython 3.1.0. When creating the new "Template" notebook, it will connect for a few seconds, and then begin a loop between "Dead kernel" and "Connected" notifications.
Thank you in advance.

Quickstart here: https://github.com/korniichuk/rk#quickstart
Next time, please, just copy here info about errors from your terminal.
I have not enough information, maybe you just forget change kernel.json file?

Setup SSH for auto login without a password:
https://github.com/korniichuk/rk#setup-ssh-for-auto-login-without-a-password
$ rk ssh
Install a template of a remote jupyter kernel:
https://github.com/korniichuk/rk#install-a-template-of-a-remote-jupyter-kernel
$ rk install-template
change the kernel.json file:
$ sudo gedit /usr/local/share/jupyter/kernels/template/kernel.json
From:

{
 "argv": [
  "rkscript",
  "python",
  "{connection_file}",
  "remote_username@remote_host"
 ],
 "display_name": "Template",
 "language": "python"
}

To:
{
 "argv": [
  "rkscript",
  "python",
  "{connection_file}",
<EMAIL_ADDRESS> ],
 "display_name": "Template",
 "language": "python"
}

Note:
For a python2 remote jupyter kernel just change value remote_username@remote_host.

Add YouTube video to Quickstart:
https://github.com/korniichuk/rk#quickstart


No longer able to interact with actual bags to change them?

Addon Version
0.1.78
World of Warcraft Version
Retail
Describe the bug
The option to show the actual bags shows them as all greyed out and thus are not intractable or replaceable with new bags.
Steps to reproduce
Use the addon.

Do you have any Lua errors? I am not able to reproduce it on v0.1.78.

Same, I can't reproduce this.

Looks like your latest release 0.1.79 fixed the issue. Thanks.


can you add a people widget for lab members?

such as http://slslab.org/#people, thus can include all lab members in the widget

have you tried reusing the projects widget for people? For the way it is currently structured, that might work.

This is a duplicate of #274 - please add your feedback to that thread.
In the meantime, many users are using an instance of the projects widget for this purpose, similar to what @jananiravi suggested. You may find my comment at https://github.com/gcushen/hugo-academic/issues/575#issuecomment-399283020 useful for some tips on how to use multiple instances of the projects widget for different purposes.

Anyone can help me how can I add the people widget?  I created a user called principal, but output is not rendering, below are the steps I did,
created md file inside home folder

created user called principal

added user-group inside the principal's _index.md file


Documentation on configuring authentication for a web application

This task seems to be the only task (compared to IIS Web App Deploy) that allows you to specify the supported authentication options (IIS Authentication) for a web app. However the documentation indicates that these settings only work for web sites, not web apps. If you set the configuration type to IIS Web App then the option to configure authentication goes away.
How can this task be used to set the IIS Authentication settings for a web application (not a site) in IIS. All of our internal sites are configured to use Win auth at the application level while the default website (under which they are hosted) only has anonymous configured. Clearly IIS supports this but we cannot figure out how to get it to work with this task.
We are currently using IIS Web App Deploy and it has the same issue. So even a solution involving WebDeploy would be sufficient.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 244121ae-5de4-14cd-a9c3-d9cda5ad0a3b
Version Independent ID: 313238b7-d67b-bc60-1188-cbe305179600
Content: IIS Web App Manage task - Azure Pipelines
Content Source: docs/pipelines/tasks/deploy/iis-web-app-management-on-machine-group.md
Product: devops
Technology: devops-cicd
GitHub Login: @RoopeshNair
Microsoft Alias: ronai

Thanks for your question. It looks like your question isn't an issue with the documentation, but an issue or question specific to your implementation or a bug in the product itself.
For help with a specific task, here are a couple of options where you might consider asking your question:

Azure DevOps on Stack Overflow
Azure DevOps Support Bot

If you've encountered an issue with the product itself, to make sure it gets in front of the right people, please submit your bug here.
If you have an idea for improving the product, please submit your idea here.


[GB18030]With a name using GB18030 encoded, create .NET Core-1.x project, Add Migration will meet an error: Unrecognized escape sequence.

REPRO MACHINE: WNVM30593
REPRO ACCOUNT: redmond\fwtlaba
REPRO Configuration: Win10 x64 CHS +  Dev15 (Release channel : pre-release-build) VSE CHS (Web  workload, ASP.NET Core 1.0-1.1 dev tools checked)
INSTALL STEPS

Clean machine: Win10 x64 CHS OS
2.Install Dev15.5RTW VSE
from: \cpvsbuild\drops\VS\d15rel\products\27128.00\channels\internal\Release\bootstrappers\fixed
Select Web workload

REPRO STEPS

New project  >C# >Web> ASP.NET Core App > ASP.NET Core 1.1 - .NET Core >Web

App(MVC) with IndAuth >Name this peoject "①Ⅻㄨㄩ 啊阿鼾齄丂丄狚狛狜狝﨨﨩ˊˋ˙– ⿻〇㐀㐁䶴䶵" > OK.

In Solution Explorer, right click the Models folder > Add > Class. Name the class Movie and add the following properties

public class Movie
{
public int ID { get; set; }
public string Title { get; set; }
public DateTime ReleaseDate { get; set; }
public string Genre { get; set;}
public decimal Price { get; set; }
}
3.Right-click the Controllers folder > Add > Controller
In the Add Scaffold dialog, tap MVC Controller with views, using Entity Framework > Add
4.Complete the Add Controller dialog
Model class: Movie(MvcMovie.Models - your namespace)
Data context class: ApplicationDbContext

Views: Keep the default of each option checked
Controller name: Keep the default MoviesController
Tap Add
5.Verify Output window - Code Generation Log pane shows that scaffolding completed successfully, and the MoviesController and cshtml files under
Views\Movies got added in Solution Explorer

Open Package Manager Console (PMC) , type "Add-Migration AddMovies"

Meet an error: Unrecognized escape sequence.

Possible duplicate of #10532. Could consider porting to 1.1.x

This does not meet the bar for patching, so closing.


take a look at the employeeapp - should content be storage or link reference?

mut be updated both in dmss (blueprint) and in typing in dm-core-packges

it should be storage, since if delete file the content (binary) should be deleted

correct look

--

not needed, better to create a new example app


move_code_into_helper_review.rb:56:in complex_select_options?': undefined method sexp_type' for nil:NilClass

hey, on rails 3.1, ruby 1.9.2 I get:
bundle exec rails_best_practices .                                                                                1 1041 christoph
/home/christoph/.rvm/gems/ruby-1.9.2-p180@endeavour/gems/rails_best_practices-1.0.1/lib/rails_best_practices/reviews/move_code_into_helper_review.rb:56:in complex_select_options?': undefined method sexp_type' for nil:NilClass (NoMethodError)
from /home/christoph/.rvm/gems/ruby-1.9.2-p180@endeavour/gems/rails_best_practices-1.0.1/lib/rails_best_practices/reviews/move_code_into_helper_review.rb:45:in start_method_add_arg' 	from /home/christoph/.rvm/gems/ruby-1.9.2-p180@endeavour/gems/rails_best_practices-1.0.1/lib/rails_best_practices/core/check.rb:40:in node_start'
from (eval):6:in block in review' 	from (eval):4:in each'
from (eval):4:in review' 	from /home/christoph/.rvm/gems/ruby-1.9.2-p180@endeavour/gems/rails_best_practices-1.0.1/lib/rails_best_practices/core_ext/sexp.rb:16:in review'
how can I fix this?

I'm getting this same error, and I can't figure it out.  I've tried various rubies ree-1.8.7, 1.9.3, perhaps others.
Here's the last file parse and the first line of the stack trace with rails_best_practices -d:
./app/views/staff/notes/_form.html.haml
/Users/gee/.rvm/gems/ree-1.8.7-2012.02@AdvisingConnect/gems/rails_best_practices-1.8.0/lib/rails_best_practices/core/check.rb:322:in `mark_used': undefined method `sexp_type' for nil:NilClass (NoMethodError)

And here's the content of that view file:
:javascript
  $(function() {
    $("a.show-private").click(function() {
      $("#advisor_only").show(300);
      /* $(this).text($(this).text() == 'Add a Note Visible to Only Advisors' ? 'Hide Advisor Only Note Field' : 'Add a Note Visible to Only Advisors'); */
      $(this).hide();
      return false;
    });
  });

- if @note.advisor_summary.present?
  :javascript
    $(function() {
      $('#advisor_only').show();
      $("a.show-private").hide();
    });

=form_for [:staff, @student, @note], :html => {:class => 'form-vertical'} do |f|
  %legend
<EMAIL_ADDRESS>? 'Create a New' : 'Edit') + ' Note'
  .row
    .span12
      = render 'shared/error_messages', :target => @note
  - if @student.prospective?
    .row
      .span4
        %fieldset{'class' => 'control-group'}
          =f.label :prospective_name, :class => 'control-label'
          .controls
            = f.text_field :prospective_name, :value => @note.prospective_name
      .span4
        %fieldset{'class' => 'control-group'}
          =f.label :prospective_email, :class => 'control-label'
          .controls
            = f.text_field :prospective_email, :value => @note.prospective_email
      .span4
        %fieldset{'class' => 'control-group'}
          =f.label :birth_month, 'Birthday', :class => 'control-label'
          .controls
            =<EMAIL_ADDRESS>{:prefix => 'note', :field_name => 'birth_month', :include_blank => true}, 'data-placeholder' => 'Month', :class => 'chosen birth-month')
            =<EMAIL_ADDRESS>{:prefix => 'note', :field_name => 'birth_day', :include_blank => true}, 'data-placeholder' => 'Day', :class => 'chosen birth-day')
  .row
    .span4
      %fieldset{'class' => 'control-group'}
        =f.label :contact_on, :class => 'control-label'
        .controls
          =f.text_field :contact_on, :value => readable_timestamp(@note.contact_on), :class => 'datepicker'
    .span4
      %fieldset{'class' => 'control-group'}
        =f.label :contact_type_id, :class => 'control-label'
        .controls
          =f.collection_select :contact_type_id, ContactType.all, :id, :name,{},:class => 'chosen'
    .span4
      %fieldset{'class' => 'control-group'}
        %label Attendance
        .controls
          %label.radio
            = f.radio_button :noshow, false, :checked =><EMAIL_ADDRESS>            %span The student showed up
          %label.radio
            = f.radio_button :noshow, true, :checked => @note.noshow?
            %span The student did not show up
  .row
    .span12
      %fieldset{'class' => 'control-group'}
        %label{'class' => 'control-label'} Reasons
        .controls
          = select_reasons(:note, @note.reasons)
      %fieldset{'class' => 'control-group'}
        .controls
          .alert.alert-block.alert-success.note-field
            %p
              %strong Advising Note:
              ="Visible to<EMAIL_ADDRESS>and Advisors"
            =f.text_area :public_summary, :class => "expand", :rows => "12"
      #advisor_only{'style' => 'display:none;'}
        %fieldset{'class' => 'control-group'}
          .controls
            .alert.alert-block.alert-error.note-field
              %p.pull-left
                %strong Advisor-Only Note:
                Visible to Only Advisors
              %p.pull-right.muted
<EMAIL_ADDRESS>may still formally request access to this note.)"
              =f.text_area :advisor_summary, :class => "expand", :rows => "12", :style => 'margin-top:10px;'
              =hidden_field_tag 'appointment_id', @note.appointment.try(&:id)
      .pull-left
        %a{'class' => 'show-private'} Add a Note Visible to Only Advisors
      .pull-right
        =f.submit @note.new_record? ? "Create Note" : "Update Note", :id => 'confirm_submit', :class => "btn success"
        or
        = link_to 'Cancel', staff_student_path(@student)

Any ideas would be greatly appreciated.
Thanks!

I just updated HEAD code in master branch, which will print some useful debug info when parsing sexp node failed, could you try it, and tell me the outputs?

Here's the relevant output with that change:
./app/views/staff/notes/_form.html.haml
find error in file: ./app/views/staff/notes/_form.html.haml line: 87
find error in file: ./app/views/staff/notes/_form.html.haml line: method_add_arg
find error in file: ./app/views/staff/notes/_form.html.haml line:
find error in file: ./app/views/staff/notes/_form.html.haml line: 87
find error in file: ./app/views/staff/notes/_form.html.haml line: command
find error in file: ./app/views/staff/notes/_form.html.haml line: stmts_add
find error in file: ./app/views/staff/notes/_form.html.haml line: string_embexpr
find error in file: ./app/views/staff/notes/_form.html.haml line: @tstring_content
find error in file: ./app/views/staff/notes/_form.html.haml line: string_embexpr
find error in file: ./app/views/staff/notes/_form.html.haml line: @tstring_content
find error in file: ./app/views/staff/notes/_form.html.haml line: string_embexpr
find error in file: ./app/views/staff/notes/_form.html.haml line: @tstring_content
find error in file: ./app/views/staff/notes/_form.html.haml line: string_add
find error in file: ./app/views/staff/notes/_form.html.haml line: 44
find error in file: ./app/views/staff/notes/_form.html.haml line: binary
find error in file: ./app/views/staff/notes/_form.html.haml line: stmts_add
find error in file: ./app/views/staff/notes/_form.html.haml line: 19
find error in file: ./app/views/staff/notes/_form.html.haml line: method_add_block
find error in file: ./app/views/staff/notes/_form.html.haml line: assign
find error in file: ./app/views/staff/notes/_form.html.haml line: binary
find error in file: ./app/views/staff/notes/_form.html.haml line: stmts_add
(eval):23:in `throw': uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<ArgumentError: uncaught throw #<NoMethodError: undefined method `sexp_type' for nil:NilClass>>>>>>>>>>>>>>>>>>>>> (ArgumentError)
	from (eval):23:in `rescue in review'
	from (eval):2:in `review'
	from /Users/gee/.rvm/gems/ruby-1.9.3-p125@global/gems/rails_best_practices-1.8.0/lib/rails_best_practices/core_ext/sexp.rb:16:in `review'

Looks like the 'throw' you added should have been 'raise'.

Now I'm getting:
./app/views/staff/notes/_form.html.haml
find error in file: ./app/views/staff/notes/_form.html.haml line: 87
find error in file: ./app/views/staff/notes/_form.html.haml line: method_add_arg
find error in file: ./app/views/staff/notes/_form.html.haml line:
find error in file: ./app/views/staff/notes/_form.html.haml line: 87
find error in file: ./app/views/staff/notes/_form.html.haml line: command
find error in file: ./app/views/staff/notes/_form.html.haml line: stmts_add
find error in file: ./app/views/staff/notes/_form.html.haml line: string_embexpr
find error in file: ./app/views/staff/notes/_form.html.haml line: @tstring_content
find error in file: ./app/views/staff/notes/_form.html.haml line: string_embexpr
find error in file: ./app/views/staff/notes/_form.html.haml line: @tstring_content
find error in file: ./app/views/staff/notes/_form.html.haml line: string_embexpr
find error in file: ./app/views/staff/notes/_form.html.haml line: @tstring_content
find error in file: ./app/views/staff/notes/_form.html.haml line: string_add
find error in file: ./app/views/staff/notes/_form.html.haml line: 44
find error in file: ./app/views/staff/notes/_form.html.haml line: binary
find error in file: ./app/views/staff/notes/_form.html.haml line: stmts_add
find error in file: ./app/views/staff/notes/_form.html.haml line: 19
find error in file: ./app/views/staff/notes/_form.html.haml line: method_add_block
find error in file: ./app/views/staff/notes/_form.html.haml line: assign
find error in file: ./app/views/staff/notes/_form.html.haml line: binary
find error in file: ./app/views/staff/notes/_form.html.haml line: stmts_add
/Users/gee/.rvm/gems/ruby-1.9.3-p125@global/gems/rails_best_practices-1.8.0/lib/rails_best_practices/core/check.rb:327:in `mark_used': undefined method `sexp_type' for nil:NilClass (NoMethodError)

Same error as my first comment, more debugging stuff.
I'm not sure how to read the extra info you've added.

@mikegee my fault, raise for ruby and throw for java

@mikegee please try the HEAD code again, the issue should be fixed.

Indeed it is fixed.  The last line of the output is now:
Found 147 warnings.

But, that's my problem!  :-)
Thanks so much for your help!

my pleasure and thanks for helping me debug

btw, you can use http://railsbp.com to get online rails project code review service.


When file properties are empty, view-count is inserted in the wrong place.

When file properties are empty, view-count is inserted in the wrong place. like:

By the way, hopefully it will exclude some folders from updating viewcount, such as template files.
Thanks.

Hi @ku-zhen. Thanks for the feature request.
Would you mind explaining what you mean by in the wrong place? I do not understand from this screenshot.
Yes, excluded files would be good. Would it be helpful to have a setting that would allow you to exclude a list of folders? Template files as far as I understand don't have any special extension or format.

I have decided to rework the plugin. View count is now stored in an external file instead of the view-count property. All files are now tracked as well. The view count will appear in the status bar.


[GHSA-m5hf-m3r2-xq53] hutool-core was discovered to contain a stack overflow via NumberUtil.toBigDecimal method

Updates

Affected products

Comments
The new version 5.8.25 adds an additional assertion to prevent the stack overflow. I have tested the fix with the unit test provided in https://github.com/dromara/hutool/issues/3423.

Hi @henrikplate! Thank you so much for contributing to the GitHub Advisory Database. This database is free, open, and accessible to all, and it's people like you who make it great. Thanks for choosing to help others. We hope you send in more contributions in the future!


Setup interactive testing for windows

/area testing
Which functionality do you think we should add?
Currently, we don't have a working interactive testing framework on Windows. We need to enable the interactive testing for windows to increase the coverage of our tests.

We are using github.com/kr/pty to create a pseudo tty to test the interactive mod, as github.com/kr/pty doesn't support creation of pseudo Windows tty, the more actively maintained forked lib github.com/creack/pty also doesn't support windows pty,  there is PR to make support for it using ConPty. We can also try to use the ConPty directly for the test to create pty.