improvements to (i)starts_with and (i)ends_with performance

Which issue does this PR close?
Related to (but not closing) #6107.
Rationale for this change
Lots of context in #6107, this makes LIKE and ILIKE queries which are simply "starts with" and "ends with" significantly faster.
Running
cargo bench -p arrow --bench comparison_kernels -F test_utils -- like

Gives notably:

Test
Change

like_utf8 scalar ends with
-53.921%

like_utf8 scalar starts with
-53.883%

like_utf8view scalar ends with
-26.079%

like_utf8view scalar starts with
-24.864%

nlike_utf8 scalar ends with
-53.944%

nlike_utf8 scalar starts with
-53.610%

ilike_utf8 scalar starts with
-21.921%

nilike_utf8 scalar starts with
-22.727%

Full output
   Compiling arrow-string v52.1.0 (/Users/samuel/code/arrow-rs/arrow-string)
   Compiling arrow v52.1.0 (/Users/samuel/code/arrow-rs/arrow)
    Finished `bench` profile [optimized] target(s) in 4.13s
     Running benches/comparison_kernels.rs (target/release/deps/comparison_kernels-b61fe744923f27b6)
like_utf8 scalar equals time:   [144.33 µs 144.47 µs 144.69 µs]
                        change: [-0.2401% -0.0477% +0.1358%] (p = 0.63 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  4 (4.00%) high severe

like_utf8 scalar contains
                        time:   [195.98 µs 196.10 µs 196.23 µs]
                        change: [-0.2527% -0.0384% +0.1985%] (p = 0.75 > 0.05)
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

like_utf8 scalar ends with
                        time:   [66.456 µs 66.538 µs 66.628 µs]
                        change: [-53.995% -53.921% -53.817%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

like_utf8 scalar starts with
                        time:   [67.058 µs 67.093 µs 67.125 µs]
                        change: [-54.038% -53.883% -53.755%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

like_utf8 scalar complex
                        time:   [124.83 µs 124.89 µs 124.96 µs]
                        change: [-2.2102% -1.9432% -1.6682%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe

like_utf8view scalar equals
                        time:   [15.993 ms 16.022 ms 16.052 ms]
                        change: [-0.1739% +0.0490% +0.2773%] (p = 0.68 > 0.05)
                        No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking like_utf8view scalar contains: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 18.1s, or reduce sample count to 20.
like_utf8view scalar contains
                        time:   [179.98 ms 180.36 ms 180.76 ms]
                        change: [-0.9752% -0.6832% -0.3939%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 23 outliers among 100 measurements (23.00%)
  9 (9.00%) low mild
  7 (7.00%) high mild
  7 (7.00%) high severe

like_utf8view scalar ends with
                        time:   [21.398 ms 21.473 ms 21.552 ms]
                        change: [-26.453% -26.079% -25.713%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

like_utf8view scalar starts with
                        time:   [21.710 ms 21.790 ms 21.889 ms]
                        change: [-25.151% -24.864% -24.473%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

Benchmarking like_utf8view scalar complex: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 23.3s, or reduce sample count to 20.
like_utf8view scalar complex
                        time:   [231.47 ms 231.93 ms 232.49 ms]
                        change: [+0.6295% +0.8893% +1.1705%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

nlike_utf8 scalar equals
                        time:   [145.37 µs 145.53 µs 145.70 µs]
                        change: [-0.0855% +0.1646% +0.3920%] (p = 0.18 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

nlike_utf8 scalar contains
                        time:   [197.16 µs 197.41 µs 197.71 µs]
                        change: [-0.7353% -0.3336% +0.0168%] (p = 0.09 > 0.05)
                        No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

nlike_utf8 scalar ends with
                        time:   [67.028 µs 67.212 µs 67.443 µs]
                        change: [-54.194% -53.944% -53.720%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

nlike_utf8 scalar starts with
                        time:   [67.260 µs 67.327 µs 67.396 µs]
                        change: [-53.731% -53.610% -53.498%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

nlike_utf8 scalar complex
                        time:   [125.99 µs 126.10 µs 126.23 µs]
                        change: [-1.2656% -1.0550% -0.8713%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

ilike_utf8 scalar equals
                        time:   [103.50 µs 103.64 µs 103.79 µs]
                        change: [+2.2061% +2.5400% +2.8621%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

ilike_utf8 scalar contains
                        time:   [664.73 µs 665.66 µs 666.72 µs]
                        change: [+0.4162% +0.6823% +0.9311%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

ilike_utf8 scalar ends with
                        time:   [104.73 µs 104.87 µs 105.01 µs]
                        change: [-0.0506% +0.2329% +0.5070%] (p = 0.11 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild

ilike_utf8 scalar starts with
                        time:   [93.878 µs 94.077 µs 94.287 µs]
                        change: [-22.114% -21.921% -21.735%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

ilike_utf8 scalar complex
                        time:   [136.51 µs 136.59 µs 136.68 µs]
                        change: [-0.8583% -0.6439% -0.4482%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe

nilike_utf8 scalar equals
                        time:   [103.23 µs 103.56 µs 104.04 µs]
                        change: [+2.3182% +2.5182% +2.7169%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

nilike_utf8 scalar contains
                        time:   [664.85 µs 665.41 µs 665.99 µs]
                        change: [+0.9991% +1.1622% +1.3429%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

nilike_utf8 scalar ends with
                        time:   [104.22 µs 104.36 µs 104.50 µs]
                        change: [-0.0951% +0.1088% +0.3029%] (p = 0.31 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild

nilike_utf8 scalar starts with
                        time:   [94.197 µs 94.387 µs 94.575 µs]
                        change: [-23.029% -22.727% -22.441%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

nilike_utf8 scalar complex
                        time:   [136.32 µs 136.47 µs 136.63 µs]
                        change: [-2.8598% -2.4438% -2.0479%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

like_utf8_scalar_dyn dictionary[10] string[4])
                        time:   [29.504 µs 29.550 µs 29.603 µs]
                        change: [+1.1570% +1.4762% +1.8389%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

ilike_utf8_scalar_dyn dictionary[10] string[4])
                        time:   [29.404 µs 29.436 µs 29.471 µs]
                        change: [+0.1443% +0.5140% +0.8086%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

What changes are included in this PR?

new implementation of starts_with_ignore_ascii_case and ends_with_ignore_ascii_case, these showed significant improvements (~20%) over the previous implementations
new methods crate::predicate::starts_with and crate::predicate::ends_with that show a 2 or 3x improvement over str.starts_with and str.ends_with

Are there any user-facing changes?
Shouldn't be. I fuzzed all the implementations against the default methods here

Here is my next run
++ critcmp master starts_with-ends_with-improvements
group                                              master                                 starts_with-ends_with-improvements
-----                                              ------                                 ----------------------------------
ilike_utf8 scalar complex                          1.00    301.7±1.10µs        ? ?/sec    1.00    301.5±1.04µs        ? ?/sec
ilike_utf8 scalar contains                         1.00   1554.5±6.20µs        ? ?/sec    1.03   1595.8±4.49µs        ? ?/sec
ilike_utf8 scalar ends with                        1.00    218.5±0.39µs        ? ?/sec    1.13    247.1±4.53µs        ? ?/sec
ilike_utf8 scalar equals                           1.16    248.8±0.40µs        ? ?/sec    1.00    214.8±0.66µs        ? ?/sec
ilike_utf8 scalar starts with                      1.00    279.4±0.36µs        ? ?/sec    1.07    298.5±0.46µs        ? ?/sec
ilike_utf8_scalar_dyn dictionary[10] string[4])    1.00     88.1±0.14µs        ? ?/sec    1.00     88.2±0.18µs        ? ?/sec
like_utf8 scalar complex                           1.01    283.4±0.82µs        ? ?/sec    1.00    282.0±0.96µs        ? ?/sec
like_utf8 scalar contains                          1.00    347.6±0.52µs        ? ?/sec    1.00    347.4±0.76µs        ? ?/sec
like_utf8 scalar ends with                         1.41    219.4±0.55µs        ? ?/sec    1.00    155.3±2.99µs        ? ?/sec
like_utf8 scalar equals                            1.00    217.6±0.53µs        ? ?/sec    1.00    217.9±0.73µs        ? ?/sec
like_utf8 scalar starts with                       1.34    232.8±0.29µs        ? ?/sec    1.00    173.5±0.30µs        ? ?/sec
like_utf8_scalar_dyn dictionary[10] string[4])     1.00     88.1±0.17µs        ? ?/sec    1.00     88.1±0.14µs        ? ?/sec
like_utf8view scalar complex                       1.00    531.3±2.04ms        ? ?/sec    1.00    531.0±2.43ms        ? ?/sec
like_utf8view scalar contains                      1.00    378.6±0.36ms        ? ?/sec    1.01    380.6±1.38ms        ? ?/sec
like_utf8view scalar ends with                     1.13     59.6±0.24ms        ? ?/sec    1.00     52.7±0.23ms        ? ?/sec
like_utf8view scalar equals                        1.00     37.0±0.47ms        ? ?/sec    1.00     36.9±0.09ms        ? ?/sec
like_utf8view scalar starts with                   1.06     60.0±0.48ms        ? ?/sec    1.00     56.8±0.23ms        ? ?/sec
nilike_utf8 scalar complex                         1.00    301.9±1.19µs        ? ?/sec    1.00    300.4±2.48µs        ? ?/sec
nilike_utf8 scalar contains                        1.00   1551.3±4.82µs        ? ?/sec    1.03   1594.3±4.71µs        ? ?/sec
nilike_utf8 scalar ends with                       1.00    219.7±3.41µs        ? ?/sec    1.12    246.3±0.86µs        ? ?/sec
nilike_utf8 scalar equals                          1.16    248.8±0.53µs        ? ?/sec    1.00    214.5±0.33µs        ? ?/sec
nilike_utf8 scalar starts with                     1.00    279.6±1.02µs        ? ?/sec    1.07    298.4±0.46µs        ? ?/sec
nlike_utf8 scalar complex                          1.01    283.2±1.05µs        ? ?/sec    1.00    281.1±1.19µs        ? ?/sec
nlike_utf8 scalar contains                         1.00    347.5±0.63µs        ? ?/sec    1.00    347.2±0.49µs        ? ?/sec
nlike_utf8 scalar ends with                        1.42    219.7±1.25µs        ? ?/sec    1.00    154.8±0.24µs        ? ?/sec
nlike_utf8 scalar equals                           1.00    217.7±0.45µs        ? ?/sec    1.00    217.7±0.47µs        ? ?/sec
nlike_utf8 scalar starts with                      1.34    233.0±0.85µs        ? ?/sec    1.00    173.6±0.48µs        ? ?/sec

BTW I am running this on a c2-standard-8 GCP instance
Script
pushd ~/arrow-rs

#git remote add samuelcolvin https://github.com/samuelcolvin/arrow-rs.git
git fetch -p samuelcolvin
BENCH_COMMAND="cargo bench -p arrow --bench comparison_kernels -F test_utils"
BENCH_FILTER="like"
REPO_NAME="samuelcolvin"
BRANCH_NAME="starts_with-ends_with-improvements"

# remove old test runs
rm -rf target/criterion/

git checkout $BRANCH_NAME
git reset --hard "$REPO_NAME/$BRANCH_NAME"

# Run on test branch
$BENCH_COMMAND -- --save-baseline ${BRANCH_NAME} ${BENCH_FILTER}

# Run on master
MERGE_BASE=$(git merge-base HEAD apache/master)
echo "** Comparing to ${MERGE_BASE}"

git checkout ${MERGE_BASE}
$BENCH_COMMAND -- --save-baseline master  ${BENCH_FILTER}

critcmp master ${BRANCH_NAME}

popd

My reading of this is that (contrary to what I found) my "istarts_with" and
"iends_with" are slower that what was here before.
Very easy to revert, but we should just check the benchmarks are really
representative of the most common queries.
Samuel Colvin
On Sat, 27 Jul 2024, 12:55 Andrew Lamb, @.***> wrote:

Here is my next run
++ critcmp master starts_with-ends_with-improvements
group                                              master                                 starts_with-ends_with-improvements

ilike_utf8 scalar complex                          1.00    301.7±1.10µs        ? ?/sec    1.00    301.5±1.04µs        ? ?/sec
ilike_utf8 scalar contains                         1.00   1554.5±6.20µs        ? ?/sec    1.03   1595.8±4.49µs        ? ?/sec
ilike_utf8 scalar ends with                        1.00    218.5±0.39µs        ? ?/sec    1.13    247.1±4.53µs        ? ?/sec
ilike_utf8 scalar equals                           1.16    248.8±0.40µs        ? ?/sec    1.00    214.8±0.66µs        ? ?/sec
ilike_utf8 scalar starts with                      1.00    279.4±0.36µs        ? ?/sec    1.07    298.5±0.46µs        ? ?/sec
ilike_utf8_scalar_dyn dictionary[10] string[4])    1.00     88.1±0.14µs        ? ?/sec    1.00     88.2±0.18µs        ? ?/sec
like_utf8 scalar complex                           1.01    283.4±0.82µs        ? ?/sec    1.00    282.0±0.96µs        ? ?/sec
like_utf8 scalar contains                          1.00    347.6±0.52µs        ? ?/sec    1.00    347.4±0.76µs        ? ?/sec
like_utf8 scalar ends with                         1.41    219.4±0.55µs        ? ?/sec    1.00    155.3±2.99µs        ? ?/sec
like_utf8 scalar equals                            1.00    217.6±0.53µs        ? ?/sec    1.00    217.9±0.73µs        ? ?/sec
like_utf8 scalar starts with                       1.34    232.8±0.29µs        ? ?/sec    1.00    173.5±0.30µs        ? ?/sec
like_utf8_scalar_dyn dictionary[10] string[4])     1.00     88.1±0.17µs        ? ?/sec    1.00     88.1±0.14µs        ? ?/sec
like_utf8view scalar complex                       1.00    531.3±2.04ms        ? ?/sec    1.00    531.0±2.43ms        ? ?/sec
like_utf8view scalar contains                      1.00    378.6±0.36ms        ? ?/sec    1.01    380.6±1.38ms        ? ?/sec
like_utf8view scalar ends with                     1.13     59.6±0.24ms        ? ?/sec    1.00     52.7±0.23ms        ? ?/sec
like_utf8view scalar equals                        1.00     37.0±0.47ms        ? ?/sec    1.00     36.9±0.09ms        ? ?/sec
like_utf8view scalar starts with                   1.06     60.0±0.48ms        ? ?/sec    1.00     56.8±0.23ms        ? ?/sec
nilike_utf8 scalar complex                         1.00    301.9±1.19µs        ? ?/sec    1.00    300.4±2.48µs        ? ?/sec
nilike_utf8 scalar contains                        1.00   1551.3±4.82µs        ? ?/sec    1.03   1594.3±4.71µs        ? ?/sec
nilike_utf8 scalar ends with                       1.00    219.7±3.41µs        ? ?/sec    1.12    246.3±0.86µs        ? ?/sec
nilike_utf8 scalar equals                          1.16    248.8±0.53µs        ? ?/sec    1.00    214.5±0.33µs        ? ?/sec
nilike_utf8 scalar starts with                     1.00    279.6±1.02µs        ? ?/sec    1.07    298.4±0.46µs        ? ?/sec
nlike_utf8 scalar complex                          1.01    283.2±1.05µs        ? ?/sec    1.00    281.1±1.19µs        ? ?/sec
nlike_utf8 scalar contains                         1.00    347.5±0.63µs        ? ?/sec    1.00    347.2±0.49µs        ? ?/sec
nlike_utf8 scalar ends with                        1.42    219.7±1.25µs        ? ?/sec    1.00    154.8±0.24µs        ? ?/sec
nlike_utf8 scalar equals                           1.00    217.7±0.45µs        ? ?/sec    1.00    217.7±0.47µs        ? ?/sec
nlike_utf8 scalar starts with                      1.34    233.0±0.85µs        ? ?/sec    1.00    173.6±0.48µs        ? ?/sec
BTW I am running this on a c2-standard-8 GCP instance
Script
pushd ~/arrow-rs
#git remote add samuelcolvin https://github.com/samuelcolvin/arrow-rs.git
git fetch -p samuelcolvin
BENCH_COMMAND="cargo bench -p arrow --bench comparison_kernels -F test_utils"
BENCH_FILTER="like"
REPO_NAME="samuelcolvin"
BRANCH_NAME="starts_with-ends_with-improvements"
remove old test runs
rm -rf target/criterion/
git checkout $BRANCH_NAME
git reset --hard "$REPO_NAME/$BRANCH_NAME"
Run on test branch$BENCH_COMMAND -- --save-baseline ${BRANCH_NAME} ${BENCH_FILTER}
Run on master
MERGE_BASE=$(git merge-base HEAD apache/master)echo "** Comparing to ${MERGE_BASE}"
git checkout ${MERGE_BASE}$BENCH_COMMAND -- --save-baseline master  ${BENCH_FILTER}
critcmp master ${BRANCH_NAME}
popd
—
Reply to this email directly, view it on GitHub
https://github.com/apache/arrow-rs/pull/6118#issuecomment-2254113064,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA62GGP2BI2NTQKNKSNMXZDZON4BBAVCNFSM6AAAAABLPEFVR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJUGEYTGMBWGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

I think the behavior here is again related to the unrealistically short (4 character) haystack used in benchmarks as I explained  in https://github.com/apache/arrow-rs/pull/6128#issuecomment-2254513573.

With haystack length=4 (default)
group                               master                                 starts_with-ends_with-improvements
-----                               ------                                 ----------------------------------
ilike_utf8 scalar ends with         1.00    234.5±0.69µs        ? ?/sec    1.05    247.4±0.70µs        ? ?/sec
ilike_utf8 scalar starts with       1.00    251.4±0.39µs        ? ?/sec    1.19    298.0±1.11µs        ? ?/sec
like_utf8 scalar ends with          1.52    234.5±0.34µs        ? ?/sec    1.00    154.6±0.58µs        ? ?/sec
like_utf8 scalar starts with        1.22    253.7±0.44µs        ? ?/sec    1.00    207.8±2.90µs        ? ?/sec
like_utf8view scalar ends with      1.25     61.2±0.20ms        ? ?/sec    1.00     48.9±0.22ms        ? ?/sec
like_utf8view scalar starts with    1.07     58.1±0.17ms        ? ?/sec    1.00     54.3±0.15ms        ? ?/sec
nilike_utf8 scalar ends with        1.00    234.7±0.81µs        ? ?/sec    1.06    247.6±0.84µs        ? ?/sec
nilike_utf8 scalar starts with      1.00    251.4±0.34µs        ? ?/sec    1.19    298.2±1.02µs        ? ?/sec
nlike_utf8 scalar ends with         1.52    234.4±0.25µs        ? ?/sec    1.00    154.7±1.81µs        ? ?/sec
nlike_utf8 scalar starts with       1.22    253.8±0.72µs        ? ?/sec    1.00    207.3±0.55µs        ? ?/sec

With random haystack length=0..400
group                                     master                                 starts_with-ends_with-improvements
-----                                     ------                                 ----------------------------------
ilike_utf8 scalar ends with               1.10  1042.3±49.28µs        ? ?/sec    1.00   947.9±45.76µs        ? ?/sec
ilike_utf8 scalar starts with             1.02  1056.6±29.14µs        ? ?/sec    1.00  1036.7±54.46µs        ? ?/sec
like_utf8 scalar ends with                1.75   480.0±14.59µs        ? ?/sec    1.00   274.8±16.16µs        ? ?/sec
like_utf8 scalar starts with              1.39    499.2±6.69µs        ? ?/sec    1.00    358.8±5.57µs        ? ?/sec
like_utf8view scalar ends with            1.16     56.5±0.28ms        ? ?/sec    1.00     48.6±0.17ms        ? ?/sec
like_utf8view scalar starts with          1.08     57.5±0.17ms        ? ?/sec    1.00     53.3±0.24ms        ? ?/sec
nilike_utf8 scalar ends with              1.05   995.0±19.04µs        ? ?/sec    1.00   951.9±49.82µs        ? ?/sec
nilike_utf8 scalar starts with            1.00  1056.0±40.94µs        ? ?/sec    1.00  1050.8±34.07µs        ? ?/sec
nlike_utf8 scalar ends with               1.76    479.3±9.99µs        ? ?/sec    1.00   272.0±25.31µs        ? ?/sec
nlike_utf8 scalar starts with             1.36    499.1±5.47µs        ? ?/sec    1.00    367.9±8.76µs        ? ?/sec

(I also rebased to latest master, which I guess could have had an affect, although I doubt it)

Bechmarks after merging master and fixing conflicts:
+ critcmp master starts_with-ends_with-improvements
group                               master                                 starts_with-ends_with-improvements
-----                               ------                                 ----------------------------------
ilike_utf8 scalar ends with         1.06    584.7±2.31µs        ? ?/sec    1.00    553.6±6.07µs        ? ?/sec
ilike_utf8 scalar starts with       1.03    578.1±5.63µs        ? ?/sec    1.00    559.8±7.77µs        ? ?/sec
like_utf8 scalar ends with          1.62    150.1±2.55µs        ? ?/sec    1.00     92.9±1.10µs        ? ?/sec
like_utf8 scalar starts with        1.63    150.2±4.11µs        ? ?/sec    1.00     92.2±1.39µs        ? ?/sec
like_utf8view scalar ends with      1.36     29.3±0.32ms        ? ?/sec    1.00     21.6±0.28ms        ? ?/sec
like_utf8view scalar starts with    1.38     28.9±0.21ms        ? ?/sec    1.00     21.0±0.38ms        ? ?/sec
nilike_utf8 scalar ends with        1.07   593.0±28.32µs        ? ?/sec    1.00    552.4±4.85µs        ? ?/sec
nilike_utf8 scalar starts with      1.04    578.1±2.64µs        ? ?/sec    1.00    557.9±7.13µs        ? ?/sec
nlike_utf8 scalar ends with         1.62    150.9±4.47µs        ? ?/sec    1.00     93.2±1.68µs        ? ?/sec
nlike_utf8 scalar starts with       1.60    149.5±3.35µs        ? ?/sec    1.00     93.2±2.92µs        ? ?/sec

🚀


Release/1.7.5

I'm sure we talked about this, but do you remember where we landed on operators on non-literal nulls?  (Or, rather, the rationale behind not automagically checking for them unless there's an attribute making them valid.)  Was it just efficiency?  Or worries of it breaking things (maybe with actual examples in the standard library)?  I'm sure we had a rationale, but looking at the commit, it still seems like a basically good idea.
Are all the other return/exit/break type potential gotchas with temporarily also handled?  I hadn't thought of them before; durr.  Offhand, maybe the only special thing is that they should restore the original value when exiting, right?

I'm sure we talked about this, but do you remember where we landed on operators on non-literal nulls? (Or, rather, the rationale behind not automagically checking for them unless there's an attribute making them valid.) Was it just efficiency? Or worries of it breaking things (maybe with actual examples in the standard library)? I'm sure we had a rationale, but looking at the commit, it still seems like a basically good idea.

I wouldn't want to add automatic null checks for reasons of both efficiency and flexibility. Developers can make null-safe global operator methods and those are useful in nonstandardizable ways. As an example, Plasmacore's "Action" has a global operator method for += that allows this:
local actions = null : Action  # redundant null assignment for clarity
...
actions += ActionSubclass(...) # works when 'actions' is null.

We could add null checks only if an operator doesn't resolve to a global method etc., but to me that's premature optimization... let's wait until people are getting seriously tripped up by the lack of it, if ever.

Are all the other return/exit/break type potential gotchas with temporarily also handled? I hadn't thought of them before; durr. Offhand, maybe the only special thing is that they should restore the original value when exiting, right?

Yeah, all exits from temporarily are covered in the InjectCmdBeforeReturnVisitor it uses (Visitors.rogue).

Right, I think intentional null-safe usage was why I initially suggested the attribute to override, but I don't remember why that doesn't apply to non-literal cases also, especially since I think the combination of performing an operation on a null reference accidentally and not having handled the null case in the operation is a pretty easy mistake to make -- and potentially an exploitable one from a security standpoint, where catching it and terminating would be a big improvement.  We don't need to address this now, but I thought we should have some sort of record, and I still suspect it might be a good idea.
👍 on temporarily.


Increase number of outputs for Multi-Out version

Since we follow sfz rule, output paths are counted as stereo pair.
Sfizz currently has 16 outputs (mono), thus just 8 stereo outputs. Will be enough if we can have 16 stereo outs (32 mono).
Most use cases are for splitting drum mics outputs or drum kit-pieces, also percussion kits within single sfz file. They are often more than 8 microphones.
Possibly drum machine kit has more than 8 kit-pieces too.

I was told by one of Calfbox developers that they have no limits in outputs number, which are set from the sfz file (and cannot be changed once loaded).
I like the idea that outputs should not have a fixed number and UI might display as much are requested.


Add International Language Business Center (ILBC)

Added the official domain of International Language Business Center (ILBC).
ILBC is an IGCSE & A-Level school in Myanmar.

@shinthantkg First stage of verification passed, official domain is ilbc.edu.mm as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@shinthantkg Please provide us with an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course. Thank you!

I believe there isn't a page of the course listings, however, ILBC does offer IGCSE Computer Science, IGCSE ICT and A-Level IT.
As proof, here is the examination timetable for 10th grade that offers the IGCSE CS & ICT courses.

And yes, all three courses are two-semesters long, so that equates to a year.

A-Level Information Technology.

@shinthantkg Thank you for the clarification and your patience while waiting for a review and processing your request. Pull request merged.


Not working for Next.js typical file paths

In Next.js, you'll typically have folders that contain the following characters (, [, ..., ], )
This seems to be causing issues with the extension. As in a Next.js project structure, Local History picks up changes for files at the root of the project, but it does not pick up any changes contained as direct, or descendent children of folders containing the characters mentioned above.
Here is some documentation on the various types of folder structures.
I'm guessing that somewhere in internalDecodeFile this breaks expected behavior.
I will probably take a look at this sometime soon, but just in case you know off the top of your head how to fix this easily, I'm sending in the issue.

i havent tried that b4, but if the file save doesnt work then we should start with https://github.com/ctf0/local-history/blob/14b5a9302ae9b521e27405014e00fc0ce1a57e0b/src/Provider/Controller.ts#L173
and figure out why we cant replicate the file path structure under the .history folder.
of if i misunderstand your point, plz give me some steps to reproduce with.

Looking into this further, the files are saved in the .history directory correctly.
But for these nested files, when they are selected as the active editor file, the Local History tree view says No History, but it does show history for the file that is not nested under these folders.
I will setup a minimum replicatable repo in a bit here and send to you.

Alright, tested some more. It seems to be specifically any folders that contain a square bracket [someFolder] that cause the issue.
I created a test repo here. Pretty much anything outside a square bracket works.
https://github.com/wrgrant/bug-report
Interestingly, the files are being captured in the .history folder no matter what.
They just aren't being shown in the history browser.
Trying to open some of the nested ones will cause the browser to hang. But I can't always reproduce that.

ok i tested it and atm the files are being saved correctly but not listed in the browser.
am a bit busy atm but will see what i can do in the next days.
in the meantime, have u tried the builtin feature timeline ?


front page "Recipe information"

Bug Description
Don't know if it's due to the theme that I'm using or it's the same every where , see screenshot

The icons aren't vertically aligned with the text ...
Also there are no icon for the "difficulty" , and if you can add a "info bubble" to define what the icon represent ?
I had to check the html code to understand that the "cup" was in fact the author rating (well I guess, as it say only "rating")
And while I'm at it, at the bottom of the recipe, when a user give a rating

the stars should be on the same line
And from where this "Thanks for voting!" is coming ?
I searched in the plugin, and I couldn't found it.
In the language folder, I found the same text for "RATELAN_3" , but again, I couldn't found it in the plugin
And as I said, my site is in French and "RATELAN_3" is translated

Thanks, indeed some bugs.
What is the name of the theme you are using? So I can do some proper testing :)

I'm using a theme "custom build" ...
I used theme "Black&White" from Jimako and more or less revert the colors ...
I see that he classed it in "e107-old-themes", however it work perfectly for me. May be I had corrected some stuff, I can't remember
Basically, I just have changed the colors, added a background image and a little bit of transparency
ericc.eu
This site is temporary. It's free and with a very limited disk space ... I did this in emergency when my previous hosting company goes bankrupt without notice and suddenly my site was not reachable anymore .
I tried to give to cookbook a "full page" but I see the same visual problems


Replace coreferences within the text

Is there any build-in option to replace / resolve the identified coreference within the text.
Example in neuralcoref:
doc = nlp(u'Deepika has a dog. She loves him. The movie star has always been fond of animals')
doc._.coref_resolved
Output:
'Deepika has a dog. Deepika loves a dog. Deepika has always been fond of animals'

Not yet, although the resolve() function could be used to implement such a feature fairly easily:

print(doc[7])
him
print(doc..coref_chains.resolve(doc[7]))
[dog]
print(list(doc..coref_chains.resolve(doc[7])[0].subtree))
[a, dog]


Split.io feature flagging for backend

Follow up of #619 to add Split.io feature flagging to the backend
Tech Spec
Documentation

Link: https://www.split.io/product/languages/python-sdk/

Sample Code:
from splitio.exceptions import TimeoutException
factory = get_factory('YOUR_API_KEY')
try:
    factory.block_until_ready(5) # wait up to 5 seconds
except TimeoutException:
    # Now the user can choose whether to abort the whole execution, or just keep going
    # without a ready client, which if configured properly, should become ready at some point.
    pass
split = factory.client() ```

1: pip install 'splitio_client[redis,cpphash]==9.1.3'
2: Instantiate the SDK client with Redis enabled

Our backend does not use Redis - look into alternative option

For future work, we would need to use Multi-process mode so that the API is a lambda and can have multiple different executions happening at a time, and Redis would be the only way to share the feature flag calls

Action Item 1: Create a future work ticket to capture https://github.com/chanzuckerberg/napari-hub/issues/692#issuecomment-1358240768

Action Item 2:  Follow step 1 and step 2 in https://github.com/chanzuckerberg/napari-hub/issues/692#issuecomment-1357948009 to set up split.io for backend

Action Item 3: Follow step 3 in https://github.com/chanzuckerberg/napari-hub/issues/692#issuecomment-1357948009 to start using the SDK for our use case(s)

https://github.com/chanzuckerberg/napari-hub/issues/810 has been created to capture Setup split.io python client in napari hub backend

https://github.com/chanzuckerberg/napari-hub/issues/811 has been created to capture Validate feature flagging works by evaluating treatments for an API endpoint

@klai95 I'm going to archive this ticket from the board as you've created the follow-up tickets needed!

@richaagarwal Sounds good!


[cart] document instrumentation

Adds documentation for cart service, focused on instrumentation code.

@puckpuck what are your thoughts on adding language to the file names / titles?
I feel like a user would prefer to see a language and go directly there rather than consult our architecture for direction

I feel like a user would prefer to see a language and go directly there rather than consult our architecture for direction

Seems like a great idea! The language could also be mentioned in the top section of each doc, for example, next to the source code link.

@puckpuck what are your thoughts on adding language to the file names / titles?
I feel like a user would prefer to see a language and go directly there rather than consult our architecture for direction

I think this is a great idea.
<servicename>_<language>.md make sense for the file names?
I will also add it to the title for each one also.  I need to make a few small changes to this PR so will do that here, and submit another one to clean up all the other docs we wrote thus far.

Awesome!

LGTM. Please do replace auto-instrumentation with instrumentation library.

We'll want to update this in all service docs but can wait until we finish adding them


Can I add custom js?

Like above. Reason? For example for adding cookie consent.

Hi @lxndrblz ! Thank you so much for your work and engagement! I tested new changes and it seems to work with scripts which are stored in assets/js/ folder, but when I try to add external scripts, it doesn't work:
customJs = [          "https://cdn.jsdelivr.net/npm/cookie-bar/cookiebar-latest.min.js?always=1"     ]
I checked it on Chrome and Firefox. There are no errors in the console.

@PiotrWachulec  That's correct. Hugo's resources.Get function can only load local resources and process (fingerprint, minify) them. I've chosen it to be this way as it allows me to keep everything in one repo and have a better control over the security.
You won't be able to see an error in your browser's console as these scripts won't be included by Hugo while building your site. Indeed, Hugo should show multiple errors when trying to include the link in your customJs.

@lxndrblz thanks for your response. What is interesting, when I add a line from the issue description, I don't get any errors in Hugo, all compile successfully.
So, could you give me any advice, how can I add external scripts to Hugo? Thanks in advance!

@PiotrWachulec  I can't quite believe that you are not getting any errors, but let's come back to your original question.
The way the cookie script works is a bit unique, as it can't just be downloaded and included locally because it's heavily depending on the parametrization. If you want to include that cookiebar into your site and can only be done by manually adding it to the head or footer partial, but be aware of any merging conflicts if you are using this theme as a submodule. If you have cloned it, it obviously wouldn't matter.
I've seen that some other templates create an dummy partial, which is included and doesn't change, but can be overwritten by the user without breaking anything. That might also be an option to consider.

You're right, I got an error - but not locally, I got below error in my CI&CD pipeline:
ERROR 2020/07/19 11:52:59 render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "D:\a\1\s\themes\anatole\layouts\partials\head.html:31:32": execute of template failed: template: partials/head.html:31:32: executing "partials/head.html" at <resources.Get>: error calling Get: CreateFile D:\a\1\s\assets\https:\cdn.jsdelivr.net\npm\cookie-bar\cookiebar-latest.min.js?always=1: The filename, directory name, or volume label syntax is incorrect.
ERROR 2020/07/19 11:52:59 render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "D:\a\1\s\themes\anatole\layouts\partials\head.html:31:32": execute of template failed: template: partials/head.html:31:32: executing "partials/head.html" at <resources.Get>: error calling Get: CreateFile D:\a\1\s\assets\https:\cdn.jsdelivr.net\npm\cookie-bar\cookiebar-latest.min.js?always=1: The filename, directory name, or volume label syntax is incorrect.
ERROR 2020/07/19 11:52:59 render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "D:\a\1\s\themes\anatole\layouts\partials\head.html:31:32": execute of template failed: template: partials/head.html:31:32: executing "partials/head.html" at <resources.Get>: error calling Get: CreateFile D:\a\1\s\assets\https:\cdn.jsdelivr.net\npm\cookie-bar\cookiebar-latest.min.js?always=1: The filename, directory name, or volume label syntax is incorrect.
ERROR 2020/07/19 11:52:59 render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "D:\a\1\s\themes\anatole\layouts\partials\head.html:31:32": execute of template failed: template: partials/head.html:31:32: executing "partials/head.html" at <resources.Get>: error calling Get: CreateFile D:\a\1\s\assets\https:\cdn.jsdelivr.net\npm\cookie-bar\cookiebar-latest.min.js?always=1: The filename, directory name, or volume label syntax is incorrect.
Total in 317 ms
Error: Error building site: failed to render pages: render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "D:\a\1\s\themes\anatole\layouts\partials\head.html:31:32": execute of template failed: template: partials/head.html:31:32: executing "partials/head.html" at <resources.Get>: error calling Get: CreateFile D:\a\1\s\assets\https:\cdn.jsdelivr.net\npm\cookie-bar\cookiebar-latest.min.js?always=1: The filename, directory name, or volume label syntax is incorrect.

Thanks for your response! I'm using your theme as a submodule, so probably this way is not good. I think it's worthy to consider, because things like cookie consent is required by EU law.
In the other theme which I'm using on a different website, it is solved by:
{{ range .Site.Params.custom_js }}
      <script src="{{ . | relURL }}"></script>
{{ end }}

but as I understood it isn't the way which you are interested in.

@lxndrblz you are amazing! I really, really appreciate your engagement in solving my problem! Thanks a lot for your time and great work!


通用 我把截图出来了，请看下面的截图,第三张图显示丢失参数 我把截图出来了，请看下面的截图,第三张图显示丢失参数 I have the screenshot out. Please see the screenshot below. The third one shows the missing parameters

@hpyeah ok what exactly is the issue here?

No follow up, closing.


Review

It seems to re-run the code every time you change the text, rather than waiting until you press the play button. Is that intended?
I'm curious what this is for?
https://github.com/browneyedsoul/RemNote-RunJS/blob/636ddf241d918152d059b2c5fd21f1af39aaebdf/src/widgets/index.tsx#L53C1-L56

①  Yes, I intended to use it to track the latest state before pressing the button. I'm hoping to find a more convenient way to do this without the procedure, but for now it seems necessary.
②  Without the codes, the code block cannot be properly created due to delimiter issues in RemNote that may differ from other editors.
Thank you for the review! Please let me know if any changes need to be made to the codes :)

ok sounds good, I approved it :)

Thx James! 😄


Agent produces DDS error

Agent produces DDS error + mbed board does not appear in node list

Hardware description: ST B-L475E-IOT01A Discovery kit running the publisher example connected to a Dell Inspiron 13 Laptop running Ubuntu 20.04.
RTOS: Mbed OS with CLI2
Installation type: as seen here also tried with this agent and this one.
Version or commit hash: ROS2 foxy, hash -> f702c68500cd48d506c36016bf209869cbae8c80

Steps to reproduce the issue
1.Install ROS2 on your system (Ubuntu 20.04) by following these steps.
2. Install Mbed CLI2, the project dependencies and compile this example onto the board.
3.  Open a terminal, compile (using colcon) and run this Python listener which subscribes to "mbed_int32_publisher".
4. Open another terminal, run the agent sudo ./MicroXRCEAgent serial --dev /dev/ttyACM0 -v6
Actual behavior
DDS error:

Expected behavior
From the Portenta example
By simply using the portenta instead and changing the topic name to "micro_ros_arduino_node_publisher" on the listener script we get the correct output on the top right terminal:

Additional information
I tried with both the Micro-XRCE agent (as seen above) and with the micro-ros-agent used as seen here but with "snapd:st-linkv21" instead of "snapd:pico.

Hello @spartacoos, Which agent branch are you using for the Micro-XRCE agent?
The snap agent version needs to be updated to fix this problem, could you try with the dockerized version?
docker run -it --rm --net=host -v /dev:/dev -v /dev/shm:/dev/shm --privileged microros/micro-ros-agent:foxy serial --dev /dev/ttyACM0 -v6

Hi @Acuadros95, I just tried with your approach and this makes it so that on the agent side there are no dds errors. However, the listener still can't receive the messages. Oddly enough, If I run ros2 node list I get an empty output. If I run ros2 topic list I only get "
/parameter_events
/rosout

but if I try to publish to it by running ros2 topic pub /mbed_int32_publisher std_msgs/msg/Int32 "data: 1" this works and the listener publishes I heard "1". Any idea what the issue could be here? It seems to me like the listener and publisher are not being discovered.

Also, if I try to run ros2 topic pub /mbed_int32_publisher std_msgs/msg/Int32 "data: 1" while the agent is running this makes it so that the listener doesn't output anything. If I try when the agent is not running then it works.

I can't replicate that issue on my computer, looks like a communication problem with the dockerized agent.
You can also try to recompile the agent, making sure you are using the latest version and then running it like on your first post: sudo ./MicroXRCEAgent serial --dev /dev/ttyACM0 -v6
Just make sure you build the latest commit from the foxy branch.

Fantastic! Thank you @Acuadros95, that seems to have worked!


[runner integration - 1] Add support for test report generation

Adds the initial format and integration point for direct test runner integration.
See

https://github.com/trunk-io/analytics-cli/pull/254
https://github.com/trunk-io/analytics-cli/pull/255

/trunk merge


api: checkpoint api optimization

Signed-off-by: Ace-Tang<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how you did it
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

LGTM if the CIs are all green.

I do not think the CI failure is related to the code change
warning: no packages being tested depend on github.com/alibaba/pouch/apis/filters warning: no packages being tested depend on github.com/alibaba/pouch/credential warning: no packages being tested depend on github.com/alibaba/pouch/daemon/events warning: no packages being tested depend on github.com/alibaba/pouch/pkg/term install install bin/pouch /usr/local/bin install bin/pouchd /usr/local/bin download_dependencies hack/install/install_ci_related.sh instsall CI related tools... package golang.org/x/tools/cover: unrecognized import path "golang.org/x/tools/cover" (https fetch: Get https://golang.org/x/tools/cover?go-get=1: dial tcp <IP_ADDRESS>:443: i/o timeout)


get versionName of osmdroid library programmatically

Issue Type
[x] Question
Description and/or steps/code to reproduce the problem
Is there an easy way to retrieve the current versionName (e.g. "6.1.2") of the osmdroid library? I tried the following, but in the last step, an exception is thrown.
String versionName = "?";	// default, unknown

MapView mapView    = findViewById(R.id.mapview);
Class mapClass     = mapView.getClass();
Package mapPackage = mapClass.getPackage();
String packageName = mapPackage.getName();    // = "org.osmdroid.views"
PackageManager pManager = this.getPackageManager();

try
{	// the next line throws NameNotFoundException
	PackageInfo pInfo = pManager.getPackageInfo(packageName, 0);
	versionName = pInfo.versionName;
}
catch (PackageManager.NameNotFoundException e)
{
	e.printStackTrace();
}

Perhaps, I'm thinking too complicated, but an easy way - something like
String versionName = mapView.getVersionName();
doesn't seem to exist. Any ideas appreciated.
Environment
Huawei P10 Lite, Android 8.0.0, SDK_INT 26 ("Oreo")
Version of osmdroid the issue relates to:
6.1.2

@spyhunter99
@mat8854
@pasniak
@linquize
Does anybody of have an idea?

There currently isn't, but i can add it..

@spyhunter99
That would be great!

any other properties you want in it?

@mikebravoyes Oh, I tried the code snippet a few days ago out of curiosity. I got the same NameNotFoundException exception. First I thought it is related to Proguard / R8 enabled in a release build. But then the Exception would be ClassNotFoundException.

@spyhunter99
No! For me just the String versionName (e.g. "6.1.6") would be perfect.

org.osmdroid.OsmdroidBuildInfo.VERSION  returns 6.1.7-SNAPSHOT or whatever the current build version is.

@spyhunter99
Thanks for your effort!
But what is the correct entry in "build.gradle" now? Until yesterday it was:
dependencies {
    implementation group: 'org.osmdroid', name: 'osmdroid-android', version: '6.1.6'
}

I tried '6.1.7' and '6.1.7-SNAPSHOT', but none of them was working.
Error message in the Build Tab:

ERROR: Failed to resolve: org.osmdroid:osmdroid-android:6.1.7
Show in Project Structure dialog
Affected Modules: app


Min py vers

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.
Here's what I've got...
For recipe:

requirements: host: python >= 3.7 should not contain a space between relational operator and the version, i.e. python >=3.7
requirements: run: python >= 3.7 should not contain a space between relational operator and the version, i.e. python >=3.7

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@jrollins, can you add a call to ndschans --help in the test/commands section, just to sanity check that script?

@jrollins, the additions look fine, can you add a call to ndschans --help in the test/commands section, just to sanity check that script?

added


It would be great if bignumber.js supported buffers.

It would be great if bignumber.js could support reading a number from a buffer and similarly returning a buffer.

More detail required.
And I would be reluctant to lose ECMAScript 3 compatibility.


CASSANDRA-18676: Catch negative deletion time early and treat AssertionError as corruption

Thanks for sending a pull request! Here are some tips if you're new here:

Ensure you have added or run the appropriate tests for your PR.
Be sure to keep the PR description updated to reflect all changes.
Write your PR title to summarize what this PR proposes.
If possible, provide a concise example to reproduce the issue for a faster review.
Read our contributor guidelines
If you're making a documentation change, see our guide to documentation contribution

Commit messages should follow the following format:
<One sentence description, usually Jira title or CHANGES.txt summary>

<Optional lengthier description (context on patch)>

patch by <Authors>; reviewed by <Reviewers> for CASSANDRA-#####

Co-authored-by: Name1 <email1>
Co-authored-by: Name2 <email2>

The Cassandra Jira

The code is ready for review. Test run before rebasing here.

The new generated CircleCI config does not have repeated tests. I guess something went wrong with the script in the JDK 8 drop.
The pre-rebase run above did do the repeated test.

+1

I just noticed your repeat is only 500 times, which the test used to pass. If the missing circle repeat in trunk comes back I'd suggest going higher than that.

Latest run here: https://app.circleci.com/pipelines/github/blambov/cassandra?branch=CASSANDRA-18676
LGTM +1

Committed as a565711056c859398d0b26081b46e71d2076de1d


update and expand girder utilities and workflows code and docs

1- add documentation for using girder API, 2- add a few girder convenience utilities, 3- update workflows. [TODO] - make sure utils test and workflows tests follow new standard

passes all checks now!


[ACM] Support structured repository

Currently our ACM story doesn't use a structured repository. The main reason was because
we would need to refactor the manifests; e.g. put ClusterScoped resources in one directory and namespace resources in a different directory.
Could we reorganize the files using a kustomize function?
Using a flat repository will get pretty unmanageable.
Can we use subdirectories with an unstructured repositories?

Fixed by #109


PTCD-255 Keep Historical Record of FATP Exporter

Added Azure blob storage export for bulk providers endpoint

It's worth noting that by uploading the data export to Azure blob as part of a synchronous HTTP request will increase the request duration by around 30 seconds.


This should fix #121.

I've updated some tests, it works for me now.

@prudnikov thanks for the PR! Can you update it now that #86 is merged? cc @jaitaiwan

@prudnikov and squash the commits too please.

Does it make sense to merge it after 10 years? If no, could you please delete this PR (I have no permission)? It still hangs out on my Pull Requests page for no reason.


Handle absent sender_short_name in zulip-hook

Zulip private messages to triagebot recently started failing with "Failure! Third party responded with 400".

In my testing it looks like Zulip is no longer always sending a sender_short_name in the outgoing hook request. Possibly related to https://github.com/zulip/zulip/commit/b375581f58447ee49e721b80a2dd462fdf318e87.
The payload getting posted to /zulip-hook looks like:
{
  "data": "ping",
  "message": {
    "id": 205745202,
    "sender_id": 327146,
    "content": "ping",
    "recipient_id": 522016,
    "timestamp":<PHONE_NUMBER>,
    "client": "website",
    "subject": "",
    "topic_links": [],
    "rendered_content": "
ping
",
    "is_me_message": false,
    "reactions": [],
    "submessages": [],
    "sender_full_name": "David Tolnay",
    "sender_email"<EMAIL_ADDRESS>    "sender_realm_str": "dtolnay",
    "display_recipient": [
      {
        "email"<EMAIL_ADDRESS>        "full_name": "David Tolnay",
        "id": 327146,
        "is_mirror_dummy": false
      },
      {
        "id": 327153,
        "email"<EMAIL_ADDRESS>        "full_name": "dtolnay-bot",
        "is_mirror_dummy": false
      }
    ],
    "type": "private",
    "avatar_url": "https://secure.gravatar.com/avatar/cb99289473f6393b89474785f2d294d1?d=identicon&version=1",
    "content_type": "text/x-markdown"
  },
  "bot_email"<EMAIL_ADDRESS>  "token": "C5aBFtWr7HIMrWzj3dF7t8LBs85SzbGZ",
  "trigger": "private_message"
}

and Triagebot currently fails to deserialize that.
HTTP/1.1 400 Bad Request
x-request-id: f18e7a64-c31a-474f-9ed5-0a5bc80a6b93
content-length: 87
date: Sun, 02 Aug 2020 21:40:58 GMT

Did not send valid JSON request: missing field `sender_short_name` at line 1 column 786

Thanks! I just got pinged earlier today, I think with something like this.

In my testing it looks like Zulip is no longer always sending a sender_short_name in the outgoing hook request. Possibly related to zulip/zulip@b375581.

Sorry for reporting a leak :)
https://blog.zulip.com/2020/07/30/zulip-server-3-1-release/

Removed unused short_name field from the User model.  This field had no purpose and could leak the local part of email addresses when email address visibility was restricted.


Option to avoid npm dependency

Hi, this is a great plugin. Thanks for making it!
One issue I ran into was running GoInstallBinaries failed due to the dependency on npm and not having npm installed locally.
It would be great if there was some way to configure the plugin to avoid the dependency.
My workaround for now is to manually delete this line.

The JSON generation feature is powered by quicktype and it's written in Node. So npm, pnpm or yarn is required to install that tool.
If you intend to do so, GoQuickType will fail. I think it okay to add an option to disable installation of certain dependency though.


Update rn-fetch-blob and CBA support for iOS

Summary
The current react-native-fetch-blob library is no longer maintained and rn-fetch-blob is now active, we ended up upgrading as we were getting a lot of sentry reports for RNFetchBlob.
The updated rn-fetch-blob is still a fork that includes the changes for Certificate Base Authentication support.
Changes:

Updated Android target SDK to be 26 instead of 23 new Playstore Requirement
Changed the way emoji's are requested so a certificate can be used
Fix uploads and profile picture upload to include the client certificate if one is available
Cleared the preference for the certificate before connecting to a server in the select server screen
Fix network connection detection by using the certificate if one is available when pinging the server
Removed the local config file for ExperimentalClientSideCertEnable so it no longer requires a custom build (the library will emit an event if a cert is needed)

Ticket Link
https://mattermost.atlassian.net/browse/MM-11320
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] Added or updated unit tests (required for all new features)

Device Information
This PR was tested on: iOS and Android

We may be too late to have this for 1.12 but that would be awesome if we do

If Nginx is configured so it requires a client cert the app will prompt for one regardless of the server configuration. It is done that way cause there is no way to get the server config without a cert.
Then the server config is applied to allow the login


Improving Caching Headers & Hit Counters

Ref: #404
Based on https://github.com/PUGX/badge-poser/pull/386

Codecov Report

Merging #407 into master will decrease coverage by 1.18%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master     #407      +/-   ##
============================================
- Coverage     86.40%   85.21%   -1.19%
+ Complexity      269      242      -27
============================================
  Files            43       39       -4
  Lines           787      717      -70
============================================
- Hits            680      611      -69
+ Misses          107      106       -1

Impacted Files
Coverage Δ
Complexity Δ

src/Controller/Badge/CircleCiController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/ComposerLockController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/DependentsController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/DownloadsController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/GitAttributesController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/LicenseController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/SuggestersController.php
87.50% <100.00%> (+4.16%)
2.00 <0.00> (ø)

src/Controller/Badge/VersionController.php
88.88% <100.00%> (+3.17%)
2.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b164019...fccdd35. Read the comment docs.

@fabiocicerchia please resolve conflicts!

I meant why this specific / magic value .. 512, 1024 would be good too,
right?
Anyway, a decent number of max conn is not a bad thing, normally I use 2048
or 4096 as a starting value.
Il giorno mar 20 ott 2020 alle ore 13:23 Fabio Cicerchia <
<EMAIL_ADDRESS>ha scritto:

@fabiocicerchia commented on this pull request.
In docker-compose.yml
https://github.com/PUGX/badge-poser/pull/407#discussion_r508420264:

 ports:

   - 6379:6379

sysctls:

 net.core.somaxconn: 511

it's to suppress a warning that it's raised by redis when running on
docker-compose.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/PUGX/badge-poser/pull/407#discussion_r508420264, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AALAHDDT5TZYMHHJYQKAHI3SLVXLFANCNFSM4SWOX5ZQ
.

--
Emiliano Gabrielli