introduce the roaring multiop in milli

Fix #582

Current benchmark.
From what I understand, we’re missing two important functions.
First, we need to implement a try_xversion of all the operators because, without it, we need to collect all the roaring bitmap in a Result<Vec<_>> that we can then unwrap before calling the op. That’s already pretty bad.
But it becomes worse when we need to merge roaring bitmaps lazily because there is too much of it, and we don’t want to collect everything in place.
Thus I think we need something else that try to not collect all the roaring bitmap if the iterator is unsized for example 🤔
 % ./compare.sh indexing_test-roaring-multiop_0c74cbf4.json indexing_main_ebddfdb9.json
group                                                                     indexing_main_ebddfdb9                 indexing_test-roaring-multiop_0c74cbf4
-----                                                                     ----------------------                 --------------------------------------
indexing/-geo-delete-facetedNumber-facetedGeo-searchable-                 1.16      2.2±1.22ms        ? ?/sec    1.00  1870.0±346.85µs        ? ?/sec
indexing/-movies-delete-facetedString-facetedNumber-searchable-           1.20     10.9±5.34ms        ? ?/sec    1.00      9.1±2.84ms        ? ?/sec
indexing/-movies-delete-facetedString-facetedNumber-searchable-nested-    1.00     14.2±4.58ms        ? ?/sec    1.14     16.2±8.05ms        ? ?/sec
indexing/-songs-delete-facetedString-facetedNumber-searchable-            1.00    53.5±10.97ms        ? ?/sec    1.19    63.4±17.39ms        ? ?/sec
indexing/-wiki-delete-searchable-                                         1.05   313.7±11.20ms        ? ?/sec    1.00   297.4±16.71ms        ? ?/sec
indexing/Indexing geo_point                                               1.05      61.2±0.60s        ? ?/sec    1.00      58.2±0.58s        ? ?/sec
indexing/Indexing movies in three batches                                 1.00      19.3±0.17s        ? ?/sec    1.00      19.3±0.10s        ? ?/sec
indexing/Indexing movies with default settings                            1.01      18.8±0.14s        ? ?/sec    1.00      18.6±0.15s        ? ?/sec
indexing/Indexing nested movies with default settings                     1.00      25.8±0.17s        ? ?/sec    1.00      25.7±0.18s        ? ?/sec
indexing/Indexing nested movies without any facets                        1.00      24.7±0.13s        ? ?/sec    1.00      24.6±0.14s        ? ?/sec
indexing/Indexing songs in three batches with default settings            1.01      66.4±0.64s        ? ?/sec    1.00      65.5±0.72s        ? ?/sec
indexing/Indexing songs with default settings                             1.07      58.9±1.27s        ? ?/sec    1.00      55.0±1.56s        ? ?/sec
indexing/Indexing songs without any facets                                1.07      54.0±0.87s        ? ?/sec    1.00      50.6±1.17s        ? ?/sec
indexing/Indexing songs without faceted numbers                           1.08      58.1±1.17s        ? ?/sec    1.00      53.6±1.21s        ? ?/sec
indexing/Indexing wiki                                                    1.03   1067.3±13.56s        ? ?/sec    1.00   1036.9±21.26s        ? ?/sec
indexing/Indexing wiki in three batches                                   1.00    1191.5±9.27s        ? ?/sec    1.00   1195.0±10.50s        ? ?/sec
indexing/Reindexing geo_point                                             1.06      67.9±0.41s        ? ?/sec    1.00      64.0±0.85s        ? ?/sec
indexing/Reindexing movies with default settings                          1.01      18.9±0.19s        ? ?/sec    1.00      18.7±0.11s        ? ?/sec
indexing/Reindexing songs with default settings                           1.08      61.9±0.71s        ? ?/sec    1.00      57.3±1.13s        ? ?/sec
indexing/Reindexing wiki                                                  1.02   1770.6±32.30s        ? ?/sec    1.00   1736.7±20.85s        ? ?/sec

With a new batch of optimisations the indexing parts is now faster than on main;
% ./compare.sh indexing_main_a79ff8a1.json indexing_test-roaring-multiop_36e27e21.json
group                                                                     indexing_main_a79ff8a1                 indexing_test-roaring-multiop_36e27e21
-----                                                                     ----------------------                 --------------------------------------
indexing/-geo-delete-facetedNumber-facetedGeo-searchable-                 1.14    44.6±13.36ms        ? ?/sec    1.00     39.1±5.16ms        ? ?/sec
indexing/-movies-delete-facetedString-facetedNumber-searchable-           1.00      9.7±3.06ms        ? ?/sec    1.12     10.9±3.61ms        ? ?/sec
indexing/-movies-delete-facetedString-facetedNumber-searchable-nested-    1.06     14.4±3.87ms        ? ?/sec    1.00     13.6±4.24ms        ? ?/sec
indexing/-songs-delete-facetedString-facetedNumber-searchable-            1.05    52.8±15.61ms        ? ?/sec    1.00    50.1±14.69ms        ? ?/sec
indexing/-wiki-delete-searchable-                                         1.08   287.8±10.67ms        ? ?/sec    1.00   266.7±11.59ms        ? ?/sec
indexing/Indexing geo_point                                               1.04      47.6±0.14s        ? ?/sec    1.00      45.9±0.62s        ? ?/sec
indexing/Indexing movies in three batches                                 1.04      13.8±0.39s        ? ?/sec    1.00      13.3±0.14s        ? ?/sec
indexing/Indexing movies with default settings                            1.06      11.3±0.22s        ? ?/sec    1.00      10.7±0.09s        ? ?/sec
indexing/Indexing nested movies with default settings                     1.09       8.4±0.16s        ? ?/sec    1.00       7.7±0.14s        ? ?/sec
indexing/Indexing nested movies without any facets                        1.08       7.9±0.20s        ? ?/sec    1.00       7.4±0.34s        ? ?/sec
indexing/Indexing songs in three batches with default settings            1.00      47.6±1.06s        ? ?/sec    1.01      48.2±1.20s        ? ?/sec
indexing/Indexing songs with default settings                             1.06      45.9±1.28s        ? ?/sec    1.00      43.2±1.39s        ? ?/sec
indexing/Indexing songs without any facets                                1.05      43.4±0.92s        ? ?/sec    1.00      41.3±1.26s        ? ?/sec
indexing/Indexing songs without faceted numbers                           1.07      45.3±1.22s        ? ?/sec    1.00      42.5±0.85s        ? ?/sec
indexing/Indexing wiki                                                    1.02     878.2±9.30s        ? ?/sec    1.00    863.6±21.88s        ? ?/sec
indexing/Indexing wiki in three batches                                   1.00     936.9±6.64s        ? ?/sec    1.01     945.4±5.52s        ? ?/sec
indexing/Reindexing geo_point                                             1.02      16.1±0.25s        ? ?/sec    1.00      15.7±0.26s        ? ?/sec
indexing/Reindexing movies with default settings                          1.22   323.1±21.42ms        ? ?/sec    1.00   265.3±15.75ms        ? ?/sec
indexing/Reindexing songs with default settings                           1.01       4.3±0.13s        ? ?/sec    1.00       4.3±0.11s        ? ?/sec
indexing/Reindexing wiki                                                  1.00    1470.2±8.37s        ? ?/sec    1.01   1492.0±25.25s        ? ?/sec

That's cool. The search on wiki (thus, when there is a lot of words?) also improved;
TODO

Whatever, the real issue is that the search on songs lost a lot of performances on a lot of benchmarks;
% ./compare.sh search_songs_main_a79ff8a1.json search_songs_test-roaring-multiop_36e27e21.json
group                                                                                search_songs_main_a79ff8a1             search_songs_test-roaring-multiop_36e27e21
-----                                                                                --------------------------             ------------------------------------------
smol-songs.csv: asc + default/Notstandskomitee                                                           1.06      3.1±0.65ms        ? ?/sec    1.00      3.0±0.01ms        ? ?/sec
smol-songs.csv: asc + default/charles                                                                    1.00      2.2±0.01ms        ? ?/sec    1.23      2.7±0.01ms        ? ?/sec
smol-songs.csv: asc + default/charles mingus                                                             1.00      3.1±0.01ms        ? ?/sec    1.35      4.2±0.02ms        ? ?/sec
smol-songs.csv: asc + default/david                                                                      1.00      2.9±0.01ms        ? ?/sec    1.18      3.4±0.03ms        ? ?/sec
smol-songs.csv: asc + default/david bowie                                                                1.00      4.6±0.55ms        ? ?/sec    1.22      5.6±0.02ms        ? ?/sec
smol-songs.csv: asc + default/john                                                                       1.00      3.1±0.01ms        ? ?/sec    1.20      3.8±0.01ms        ? ?/sec
smol-songs.csv: asc + default/marcus miller                                                              1.00      5.0±0.02ms        ? ?/sec    1.21      6.0±0.02ms        ? ?/sec
smol-songs.csv: asc + default/michael jackson                                                            1.00      4.7±0.40ms        ? ?/sec    1.16      5.5±0.03ms        ? ?/sec
smol-songs.csv: asc + default/thelonious monk                                                            1.00      4.4±0.02ms        ? ?/sec    1.53      6.8±0.03ms        ? ?/sec
smol-songs.csv: asc/charles mingus                                                                       1.00   783.5±68.03µs        ? ?/sec    1.11    871.9±3.18µs        ? ?/sec
smol-songs.csv: asc/david bowie                                                                          1.00   1121.2±7.70µs        ? ?/sec    1.06   1191.0±7.29µs        ? ?/sec
smol-songs.csv: asc/michael jackson                                                                      1.00   1061.7±9.80µs        ? ?/sec    1.06  1130.6±11.77µs        ? ?/sec
smol-songs.csv: asc/thelonious monk                                                                      1.10      3.0±0.01ms        ? ?/sec    1.00      2.7±0.01ms        ? ?/sec
smol-songs.csv: basic with quote/"Notstandskomitee"                                                      1.22    188.8±1.47µs        ? ?/sec    1.00    154.7±0.74µs        ? ?/sec
smol-songs.csv: basic with quote/"charles"                                                               1.00    163.7±6.29µs        ? ?/sec    1.14    186.9±1.38µs        ? ?/sec
smol-songs.csv: basic with quote/"david"                                                                 1.00    233.0±1.23µs        ? ?/sec    1.19    277.8±2.02µs        ? ?/sec
smol-songs.csv: basic with quote/"david" "bowie"                                                         1.00  1386.6±33.56µs        ? ?/sec    1.22   1691.7±7.20µs        ? ?/sec
smol-songs.csv: basic with quote/"john"                                                                  1.00    349.6±1.91µs        ? ?/sec    1.14    400.0±2.23µs        ? ?/sec
smol-songs.csv: basic with quote/"michael" "jackson"                                                     1.00  1337.4±31.72µs        ? ?/sec    1.13   1513.7±8.17µs        ? ?/sec
smol-songs.csv: basic with quote/"thelonious" "monk"                                                     1.00  1236.8±11.12µs        ? ?/sec    1.10  1362.0±10.19µs        ? ?/sec
smol-songs.csv: basic without quote/charles                                                              1.00    273.6±2.37µs        ? ?/sec    1.33    363.1±3.19µs        ? ?/sec
smol-songs.csv: basic without quote/charles mingus                                                       1.00      2.3±0.01ms        ? ?/sec    1.08      2.5±0.01ms        ? ?/sec
smol-songs.csv: basic without quote/david                                                                1.00    432.0±2.56µs        ? ?/sec    1.23    530.3±3.53µs        ? ?/sec
smol-songs.csv: basic without quote/david bowie                                                          1.00      5.6±0.02ms        ? ?/sec    1.10      6.1±0.03ms        ? ?/sec
smol-songs.csv: basic without quote/john                                                                 1.12  1297.8±11.98µs        ? ?/sec    1.00   1154.8±4.19µs        ? ?/sec
smol-songs.csv: basic without quote/tamo                                                                 1.11    811.8±8.56µs        ? ?/sec    1.00    731.5±4.36µs        ? ?/sec
smol-songs.csv: basic without quote/thelonious monk                                                      1.11      3.8±0.01ms        ? ?/sec    1.00      3.4±0.01ms        ? ?/sec
smol-songs.csv: big filter/charles mingus                                                                1.00    649.1±3.51µs        ? ?/sec    1.10    714.0±6.44µs        ? ?/sec
smol-songs.csv: big filter/david                                                                         1.00  1015.5±18.94µs        ? ?/sec    1.20  1214.3±33.74µs        ? ?/sec
smol-songs.csv: big filter/david bowie                                                                   1.00   1910.2±9.33µs        ? ?/sec    1.20      2.3±0.01ms        ? ?/sec
smol-songs.csv: big filter/john                                                                          1.00    871.3±4.01µs        ? ?/sec    1.22  1058.9±15.94µs        ? ?/sec
smol-songs.csv: big filter/marcus miller                                                                 1.00    716.8±3.93µs        ? ?/sec    1.08   775.7±17.11µs        ? ?/sec
smol-songs.csv: big filter/michael jackson                                                               1.00   1657.9±9.33µs        ? ?/sec    1.11  1840.2±15.93µs        ? ?/sec
smol-songs.csv: desc + default/charles                                                                   1.00  1605.9±10.44µs        ? ?/sec    1.20   1919.7±6.58µs        ? ?/sec
smol-songs.csv: desc + default/charles mingus                                                            1.00      2.4±0.01ms        ? ?/sec    1.08      2.5±0.01ms        ? ?/sec
smol-songs.csv: desc + default/david                                                                     1.00      5.7±0.02ms        ? ?/sec    1.18      6.7±0.03ms        ? ?/sec
smol-songs.csv: desc + default/david bowie                                                               1.00      9.0±0.57ms        ? ?/sec    1.26     11.3±0.04ms        ? ?/sec
smol-songs.csv: desc + default/john                                                                      1.00      4.6±0.95ms        ? ?/sec    1.13      5.2±0.02ms        ? ?/sec
smol-songs.csv: desc + default/marcus miller                                                             1.00      3.8±0.01ms        ? ?/sec    1.38      5.3±0.02ms        ? ?/sec
smol-songs.csv: desc + default/michael jackson                                                           1.00      6.5±0.03ms        ? ?/sec    1.35      8.8±0.03ms        ? ?/sec
smol-songs.csv: desc + default/thelonious monk                                                           1.00      4.5±0.02ms        ? ?/sec    1.52      6.8±0.02ms        ? ?/sec
smol-songs.csv: desc/charles                                                                             1.00    479.6±7.86µs        ? ?/sec    1.06    508.1±4.09µs        ? ?/sec
smol-songs.csv: desc/charles mingus                                                                      1.00  794.9±118.88µs        ? ?/sec    1.10    875.4±6.00µs        ? ?/sec
smol-songs.csv: desc/david bowie                                                                         1.00  1121.5±12.35µs        ? ?/sec    1.06  1183.8±11.13µs        ? ?/sec
smol-songs.csv: desc/michael jackson                                                                     1.00   1064.2±9.50µs        ? ?/sec    1.06  1125.1±14.91µs        ? ?/sec
smol-songs.csv: desc/thelonious monk                                                                     1.14      3.1±0.42ms        ? ?/sec    1.00      2.7±0.02ms        ? ?/sec
smol-songs.csv: prefix search/x                                                                          1.00    284.8±1.76µs        ? ?/sec    1.05    300.4±1.43µs        ? ?/sec
smol-songs.csv: proximity/7000 Danses Un Jour Dans Notre Vie                                             1.00      4.9±0.98ms        ? ?/sec    1.20      5.9±0.02ms        ? ?/sec
smol-songs.csv: proximity/The Disneyland Sing-Along Chorus                                               1.00      5.6±0.02ms        ? ?/sec    1.14      6.3±0.02ms        ? ?/sec
smol-songs.csv: proximity/Under Great Northern Lights                                                    1.00      2.5±0.01ms        ? ?/sec    1.40      3.4±0.01ms        ? ?/sec
smol-songs.csv: proximity/black saint sinner lady                                                        1.16      4.8±0.02ms        ? ?/sec    1.00      4.2±0.01ms        ? ?/sec
smol-songs.csv: words/7000 Danses / Le Baiser / je me trompe de mots                                     1.00     21.0±0.12ms        ? ?/sec    2.03     42.5±0.12ms        ? ?/sec
smol-songs.csv: words/Bring Your Daughter To The Slaughter but now this is not part of the title         1.00     48.4±0.12ms        ? ?/sec    1.53     74.0±0.20ms        ? ?/sec
smol-songs.csv: words/The Disneyland Children's Sing-Alone song                                          1.00     13.9±0.06ms        ? ?/sec    1.12     15.5±0.07ms        ? ?/sec
smol-songs.csv: words/seven nation mummy                                                                 1.00  1059.0±13.98µs        ? ?/sec    1.08  1147.5±23.15µs        ? ?/sec
smol-songs.csv: words/whathavenotnsuchforth and a good amount of words to pop to match the first one     1.00     66.3±0.36ms        ? ?/sec    1.51    100.2±0.20ms        ? ?/sec

That's not acceptable, the next step is to profile milli where it comes from.

Thinking out loud: Could we integrate typical indexing datasets and workflows into roaring-rs performance benchmarks? IMO Preventing surprise perf regressions like this would be beneficial to both projects.

@irevoire is it still relevant?


Example doesn't work ...

Hi,
Error in example ... how correct ?

Because your db object is defined in the do structure, you need to wrap everything in it (db is created in it so stays in it)
do {
    let db = try Connection("financeiro.db")

    // Move your code here
    let users = Table("users")
    // ...
} catch {
    print (error)
}


Set the field size related to the amount to the maximum value

While checking the cause of Issue #442 , I confirmed that the field was an error caused by insufficient size to store the amount.
So I set the size of the field to the maximum size that the corresponding value can have.

@TrustHenry Thank for your review


[Bug]: Cannot install on Ubuntu 20.04.6 LTS python 3.10.8

What happened?
I was unable to install the package using pipx
Version check

[X] Yes I was.

Relevant log output
~/build/fabric$ pipx install .
ERROR: Ignored the following versions that require a different python version: 0.1.14 Requires-Python >=3.9,<4.0; 0.1.15 Requires-Python >=3.9,<4.0; 0.1.16 Requires-Python >=3.9,<4.0; 0.1.17 Requires-Python >=3.9,<4.0; 0.1.23 Requires-Python >=3.9,<4.0; 0.1.24 Requires-Python >=3.9,<4.0; 0.1.32 Requires-Python >=3.9,<4.0; 0.10.0 Requires-Python >=3.10,<4.0; 0.11.0 Requires-Python >=3.10,<4.0; 0.11.1 Requires-Python >=3.10,<4.0; 0.11.2 Requires-Python >=3.10,<4.0; 0.14.0 Requires-Python >=3.10,<=3.13; 0.14.0rc0 Requires-Python >=3.10,<3.12; 0.14.0rc1 Requires-Python >=3.10,<=3.13; 0.14.1 Requires-Python >=3.10,<=3.13; 0.14.3 Requires-Python >=3.10,<=3.13; 0.14.4 Requires-Python >=3.10,<=3.13; 0.16.0 Requires-Python >=3.10,<=3.13; 0.16.1 Requires-Python >=3.10,<=3.13; 0.16.2 Requires-Python >=3.10,<=3.13; 0.16.3 Requires-Python >=3.10,<=3.13; 0.17.0 Requires-Python >=3.10,<=3.13; 0.17.0rc0 Requires-Python >=3.10,<=3.13; 0.17.0rc1 Requires-Python >=3.10,<=3.13; 0.17.0rc2 Requires-Python >=3.10,<=3.13; 0.19.0 Requires-Python >=3.10,<=3.13; 0.22.0 Requires-Python >=3.10,<=3.13; 0.22.2 Requires-Python >=3.10,<=3.13; 0.22.3 Requires-Python >=3.10,<=3.13; 0.22.4 Requires-Python >=3.10,<=3.13; 0.22.5 Requires-Python >=3.10,<=3.13; 0.27.0 Requires-Python <=3.13,>=3.10; 0.27.0rc0 Requires-Python <=3.13,>=3.10; 0.27.0rc1 Requires-Python <=3.13,>=3.10; 0.27.0rc2 Requires-Python <=3.13,>=3.10; 0.27.0rc3 Requires-Python <=3.13,>=3.10; 0.27.1 Requires-Python <=3.13,>=3.10; 0.27.2 Requires-Python <=3.13,>=3.10; 0.28.0 Requires-Python <=3.13,>=3.10; 0.28.1 Requires-Python <=3.13,>=3.10; 0.28.2 Requires-Python <=3.13,>=3.10; 0.28.3 Requires-Python <=3.13,>=3.10; 0.28.4 Requires-Python <=3.13,>=3.10; 0.28.5 Requires-Python <=3.13,>=3.10; 0.28.6 Requires-Python <=3.13,>=3.10; 0.28.7 Requires-Python <=3.13,>=3.10; 0.28.8 Requires-Python <=3.13,>=3.10; 0.5.0 Requires-Python >=3.9,<4.0; 0.5.2 Requires-Python >=3.10,<4.0; 0.5.3 Requires-Python >=3.10,<4.0; 0.5.5 Requires-Python >=3.10,<4.0
ERROR: Could not find a version that satisfies the requirement crewai<0.23.0,>=0.22.5 (from fabric) (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.5, 0.1.6, 0.1.7)
ERROR: No matching distribution found for crewai<0.23.0,>=0.22.5

Relevant screenshots (optional)
No response

Verbose log:
pipx install . --verbose
pipx (mkdir:35): creating directory /home/eso/.local/pipx/venvs
pipx (run_pipx_command:125): Virtual Environment location is /home/eso/.local/pipx/venvs
pipx (_run:124): running /usr/bin/python3 -m venv /home/eso/.local/pipx/venvs
pipx (_run:124): running /home/eso/.local/pipx/venvs/bin/python -m pip install --upgrade pip
Collecting pip
Using cached pip-24.0-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Uninstalling pip-20.0.2:
Successfully uninstalled pip-20.0.2
Successfully installed pip-24.0
pipx (_run:124): running /home/eso/.local/pipx/venvs/bin/python -m pip install .
Processing /home/eso/build/fabric
donestalling build dependencies ... -
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting anthropic<0.19.0,>=0.18.1 (from fabric==1.2.0)
Using cached anthropic-0.18.1-py3-none-any.whl.metadata (15 kB)
INFO: pip is looking at multiple versions of fabric to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following versions that require a different python version: 0.1.14 Requires-Python >=3.9,<4.0; 0.1.15 Requires-Python >=3.9,<4.0; 0.1.16 Requires-Python >=3.9,<4.0; 0.1.17 Requires-Python >=3.9,<4.0; 0.1.23 Requires-Python >=3.9,<4.0; 0.1.24 Requires-Python >=3.9,<4.0; 0.1.32 Requires-Python >=3.9,<4.0; 0.10.0 Requires-Python >=3.10,<4.0; 0.11.0 Requires-Python >=3.10,<4.0; 0.11.1 Requires-Python >=3.10,<4.0; 0.11.2 Requires-Python >=3.10,<4.0; 0.14.0 Requires-Python >=3.10,<=3.13; 0.14.0rc0 Requires-Python >=3.10,<3.12; 0.14.0rc1 Requires-Python >=3.10,<=3.13; 0.14.1 Requires-Python >=3.10,<=3.13; 0.14.3 Requires-Python >=3.10,<=3.13; 0.14.4 Requires-Python >=3.10,<=3.13; 0.16.0 Requires-Python >=3.10,<=3.13; 0.16.1 Requires-Python >=3.10,<=3.13; 0.16.2 Requires-Python >=3.10,<=3.13; 0.16.3 Requires-Python >=3.10,<=3.13; 0.17.0 Requires-Python >=3.10,<=3.13; 0.17.0rc0 Requires-Python >=3.10,<=3.13; 0.17.0rc1 Requires-Python >=3.10,<=3.13; 0.17.0rc2 Requires-Python >=3.10,<=3.13; 0.19.0 Requires-Python >=3.10,<=3.13; 0.22.0 Requires-Python >=3.10,<=3.13; 0.22.2 Requires-Python >=3.10,<=3.13; 0.22.3 Requires-Python >=3.10,<=3.13; 0.22.4 Requires-Python >=3.10,<=3.13; 0.22.5 Requires-Python >=3.10,<=3.13; 0.27.0 Requires-Python <=3.13,>=3.10; 0.27.0rc0 Requires-Python <=3.13,>=3.10; 0.27.0rc1 Requires-Python <=3.13,>=3.10; 0.27.0rc2 Requires-Python <=3.13,>=3.10; 0.27.0rc3 Requires-Python <=3.13,>=3.10; 0.27.1 Requires-Python <=3.13,>=3.10; 0.27.2 Requires-Python <=3.13,>=3.10; 0.28.0 Requires-Python <=3.13,>=3.10; 0.28.1 Requires-Python <=3.13,>=3.10; 0.28.2 Requires-Python <=3.13,>=3.10; 0.28.3 Requires-Python <=3.13,>=3.10; 0.28.4 Requires-Python <=3.13,>=3.10; 0.28.5 Requires-Python <=3.13,>=3.10; 0.28.6 Requires-Python <=3.13,>=3.10; 0.28.7 Requires-Python <=3.13,>=3.10; 0.28.8 Requires-Python <=3.13,>=3.10; 0.5.0 Requires-Python >=3.9,<4.0; 0.5.2 Requires-Python >=3.10,<4.0; 0.5.3 Requires-Python >=3.10,<4.0; 0.5.5 Requires-Python >=3.10,<4.0
ERROR: Could not find a version that satisfies the requirement crewai<0.23.0,>=0.22.5 (from fabric) (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.5, 0.1.6, 0.1.7)
ERROR: No matching distribution found for crewai<0.23.0,>=0.22.5
pipx (rmdir:25): removing directory /home/eso/.local/pipx/venvs
'/home/eso/.local/pipx/venvs/bin/python -m pip install .' failed

The error message suggests that the problem is your python version. Which version of python are you running?

The error message suggests that the problem is your python version. Which version of python are you running?

Python 3.10.8

ok, so I can't reproduce this error. Is there a chance that your python environment includes another instance running < python 3.9? because this is the main error that you are getting 'ERROR: Ignored the following versions that require a different python version: 0.1.14 Requires-Python >=3.9'

I'm seeing similar with Python 3.9.12 and running pip install .

I was able to get it working on MacOS 14.4.1 with these steps:
virtualenv .venv
. .venv/bin/activate
pip3 install .
python --version

Using Python 3.12.2 from this. Python isn't my main language, so I know how to debug it and get it working, but can't quite rationalize why it wasn't working as expected before. I didn't use pipx on this, but I anticipate it would work similarly.

This still does not work for me.  Tried this command which should make sure the pipx virtual environment is also using the appropriate python version.
pipx install --python=$(which python) .
It seems to get past the original error msgs, but still does not install.
eso@eso-OptiPlex-7080:~/build/fabric$ python --version
Python 3.12.0
eso@eso-OptiPlex-7080:~/build/fabric$ pipx install --python=$(which python) .
Installing to existing directory '/home/eso/.local/pipx/venvs'
WARNING: typer 0.12.3 does not provide the extra 'all'
WARNING: typer 0.12.2 does not provide the extra 'all'
WARNING: typer 0.12.1 does not provide the extra 'all'
WARNING: typer 0.12.0 does not provide the extra 'all'
:30: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
Could not find package .. Is the name correct?
eso@eso-OptiPlex-7080:~/build/fabric$ fabric --setup
fabric: command not found


Publish metadata that describes the size and hash of each published file

Add some additional information to the module metadata file that is published to Maven modules, to specify the size of the file in bytes and MD5/SHA1 hash of the file.
This information would allow Gradle to avoid downloading files that it already has available, and allows some interesting linking in build scans.
Split out of #143.

This is done, thanks to recent long haul flights.


[Crosswalk-16][Cordova] enable the camera uscase selection

Impacted tests(approved): new 0, update 1, delete 0
Unit test platform: Crosswalk Project for android 18.46.457.0
Unit test result summary: pass 1, fail 0, block 0
https://crosswalk-project.org/jira/browse/XWALK-4704

LGTM


Upgrade actions/upload-artifact v4.3.1 -> v4.3.2

This PR contains the following updates:

Package
Type
Update
Change

actions/upload-artifact
action
patch
v4.3.1 -> v4.3.2

Release Notes

actions/upload-artifact (actions/upload-artifact)
v4.3.2
Compare Source
What's Changed

Update release-new-action-version.yml by @konradpabjan in https://github.com/actions/upload-artifact/pull/516
Minor fix to the migration readme by @andrewakim in https://github.com/actions/upload-artifact/pull/523
Update readme with v3/v2/v1 deprecation notice by @robherley in https://github.com/actions/upload-artifact/pull/561
updating @actions/artifact dependency to v2.1.5 and @actions/core to v1.0.1 by @eggyhead in https://github.com/actions/upload-artifact/pull/562

New Contributors

@andrewakim made their first contribution in https://github.com/actions/upload-artifact/pull/523

Full Changelog: https://github.com/actions/upload-artifact/compare/v4.3.1...v4.3.2

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade actions/upload-artifact v4.3.1 -> v4.3.2


Between one and five of these:
Error: Request workspace/executeCommand failed with message: connect ECONNREFUSED <IP_ADDRESS>:15156
    at new ResponseError (C:\Users\Gary\.atom\packages\ide-purescript\node_modules\vscode-jsonrpc\lib\messages.js:46)
    at handleResponse (C:\Users\Gary\.atom\packages\ide-purescript\node_modules\vscode-jsonrpc\lib\main.js:430)
    at processMessageQueue (C:\Users\Gary\.atom\packages\ide-purescript\node_modules\vscode-jsonrpc\lib\main.js:258)
    at Immediate.<anonymous> (C:\Users\Gary\.atom\packages\ide-purescript\node_modules\vscode-jsonrpc\lib\main.js:242)
    at runCallback (timers.js:651)
    at tryOnImmediate (timers.js:624)
    at processImmediate [as _immediateCallback] (timers.js:596)

kind of errors pop up together. This also prevents the restart server action from working, although after this explodes you can use the start server action and it seems to work - so it does actually shut the server down.

Is this when the server was running happily at the time? Or were you attempting to restart from a position of something being broken?

Yeah, it happens when everything is running happily. It actually behaves the same way regardless of whether the server is running or not.

Can you check this again, there are some relevant changes - and in particular restart should do the start even if the stop part fails

I think this is good now, it's a little hard for me to tell since there's no feedback at all now - I assume that means it is working though!
I still see an error when stopping the server, but both start and restart do get things running again.
I'll comment back here again if it fails next time I have a need to restart the server (I'll know for sure if it works then, as restarting should fix the problem if it has some cached state or something).


Unable to run juno in p2p mode over public network

With the current implementation of the p2p mode, one is required to provide the current node address via --p2p-addr.
This address will later on be gossiped to the network HERE.
In order to use the p2p mode over a public network, I tried to set the p2p addr to a public one, but it failed:
# Of course the ip I used doesn't contain XXX, I just replaced for security reasons
juno ... -p2p-addr=/ip4/35.237.XXX.XXX/tcp/7777 ...

Error: set up p2p service: failed to listen on any addresses: [listen tcp4 35.237.XXX.XXX:7777: bind: cannot assign requested address]

Having a quick look at the code, it seems that there is no way to tell Juno what's my public IP.
Looking at what other people have done using the same lib, I found some examples documented to do something similar to what we are trying to do https://github.com/libp2p/go-libp2p/tree/master/examples/routed-echo
I've also found an up-to-date maintained project that is trying to do the same, p2p over public network: https://discuss.libp2p.io/t/minimal-public-network/1651, and on their implementation they do expose a flag to set the external address: https://gitlab.com/accumulatenetwork/accumulate/-/blob/main/pkg/api/v3/p2p/p2p.go#L99
So this issue is to implement a way to be able to use the p2p option over private AND public networks

Can't imagine that each node user gonna set external IP address. If not then, almost every node start to gossip not reachable node. If there gonna be a lot of such nodes, whole network start to be slow. For sure there is some solution for this, maybe something like here: https://docs.libp2p.io/concepts/nat/autonat/ (?)

I found some good documentation around how to fix this issue: https://github.com/libp2p/specs/blob/master/connections/hole-punching.md#vision
Basically we will need to consider and implement a few ways to go about it.  I found another example that uses a few interesting options from libp2p
https://github.com/taubyte/tau/blob/main/p2p/helpers/libp2p.go and https://github.com/taubyte/tau/blob/main/p2p/peer/peer.go
Some points to consider from libp2p:

EnableHolePunching
EnableAutoRelayWithPeerSource
EnableAutoNATv2
EnableNATService
EnableRelayService


Problem With Glyphs

I installed airline and set my ~/.vimrc accordingly. but I get these Chinese glyph instead.
I use PragmataPro font.

well instead of those nice symbols I have Chinese glyphs. What is exactly wrong with my vim?

This problem can be fixed by adding the following line to your vimrc:
let g:airline_powerline_fonts=1
This solved the issue for me using Essential PragmataPro.

(Hope maintainers would forgive me commenting on this closed issue)
I do encounter the exact problem a while ago, with the Chinese characters clobbering the nice symbols one should have, even after installing patched font. While @bling 's suggestion worked, there is one caveat:
set encoding=utf-8
MUST be specified before
let g:airline_powerline_fonts=1.
Order is important in vim config. For reason I haven't investigated, under default encoding used in Windows (which is probably multibyte codepage under original reporter's environment), system fallback multibyte font would be looked up instead of the patched font.

Well, in general, setting encoding always has to be the very first item in your .vimrc, because this will possibly invalidate all strings in the Vim core, e.g. registers, variables and similar.


add UI input element to add formula roll for each tableResult

instead of having the text field with syntax 1d6 [potion roll] we can have entity / rolltable / and a field with formula and then the text field for the rolltable name

already in better table type. coming to loot tables in 1.5 (currently in dev)


Add default properties file for bootstrapping

Add some default 'ravel.properties' that can be overridden by command-line/environment variables to load default DB/redis/oauth/... information.

The plan is to implement this as a separate project, in the form of a yeoman generator complete with:

a directory structure
default modules (ravel-google-oauth2-provider, ravel-mysql-provider)
default properties
a dev Vagrant vm (redis + mysql)
a build file (grunt/gulp) which helps bootstrap both development and test suite creation

Aside from eww yeoman, sure.  :)


F-Droid releases

Is your feature request related to a problem? Please describe.
I'm always frustrated when I use proprietary Google Play application.
Describe the solution you'd like
Since the code is open-source, it would be nice to have it released in open-source friendly platform as well.
Describe alternatives you've considered
Compiling from source.
Additional context
http://f-droid.org/

+1
I hope there would be a fdroid version without google's trackers

https://github.com/kiwibrowser/src.next/issues/91


If node that runs the taskrun pod shutdown then retry will not work as except

Expected Behavior
When the node shutdown and the tasrkrun pod running on the node will be failed. And if we set retry on taskrun we would expect the retried pod to start in a normal node and taskrun can continue to work.
Actual Behavior
Taskrun will use the  failed pod as the work pod and will not create a new pod for retry.
Steps to Reproduce the Problem

create a taskrun with retry in a k8s cluster that has multi nodes
shutdown  the node which running the pod
check the status of taskrun

Additional Info

Kubernetes version:
Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.6", GitCommit:"ff2c119726cc1f8926fb0585c74b25921e866a28", GitTreeState:"clean", BuildDate:"2023-01-18T19:22:09Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.6", GitCommit:"ff2c119726cc1f8926fb0585c74b25921e866a28", GitTreeState:"clean", BuildDate:"2023-01-18T19:15:26Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}

Tekton Pipeline version:

v0.41.0

I find it's because we only check the status of container and when node shutdown the container status in  pod will not change.

And I find when I execute kubectl get po to get the pod which hosted node has shut down I find the pod status is Terminating. I think that k8s recognizes that the pod is in an unnormal status and read the related code.
If pod.DeletionTimestamp != nil and k8s will think this pod is in Terminating status.

I think maybe we can change the logic of detecting the taskrun pod status like this to resolve the problem.
// DidTaskRunFail check the status of pod to decide if related taskrun is failed
func DidTaskRunFail(pod *corev1.Pod) bool {
	f := pod.Status.Phase == corev1.PodFailed
	for _, s := range pod.Status.ContainerStatuses {
		if IsContainerStep(s.Name) {
			if s.State.Terminated != nil {
				f = f || s.State.Terminated.ExitCode != 0 || isOOMKilled(s)
			}
		}
	}
	// when the node lost the pod will be in Terminating
	f = f || pod.DeletionTimestamp != nil

	return f
}

/assign

I think it's worth considering whether we want a TaskRun to handle node failures in general or only in the case of retries. FYI @pritidesai @skaegi

Thank you @yuzp1996 for reporting this issue 🙏

Why are these pod names are same for multiple attempts? @XinruZhang @lbernick can we please reproduce this 🙏
As per our retry strategies, the pod names must be unique for each retry attempt if I am not mistaken.

@yuzp1996 behavior of retries was modified in v0.43.0. Are you able to reproduce this bug on v0.43.0 or newer?

I reproduced the error -- it seems like this behavior is unrelated to the "retries" feature: evicting the pod of the taskrun that is created out of the following pipelinerun has the same result:
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: pr-retries-
spec:
  serviceAccountName: 'default'
  pipelineSpec:
    tasks:
      - name: task-abnormal-pod
        taskSpec:
          steps:
          - name: echo
            image: alpine
            script: |
                sleep 50000 && exit 1

We discussed this issue in the API WG today: do we want to retry a taskrun that contains an unfinished pod that is evicted or terminated unexpectedly (node deletion etc)? People in the WG meeting generally agree that we should not retry it.
cc @yuzp1996 @pritidesai  @lbernick

thanks a bunch @XinruZhang 🙏

evicting the pod of the taskrun that is created out of the following pipelinerun has the same result

please explain what does the same result mean here. What happens to such pipelineRun without retry? Is pipelineRun kind of hangs until it times out?

Most likely here is the case:
I've tested the following behavior:
When the pod is scheduled in the same node that tekton-pipelines-controller and tekton-pipelines-webhook are running on, then if we drain the node, all pods on that node are evicted, the reported behavior happens.
But if we only evict the pod created for the taskrun, the taskrun and pipelinerun faills:
Events:
  Type     Reason         Age                    From     Message
  ----     ------         ----                   ----     -------
  Normal   Started        15m                    TaskRun
  Normal   Pending        15m                    TaskRun  Pending
  Normal   Pending        15m                    TaskRun  pod status "Initialized":"False"; message: "containers with incomplete status: [prepare place-scripts]"
  Normal   Pending        15m                    TaskRun  pod status "Initialized":"False"; message: "containers with incomplete status: [place-scripts]"
  Normal   Pending        15m                    TaskRun  pod status "Ready":"False"; message: "containers with unready status: [step-echo]"
  Normal   Running        15m (x2 over 15m)      TaskRun  Not all Steps in the Task have finished executing
  Warning  Failed         6m27s                  TaskRun  "step-echo" exited with code 255 (image: "docker.io/library/alpine@sha256:124c7d2707904eea7431fffe91522a01e5a861a624ee31d03372cc1d138a3126"); for logs run: kubectl -n default logs pr-retries-z5mdd-task-abnormal-retries-pod -c step-echo
  Warning  InternalError  6m26s (x2 over 6m26s)  TaskRun  pods "pr-retries-z5mdd-task-abnormal-retries-pod" not found

I don't quite understand why "creating the pod" and "updating the podname in taskrun" would happen in two different reconcile loops -- why is there a case that PodName is not empty while the Pod is not found.
https://github.com/tektoncd/pipeline/blob/8e8c163446e8bef885bd628072320f9805529ac9/pkg/reconciler/taskrun/taskrun.go#L437-L448

evicting the pod of the taskrun that is created out of the following pipelinerun has the same result

Please explain what does the same result mean here. What happens to such pipelineRun without retry? Is pipelineRun kind of hangs until it times out?

The same result means: the pod will be recreated (with the same PodName as this issue reports), and the PipelineRun will eventually succeed (or fail if that's what's defined by the PipelineTask).

Looking at the history, this check on tr.Status.PodName was deleted but then it was reverted back.
Introduced in https://github.com/tektoncd/pipeline/commit/0f20c3539f25ede46dfe58b83924e28db1fd783e to stop looking up the pod for a taskRun by name but instead only look up by labelSelector.

This adds Reconciler.getPod, which looks up the Pod for a TaskRun by performing a label selector query on Pods, looking for the label we apply to Pods generated by TaskRuns.

If zero Pods are returned, it's the same as .status.podName being "". If multiple Pods are returned, that's an error.

This commit was reverted in https://github.com/tektoncd/pipeline/issues/1944 because back then we had multiple pods associated to single taskRun object and was not easy to identify (back in 2019) when to declare done for a taskRun.
Further reference: https://github.com/tektoncd/pipeline/issues/1689
We can certainly update pod creation implementation since we have ownerReference implemented now but I think that could be something nice to have and not causing any issues here.

When the pod is scheduled in the same node that tekton-pipelines-controller and tekton-pipelines-webhook are running on, then if we drain the node, all pods on that node are evicted, the reported behavior (the same pod will be created for the taskrun -- simply because the controller logic needs to be re-run [1]

Let's focus our testing for this bug report in which the controller and taskRun pods are on different nodes. There is generally a separation of duties and access controls in actual deployments.

But if we only evict the pod created for the taskrun, the taskrun and pipelinerun faills:

Warning  Failed         6m27s                  TaskRun  "step-echo" exited with code 255 (image: "docker.io/library/alpine@sha256:124c7d2707904eea7431fffe91522a01e5a861a624ee31d03372cc1d138a3126"); for logs run: kubectl -n default logs pr-retries-z5mdd-task-abnormal-retries-pod -c step-echo
Warning  InternalError  6m26s (x2 over 6m26s)  TaskRun  pods "pr-retries-z5mdd-task-abnormal-retries-pod" not found

I am interpreting the taskRun and pipelineRun fails without any retries based on the status updates here ⬆ which is as expected, right?
@yuzp1996 can you please try this with the controller beyond 0.43 as @lbernick requested?

Thank you @pritidesai  for tracing back to the history.

I am interpreting the taskRun and pipelineRun fails without any retries based on the status updates here ⬆ which is as expected, right?

Yes yes, the TaskRun and PipelineRun failed directly which is expected because no Retries was specified.
Once I specify the Retries field, and then the pod is retried, see the status tansitions here:
pr-retries-vmbdz-task-abnormal-retries-pod   1/1     Running   0          41s
pr-retries-vmbdz-task-abnormal-retries-pod   1/1     Terminating   0          51s
pr-retries-vmbdz-task-abnormal-retries-pod   0/1     Terminating   0          52s
pr-retries-vmbdz-task-abnormal-retries-pod   0/1     Terminating   0          52s
pr-retries-vmbdz-task-abnormal-retries-pod   0/1     Terminating   0          52s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   0/1     Pending       0          0s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   0/1     Pending       0          0s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   0/1     Init:0/2      0          1s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   0/1     Init:1/2      0          2s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   0/1     PodInitializing   0          3s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   1/1     Running           0          4s
pr-retries-vmbdz-task-abnormal-retries-pod-retry1   1/1     Running           0          4s

P.S. I'm testing the behavior on the latest Tekton Pipelines code. cc @yuzp1996

Thank you for your response! I'll try this with a controller above 0.43 and see if the problem persists.


Starting equipment arcane foci are repeated/redundant

In the Starting Equipment endpoint, every choice thats allow an Arcane focus to be chosen list the equipment items recommended by the SRD.
This raises two key issues:

the list is repeated several times. This means any changes to the list have to be made in several places, which is a hit to maintainability.
the list contains only the equipment items explicitly mentioned by the SRD as possible foci:

An arcane focus is a special item -
an orb, a crystal, a rod, a specially constructed staff, a wand-like length of wood, or some similar item -
designed to channel the power of arcane spells. A sorcerer, warlock, or wizard can use such an item as a spellcasting focus.

The first issue could be resolved by moving the suggested arcane foci into an arcane-foci equipment category. This would allow easier maintenance and addition of homebrew data, and would make the choice more consistent with other starting equipment options.
The second issue is trickier; given that any item can be a focus, with suitable fluff to back it up, an equipment category cannot contain all possible arcane foci. The question then becomes, how likely is it that such a non-standard focus will be chosen by the user, and is that event worth supporting within the data itself, if possible?
It would be most straightforward, and most inline with the SRD source text, to offer a selection of arcane foci from an equipment category containing the items referenced in the SRD description of am arcane focus. If a user desires an alternative focus, they may swap one of the default options for another item manually, after choosing their starting equipment.
This issue was raised after discussion on PR #230 in this comment thread

If I check the SRD, some classes can use an arcane focus, some other a druidic focus.
The SRD says on page 66-67 what an arcane focus can be and on page 67 what a druidic focus is, but it defines on page 69 the list of items that are available in the SRD and that are an arcane or druidic focus.
So to me, we create two new equipment categories: arcane foci and druidic foci.
In my view it's rather clear.

@ogregoire Agreed. I think this is simply a case of moving druidic and arcane focuses into their own equipment categories, and updating the starting equipment entries to reference these.
Currently, arcane and druidic foci are simply gear categories, rather than standalone equipment categories.

Yup! That's what @fergcb and I came up with up with as well. It seems pretty straight forward.

@ogregoire @bagelbits Should Holy symbols be included in this PR, too? I.e. moved to their own equipment category?

@ogregoire @bagelbits Should Holy symbols be included in this PR, too? I.e. moved to their own equipment category?

Yes since both the cleric and the paladin use those, and get those at character creation.


archive/zip: FileHeader.Extra API is problematic

The FileHeader.Extra field is used by the Writer to write the "extra" field for the local file header and the central-directory file header. This is problematic because the Go implementation assumes that the extra bytes used in the two headers are the same. While is this is often the case, it is not always true.
See http://mdfs.net/Docs/Comp/Archiving/Zip/ExtraField and you will notice that it frequently describes a "Local-header version" and a "Central-header version", where the formats sometimes differ.
The Reader does not have this problem because it entirely ignores the local headers.
I haven't thought much about what the right action is moving forward, whether to deprecate this field or add new API. I just want to file this issue, so I remember to address it later.

@dsnet This is currently set for the 1.11 milestone.  Please update as appropriate.  Thanks.

I suggest we add ExtraCentral []byte, where a writer treats nil to mean "same as Extra".

My use case is to read the Extra field from local file headers, as httrack only writes data to the local file headers and the extra data in central directory record is empty. See also https://github.com/golang/go/issues/40354#issuecomment-798846397
How would adding ExtraCentral field work with readers? Would ExtraCentral be always nil when reading from the archive?
Currently when reading, we read the central directory only. Reading the local file header in addition to it is expensive as we'd need to seek to read the data for each file and most of the time, users don't care about the differences in local file header/central directory. So I think we don't want to read the local file headers in Reader.init.
However, if we used ExtraCentral also for reading, then we'd probably need to change Extra to contain data from local file header instead of central directory record. That could break some existing code if an archive contains different data in local file header and central directory record.
If we add ExtraLocal field instead of ExtraCentral, we won't have this problem and the semantics when reading would be more clear (Extra is from central directory, ExtraLocal is nil because we didn't read the local file header).


Infinite loop with case::

The double colon in this code will cause an infinite loop:
switch (a)
{
     case 7::
         break;
}

Hi,
see #330.
It should be fixed already...

Indeed, somehow I had an old version. Sorry for the hassle.


endpoint: Fix packets to host dropped with the chaining mode and host firewall

This PR fixes the issue Cilium drops packets to the host when it's configured with the chaining mode and host firewall by populating the policy map regardless of the per-endpoint routes setting. All routing is performed by the Linux stack with the chaining mode even if the per-endpoint route is disabled in the agent.
Please ensure your pull request adheres to the following guidelines:

[x] For first time contributors, read Submitting a pull request
[x] All code is covered by unit and/or runtime tests where feasible.
[x] All commits contain a well written commit description including a title,
description and a Fixes: #XXX line if the commit addresses a particular
GitHub issue.
[x] All commits are signed off. See the section Developer’s Certificate of Origin
[x] Provide a title or release-note blurb suitable for the release notes.
[x] Thanks for contributing!

Fixes: #18942

Added @pchaigno as a reviewer, since it seems that he has some context.

/test

It seems that some tests are failing. I don't see those related to this change. Should I rebase my branch?
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.21-kernel-5.4/1296/testReport/Suite-k8s-1/21/K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master/
Stderr:
 	 WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jenkins/workspace/Cilium-PR-K8s-1.21-kernel-5.4/src/github.com/cilium/cilium/test/vagrant-kubeconfig
	 WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jenkins/workspace/Cilium-PR-K8s-1.21-kernel-5.4/src/github.com/cilium/cilium/test/vagrant-kubeconfig
	 Error: INSTALLATION FAILED: rendered manifests contain a resource that already exists. Unable to continue with install: Service "hubble-peer" in namespace "kube-system" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-name" must equal "cilium-preflight": current value is "cilium"

/home/jenkins/workspace/Cilium-PR-K8s-1.21-kernel-5.4/src/github.com/cilium/cilium/test/k8s/updates.go:127

Should I rebase my branch?

@ysksuzuki Yes, please. The K8sUpdates test failures should be fixed by https://github.com/cilium/cilium/pull/19759.

@pchaigno Rebased my branch. Please rerun the tests

/test

Something wrong with the test environment. I can't imagine that my patch would affect it.
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9/1596/testReport/junit/Suite-k8s-1/16/K8sPolicyTest_Multi_node_policy_test_with_L7_policy_using_connectivity_check_to_check_datapath/
Stderr:
 	 The Service "echo-b" is invalid: spec.ports[0].nodePort: Invalid value: 31414: provided port is already allocated

k8s-1.16-kernel-4.9 failed with known flake https://github.com/cilium/cilium/issues/13071. Other required tests are passing.
We're missing a review from @cilium/endpoint but I've reviewed and am the author of the specific changed lines under @cilium/endpoint's ownership so fine to ignore the missing review IMO.
Marking ready to merge.


[GHSA-mmrq-6999-72v8] Incorrect value comparison in Ruby openssl

Updates

Affected products

Comments
The official blog post states:

The openssl 2.1.2 gem or later includes a fix for the vulnerability, so upgrade the openssl gem to the latest version if you are using Ruby 2.4 or a later series.

Interesting. We've got a fix commit that seems to indicate that it went out on 2.0.9 based on the tags.
https://github.com/ruby/openssl/commit/f653cfa43f0f20e8c440122ea982382b6228e7f5
Give me a bit to read into this one 👍


Update readme with dev doc link

Adding an intro and link to the docs so that the readme is more similar to the other SDK's readmes.

Hi @trishahanlon
It seems, there is a conflict in the README.md! Also please make sure the changes are still valid.

Internal JIRA created to track to completion of this task: https://jira.sso.episerver.net/browse/FSSDK-9368

This can be closed, as this PR was created before the readme markdown files were updated for Full Stack becoming legacy.


Add optional infine-looping command-line parameter for WindowsTerminal profiles

Description of the new feature/enhancement
Add boolean parameter if WindowsTerminal profile's command line should repeat every 1s in case command finishes. This would be useful for automatically restarting openssh sessions on the same tab. Currently you need to clone tab and reopen it again.
Proposed technical implementation details (optional)
E.g. I tried to implement this by typing this command-line profile: while ($true) { ssh<EMAIL_ADDRESS>-p 22 -i ~/.ssh/id_ed25519; start-sleep -seconds 1}
But it doesn't work. I get [error 0x80070002 when launching while ($true) { ssh<EMAIL_ADDRESS>-p 22 -i ~/.ssh/id_ed25519; start-sleep -seconds 1}] error.
If profile's command-line infine-loop+delay would be integrated inside WindowsTerminal, then it would be easier appliable
Thank you for creating this greater alternative tool to PuTTY.

You know, I think the even easier way to implement this would be with #4379 or #3726. No need to have an infinite loop to keep trying the commandline, just a button would do well ☺
/dup #3726
/dup #4379
Also, you might try setting the commandline to powershell -c while ($true) { ssh<EMAIL_ADDRESS>-p 22 -i ~/.ssh/id_ed25519; start-sleep -seconds 1} or something like that. You need to make sure to set the commandline to powershell running that script


Move prompting on repo status into VCS client

For example: https://github.com/expo/eas-cli/blob/e96cee1c3b53627e64a25a8452adcbea77bec86a/packages/eas-cli/src/build/utils/repository.ts#L15-L33
This results in the user still being prompted about git status:

fixed in https://github.com/expo/eas-cli/pull/481


Allow manual date input

When I try to put my own date in the YYYY-mm-dd format in the https://chmln.github.io/flatpickr/#strap demo, the calendar doesn't update with my input. It still displays today's date, regardless.
Meanwhile, I can with dbushell/Pikaday. However I still like flatpickr more seeing that you also deal with datetimes, though it'd be really nice if the calendar view updates with user input.

Hello,
You mean manual date input into the calendar.
I considered it at some point but then realized that manual input clashes with the purpose of the date picker, that is picking dates without having to type them out.
Year changing is lightning fast. Changing months - any month is at most 6 clicks away (e.g. from june to december), since months wrap around January. Selecting a date takes a click.
That having said, I will still consider manual input, and keep this issue open for now :cake:
Thanks for contributing :+1:

Thanks for the speedy response.
I don't agree that manual input clashes with the purpose of the date picker where you do so without typing them out - it's not exactly one or the other! Even with manual input functionality implemented, you can still pick dates without typing. It's just an addition, and doesn't take away anything from the original functionality.
That being said, thanks for your work!  :+1:

I agree with @eddowh . While manual date/time input seems counterintuitive, there is value in having this available. Speaking as a user of the package, I see Flatpickr's purpose in being a very clean, polished alternative to the drab native datepicker in Chrome. But not allowing access to the internal input value eliminates this as an option for many end users having particular use cases.

Came across this page after pulling a few hairs out wondering why I could not update the calendar to show the date I manually keyed in. Seems the various object properties weren't intended to be used in this way, which this thread confirms. It definitely takes away from the tool's versatility. For now, I'll see if I can find a workaround. Still a great tool.

Manual date input has been available for over a year now with the allowInput option. If you are experiencing problems with it, please open a new issue.
Thank you.


Rebasing from master branch

Signed-off-by: Raj Das<EMAIL_ADDRESS>

This is overwriting some desirable commits made into staging earlier today. You may want to pull the latest changes on staging before merging master..?

Its a newly forked repo. I have rebased the staging with master branch.


fix(cli-service): fix path RegEx error

This fixes issues 2813

I don't know why I have this problem.

This failure is due to the slow execution in CI environment and can be ignored.


Removing dependency on onEnterKeyDown and using onKeyDown as per material-ui request (onEnterKeyDown deprecated)

:+1:

@kainolophobia :+1:

What can we do to help merge this?

Sorry @kainolophobia, I've been focussed on the Material-UI 0.15.0-alpha.2 release. I'll look at this soon.

Should fix #71.

Hope this PR to be merged

Funny you should ask (was that a verbal '+1' ? :smile: ) as I started updating this about 5 minutes before your comment.

do you have any estimate to finish that update? @mbrookes I'm so excited about this repository, if you have any roadmap maybe we can help for something? 🤓


Make NSArray lastObject & firstObject atomic

Fixes #2751

This change is

lastObject was the troublemaker. After this change, repro rate of test failure ~0.6% -> 0%

oh right, you have a point there


Joints coordinates in real-time

Hello Daniil-Osokin, hello everyone ! Thank you very much for your work, it's very good. I would like to get the real-time coordinates of each joint and save them in a CSV file. Where is the place in the code I could get them ?
I already checked in keypoints.py but I don't know how I could clearly identify each joint (left wrist, right wrist, left elbow, right elbow etc etc) and get it's coordinates in real-time.
Thank you very much for your feedback, have a nice day :)

Hi! You can call convert_to_coco_format function, which returns keypoints in MS COCO format order: 'nose', 'left_eye', 'right_eye', 'left_ear', 'right_ear', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle'.

Thank you very much for your fast feedback :) !
So, if I understand well, the list coco_keypoints contains the coordinates of each joint like this : [[noseX, noseY]; [left_eyeX, left_eyeY]; [right_eyeX, right_eyeY] ... etc] ?
Also,  what does the list 'scores' contain exactly ? Thanx :)

It returns list of 17x3 entries, every entry is x, y, coordinates of corresponding keypoint and visibility (set to 1 or -1, if this type of keypoint not found). Scores is the confidence of method in particular pose.

Thank you very much for the time you accorded to respond, your answer is
very helpful to me ! Have a nice week-end :)
Best regards,
ADJEL Mohamed
Le ven. 26 avr. 2019 à 22:26, Daniil-Osokin<EMAIL_ADDRESS>a
écrit :

It returns list of 17x3 entries, every entry is x, y, coordinates of
corresponding keypoint and visibility (set to 1 or -1, if this type of
keypoint not found). Scores is the confidence of method in particular pose.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch/issues/14#issuecomment-487190600,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJWH7I77WJQUAW5KPAGPMWTPSNQQLANCNFSM4HIMXADA
.


imagestore: skip originalname annotation for non-docker images

Running CRI pods resulting in a lot of log noise:
imagestore.go:174] image "coreos.com/rkt/stage1-coreos" does not have originalname annotation

I didn't dig into the details, but we should probably skip this test for non-docker image. Or at least, for stage1 images.

Not printing a warning at all is the easy solution; it's just noise.


redis fixed (for redis cloud instance)

The Redis cache works when running in localhost. But for Redis Cloud instance it doesn't work. For it to work the REDIS_USERNAME is required in the env file. So I added that to the required places.
I've also added JOB_ENABLE env variable.

Hi @horizon-dev-group ,
Yes, u right. we need REDIS_USERNAME if we want connect to cloud.
Thank you for for u contribution.
But what purpose of added env. JOB_ENABLE ?
I didnt see u use that in the project


Missing info / incomplete documentation found

Hi,
Sorry this is my first time putting an issue with such a big company! Not sure if there's any decorum to abide by but, here is the issue:
[] (https://github.com/NVIDIA/GenerativeAIExamples/docs/api-catalog.md)
At the very end there is a missing bullet point and an incomplete sentence that would be nice to have the info for. Here is the sentence:

Next Steps

Access the web interface for the chat server. Refer to missing_info for information about using the web interface.

extra bullet or missing info

Stop the containers by running docker compose -f deploy/compose/rag-app-api-catalog-text-chatbot.yaml down and docker compose -f deploy/compose/docker-compose-vectordb.yaml down.

It wasn't your mistake, I was just not looking at the final pubic-facing page.... oops!


[incubator/mysqlha] Add the way that how to test the master and slave work well

Signed-off-by: Aisuko<EMAIL_ADDRESS>
What this PR does / why we need it:

Add the way that how to test the master and slave work well.
Add a common image pull policy for all image in the charts.

Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

/unassign @davidkarlsen @viglesiasce

/assign @jpoon

/lgtm

/ok-to-test
/lgtm

/ok-to-test

/approve

/lgtm

/assign
/lgtm

@jpoon Please have a look on this. Thanks a lot.

/lgtm


fix: make log severity type safe

Fixes #590

Codecov Report
Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (51e7a43) 58.75% compared to head (2f5e950) 59.01%.

Files
Patch %
Lines

src/logging.cc
53.33%
7 Missing :warning:

src/signalhandler.cc
0.00%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1025      +/-   ##
==========================================
+ Coverage   58.75%   59.01%   +0.26%
==========================================
  Files          18       18
  Lines        2560     2550      -10
  Branches      918      912       -6
==========================================
+ Hits         1504     1505       +1
+ Misses        707      701       -6
+ Partials      349      344       -5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Migrate from prettier standalone to eslint plugin

Instead of running prettier separately, this PR adds the prettier plugin to eslint.  This should avoid potential style conflicts between eslint and prettier.
This PR also contains fixes recommended by the prettier eslint plugin (mostly just double to single quotes on some strings)

lgtm, some test failures


Produce hierarchical dependency output

fixes #59

Changes dependency provider interface to return a map of list of dependencies where each key is the direct dependency of the application and indirect dependencies form the list of values for that key. Note that transitive / indirect dependencies are flattened into a list.
Updates golang / java dep providers to parse dependency in the expected format

In future, the same map can be used to return an adjacency list instead of an absolute list to further maintain the hierarchy within transitive / indirect deps.

/lgtm with a quick pass and I didn't run it.
The only large comment is a test for golang as well? but I didn't create one either so I get it.

@shawn-hurley added tests for go provider, updated to use relative path in main.go.

/lgtm


[Outlook] (cache) Update instructions

Docs Build status updates of commit 8217624:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/testing/clear-cache.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Can't use integrated terminal?

Which OS: macOS 10.14.3
Which VSCode version:  1.31.1
Which extension version: 1.2.1
Which adapter type: Unsure
Which LLDB version: lldb-1<IP_ADDRESS>
Can't use launch configuration with integrated console option
"console": "integratedTerminal",
Error produced:

NB: desired configuration was:
    {
      "type": "lldb",
      "request": "launch",
      "name": "Processing",
      "program": "${workspaceFolder}/.build/debug/Processing",
      "args": [],
      "cwd": "${workspaceFolder}",
      "preLaunchTask": "Processing: Build",
      "console": "integratedTerminal",
      "internalConsoleOptions": "neverOpen"
    },

The schema of launch configuration is not the same across debug extensions (though some attributes are pretty universal).  For CodeLLDB, please use "terminal": "integrated" (other options are "console" and "external")

Okay, thanks, this works.


Add full-text dataset

We should also add a full-text dataset to Rally.
Hint: @polyfractal has already started looking into it.

I have now created a subset of the PMC repository consisting of roughly half a million articles which are licensed under CC-BY. The data set has a size of 23GB.


Wrong hash returned for issuance tx

Here you can find an example of a transaction containing an issuance that I was able to broadcast on regtest with hash 4f2560d018b2600399ad960284f5d13f02a978de8673c8950bf8f9cd265ec1a6.
The method Transaction.TxHash() returns 16a2d64e1e9d0fb233169a4d08a222a0562895bd3cbbe1783c66aeadd5320f26 instead.
The bug hasn't been identified yet, but it could reside somewhere here where the input issuance is serialized.

Has this been closed? @sekulicd

Has this been closed? @sekulicd

https://github.com/vulpemventures/go-elements/pull/171


Enhancement: Improving memory efficiency by eliminating the vtable in the managed class.

By making the destructor protected - we can remove the possibility that someone tries to delete one of the derived classes by deleting an instance of the managed class, which should allow for not having a virtual destructor.
Code change checklist

[x] I have ensured that all methods and functions are fully documented using doxygen style comments
[x] My code follows the coding style guide.
[x] I tested that my change works before raising the PR.
[x] I have ensured that I did not break any existing API calls.
[x] I have not built my pull request using AI, a static analysis tool or similar without any human oversight.

why are you so obsessed with getting rid of the vtable here?
I already said it's over engineered and hard to read... theres no need to go this far to save 8 bytes.

why are you so obsessed with getting rid of the vtable here? I already said it's over engineered and hard to read... theres no need to go this far to save 8 bytes.

Because I tend to fixate on things periodically - and I disagree that it's over engineered and hard to read - I've barely changed anything. Is it slightly more complex? Yes. Is it worth the memory savings? I would say it is - though I suppose that's what is up for debate here. The final say is ultimately yours though.


Comment addition for "break"

[ ] I have read freeCodeCamp's contribution guidelines.
[ ] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the master branch of freeCodeCamp.
[ ] None of my changes are plagiarized from another source without proper attribution.
[ ] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Closed and Reopened this PR to attempt to resolve a specific Travis build failure.

@aniltirli Thank you for your contribution to the page! 👍
We're happy to accept these changes, and look forward to future contributions. 📝


Add 'maximum_size' to window builder

Presumably web will just not handle a 'maximum' window size, but it should be possible for Windows and Mac.

Implemented for Mac in https://github.com/kettle11/kapp/commit/66409b285fc1bc124fc2f59ec886f6569ad9eb0c

Ready for close? Only remaining issue seems to be tracked by #61

Yup! Forgot to close yesterday.


No longer working in 1.18.2

Just as the title says. Mod is no longer working in 1.18.2

It only supports 1.18.1 ATM.


Migrate crates to new RustCrypto trait versions

Specific dependency changes:

block-buffer 0.7 -> 0.9
block-cipher-trait 0.6 -> block-cipher 0.7
block-padding 0.1 -> 0.2 (now re-exported from block-buffer)
digest 0.8 -> 0.9
stream-cipher 0.3 -> 0.4

The MSRV is increased to 1.41, matching the MSRV for the new RustCrypto trait versions. byteorder has been removed from the RustCrypto APIs and dependencies as part of the MSRV bump; I have similarly replaced it with native APIs here.

Thanks for your contribution! MSRV bump for crates with RustCrypto APIs makes sense, but ppv-lite86 is used by rand and must not exceed the rand MSRV of 1.32. I'll update the tests and documentation to reflect the multiple MSRVs.

MSRV changes LGTM! I've pushed a commit to address your other comment.

Anything else you would like me to do to this PR?
Side-note: I notice that there is now a stream-cipher 0.6 (and associated block-cipher 0.8), but they aren't being used in the ecosystem yet. In particular, the current chacha20poly1305 release depends on stream-cipher 0.4, so a c2-chacha release that also depends on stream-cipher 0.4 is useful for interoperability with it.

cc @tarcieri for the above

@newpavlov has done some recent releases but the crates that depend on them haven't gotten bumped yet
We can probably do an across-the-board release of the block cipher and stream cipher crates again, followed by another bump of the AEADs. I think we're mostly done making changes at this point?
All that said, I'd suggest targeting the latest the latest release or you'll have to do the same. We can prioritize getting another release of chacha20poly1305 (and chacha20) out which depend on stream-cipher v0.6 if that's helpful.

Block ciphers are updated in RustCrypto/block-ciphers#138, next I plan to update stream ciphers, then MACs and finally AEADs.

I've bumped block-cipher and stream-cipher.

Rebased on master to fix a merge conflict, and then migrated to stream-cipher 0.7 (which makes a breaking change that was forgotten from 0.6).


Sort reports

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.83%. Comparing base (d009d5d) to head (b704144).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1170      +/-   ##
==========================================
+ Coverage   94.79%   94.83%   +0.03%
==========================================
  Files         104      104
  Lines       12207    12179      -28
==========================================
- Hits        11572    11550      -22
+ Misses        484      480       -4
+ Partials      151      149       -2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


fix exception No such type, issue 154

@Lana243 Can you add the simple test suite of 2 files that reflects the changes, please?

@Lana243 Can you add the simple test suite of 2 files that reflects the changes, please?

@operasfantom, should it be integration or unit test?

@Lana243 Can you add the simple test suite of 2 files that reflects the changes, please?

@operasfantom, should it be integration or unit test?

Unit test is enough, look at regression suite as an example


Does hey-pane shortcut work on Windows 10?

After installing this package, I noticed that it don't work on Windows. I mean, Shift-Ctrl-K instead of Shift-Cmd-K that seems a Mac shortcut. I modified the keymap.json file substituting shift-cmd-k with shift-ctrl-k, but ti didn't work. Any advice?

Can you post the snippet from your keymap.json with the new keymapping?
Also, you can check the keybinding by activating Keybinding Resolver (through the command palette) and trying your new shortcut. If should resolve to hey-pane:toggle-focus-of-active-pane.
Nonetheless, I should add support for Windows out of the box.

Of course, here it is:
{   "atom-text-editor": {     "shift-ctrl-k": "hey-pane:toggle-focus-of-active-pane"   } }

It should be pretty simple to edit, but it's not working on Windows 10. Thank you for your interest in this issue.

{
"atom-text-editor": {
"shift-ctrl-k": "hey-pane:toggle-focus-of-active-pane"
}
}
This is the content of hey-pane.json file. I've just substituted cmd with ctrl.

Don't modify the hey-pane.json file directly. You can assign your own shortcuts in your keymap.cson/json.
To do this, select "Open your keymap" in the Command Palette. Then a file called keymap.cson or keymap.json should open.
If it's a json, add the code you posted.
If it's a cson, add this:
'atom-text-editor':
  'shift-ctrl-k': 'hey-pane:toggle-focus-of-active-pane'

The keymap file should include a big comment block on how to edit this file properly.
To debug the shortcut, you can toggle the Key Binding Resolver (in the command palette) and press the shortcut. You should see a window telling you which command the shortcut is assigned to. If you press shift-ctrl-k, you should see that the hey-pane command get's triggered, if the key binding resolves correctly.

Hi timomeh!
Thank you for pointing me out in the right direction. I was focusing on the wrong file (I know, I'm a noob, just started learning Atom).
Curiously, I've found that Atom x64 for Windows 10 (which at least it's the one I'm using), won't accept keybinds like Shift-Ctrl-K, Shift-Ctrl-I or Shift-Ctrl-L... strange! In the Key Binding Resolver they aren't even tiped... I mean, if I press Shift-Ctrl-K, the Resolver only shows Shift-Ctrl and it seems the "K" is broken, although if I press "K" alone, of course the output is "K". Strange.
Anyway, following your directions I bound Hey Pane to Shift-Ctrl-J, which works flawlessly. Thank you for support! 👍

No problem, I'm here to help 🙂
I'm going to add a keyboard shortcut for windows out-of-the-box when I have some time, so it's very good to know that shift-ctrl-k doesn't work on Windows. Seems like some sort of reserved shortcut on Windows. Thank you for pointing that out!

@Alpha-UMi Cmd is indeed Mac-specific, however on my Linux it translates to the Windows key (called Super down here in Linuxland) and the shortcut Shift-Win-K works fine.


chore: increase windows ci test timeout minutes

Summary
The release is always timeout https://github.com/web-infra-dev/rspack/actions/runs/6080313608

Test Plan

I think this is not caused by timeout? it will increase the ci time, we should revert the timeout back


non cli usage

Since many teams collaborate using python notebooks it may be useful to expose a python module interface which allows users to specify their config with variables and allow downloading of competition datasets. Good idea?

@morenoh149 do you mean being able to use the API within kernels?  If so, that is on my long term to do list, but I don't know when it will be possible.

no, I'd like to use it within notebooks (colab included) to load datasets into memory. Mainly allowing the library to be used by passing in variables for the secret keys instead of relying on a file with the secret keys (the way it currently works).

@morenoh149 : maybe you could rename the issue into e.g. "Improve API design: proper python library not just CLI" ?

The best I could find to merge Kaggle API calls (here: submissions) into my python code was to use os.system(), like this:
import os

# submit current predictions file (-f) using its name as annotation (-m)
cmd = 'kaggle competitions submit ' + competition_name + \
    ' -f ' + local_scores_path+cur_preds_file + ' -m ' + cur_preds_file
print(cmd)

# invoke the Kaggle API CLI using a system call
returned_value = os.system(cmd)

print('Returned value:', returned_value)
assert(returned_value==0)


Adding a Custom Partition Table in a Custom CMake Project (IDFGH-3027)

Environment

Development Kit: ESP32-DevKitC
Kit version (DevKitC): [v4]
Module or chip used: ESP32-WROOM-32D
IDF version: v4.2-dev-910-g4fe04f1
Build System: CMake
Compiler version: xtensa-esp32-elf-gcc (crosstool-NG esp-2019r2) 8.2.0
Operating System: Linux
Using an IDE?: No
Power Supply: USB

Problem Description
I am trying to build a Custom CMake Project by following the documentation provided under API Guides -> Build System -> Using ESP-IDF in Custom CMake Projects. I am also following the example idf_as_lib
My requirement is to add a custom partition table using my partitions.csv file. To do this I have made a sdkconfig file by running the menuconfig tool in the hello-world example project. I was able to build the hello-world example project with the custom partitions.csv successfully.
Now I have copied the sdkconfig and partitions.csv files in the example idf_as_lib. I have edited the CMakeLists.txt file and added SDKCONFIG_DEFAULTS as below:
    idf_build_process(esp32
                    # try and trim the build; additional components
                    # will be included as needed based on dependency tree
                    #
                    # although esptool_py does not generate static library,
                    # processing the component is needed for flashing related
                    # targets and file generation
                    COMPONENTS esp32 freertos esptool_py
		    SDKCONFIG_DEFAULTS ${CMAKE_SOURCE_DIR}/sdkconfig
                    SDKCONFIG ${CMAKE_BINARY_DIR}/sdkconfig
                    BUILD_DIR ${CMAKE_BINARY_DIR})

When I compile the code I get the following error:
ninja: error: 'esp-idf/partitions.csv', needed by 'partition_table/partition-table.bin', missing and no known rule to make it

Note: If I do not use a custom partition table, the program compiles perfectly.
Expected Behavior
The compilation should be successful with a custom partition table.
When I use the esp-idf provided partition table partitions_two_ota_coredump the compilation is successful, so I assume there should not be any problem with custom partition table also.
Actual Behavior
I get compilation error:
Steps to reproduce

Generate a sdkconfig file from the hello_world example menuconfig which has a custom partition. My sdkconfig and partitions.csv files are attached
Copy these files into the example idf_as_lib
Edit the idf_as_lib CMakeLists.txt file to add the following line.

SDKCONFIG_DEFAULTS ${CMAKE_SOURCE_DIR}/sdkconfig

My CMakeLists.txt file is attached

Build the project by running the script build-esp32.sh

Debug Logs
-- The C compiler identification is GNU 8.2.0
-- Check for working C compiler: /home/test1/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /home/test1/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/bin/git (found version "2.7.4")
-- Checking Python dependencies...
Python requirements from /home/test1/esp/esp-idf/requirements.txt are satisfied.
Loading defaults file /home/test1/esp/esp-idf/examples/build_system/cmake/idf_as_lib/sdkconfig...
-- The CXX compiler identification is GNU 8.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/test1/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Check for working CXX compiler: /home/test1/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++
-- Check for working CXX compiler: /home/test1/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /home/test1/.espressif/python_env/idf4.2_py2.7_env/bin/python (found version "2.7.12")
-- Found Perl: /usr/bin/perl (found version "5.22.1")
-- App "" version:
-- Adding linker script /home/test1/esp/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs-time.ld
-- Adding linker script /home/test1/esp/esp-idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /home/test1/esp/esp-idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /home/test1/esp/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script /home/test1/esp/esp-idf/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Adding linker script /home/test1/esp/esp-idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script /home/test1/esp/esp-idf/examples/build_system/cmake/idf_as_lib/build/esp-idf/esp32/esp32_out.ld
-- Adding linker script /home/test1/esp/esp-idf/components/esp32/ld/esp32.project.ld.in
-- Adding linker script /home/test1/esp/esp-idf/components/esp32/ld/esp32.peripherals.ld
-- Configuring done
-- Generating done
-- Build files have been written to: /home/test1/esp/esp-idf/examples/build_system/cmake/idf_as_lib/build
ninja: error: 'esp-idf/partitions.csv', needed by 'partition_table/partition-table.bin', missing and no known rule to make it

Other items if possible

[x] sdkconfig file

idf_as_lib-res.zip

Hi @iqrohan, sorry for the late reply. This was diagnosed to be a bug in idf_build_process. A fix has submitted internally. In the meantime, is it possible to use the existing partition table csv to contain the partition table you want?

Hi @renzbagaporo, as of now I have edited one of the csv files inside the esp idf to fit my requirement. But this feature would be needed in the future.
Thanks

Good to know that you have found a workaround. This issue will be closed once the fix has been submitted.

I also have the same use-case. Can the fix be cherry-picked to the upcoming v4.1?

@galah92  Thanks for reporting, the fix is being back ported to v4.1, but will take some time to appear on GitHub.

@Alvin1Zhang Thanks! Is there an ETA for that? is it days/weeks/months?

@Alvin1Zhang I see this hasn't been backported to v4.1 (nor it can even be found on v4.2-dev). Can something be done regarding this? I rely heavily on this feature in order to being able to build multiple images under one CMake invocation.

@galah92 Unfortunately the backport missed the v4.1 final release, but commit d101da4ca43f44716d91c8c09e891ff97a7282f0 is part of the release/v4.1 branch and will be included in the v4.1.1 release. You could consider checking out the release branch until then.
The v4.2 backport is unfortunately still pending, due to our internal testing requirements branches are frozen and unfrozen periodically.

Hi, I've rechecked following @iqrohan's steps to reproduce and it now works with the fix on master (and on v4.1, v4.0 to which the fixes have been backported to). This might be a different use case @galah92?

@projectgus I'll probably wait for v4.2 / v4.11, thanks
@renzbagaporo I've to get the exact same use case and is able to reproduce it. I'm afraid the fix @projectgus attached (both commits) is not part of v4.1.


Version for Menuhax users

Could you please modify your project to allow the option to download the new Luma3DS.dat file from the github to update, instead of the arm9loaderhax.bin?
Some users simply do not have the time to install arm9loaderhax (like me), or do not want to take the risk of an additional downgrade. This addition to your program would be great for menuhax/coldboot users!

Sounds like a nice idea for a future version!
Since I want to release 1.3.0 to stop people from getting dev updates ASAP, I'll think about doing it for 1.4.

Added in a6b7560 and 8eb8830


Add release workflow

Release a new version only needs a tag:
git tag v0.2.3
git push origin v0.2.3

GitHub actions will help to build artifacts and create a release named Gokins v0.2.3 with those artifacts.

thanks!


Usage with IOptions*

Hello @MrDave1999,
I'm trying to incorporate you're awesome library in my application. However, with the use of IOptionsSnapshot, I endup with key property has not been initialized. There is a need to have a dynamic value at runtime, not necessarily from appsettings.json or environment but from other sources.
What I tried:
builder.Services.AddDotEnv<AppSettings>();
builder.Services.Configure<AppSettings>(builder.Configuration.GetSection("ConfigData"));

Thanks

Solved with Dotnet.Core and it's awesome documentation. Thanks

Hi @kkharji.
I am glad you solved your problem, it would also be very helpful if you could add your solution as a comment, it could be useful to other persons.
Greetings.

Great Idea. Thanks again for your amazing work
var builder = WebApplication.CreateBuilder(args);
new EnvLoader().Load();
builder.Configuration.AddEnvironmentVariables();
builder.Services.Configure<AppSettings>(builder.Configuration);


help_docs: Rename stream to channel.

Before:
$ git grep -i stream help/ | wc -l
166

After:
$ git grep -i stream help/ | wc -l
2

$ git grep -i stream help/
help/include/channels-and-topics.md:![Channels and topics](/static/images/help/streams-and-topics.png)
help/public-access-option.md:    by setting `WEB_PUBLIC_STREAMS_ENABLED = True` prior to proceeding.

We should replace the image as it uses the term 'stream' within image:
Remaining 1

WEB_PUBLIC_STREAMS_ENABLED -- I'll do it as follow-up. Some transition code is required.

Screenshots and screen captures:

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Question (Not a blocker):

Why do we use absolute link to zulip.com in /docs/production/management-command -- Due to this we are unable to update the https://zulip.com/help/archive-a-stream#unarchiving-archived-streams URL with channel . As the changes are currently not deployed so the link is invalid, hence CI fails. I haven't inspected in depth but if possible we should make it relative.

For WEB_PUBLIC_STREAMS_ENABLED, I think we should just post it in the stream as an API change to manage; there's plenty more things like it, and some may not be priorities. That one we might also just want to rename to PUBLIC_ACCESS_OPTION_ENABLED anyway.

Merged, thanks @prakhar1144!


Custom jupyter extension docs

Contains new/updated documentation for new/revised custom jupyter extensions. Directory structure was modified as well.

@grallewellyn

Should ADE Custom Extensions be beneath Platform Technical Documents like so?

This looks like what I intended and what I have locally. Here is an image of the tree from my local build: