Different tab completion rules for different Git versions

The current tab completion file is based on the manual pages from Git 2.13.1. Newer versions of Git could introduce options and arguments that we won't be able to complete. Conversely, we will offer completions to users of older versions of Git that they can't actually use.

Bumping this from v0.14 to v1.0: this is still an important feature, but I don't want to delay releasing some of the big tab completion improvements that we've done recently for something that's more complex, and arguably less important.


Add basic number formatting

Description
Only basic functionalities have been implemented so that formatting can be applied and output even when a 'float' type number comes in.
However, not all functions are fully functional(i.e, decimal rounding or inf value handling.), so it seems that additional functions will need to be supplemented in the future.
Example
## before
print('{:5n}'.format(1234567.1))
>>> Traceback (most recent call last):
    File "<wasm>", line 1, in <module>
    ValueError: Format code 'n' for object of type 'float' not implemented yet

## after
>>> 1.23457e+06

a = '{:.8n}'.format(12345.12345)

type(a)
>>> <class 'str'>

print(a)
>>> 12345.123

Related Issue
#1656

@fanninpm Thanks for your advice! It's definitely better than before 👍👍

The newly added test seems fail in CPython 3.10
Traceback (most recent call last):
  File "/home/runner/work/RustPython/RustPython/extra_tests/snippets/builtin_str.py", line 619, in <module>
    assert '{:5n}'.format(1e+3) == " 1000"
AssertionError

I don't know if this is right way, but I removed @unittest.expectedFailure from the test.test_format.FormatTest.

@notJoon it's the right way! You should also remove the # TODO comments before the decorators though since they aren't needed anymore

I thought this is done except for test decorators but seems still on working, right?


Quality of life suggestion -- add welcome on login

When you log in it's not apparent you're logged in. Would be nice to have a quick welcome

Same on logout, aka "goodbye, {user}"


Renaming the VL arguments to AVL

§5.2 clarifies that the size_t vl argument is used to specify the AVL:

The intrinsics programmer specifies an "application vector length (AVL)" using the argument size_t vl.

I can't speak for others but I've often gotten vl (the register) confused with AVL (the vsetvli argument), and it would appear that there is precedent for clarifying this in the note below:

The actual value written to the vl control register is an implementation defined behavior and is typically not known until runtime. The actual setting of vl, given the provided AVL through the parameter, follows the rules 27 in the specification.

Given this, would it be worthwhile renaming the vl arguments to avl so that we call it what it is?
If it's just a matter of renaming an argument I think this should be backwards compatible.

For most use cases it probably isn't really the AVL because it came from a vsetvli intrinsic that already clipped it. Though I don't have a better name.

For most use cases it probably isn't really the AVL because it came from a vsetvli intrinsic that already clipped it.

Isn't that still the AVL though? The compiler might fold two equivalent vsetvlis into one, but IIUC that's still the same as setting the AVL to the output VL of a vsetvli.

I was thinking about it from the perspective of the programmer that probably wrote something like
while (n != 0) {
  size_t vl = vsetvli_e8m1(n);
  n -= vl;

  a = vadd(a, b, vl);
}

So to them it is the vl because there is a separate AVL they passed to the vsetvli intrinsic. Does the argument to the intrinsics cause confusion since its not the AVL from the programmers perspective in most cases.
I don't think I recommend passing an unclipped AVL to the intrinsics even though it is supported. Do our vsetvli optimizations work as well if you do that?

I don't think I recommend passing an unclipped AVL to the intrinsics even though it is supported. Do our vsetvli optimizations work as well if you do that?

I'm not sure. I agree though that clipping the AVL beforehand is what programmers should continue to do, if we were to rename the argument I was thinking that they would still write the same
while (n != 0) {
  size_t vl = vsetvli_e8m1(n);
  n -= vl;

  a = vadd(a, b, vl);
}

But the intrinsic signature would just make it more clear that they're setting the AVL to the precalculated vl.

Does the argument to the intrinsics cause confusion since its not the AVL from the programmers perspective in most cases.

I think it's more likely to cause confusion if they're not made aware that it's the AVL, e.g. if they accidentally change the SEW/LMUL ratio:
size_t vl = vsetvli_e32m1(n);
__riscv_vadd_vv_i32m1(a, b, vl); // vl guaranteed to equal vl
__riscv_vadd_vv_i64m1(c, d, vl); // vl not guaranteed to equal vl

We typically use vsetvli in most of our rvv applications so for me it makes sense to call that operand vl.
But I see how it can be confusing in some cases. Especially when using fixed vectors and an assumed VLEN, I always find a bit jarring, even if perfectly correct, that I can just use an integer literal as the vl operand, given how used I am to vsetvli.
I guess that vl operand is conceptually more like "a value in the corresponding range [0, vlmax(sew, lmul)] of the intrinsic" (otherwise the value may be clipped and it is not obvious what happens with the operation). That said, I don't have a better name though for me this is closer to vl than to AVL.

Can we revisit this after 1.0?


Akshat 5 august2021 add blog top 10 songs of all time

I'm closing this PR as this has a feature and blog together. You accidently pushed Quotes.jsx which also has some errors. Please make a clean branch from main


Option to change modifier keys

I find that the Command key is larger than the Option key.
Is there a way that the keys could be swapped around, maybe as a user selection?

Are you talking about in macOS? You want to switch the modifier key from say for example Command-N to Option-N?
This is probably outside the scope of this project, but you could try referring to https://osxdaily.com/2017/08/08/create-custom-keyboard-shortcut-mac/ and see if that helps as it isn't specific to just NetNewsWire

This issue isn't specific to NNW, so I'm going to go ahead and close this issue.


project files should be saved in custom user locations

Description
As a user I want to be able to save my project files in a custom location.
Acceptance Criteria

[ ] add "save" menu item
[ ] add "save as" menu item
[ ] add  "open recent" menu item
[ ] the tool should persist the preferred project save location

Did stuff here 7e8ed82045dfd62b07d87da281819aab749a2709


Remove overriden createJSModules

Since of recently in master the unused createJSModules has been removed and ReactPackage's no longer contain the definition of createJSModules. There for it must be removed in order to not yield a compile error. This is breaking change.
https://github.com/facebook/react-native/commit/ce6fb337a146e6f261f2afb564aa19363774a7a8

Thanks, @ptomasroos. I'll need do a bit of ceremony to make a new major revision.

After a bit of back and forth, we've got a non-breaking way to solve this in #120. Shipping imminently 🎉


Question about fragments and graphql-tag/loader

The recent change to deprecate specifying fragments in options of a query raises the question about using graphql-tag/loader. Currently in my app I save both queries and fragments in separate files (.gql) and use a webpack loader (graphql-tag/loader) to preload them into a bundle. If from now on I have to reference fragments in a query using ${...} notation this will definitely break, won't it? This will force me to abondon storing fragments separately and always have them embedded into a query. Is there something I'm missing?

🤔 good question!
@tmeasday @stubailo Is there also an alternative way for defining fragments with graphql-tag?

Unless there is some way to require inside a .graphql file?
Perhaps we need to some utility to attach a fragment to a document after the fact?
import myQuery from './myQuery.graphql';

const fullQuery = attachFragment(myQuery, FooComponent.fragments.barFragment);

I'm not sure if it's a good thing to have an incomplete myQuery.graphql in this scenario, and rely on the consumer to attach the fragment each time. If myQuery is referencing FooBarFragment, then you would need to run the attachFragment any time you use it, in which case it'd be better for myQuery.graphql to just do the attaching somehow (cf require-ing above).

Hmm, I'll admit that we didn't think of this when making the change, although I still think it was a good change to make. I think having some way to import fragments is the way to go, and I can imagine something extremely clever:
query myQuery {
  todos {
    name
    ...TodosFields @import(from: "../somewhere")
  }
}

The way Facebook handles this is by actually merging all .graphql files into one so that all queries and fragments are in one document, but I don't know if that fits the JS/Webpack mindset where things should be modular and declare dependencies explicitly?
Open to either solution, either way let's have this discussion at graphql-tag since I think Apollo Client shouldn't deal with fragments via options anymore: https://github.com/apollostack/graphql-tag

My current workaround is importing file content of .graphql file using raw-loader, and use it like this:
 const myQuery = gql
${myQuery}
${fragments}
;


Adding Map Property

Hi, there First of all good library. I am trying to add MapProperty in property section
The file I am trying to open is
Active.zip
It contain MapProperty So I am trying to add that property and then getting this error

The sample data chart I created is

Name
Bytes
Value

Size of Name string
13 00 00 00
19bytes

Name string
50 69 63 6b 55 70 43 6f 75 6e 74 53 65 74 74 69 6e 67 00
PickUpCountSetting\0

Size of Type string
0c 00 00 00
12bytes

Type string
4d 61 70 50 72 6f 70 65 72 74 79 00
MapProperty\0

Size of Stored data
38 00 00 00
56bytes

Padding
00 00 00 00
4 null characters

Size of StoredType string
0c 00 00 00
12bytes

StoredType string
49 6e 74 50 72 6f 70 65 72 74 79 00
IntProperty\0

Size of StoredType string
0c 00 00 00
12bytes

StoredType string
49 6e 74 50 72 6f 70 65 72 74 79 00
IntProperty\0

padding
00 00 00 00 00
5 null characters

Stored Data
06 00 00 00 89 8a 01 00 96 00 00 00 8b 8a 01 00 dc 00 00 00 8c 8a 01 00 fa 00 00 00 8d 8a 01 00 a4 00 00 00 8e 8a 01 00 fa 00 00 00 90 8a 01 00 96 00 00 00
e.g (Map([["apples", 500],["bananas", 300],["oranges", 200]]);)

The sample code I am using from upper file is

And the Code I wrote I almost like StructProperty. Sorry I don't know java script that much but can understand to some extend.
import { Property } from './index.js'
import { PropertyFactory } from '../factories/index.js';
import { Serializer } from '../../utils/Serializer.js';
import { SerializationError } from '../PropertyErrors.js';

export class MapProperty extends Property {
    constructor() {
        super();
        this.StoredPropertyType = "";
        this.Properties = [];
    }
    get Size() {
        let size = this.Name.length + 4;
        size += this.Type.length + 4;
        size += 8; // 4 byte size + 4 byte padding
        size += this.StoredPropertyType.length + 4;
        size += 5; // 5 byte padding
        for (let i = 0; i < this.Properties.length; i++) {
            size += this.Properties[i].Size;
        }
        return size;
    }
    get HeaderSize() {
        let size = this.Name.length + 4;
        size += this.Type.length + 4;
        size += 8;
        size += this.StoredPropertyType.length + 4;
        size += 5;
        return size
    }
    get Count() {
        return this.Properties.length;
    }
    deserialize(serial, size) {
        console.log(`Deserializing ${this.Name} Size: ${size}`)
        serial.seek(4);
        this.StoredPropertyType = serial.readString();
        serial.seek(5);
        let end = serial.tell + size;
        let i = 0;
        while (serial.tell < end) {
            let Name = this.StoredPropertyType;
            let Type = 'Tuple';
            let prop = PropertyFactory.create({ Name, Type })
            prop.deserialize(serial)
            this.Properties.push(prop);
            i++;
        }
        // console.log(`Done Deserializing ${this.Name} Offset: ${serial.tell}`)
        return this;
    }
    serialize() {
        let serial = Serializer.alloc(this.Size);
        serial.writeString(this.Name);
        serial.writeString(this.Type);
        serial.writeInt32(this.Size - this.HeaderSize);
        serial.seek(4);
        serial.writeString(this.StoredPropertyType);
        serial.seek(5);
        for (let i = 0; i < this.Properties.length; i++) {
            serial.write(this.Properties[i].serialize());
        }
        if (serial.tell !== this.Size)
            throw new SerializationError(this);
        return serial.Data;
    }
    static from(obj) {
        let map = new MapProperty();
        map.Name = obj.Name;
        map.Type = obj.Type;
        map.StoredPropertyType = obj.StoredPropertyType;
        map.Properties = [];
        if (obj.Properties !== undefined)
            obj.Properties.forEach((prop) => map.Properties.push(PropertyFactory.create(prop)));
        return map;
    }
}

And one more change I want you to do add a feature to allow the drop of bytes in unknown maps

It's been a while since I've done anything with this. I don't have much time to dedicate to making changes to this code.
So the error is happening on deserialize. It may be beneficial to add some console outputting what's been deserialized leading up to the error. Offset errors usually occur because you miscounted some padding or maybe there's an extra field somewhere, or sometimes the 'size' being relied upon doesn't intuitively make sense.
In the case of Structs, they have that 'StoredPropertyType' field as well as multiple 'size' fields. I'm not at my PC rn. I would suggest just trying to get the deserializer to work. I'll take a look when I can.

Thanks for reply. I already spend my whole day reading your code and understanding pattern. I send you the code snippet please take a look at it and let me know what I did wrong. I am also trying to figure out by my self thanks.

So I'm looking at the raw data, and it looks like a different structure from what you think it is.
In your example you give, Map[['apples',500],[bananas,300]...], but that's not what the data looks like. Instead, it looks more like PickupCountSetting [Int,[number,number]]. The IntProperty actually stores 2 integers. So the data you are trying to extract does not contain any strings. That said, one IntProperty will only store 13 bytes of data, two 4-byte integers and 5 bytes of padding between them.
So this might be more of a contiguous array of integers, you need to find out how many integers are actually being stored and the padding between them. Also, make sure the size of 56-bytes is correct. Where does the 56 bytes begin and end? If all 56 bytes are integers, then there could be as many as 14 different integers.
So first, you should rewrite your deserializer to work for this specific instance, even if the data is gibberish, just try to get the deserializer to work with it. Figure out what that data is, and how it's structured. Once it is able to deserialize this particular MapProperty, then you can try to figure out a way to make it generic. It looks like there is one MapProperty that stores a StrProperty near the end of the file. Don't worry about that yet, just figure out this one.
Here's what I did, and it does deserialize this instance, but idk what the data even is:
deserialize(serial, size) {
        console.log(`Deserializing ${this.Name} Size: ${size}`)
        serial.seek(4);
        this.StoredPropertyType = serial.readString();
        serial.seek(5);
        let end = serial.tell + size;
        console.log(`Stored Type: ${this.StoredPropertyType} Size: ${size}`)
        let i = 0;
        while (serial.tell < end) {
            // let Name = this.StoredPropertyType;
            // let Type = 'Tuple';
            // let prop = PropertyFactory.create({ Name, Type })
            // prop.deserialize(serial)
            let prop = serial.readInt32();
            this.Properties.push(prop);
            i++;
        }
        console.log(`Done Deserializing ${this.Name} Offset: ${serial.tell}`)
        console.log(this);
        return this;
    }

And the data from this structure is:
MapProperty {
  Name: 'PickUpCountSetting\x00',
  Type: 'MapProperty\x00',
  StoredPropertyType: 'IntProperty\x00',
  Properties: [
<PHONE_NUMBER>,<PHONE_NUMBER>,
         31092,          0,
             6,     101001,
           150,     101003,
           220,     101004,
           250,     101005,
           164,     101006
  ]
}

You may want to look at the structure of the ArrayProperty and see if it's anything like that. There's a lack of the use of None in the sav file, so I think this is just a unique data structure that can't really be derived from the StructProperty or the ArrayProperty. Also, ArrayProperty structure is different depending on the structures stored inside, which is annoying and weird.

Now it is working for the specific data like for integer but at some point it will get string Property may be both the String and Integer properties then what should I do for them?

The data is more like nested arrays


Reset all motors to factory defaults before initializing to what the code generators is configured to

Do this during the replacement of Dragonxxxx classes


Arguments get lost in compilers from env vars that don't start with known wrappers

Sorry for filing this with such a long summary, but I'm trying to fix a Firefox bug about sources compiled via Rust build scripts not using the same compiler as the rest of the Firefox build, and I found weird bustage in one of our build configurations that uses a custom compiler wrapper. I was able to create a reduced testcase to reproduce the issue.
luser@eye7:/build/snippet$ CC="$PWD/ccwrap gcc -Wall" cargo build -vv
   Compiling cc v1.0.17
     Running `rustc --crate-name cc /home/luser/.cargo/registry/src/github.com-1ecc6299db9ec823/cc-1.0.17/src/lib.rs --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=78b178596e16bae7 -C extra-filename=-78b178596e16bae7 --out-dir /build/snippet/target/debug/deps -L dependency=/build/snippet/target/debug/deps --cap-lints warn`
   Compiling snippet v0.1.4-alpha.0 (file:///build/snippet)
     Running `rustc --crate-name build_script_build build.rs --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=43dbc33b3a267a6e -C extra-filename=-43dbc33b3a267a6e --out-dir /build/snippet/target/debug/build/snippet-43dbc33b3a267a6e -C incremental=/build/snippet/target/debug/incremental -L dependency=/build/snippet/target/debug/deps --extern cc=/build/snippet/target/debug/deps/libcc-78b178596e16bae7.rlib`
     Running `/build/snippet/target/debug/build/snippet-43dbc33b3a267a6e/build-script-build`
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("0")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = Some("/build/snippet/ccwrap gcc -Wall")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running: "/build/snippet/ccwrap" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-m64" "-Wall" "-Wextra" "-o" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/test.o" "-c" "test.cpp"
cargo:warning=/build/snippet/ccwrap: 3: exec: -O0: not found
exit code: 127
thread 'main' panicked at '

Internal error occurred: Command "/build/snippet/ccwrap" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-m64" "-Wall" "-Wextra" "-o" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/test.o" "-c" "test.cpp" with args "ccwrap" did not execute successfully (status code exit code: 127).

', /home/luser/.cargo/registry/src/github.com-1ecc6299db9ec823/cc-1.0.17/src/lib.rs:2180:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: failed to run custom build command for `snippet v0.1.4-alpha.0 (file:///build/snippet)`
process didn't exit successfully: `/build/snippet/target/debug/build/snippet-43dbc33b3a267a6e/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("0")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = Some("/build/snippet/ccwrap gcc -Wall")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running: "/build/snippet/ccwrap" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-m64" "-Wall" "-Wextra" "-o" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/test.o" "-c" "test.cpp"
cargo:warning=/build/snippet/ccwrap: 3: exec: -O0: not found
exit code: 127

--- stderr
thread 'main' panicked at '

Internal error occurred: Command "/build/snippet/ccwrap" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-m64" "-Wall" "-Wextra" "-o" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/test.o" "-c" "test.cpp" with args "ccwrap" did not execute successfully (status code exit code: 127).

You can see that when I pass CC="$PWD/ccwrap gcc -Wall" cc drops the gcc -Wall bits and so everything breaks. This shows up even if you don't use a compiler wrapper but try to pass extra compiler arguments in CC, like:
luser@eye7:/build/snippet$ CC="gcc -Wall" cargo build -vv
       Fresh cc v1.0.17
   Compiling snippet v0.1.4-alpha.0 (file:///build/snippet)
     Running `/build/snippet/target/debug/build/snippet-43dbc33b3a267a6e/build-script-build`
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("0")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = Some("gcc -Wall")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running: "gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-m64" "-Wall" "-Wextra" "-o" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/test.o" "-c" "test.cpp"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
AR_x86_64-unknown-linux-gnu = None
AR_x86_64_unknown_linux_gnu = None
HOST_AR = None
AR = None
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
running: "ar" "crs" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/libtest.a" "/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out/test.o"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
cargo:rustc-link-lib=static=test
cargo:rustc-link-search=native=/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out
     Running `rustc --crate-name snippet src/main.rs --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=8031abc25a926017 -C extra-filename=-8031abc25a926017 --out-dir /build/snippet/target/debug/deps -C incremental=/build/snippet/target/debug/incremental -L dependency=/build/snippet/target/debug/deps -L native=/build/snippet/target/debug/build/snippet-47fc6a6579d8af3c/out -l static=test`
    Finished dev [unoptimized + debuginfo] target(s) in 0.40s

You can see in this case running with CC="gcc -Wall" the -Wall gets dropped.
This seems to be because when an env var isn't a single path, it gets split and the remaining args get stored in cc_wrapper_args but then those are only used if cc_wrapper_path is Some.

Oh dear, sorry about that! Definitely a bug in cc-rs

Thanks for the quick fix! If things are in a good place for you to publish a new version that would be helpful for me to finish that Firefox bug.

Done!


feat(flags): add ApplicationFlags.application_auto_moderation_rule_create

Summary
Implements discord/discord-api-docs#6021
Note that I am unsure if the name of the flag is okay, I've just used what it was called in the documentation.
Checklist

[x] If code changes were made then they have been tested.

[x] I have updated the documentation to reflect the changes.
[ ] I have run task pyright and fixed the relevant issues.

[ ] This PR fixes an issue.
[x] This PR adds something new (e.g. new method or parameters).
[ ] This PR is a breaking change (e.g. methods or parameters removed/renamed)
[ ] This PR is not a code change (e.g. documentation, README, ...)

The client does say that, however, remember that some names on the client are different on the API. Such as how there is 'guild' and 'server'


Improve Geo.JSON encoding and decoding

Fix an issue when you pass a random type to the decoding process
Create Encoder and Decoder modules following Poison code style, I think this improve the readability

Breaking changes
Geo.JSON.encode becomes Geo.JSON.encode!
Geo.JSON.decode becomes Geo.JSON.decode!
Because normally ! from code style on Elixir people would expect a error to be rise if something goes wrong.

Probably I could keep cleaning up with the other encoders as well.

Thanks!


Node.js support

I added Node.js module support but have maintained backwards compatibility for including it strait into a HTML page.
I also added a package.json so that you can publish this module on NPM. I put your name in the author field, I hope that is OK?
Thought it might be nice because I use Browserify a lot and it is really handy to handle dependencies through NPM at the same time as leveraging the encapsulation of CommonJS modules.

:+1:

Very very cool! Thanks for putting this together!

Thank you for publishing to NPM! Now I can switch my dependency reference from GitHub to NPM, great job!


ClientModel: API Updates

Create(IModel)->Create(T)
Initial ClientRequestException feedback

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.Core


[DISCUSS] Could we have a different ANN algorithm for Learned Sparse Vectors?

Description
Learned Sparse Vectors claim to combine the benefits of sparse (i.e. lexical) and dense (i.e. vector) representations
From https://en.wikipedia.org/wiki/Learned_sparse_retrieval:

Learned sparse retrieval or sparse neural search is an approach to text search which uses a sparse vector representation of queries and documents.[1] It borrows techniques both from lexical bag-of-words and vector embedding algorithms, and is claimed to perform better than either alone. The best-known sparse neural search systems are SPLADE[2] and its successor SPLADE v2

From https://zilliz.com/learn/enhancing-information-retrieval-learned-sparse-embeddings:

Learn sparse embeddings denote sparse vector representations of data crafted through sophisticated machine learning models such as SPLADE and BGE-M3. Unlike traditional sparse vectors, which rely solely on statistical methods like BM25, learned sparse embeddings enrich the sparse representation with contextual information while retaining keyword search capabilities. They can discern the significance of adjacent or correlated tokens, even if not explicitly present in the text, resulting in a "learned" sparse representation adept at capturing relevant keywords and classes.

A famous model for such sparse representations of documents is SPLADE (https://github.com/naver/splade)
Paper
Came across Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (https://arxiv.org/pdf/2404.18812) which shows promising benchmarks for KNN search over learned sparse vectors:

Experimental results show average per-query latency in microsecond territory on various sparse embeddings of Ms Marco. Impressively, Seismic outperforms the graph-based winning solutions of the BigANN Challenge by a factor of at least 3.4 at 95% accuracy on Splade and 12 on Efficient Splade, with the margin widening substantially as accuracy increases. Other baselines, including state-of-the-art inverted index-based algorithms, are consistently one to two orders of magnitude slower than Seismic

Figure 3 in the linked paper summarizes the design of the algorithm:

The design of Seismic. Inverted lists are independently partitioned into geometrically-cohesive blocks. Each block is a set of document identifiers with a summary vector. The inner product of a query with the summary approximates the inner product attainable with the documents in that block. The forward index stores the complete vectors (including values).

Learned Sparse Vectors seem to be naturally compatible with inverted indexes, and many aspects of the algorithm are already implemented in Lucene
Could we use this for faster KNN search when sparse vectors are used?

What I wonder is: how can Lucene help with this? I feel like we have all the primitives available to enable Splade-style search and retrieval, but maybe there is something missing? IIRC there needs to be a per-term score, but I think we do have the ability to store custom term frequencies and to override similarity in order to do the appropriate combination of these term scores, so I think those are the ingredients needed for this. Maybe it's a case of trying it and seeing if there are some features it would be helpful to embed in the Lucene layer, or if indeed we can build this "on top"?

There might be a better format than just terms. But I would assume the bipartite graph stuff would help here.
Additionally, I would expect the most benefits to be made at query time. Looking at the newer research out of the splade folks, they are optimizing the query time by adjusting the scoring, not really the storage.
Maybe a better query would be a good first step.

I found this recent paper by well-known people in the IR efficiency space quite interesting: https://arxiv.org/pdf/2405.01117. It builds on inverted indexes and simple/intuitive ideas:

BP reordering, that Ben alluded to and that Lucene already supports, it naturally clusters documents with similar terms together,
Block-max WAND, which Lucene supports,
Anytime ranking on document ordered indexes (https://arxiv.org/pdf/2104.08976), ie. ranking ranges of doc IDs that have the best impact scores first in order to optimize pruning. Something Lucene doesn't support at the moment but that look doable and generally useful.
Unsafe top-k search via termination conditions and skipping blocks that are barely more competitive than the current top-k-th hit.
Query term pruning, which sounds like a good idea in general for learned sparse retrieval when the model generates many terms.
Scoring high-frequency / low-scoring terms via a forward index instead of an inverted index.

I have recently been interested in this direction and plan on spending non trivial amount of time on this over the next few weeks. Assuming we haven't started dev on this, I am assigning it to myself.
What I have been hacking on is impact sorted prioritisation of docIDs during the ranking phase (especially for custom rankers), so that would probably be the first thing that comes out of this ticket.


Treecrumb

The treecrumb should start with "Federal Council", not with "Federal Administration".

SWG-83

Fixed by a592cd580acfa6a698f5ad5cf6d9e46700a5cb96


Save image to server and send image name to MySQL

Hello Friend. Fastastic your code!
How to save image to server and send image name to MySQL?
See the link: https://jsfiddle.net/ct0uk1as/2/
Example directory: /home4/host/public_html/testes/
Thank you very much.

Since this is not a bug or development request, perhaps stack overflow would be a better place to ask for development help with your project.

Hi, if you want to save the signature in database I suggest to use the format SVG. This format is very lite and only you need to keep the file path in your DB to have access.


No way to stop bad admin music

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAH

working as intended

survey says so

For context to other people here, metalgearsloth is the one who force-plays a lot of music.

grudgecode inbound

i think a volume slider for announcements, round start and round end audio, and admin sounds would be appreciated

i see no issue

Regardless of anything else here, being able to trust the game will be muted is important, it could really ruin someones day. Other then that, if we did have someone recording the game, and they wanted to upload it, or stream, they would probably get really upset if a copyrighted song started playing.

Memes aside this is an actual issue in a number of ways; If I want the game muted, I want the game muted.

ss13 just has a "disable/enable admin music" toggle

Bad music tyranny will soon be at an end.

I had to disable the startup menu song as well because none of the volume sliders affected it.


Use specified signed runtime store archives when available for installers

Addresses https://github.com/aspnet/MetaPackages/issues/155.
This will allow our builds to pick up the latest archives from Coherence-Signed instead of relying on myget which tends to pick up older builds due to lag in indexing.

Not sure exactly how this will work. Isn't Build.RS.nupkg only on the signed feeds (aspnetcore-dev), and not aspnetcore-ci-dev?

The Linux agent will have the drop share mounted and we will be grabbing the artifacts from Coherence-Signed directly instead of going through myget. This way we can avoid mismatches caused by myget delays.

Would it make more sense to use an Artifact Dependency between the builds to avoid the dotnet-restore altogether?

The dotnet-restore is only done when the drop-share isn't available. Artifact dependencies will work too, but we need to update the mechanism to fallback to myget when it's not available. Unless we don't want to preserve self contained builds without access to drop share.


Virtualise the Autocomplete component

The Autocomplete component is quite unresponsive when there is a lot of data. This can be improved by using react-virtualized, for example.

Ticket closed as autocomplete component will be updated as part of the broader refactor of the genetics-app codebase as noted in #511


Changed Psbt to SpendTransaction.

ManagerImportSendTransactionState currently stores the psbt directly in psbt_imported. This has been updated to SpendTransaction. Following this change is a chain of minor changes under impl State for the same.
The implemented test only checks if the parsing for a psbt string happens through SpendTransaction::from_psbt_str. This PR resolves #79

As suggested, state is being checked with the valid psbt. Since the daemon is currently only dealing with a single transaction, I assumed that setspendtx is being called upon it and the daemon responds with an ok. Let me know if there is something that needs to be fixed. Thanks :D

This request set_spend_tx must be mocked with a response.
Since SpendTxMessage::Import triggers update_spend_tx and not set_spend_tx, should I mock the response for this instead?

Yes your code is correct, it is 'updatespendtx' that need to be mocked.


Merge two Url with + operator

I would like to add two URL to create a new one, for example:
    Url("https://example.com/api/v1") + Url("/settings")
But, I believe that we can add one Url with a string too like that:
    func + (_ url: Url, _ complementary: String) -> Url {         Url("\(url.value ?? "")\(complementary)")      }

Great suggestion, feel free to make a PR. I agree that Url + String and Url + Url would be good.


Question of evaluate result on Waymo dataset

Hello! I wonder know whether you remove the objects that out of the front camera’s field of view in validation set when you evaluate your model on waymo dataset.

Yup that's correct, only objects in the FOV of the front camera are included during evaluation.

Thanks for help!

Hello! I wonder know whether you remove the objects that out of the front camera’s field of view in validation set when you evaluate your model on waymo dataset.

Hello, could you please tell me how to remove the objects out of fov while evaluation?


Unable to obtain cookies using gateway parameter

I'm trying to get a response from my PA Oncloud server. I used the below code for it. Initially when I use the Gateway URL in the address bar of the browser, it will take me to IBM sign in page. Once I sign in, I will be able to see a blue screen with my name at the corner (looks like they have utilized IIS).
Now when I send a get request to the IBM login page, I obtain some cookies (CAMPassport was not available) and the response code is 200.
But when I do the same to my Gateway url, I do not get any cookies and the response code is 401.
I'm using the CLIENTURI parameter in the config file as Gateway parameter.
IBM sign in url = https://idaas.iam.ibm.com/idaas/mtfim/sps/authsvc?PolicyId=urn:ibm:security:authentication:asf:basicldapuser
My Gateway parameter = https://host.planning-analytics.ibmcloud.com/ibmcognos/cgi-bin/cognosisapi.dll
The code I used:
import requests
from requests_negotiate_sspi import HttpNegotiateAuth

response = requests.get("https://idaas.iam.ibm.com/idaas/mtfim/sps/authsvc?PolicyId=urn:ibm:security:authentication:asf:basicldapuser", auth=HttpNegotiateAuth())
response1 = requests.get("https://host.planning-analytics.ibmcloud.com/ibmcognos/cgi-bin/cognosisapi.dll", auth=HttpNegotiateAuth())
print(response)
print(response1)

TM1py version - 1.3.1
How do I establish a connection in this case?

IBM cloud only supports interactive accounts, that was the issue.
I'm closing this ticket.


Not able to query the first record of a table when using Firebase as a datastore.

I created a Users table with username as a primary key and a password field and inserted three users. I'm able to query 2nd and 3rd users but not the first. I checked and double checked in case I was making any typing errors. I'm using the below query
getDB().select().from(usr)
	.where(lf.op.and(usr.username.eq(uname),  usr.password.eq(pswd)))
	.exec().then(function (rows) {
	console.log('user', rows);

Additionally, I deleted the first record from the firebase dashboard and the second record became inaccessible.
I tried querying using just the password in the where clause and it worked. Writing the query using the primary key username in the where clause is not returning the first record.

Deleting all the data on firebase and local indexeddb instance solved the problem. Closing the issue.


Update lists.ts

Add Encounters project to derivatives in lists.ts

Done, thank you!

For reference to others, etherscan link to verified contracts. Maybe you would consider adding it to the Twitter + Website? Would make it easier for everyone to verify the contracts they are using.


merge upstream/main into develop

Testing this update here https://github.com/thebrowsercompany/arc/pull/19495


Hardcoded port 80 conflicts with file transfer http server

Xochitl has a built-in HTTP server that is necessary for transferring files onto the device without having a cloud account.  The default port-80 server of pipes-and-rust conflicts.  It would be helpful to have a way to override the default, e.g., with a command line argument or config file.

Oh, I didn't notice that in my testing.
Probably because I have a cloud account.
I'll think about which way to implement the port override and implement it in the coming weeks.
I'd do it sooner, but am currently short on time.

Thanks for a great tool, btw!
I'm likewise short on time (and haven't had much of a chance to play with rust), but if I get a few moments I'll do the same.
A command-line argument and/or environment variable should be sufficient.  Both can be easily overridden in the systemd config file.

Good point!
When testing, I usually don't use systemd to start pipes-and-rust, so I forgot that the file basically serves as a config file.
I'll add the environment variable soon.
I think that should not take too much time.


FLEX-2623 ~ Removes SLFJ injection for netty.

this change will result in no logging from netty directly, instead, logback logging of component is used

All is well.


[20.03] mutt: apply patch for CVE-2020-28896

Motivation for this change
mutt has improper handling of broken IMAP connections, this could result
in authentication credentials being sent over an unencrypted connection,
without $ssl_force_tls being consulted.
https://security.archlinux.org/CVE-2020-28896
https://gitlab.com/muttmua/mutt/-/commit/04b06aaa3e0cc0022b9b01dbca2863756ebbf59a
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Tested using a basic imaps connection OK
Result of nixpkgs-review pr 104583 1

2 packages built:

grepm
mutt (mutt-with-sidebar)

Please cherry-pick this change according to https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md from release-20.09.

Please cherry-pick this change according to https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md from release-20.09.

The commit in 23db21e3138f02f5318ef7539bbf164868199351 (from #104584) was cherry picked from this commit 4586b2f0d0cce2916766dfcd1b717c0940d865ef. So I'm not quite sure how to proceed?

https://github.com/NixOS/nixpkgs/commit/4586b2f0d0cce2916766dfcd1b717c0940d865ef does not belong to any branch in nixpkgs. And since this is a backport for a commit created in #104584, please use git cherry-pick -x a631b2d169e0e7c30a9eb31d3ca67f45ee8eba1c.

I apparently missed this, but https://github.com/NixOS/nixpkgs/commit/4586b2f0d0cce2916766dfcd1b717c0940d865ef is not pointing to any commit in nixpkgs, so the reference isn't very helpful.
If https://github.com/NixOS/nixpkgs/commit/23db21e3138f02f5318ef7539bbf164868199351 was originally only commited to release-20.09, then it the backport to release-20.03 IMO should originate from there.

I apparently missed this, but 4586b2f is not pointing to any commit in nixpkgs, so the reference isn't very helpful.
If 23db21e was originally only commited to release-20.09, then it the backport to release-20.03 IMO should originate from there.

If this PR (which contains 4586b2f0d0cce2916766dfcd1b717c0940d865ef) is merged into nixpkgs, then the reference from 23db21e3138f02f5318ef7539bbf164868199351 to 4586b2f0d0cce2916766dfcd1b717c0940d865ef would resolve?
I agree that it would probably be best to port from 20.09 to 20.03, and not from 20.03 to 20.09. Sorry about that.

Oh boy, yes. Now I see. Nothing that can be done about that now.


Improved timed command execution

Older timeout command versions may not have the "-k" option.

Fixed in https://github.com/bioinform/metasv/pull/86


Too many requests

If there are many pages of transactions and all of them load simultaneously the API will return 429 (too many requests)

Only ~50 requests did respond with 200 OK.
Requests should be limited to an amount that is low enough (maybe 10) then on success continue or in case of an error wait and retry.

Potentially helpful: The API response exposes its ratelimits
x-ratelimit-limit: 60
x-ratelimit-remaining: 59


udp: fix msghdr creation on musl libc

https://github.com/rust-lang/libc/issues/2344
Right now building for musl also requires updating glommio to a git version. (post- https://github.com/DataDog/glommio/pull/590)

Thanks.


Don't throw if a reparsed list has no parent

Fixes the following edit:
f(1);
-> Edits 1 to ); g(
-> Reparse FAILS
-> Fall back to reparsing the parent
-> Reparse f(); g(); as two separate blocks
-> Finished
Sample edits that throw an exception prior to this change:

To understand why this happens, set a conditional breakpoint in Editor::reparse where parent is assigned:

Editing a socket in a way that breaks the parent block into two or more blocks hits this breakpoint:

At this point parent.type throws because parent is null:

The catch block suppresses the exception, but I believe the intent of this try/catch is to catch parser errors, not the error thrown when parent is null.


Return _meta with all responses

_meta can contain additional metadata regarding the request/response.
For now, it only includes the original request and response as objects.
Depends on https://github.com/lune-climate/openapi-typescript-codegen/pull/67

@jstasiak I've changed a few things. Re-review?


ENH: stats: add bootstrap_ci for estimating confidence intervals of n-sample statistics

Reference issue
gh-7949
gh-4399
What does this implement/fix?
This adds a function bootstrap_ci that estimates the confidence interval around a statistic. It implements three methods: the 'percentile' bootstrap, the 'basic' (reverse percentile) bootstrap, and 'BCa' (bias corrected and adjusted) bootstrap. It currently expects the statistic to be vectorized, but the docstring shows how to use np.apply_along_axis, and in a followup PR we can add the option that would do take care of this for the user automatically (e.g. using an approach like in gh-13899).
Additional information
Here's an example that generates one confidence interval, then repeats the same process to generate 1000 confidence intervals, each time with different data sampled from the same distributions. It shows that the confidence interval contains the true value of the statistic (very nearly) the expected number of times.
import numpy as np
from scipy.stats import norm, bootstrap_ci
np.random.seed(1)

n1 = 100  # size of sample 1
n2 = 120  # size of sample 2
n_resamples = 1000  # number of bootstrap resamples used to form each CI
confidence_level = 0.9

# The statistic we're interested in is the difference in means
def my_stat(data1, data2, axis=-1):
    mean1 = np.mean(data1, axis=axis)
    mean2 = np.mean(data2, axis=axis)
    return mean1 - mean2

# The true difference in the means is -0.1
dist1 = norm(loc=0, scale=1)
dist2 = norm(loc=0.1, scale=1)
stat_true = dist1.mean() - dist2.mean()

data1 = dist1.rvs(size=n1)
data2 = dist2.rvs(size=n2)

ci = bootstrap_ci((data1, data2),
                  statistic=my_stat,
                  confidence_level=confidence_level,
                  n_resamples=n_resamples,
                  method='basic',
                  axis=-1)

print(ci)  # (-0.3625395759148521, 0.06064948717410179); contains true value

# Do the same thing 1000 times. (The code is fully vectorized.)
n_replications = 1000
data1 = dist1.rvs(size=(n_replications, n1))
data2 = dist2.rvs(size=(n_replications, n2))
ci = bootstrap_ci((data1, data2),
                  statistic=my_stat,
                  confidence_level=confidence_level,
                  n_resamples=n_resamples,
                  method='basic',
                  axis=-1)

# ci contains vectors of lower and upper confidence interval bounds
ci_contains_true = np.sum((ci[0] < stat_true) & (stat_true < ci[1]))
print(f"The {confidence_level*100}% confidence interval contained the true "
      f"value of the statistic in {ci_contains_true} out of {n_replications} "
      "replications.")

produces:
(-0.3625395759148521, 0.06064948717410179)
The 90.0% confidence interval contained the true value of the statistic in 904 out of 1000 replications.

Currently, the BCa method works only for one-sample statistics. It uses the jackknife to estimate the acceleration factor, and I haven't read up on how to do jackknife resampling with multiple samples (each potentially of a different size). If you can point me to a reference, I'd appreciate it! Thoughts for extending it to multi-sample statistics added here.
Currently, there are only unit tests for input validation. I have some ideas for very slow bootstrap_ci tests, and I can also think of some private function-level tests, but I'd be interested in thoughts on reasonably fast bootstrap_ci tests that should be added. Added some basic tests of functionality. Could add unit tests of private functions. Carefully tests private functions individually.
Some features we could consider adding in a separate PR:

BCa method with multi-sample statistics.
Convenience parameter paired=False for paired-sample statistics. (I show how to do it in the docstring.)
Convenience parameter vectorized = True that indicates whether the provided function is already vectorized. If not, we can apply np.apply_along_axis for the user.
A batch parameter so the user can select the size of the batches used to calculate the statistic for all the resampled data. Currently it's all done in one batch, but that can take a ton of memory, so some sort of control would be good.
Performing the bootstrap-resampling with Generator.choice if available
Pythranize the resampling functions - especially an Nd extension of the jacknife, which would enable use of the BCa method with multi-sample statistics
Parallelize the computations. The code avoids loops, so it shouldn't be too hard make use of a GPU.
Write a decorator (e.g. like in gh-13312) to add a method for calculating confidence intervals (e.g. confidence_interval) to the result returned by existing statistical tests.
Write a class that incorporates the functionality of this PR, gh-13312, and gh-13899 to standardize and simplify the implementation of statistical tests, much like rv_discrete/rv_continuous standardizes and simplifies the implementation of a statistical distribution (currently under development, mostly waiting on merge these PRs to merge, first).

Tested again; looks good to me!

Tested again; looks good to me!

Marking as draft, but I need input on how to do BCa for multi-sample statistics and what sort of tests should be added.

I didn't find much about BCa bootstrap CIs for two-sample statistics except Quantifying effects in two-sample environmental experiments using bootstrap confidence intervals , which suggests the obvious generalizations of the jackknife for two-samples:

Of course, this generalizes to n-sample statistics, but the complexity is the product of the number of observations in each sample. This suggests that we could sum over randomly sampled (i, j) pairs instead of all combinations, or perhaps the ideas in Jackknifing two-sample statistics would be useful.
Should we implement the method suggested in Equation 2 above for this PR or leave it to a follow-up? I suppose we could always add as an option different ways of computing the acceleration factor.
On one hand, I don't want to increase the scope of the PR beyond what is necessary, but on the other hand, if you think BCa should be the default, then maybe we need to have at least a basic n-sample implementation.

There is an example of computing multiple CIs in the docstring, but it might be nice to have another that calls out this feature explicitly and shows the result

I tried to address this in 979ab7135.

@WarrenWeckesser thanks for the review; I think I addressed all the changes. The only one I didn't implement was use of ndtr(i). The overhead is negligible in context, and I value the clarity of stats.norm.cdf/pdf over the performance optimization.

@WarrenWeckesser Do you think this is close enough for an email to the mailing list? (I'd prefer to wait until you're happy with it, though.)

Thanks, @mdhaber, this looks great.
There is currently a merge conflict, but it looks like it will be easy to resolve.

OK @WarrenWeckesser shall I send a post to the mailing list?

Shall I send a post to the mailing list?

Yeah, go for it.

@jb-leger Thought you might be interested in this. After this merges, it would be great to replace the resampling functions with Pythran versions - especially jackknife (which we'd also like to extend to Nd). I'd also like to add some sort of option for setting a batch size, as large samples with a large number of bootstrap resamples can be taxing on memory.

I think the code should use a loop over n_resamples bootstrap resamples, instead of taking an (n_resamples, n) bootstrap resample. Using (n_resamples, n) is a vectorization that yields very litte with respect to performance, but is not scalable with respect to memory. As at least 10,000 bootstrap resamples are required, at least in my experience, it can quickly run out of memory. We had the same issue with permutation tests as well. I have to check but I believe we ended up opting for a loop.

Thanks @tupui; I'll reply soon!
@sturlamolden Yes, at the top I suggested this as a feature:

A batch parameter so the user can select the size of the batches used to calculate the statistic for all the resampled data. Currently it's all done in one batch, but that can take a ton of memory, so some sort of control would be good.

I would prefer to add those things in a separate PR. As you suggested in the permutation PR, let's check for correctness here and improvements can come in follow-ups. Sound good?

@tupui I think the doctest should be fixed.

Since we call statistics multiple times, I wonder if we could optionally return more information. What do you think?

Maybe could return the bootstrap distribution of the statistic. I think it's a good idea, if only to give us a reason to return an object that we can extend later.

Also I like the decorator addition you noted. This would be really great. I could give a hand here if you want.

I would appreciate your help figuring out a convenient way to update the documentation when we apply decorators. For instance, in gh-13312, I want to add axis and nan_policy options to hypothesis tests using a decorator, but I'm not sure the best way to add these to the docs. I also need to make sure that the documentation is properly passed through the decorator so that documentation in IDEs still works correctly.

@tupui I think the doctest should be fixed.

LGTM now, thanks!

Maybe could return the bootstrap distribution of the statistic. I think it's a good idea, if only to give us a reason to return an object that we can extend later. But let's make sure @WarrenWeckesser likes the idea, too, before I pull the trigger because it's going to be a pain.

👍

I would make it a dataclass. What should the attributes be called? confidence_interval and bootstrap_distribution?

Sounds good. Maybe we also want the level which is used and a name as this class would be generic.

I would appreciate your help figuring out a convenient way to update the documentation when we apply decorators. For instance, in gh-13312, I want to add axis and nan_policy options to hypothesis tests using a decorator, but I'm not sure the best way to add these to the docs. I also need to make sure that the documentation is properly passed through the decorator so that documentation in IDEs still works correctly.

To pass the documentation, you usually need to use a decorator on the wrapper:
from functools import wraps

def my_wrapper(func):

    @wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)

    return wrapper

I will have a look at the PR you linked to see if I can be of help.

To pass the documentation, you usually need to use a decorator on the wrapper:

Yes, I use @wraps(func) in gh-13312.

In case you don't know this...

Yes, that's how gh-13312 works.

FYI, I sent an email to Art about this 999 vs 1000. Otherwise, is this ready for you? On my side I don't have more comments.

I just received a reply:

There is a nice story for 999.  When you get 999 values you split the real line into 1000 pieces.  There is an argument that those pieces have equal probability content.  So 950 of them give you a 95% interval.
I would advocate for 9999 instead.  I think that the old 95% intervals are better replaced by 99% intervals.
While the story is nicer you won't see a big numerical difference in the results from using a round number versus a round number minus one.

So as you wish here.

Maybe could return the bootstrap distribution of the statistic.

I've made the mistake in the past of adding stuff to a public API because I thought it might be useful, and later regretted it.  (And I should take another look at some of my current PRs to see if I'm doing it again. 😦 )  What do we expect the user to do with the array?  I guess a histogram might be interesting (but there's that darn "might" again...).
Even without it, we might still want to reconsider the format of the return value.  Currently it is a plain tuple, (low, high).   We are gradually changing existing functions to return objects with attributes, and for new functions (at least those that implement statistical tests), that should be considered the preferred API.  So perhaps the return value should be an object with the attribute confidence_interval that @mdhaber  suggested above.  Admittedly, this seems like an awkward API when the only return value is the CI tuple.
Note that we currently have a ConfidenceInterval data class defined in "_common.py".  It is used by the new (merged, but not yet released) functions relative_risk and binomtest.  If we used that as the return value of bootstrap_ci, the user would access the low and high attributes of the result to get the CI bounds.  Or, if the return value was something like a BootstrapCIResult, the confidence_interval attribute of the result could be a ConfidenceInterval instance.
FWIW: Confidence intervals are part of the PRs for the new odds_ratio and one-way ANOVA functions, and for an enhanced pearsonr function.  (These will be updated to use the same representation of a CI as the functions that have already been merged.)  In the work that I've done related to computing CIs, the representation of a CI evolved from a plain tuple to a namedtuple to a data class.  Since relative_risk and binomtest haven't been released yet, we're not locked in to the data class representation.

Note that we currently have a ConfidenceInterval data class defined in "_common.py". It is used by the new (merged, but not yet released) functions relative_risk and binomtest. If we used that as the return value of bootstrap_ci, the user would access the low and high attributes of the result to get the CI bounds. Or, if the return value was something like a BootstrapCIResult, the confidence_interval attribute of the result could be a ConfidenceInterval instance.

+1 on my side to use this class for consistency as all the tests we add return a class now.

OK, I will make this function return a class. I'm going to ask for votes here; as I'd like to head toward a decision if possible.

Can we agree that this should return an attribute statistic (the sample test statistic)? Please vote yes/no.

I think we should go ahead and return an array bootstrap_distribution. @WarrenWeckesser asked for uses: to implement other methods for calculating the confidence interval, and to estimate the standard error and bias. Please vote yes/no.

Actually, because the distribution can be used for several things, I wonder whether each of those things should be an attribute or a method. For instance, if someone is using this function to get an estimate of the standard error, should it always go through the procedure for calculating a confidence interval? I don't think we can just call the function bootstrap (because the bootstrap for hypothesis tests would fit better as part of the function proposed in gh-13899), but it is not just for confidence intervals.

With that in mind, please vote for one of the following options for an attribute confidence_interval (if you think we're close enough to making a decision):
a. a ConfidenceInterval object
b. a method that returns a ConfidenceInterval

May I make ConfidenceInterval a bunch-like object instead of a dataclass? We all seem inclined to think of a confidence interval as a tuple (as there were approvals when this was going to just return a tuple). So it seems to me that we should allow a user to think of it and unpack it as such instead of having to separately access low and high attributes. res.confidence_interval is already a mouthful.  Please vote yes/no.

Yes, that sounds like a good idea.
Yes, I can go along with that.
Not sure.  Is the idea that the function (perhaps renamed to something like bootstrap_resample) does the resampling and returns an object with the resampled distribution, and that object has methods to compute the confidence intervals and other statistics?  That is similar to how the R boot package works (see my example above).  The function boot does the resampling and returns an object that can be passed to boot.ci  and a few other functions in the package. (In the Python version, we would use methods on the object instead of a separate functions.)
I agree that the ConfidenceInterval object does not have to be a dataclass.  It really is just a pair of values; any metadata related to the CI is probably more appropriately stored on the object that holds the CI rather than as additional attributes of the CI itself.  I think a simple namedtuple would be fine: ConfidenceInterval = namedtuple('ConfidenceInterval', ['low', 'high'])

Re: @WarrenWeckesser #3 Exactly. The only thing is that bootstrap_resample is not quite specific enough. The distinction that needs to be made is that the bootstrap samples xb and yb are resampled without replacement from the original samples x and y, respectively; they are not resampled without replacement from the pooled np.concatenate(x, y) (which is what might do for a bootstrap hypothesis test).

yes, test statistic and standard error
Maybe, unless it wastes a lot of memory
No opinion
No opinion

Yes
Yes
a
Ok for the tuple as indeed, it's just two values here. Also I would return a central value with these.

@sturlamolden

Standard error is the standard deviation of the bootstrap distribution (next question)
Far less memory than this function already uses - it's just a scalar for each bootstrap resample.

I do have an opion on hypothesis testing though. In the world of resampling statistics, that typically belong to the permutation tests. Hypotesis testing using bootstrap are still not well understood and in any case only an approximation to the “exact” result produced by permutation tests. Bootstrap is typically used for standard errors and confidence intervals, because permutation tests are clumsy for that.

@sturlamolden

I do have an opion on hypothesis testing though

We aren't going to do bootstrap hypothesis testing here. I brought up this point just about the name of this function. Should it be bootstrap_ci, or can it just be bootstrap (or bootstrap_resample as @WarrenWeckesser suggested)?

I would vote for just bootstrap.

@tupui Great! To make it easier on myself, I would also prefer that the answer to 3) is a), but I want to make sure you've considered that binomtest currently returns and the new odds_ratio is slated to return an object with a method for calculating the confidence interval.

Wondering if we would want an option to sample with/without replacement.

That is a different test. If you sample without replacement and don't pool the samples, you will not get a distribution unless the statistic depends on the order. If you pool that samples, that's a permutation test.

Wondering if we would want an option to sample with/without replacement.

Bootstrap is always done by sampling with replacement.

OK! We've made a lot of progress quickly. To summarize, this new function bootstrap will return statistic, bootstrap_distribution, and confidence_interval. The only question left, I think is 3) whether confidence_interval should be
a) an instance of ConfidenceInterval (which will be changed to a namedtuple) or
b) a method that returns an instance of ConfidenceInterval (which will be changed to a namedtuple)
Keep in mind that binomtest returns an object with method binomtest_ci and gh-13340 is adding odds_ratio, which will have a method odds_ratio_ci. That said, I am very happy to leave this alone and just make confidence_interval a ConfidenceInterval object rather than a method.

@tupui Great! To make it easier on myself, I would also prefer that the answer to 3) is a), but I want to make sure you've considered that binomtest currently returns (and the new odds_ratio is slated to return) an object with a method for calculating the confidence interval.

I am not sure we would want to follow this method logic here as the whole class is acting as this method for to me. We could have other methods to the class which would be other method and they would return the same ConfidenceInterval object (tuple or dataclass).

We could have other methods to the class which would be other method

Ah! Yes, we could. Let's leave that for an update though.

@mdhaber wrote

I would also prefer that the answer to 3) is a)

Does this mean you prefer to keep the name bootstrap_ci, and always include the computed CI as an attribute of the result?  Just to be clear, here is what I think the options (a) and (b) imply:
a.

The function keeps the name bootstrap_ci.
The return value is an instance of a class (e.g. BootstrapCIResult) with attributes statistic,  confidence_interval (holding an instance of ConfidenceInterval, whatever that is), and boostrap_distribution.
Currently there are no additional computational methods proposed for returned object.

b.

The function gets a new name (maybe just bootstrap).
The return value is an instance of a class (e.g. BootstrapResult) with attributes statistic and bootstrap_distribution.
The returned object has a method (e.g. confidence_interval(self, confidence_level=0.95)) that returns an instance of ConfidenceLevel.

Is that correct?  And (a) is your preference?

OK! We've made a lot of progress quickly. To summarize, this new function bootstrap will use 9999 resamples by default and return an object with attributes statistic, bootstrap_distribution, and confidence_interval. The only question left, I think is 3) whether confidence_interval should be
a) an instance of ConfidenceInterval (which will be changed to a namedtuple) or
b) a method that returns an instance of ConfidenceInterval (which will be changed to a namedtuple)
Keep in mind that binomtest returns an object with method proportion_ci and gh-13340 is adding odds_ratio, which will have a method odds_ratio_ci. That said, I am very happy to leave this alone and just make confidence_interval a ConfidenceInterval object rather than a method. Unless using method='BCa', getting the confidence interval from the bootstrap distribution is inexpensive, and even with method='BCa' it's relatively inexpensive for one-sample statistics (proportional to sample size).

To me a) would be fine. It removes a layer which here is questionable as for me this method is equivalent to proportion_ci(..., method) which returns ConfidenceInterval.

No, I think a) was compatible with the function name bootstrap.

The function's name is bootstrap.
The return value is an instance of a class (e.g. BootstrapCIResult) with attributes statistic, confidence_interval (holding an instance of ConfidenceInterval, a namedtuple), and bootstrap_distribution.
Currently there are no additional computational methods proposed for returned object.

okay I am so looking forward to this

@bashtage Is this looking acceptable to you? I see two approving reviews but you have some recent thoughts/reviews above.

@tylerjereddy I think there were three issues that appear surprising. In order of importance (most first)

Quadratic in memory algorithms should only be used with extreme caution.  Creating a situation where an end user could DoS themselves in the name of performance does not look like good practice. I suspect that in cases where the memory allocation can succeed but is large, allocating such a large chunk will also not be very performant, at least when compared to a chunked implementation. When I look through the tests I noticed there is a skip on 32-bit since the test exhausts the memory available on the 32-bit worker. This suggests to me that it isn't that hard to get to an out-of-memory error, or worse, sending a process into scratch/pagefile.
The hard requirement that the function support axis.  This makes it harder to write UDFs that implement bootstraps using more complex estimators. Here is an example where a bootstrap is applied to a Probit (like a Logit if unfamiliar).  It is simple to write a wrapper than will output a vector of parameters.  Handling axis internally by the wrapper to that the function must output a 2-d array pushes a lot of work downstream.
Using RandomState.  This one sounds settled - use random_state and mass migrate in the future.

Of the three, I think the first really should be solved before the code is included as part of a release.  This issue may stop bootstrap from being useful in some contexts. Whether this happens in the PR or a later one is just a matter of preference.  This second is a matter of taste since it is not a former barrier to using the method even if it requires more complex wrapper functions.

I will implement 1 and 2 later this week. I would prefer that  it be a separate PR for reasons explained above, but we'll let @WarrenWeckesser / @tupui decide that.

I will implement 1 and 2 later this week. I would prefer that it be a separate PR for reasons explained above, but we'll let @WarrenWeckesser / @tupui decide that.

On my side the current PR is complete and further improvements could wait. This is a completely new feature and these improvements would be transparent for users. So we are not really risking to break anyone's code. First versions are not meant to be perfect (and IMO should not) and in any case putting it out there will allow to get feedback on how we can improve it better. To me, the API is what matters the most and here it seems sensible enough so we are free to evolve the feature set (* is a nice thing to have in that regard).

It looks like the random_state issue is resolved, and options for working in batches and for making the use of axis optional (with a vectorized parameter) will be implemented in followup pull requests.  So this still looks good to me.  I'll merge this tonight (eastern US time), unless there are new objections before then.

The refguide check failure is real.  With the default method changed to BCa, the output in the examples must also be updated.

Thanks @tupui. Took care of it locally because there were a couple other things that needed to be changed.

Merged.  Thanks @mdhaber and all reviewers!


Scoped modules

Lets try this again.
@mafintosh Your idea with doing mkdirp made things so much easier. After all, we are only interested in the actual .tar.gz file when packing the file and uploading it. Also, while downloading we don't care about the scope, the github repo is still unique.

:+1:


REFACTOR-#6046: remove code duplication for progress_bar_wrapper

What do these changes do?

[x] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

[x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] signed commit with git commit -s
[x] Resolves #6046
[x] tests added and passing
[x] module layout described at docs/development/architecture.rst is up-to-date

@dchigarev let's merge it


Support changing the phone number in the admin back end

It should be possible for admins to change the phone number in the credentials tab of a user.

Since the phone number is saved in a credential (not a user attribute), we can neither change it via the admin ui nor the admin api (they removed credentials update).
Options to fix this would in my opinion be:

add configuration to use the phone number from a user attribute instead of the credentials
provide admin api endpoint to change credentials of type "mobile-number" (but no other credentials)

Thoughts on this?

To be honest, I don't think that this repo is the right place for this request. If keycloak decides for security reasons to leave credential updates up to the user and we decide to turn around things, we would rather fight the framework instead of letting it help us.

I guess the question is: Is the phone number a credential in that situation.
I think we should not fight the framework by re-implementing what the community decided to remove for sec reasons (understandably).
Changing the phone number from a credential to a user attribute would still be a viable option IMHO, since the phone number is in fact not a credential but rather another additional information on the user entity that an administrator should be able to change.
Is it needed to store the phone number as a credential or could we technically also use user attributes? I'm wondering if its even feasible to change that to user attributes.

Unfortunately phone number must be a credential for technical reasons.
Here for example is the setup/update/delete functionality for the account console defined. It sits inside the CredentialProvider and connects it to RequiredActionProvider.
https://github.com/netzbegruenung/keycloak-mfa-plugins/blob/ecb21536323fc2df3f476dd46a3592a7dfb2933e/sms-authenticator/src/main/java/netzbegruenung/keycloak/authenticator/SmsAuthCredentialProvider.java#L107
The difference to other user attributes is, that it is used for 2FA. The whole authentication process was designed with the idea of orderable 2FA credentials, which is another reason to stick with the CredentialProvider.
See discussion and original design: https://github.com/keycloak/keycloak/issues/12102#issuecomment-1229914623

Should we close this issue then?


Fixes the currently failing json-schema-test-suite tests

Currently using jsonschema at work and interested in getting draft6 support finalised. To jump in, I've provided a couple of quick fixes for the previously failing tests.
Before:
FAILED (skips=47, failures=3, errors=10, successes=1209)
After:
PASSED (skips=47, successes=1254)
Note that I pulled in the latest tests from the JSON schema test suit repo.
I was hesitant to have RefResolver needing to know about id vs $id, but I could see no other way without removing RefResolver.from_schema() from the interface. I also briefly looked at the tests being skipped, which appear to require RefResolver to be able to parse the id attribute, but I'll comment on that separately.

Hey! Awesome, thanks so much for trying to move this along, these 3 things (draft 6, ref v2, and perf) have all hit little walls which has left them hanging there :/, so definitely definitely appreciate some help moving them to completion.
Gonna have to leave some detailed comments, for now just doing a first pass.

@Julian Hi, could this be merged? It should help to resolve other pull requests, I reckon.

@manycoding will have another look now but I'm not even sure anything in this PR is relevant anymore, all the tests now pass already and a lot of the specific fixes here went in separately.
What specifically from this one are you looking at/for?


Update testcontainers-scala to 0.40.17

About this PR
📦 Updates com.dimafeng:testcontainers-scala from 0.40.14 to 0.40.17
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.dimafeng", artifactId = "testcontainers-scala" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.dimafeng", artifactId = "testcontainers-scala" }
}]

labels: test-library-update, early-semver-minor, semver-spec-patch, commit-count:1

solved with 8679f1f.


pull request for e2e test at 02/03/2017 21:26:07

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Didn't work for me, followed the steps, getting   playwright._impl._api_types.Error: value: expected string, got undefined

playwright._impl._api_types.Error: value: expected string, got undefined

Updated doc 2aeedd83a1e05a557afd1b212e9b85bc442a74b7


Update MistralLLM to work with latest Mistral AI client

Description
This PR updates the MistralLLM class in mistral.py to work with the latest version of the Mistral AI client. The main changes include:

Updated import statements for the new Mistral AI client structure.
Replaced MistralAsyncClient with MistralClient and AsyncMistralClient.
Updated the load() method to initialize both sync and async clients.
Modified the agenerate() method to use the new client API.

Motivation and Context
The previous implementation was using a deprecated version of the Mistral AI client, which was causing NotImplementedError. This update ensures compatibility with the latest Mistral AI client and resolves the error.
How Has This Been Tested?

[ ] Tested with the latest Mistral AI client version
[ ] Verified that the MistralLLM class can be initialized without errors
[ ] Tested the agenerate() method with both structured and unstructured outputs

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Additional Notes
Please make sure to update the requirements.txt or setup.py to specify the latest version of the Mistral AI client.

In PR #900 the client was updated, closing this


Dropwizard has a stable and mature Apache HTTP client with an extensive configuration. Rather than trying to simulate configuration with Jersey properties, we can just use a built Apache client and write a custom connector that does a glue work between Jersey and Apache client's abstractions.
With this approach we have a single place of configuration. That reduces burden of supporting a new feature both in Apache and Jersey clients.
This also reverts the approach taken in #791 with a base class that contains common configuration.
It would be great if someone could try to test this more thoroughly because I did only sanity tests.

Coverage increased (+1.24%) to 68.51% when pulling 75faffdf1015127f89d97a555469e2d96aaa2fa9 on arteam:dropwizard_jersey_apache into c6b4c3257b22d47c1fc11890fdb5ea323e34ab81 on dropwizard:master.

+1

+1

Coverage increased (+1.24%) to 68.5% when pulling 3ec7f18fc5ee59520706adae624f05e5c8729e3c on arteam:dropwizard_jersey_apache into 49fe243f0e472fea34355a183fba253409a68520 on dropwizard:master.

Coverage increased (+0.21%) to 67.47% when pulling 48fdb73ebb24aafd4ed18e968e01363ea6abada0 on arteam:dropwizard_jersey_apache into 49fe243f0e472fea34355a183fba253409a68520 on dropwizard:master.

LGTM

Thanks, @arteam! And thanks @Trundle for reviewing.


Reliability: Update content checker relative link checker

The guiding principle of content checker is the link should work in github.
Secondary the developer portal will interpret the links so they work in developer portal. We want to get away from making doc updates to fix links in dev portal only and break the natural linking in github.

[ ] content checker should understand the repos docs root folder, it's in flow-docs.json and in dev portal docCollections files
[ ] add checking against fully qualified links in the updated docs.

Fix Hinting

[ ] currently some relative links are marked as invalid and are not
[ ] provide more suggestions rather than just one give 2-3

Future Support

[ ] allow authors to use ../ in relative links

What does it mean that the link should work in github?
Why is there a distinction between a link working in github and on the dev portal, shouldn't it work in both of them?

Issue with low confidence in content checker is the scope is too large. In order to address confidence and stability the scope needs to be narrowed. "work in github" means the links in the markdown should be navigable from within github. The dev portal should massage the markdown to fit it's own needs.
There should not be broken links in the markdown as it exists within in github.

closing, content checker has been updated and we'll create specific ticket as needed.


Fix ReactProp dependency

See related issue https://github.com/facebook/react-native/issues/5649

Thanks for the PR! 👍

(writing here as you have issues disabled)
Actually I did not succeed to get it working with RN 0.33. I have
Attempt to invoke virtual method 'android.view.View. android.app.activity.findViewById(int)' on a null object reference.

at BlurryOverlayView.java:44.
Is it working for you or you?

@priezz I haven't tested the library on RN 0.33 yet, maybe something in the passing of the current Activity has changed (again), which causes the NPW.
I have created an issue for this now, I will try to fix the issue in the next few days. :)


add open in browser action for functions

Fix: #169

Codecov Report

Merging #170 (c4c0089) into main (6b4949e) will decrease coverage by 3.45%.
The diff coverage is 81.08%.

@@            Coverage Diff             @@
##             main     #170      +/-   ##
==========================================
- Coverage   96.55%   93.09%   -3.46%
==========================================
  Files          51       72      +21
  Lines        2233     2867     +634
  Branches      336      458     +122
==========================================
+ Hits         2156     2669     +513
- Misses         77      198     +121

Impacted Files
Coverage Δ

src/util/constants.ts
0.00% <0.00%> (ø)

src/cli/cmdCli.ts
49.27% <20.00%> (-2.24%)
:arrow_down:

src/functions/function-command/get-url-function.ts
23.07% <23.07%> (ø)

src/util/existing-workspace-folder-pick.ts
33.33% <33.33%> (ø)

.../functions/function-tree-view/functionsTreeItem.ts
34.69% <34.69%> (ø)

src/cli/cli-config.ts
76.34% <50.00%> (-4.38%)
:arrow_down:

src/extension.ts
82.35% <52.63%> (-17.65%)
:arrow_down:

src/cli/func-api.ts
65.90% <65.90%> (ø)

src/functions/active-namespace.ts
76.92% <76.92%> (ø)

src/functions/function-command/create-function.ts
79.31% <79.31%> (ø)

... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b4949e...c4c0089. Read the comment docs.


Typo in KarafTestSupport

Typo in getAvailablePort calling for rmiRegistryPort: the second argument should be the MAX_RMI_REG_PORT, not the MIN_RMI_REG_PORT.

R: @jbonofre

For the record, please open a pull request only on master: it's the only branch monitored by the github integration.

I understand but the problem was that the issue was not on the main branch. Anyway thanks for fixing it.

No worries, I did an empty commit on master to automatically close the PR. For issue on a specific branch, please, create a Jira first attaching a patch. Thanks !


Website: update talk links

Signed-off-by: Juraj Michalek<EMAIL_ADDRESS>

[ ] I added CHANGELOG entry for this change.
[x] Change is not relevant to the end user.

Changes
Updated Talks section on website.

Well I am pretty sure my change didn't break e2e tests.

Yay!

Well I am pretty sure my change didn't break e2e tests.

Don't be so sure! 🙃


Incorporate .well-known into build process

Incorporate .well-known into build process for now.sh

Resolved with https://github.com/spences10/spences10.github.io.react/commit/4e5f07c025478b3dbb46f8a06e843f6872016d02


Inconsistency in eShopLite Solutions Across Aspire Repositories

I've noticed a discrepancy in the eShopLite solutions presented in different parts of the Aspire repositories. Specifically, there are two versions of the eShopLite solution, and they appear to be at different stages of development or feature completeness.

Main Aspire Repository: The first version of eShopLite is located in the main Aspire repository, under the samples directory. This version seems more comprehensive and feature-rich. It can be found here: Main Aspire eShopLite Sample.

Aspire-Samples Repository: The second version is in the Aspire-Samples repository. It's also under a samples directory but appears to be a smaller and less feature-complete version of the eShopLite solution. This can be viewed here: Aspire-Samples eShopLite.

Issue Summary:
There seems to be an inconsistency between these two versions of the eShopLite solution. The version in the main Aspire repo is more developed compared to the one in the Aspire-Samples repo.

Agree, please share the roadmap here between these two implementations of eShopLite.  My guess in looking at them, is the "aspire-samples" version is more advanced (e.g. works with azd init/up) with a smaller surface area, and over time it will be the surviving version.  The version in this repo (aspire), has a larger surface area, but has not been updated to work with azd init/up.
(also while we're on the subject of eShop implementations.  Is there a roadmap that includes the gradual maturation of eShopLite to eventually replace the full eShop?)

I've noticed that the Aspire repository contains some unique services in the eShop sample, such as an API Gateway project (found at https://github.com/dotnet/aspire/tree/main/samples/eShopLite/ApiGateway). In addition, one of the projects demonstrates deploying to a local Kubernetes cluster, which is a valuable feature for development and testing purposes. These services and capabilities seem to be exclusive to the Aspire repository and are not present in other related repositories like the eShop sample repository (https://github.com/dotnet/aspire-samples) or the main eShop repository (https://github.com/dotnet/eShop).
Including such practical deployment examples in the main or eshop lite sample repository could provide significant value. While there are various methods to implement and manage deployments, showcasing how to deploy to a local Kubernetes cluster directly within these examples would be highly beneficial. It would give a clearer picture of the capabilities and integrations possible within Aspire, particularly for developers who are exploring or transitioning to this architecture.
Incorporating these unique components, including the API Gateway and the local Kubernetes deployment process, into the other eShop sample repositories would ensure that valuable information and nuanced features are not overlooked.

The version in the main aspire repo is less a true sample and more a solution that can be used while working on aspire features (hence it being in the main repo) to quickly verify and experience features as they're being worked on. The version in the aspire-samples repo is intended to be a standalone example that can be consumed and reasoned about without any knowledge of the aspire source itself.
If there are features/implementations from one version that you think should be in the other, go ahead and log a specific issue in the relevant repo to suggest it be added.

Is there guidance/roadmap on what the goal is for how far the functionality in eShopLite (in aspire-samples) should evolve towards the functionality of the full eShop (https://github.com/dotnet/eShop).

No, other than to say I don't expect it to grow much beyond what it has now. It's just intended to be a "lighter" version with much less going on but enough to demonstrate a more than just a frontend talking to a backend API with a Redis cache.

Ok, thanks @DamianEdwards, this is what I needed know.  I'll follow up on where would be the best place to bring in the multi-tenant concept so we can demonstrate Azure Database Fleet Manager (https://aka.ms/azdbfleet).  If someone is already looking at multi-tenancy, please let me know.

fyi @adityamandaleeka


One school ensure population of template with no filtering

Function for ensuring the template is filled with one school. All working and checked its right for school_id = RRS2030250 :)
Formatting is still a bit funny on the templates as some tables have borders and some don't, have made sure all the tables are the same in the template but will have a further think about what else it could be 🤔
Also still need to replace the school name!

@stefpiatek @harryjmoss few bits to review 🙂:

changed the name tables.py -> schools.py
function to replace the school name in the template which keeps the formatting
v small changes to the template file to make sure each table is in the same format (I thought this may help the inconsistencies with the table borders but alas it did not 😬)

Just merged in main and resolved the commits, will take a look at the changes now!

Looks good so far! Mainly a couple of typos and nit-picky changes, there is one on L211 in schools.py that will need changing before the tests pass though.

ah great, thank you! will tweak and push :)


MigPlan doesn't notice after "Namespaces not found on ... cluster" is resolved

This message doesn't go away after the namespace in question is created.
status:
  conditions:
  - category: Critical
    lastTransitionTime: 2019-05-22T18:34:11Z
    message: Namespaces [nginx-example] not found on the source cluster.
    reason: NotFound
    status: "True"
    type: NamespaceNotFoundOnSourceCluster

Should be fixed by: https://github.com/fusor/mig-controller/pull/111.


PRUDP connection skips sequence IDs on the server if the client ACKs a packet it has not received

There seems to be some undocumented behavior about how sequence IDs work with PRUDP, at least with PRUDPv1. We have observed several cases where a client will send an acknowledgment packet for sequence IDs it has not received from the server, and in doing so the server then skips those IDs in it's counter
This example was found using Steeldiver Subwars by @DaniElectra. We have verified this happens both using Wireshark and a homebrew app which dumps PRUDP traffic right from the console, so this is not a matter of dropped/lost packets it seems?
The example is as follows:

The client has sent packets up to sequence ID 0xD to the server
The server has sent packets 0xA and 0xB to the client
The server sends a multi-ACK packet to the client ACKing up to 0xD (no issues so far)
The client sends a multi-ACK packet to the server ACKing up to 0xD (the server has not sent packets 0xC or 0xD)
The next packet the server sends is sequence ID 0xE, skipping 2 places on the servers sequence ID counter

I have not provided a network dump of this behavior here as it was made using Dani's account, and I am not sure he is comfortable posting his NEX credentials on here

Interestingly, the behavior described above has only shown when using Matchmaking protocols so far. Also, the acknowledgement skip isn't consistent (it can be one packet, it can be 8).
The example from above is on Steeldiver Subwars, but this also appears on other games like on Minecraft for the Wii U and Wii Sports Club.

It is also unclear if this skipping only goes one way. We have not observed any client sending ACKs for packets it has already sent an ACK for, so we do not know if this would send the servers counter backwards or not (I highly doubt it would, but just making it clear that we haven't seen it happen or tried to force it). We also have not seen a case where the server forces the client to skip sequence IDs (to my knowledge), so it's unclear if this also works in that case

That sounds weird. I just looked at the PRUDP implementation in DKC:TF and can't figure out how this is possible. It looks like the outgoing sequence id is only ever incremented by 1. I also know that the server uses the same implementation as the client.
I also can't reproduce it with a custom client. The server uses the expected sequence id even if I acknowledge a future packet. I tried both regular ack and aggregate ack packets on the Wii Sports Club server.
If you would like me take a look at your network dump, you can send it to me in private.
Here's the script that I used. It sends an aggregate acknowledgement packet that skips 10 sequence ids.

from nintendo.nex import common, prudp, rmc, settings
import anyio
import struct

import logging
logging.basicConfig(level=logging.DEBUG)

async def send_ack(self, packet):
	ack = prudp.PRUDPPacket(prudp.TYPE_DATA, prudp.FLAG_MULTI_ACK | prudp.FLAG_HAS_SIZE)
	ack.substream_id = 1
	ack.payload = struct.pack("<BBH", 0, 0, packet.packet_id + 10)
	await self.send_packet(ack)
prudp.PRUDPClient.send_ack = send_ack

async def main():
	s = settings.default()
	s.configure("4d324052", 30400)
	async with rmc.connect(s, "<IP_ADDRESS>", 43480) as client:
		await client.request(10, 5, struct.pack("<I", 1))
		await anyio.sleep(3)
		await client.request(10, 5, struct.pack("<I", 1))
		await anyio.sleep(3)
		await client.disconnect()

anyio.run(main)

And this is the debug output. The server does not skip any sequence id.
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:nintendo.nex.rmc:Connecting RMC client to <IP_ADDRESS>:43480:1
DEBUG:nintendo.nex.prudp:Connecting PRUDP transport to <IP_ADDRESS>:43480
DEBUG:anynet.udp:Connecting UDP client to <IP_ADDRESS>:43480
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_SYN flags=NEED_ACK seq=0 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_SYN flags=ACK,HAS_SIZE seq=0 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_CONNECT flags=NEED_ACK,RELIABLE,HAS_SIZE seq=1 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_CONNECT flags=ACK,HAS_SIZE seq=1 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_DATA flags=NEED_ACK,RELIABLE,HAS_SIZE seq=2 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DATA flags=NEED_ACK,RELIABLE,HAS_SIZE seq=1 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_DATA flags=MULTI_ACK,HAS_SIZE seq=0 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DATA flags=MULTI_ACK,HAS_SIZE seq=0 frag=0>
DEBUG:nintendo.nex.rmc:Received RMC response: protocol=10 method=5 call=1
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_DATA flags=NEED_ACK,RELIABLE,HAS_SIZE seq=3 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DATA flags=NEED_ACK,RELIABLE,HAS_SIZE seq=2 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_DATA flags=MULTI_ACK,HAS_SIZE seq=0 frag=0>
DEBUG:nintendo.nex.rmc:Received RMC response: protocol=10 method=5 call=2
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DATA flags=MULTI_ACK,HAS_SIZE seq=0 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_PING flags=NEED_ACK,RELIABLE seq=4 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_PING flags=ACK,HAS_SIZE seq=4 frag=0>
DEBUG:nintendo.nex.prudp:[135] Closing PRUDP connection
DEBUG:nintendo.nex.prudp:[CLI] Sending packet: <PRUDPPacket type=TYPE_DISCONNECT flags=NEED_ACK,RELIABLE seq=5 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DISCONNECT flags=ACK,HAS_SIZE seq=5 frag=0>
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DISCONNECT flags=ACK,HAS_SIZE seq=5 frag=0>
DEBUG:nintendo.nex.prudp:PRUDP connection is closed
DEBUG:nintendo.nex.prudp:[CLI] Received packet: <PRUDPPacket type=TYPE_DISCONNECT flags=ACK,HAS_SIZE seq=5 frag=0>
DEBUG:nintendo.nex.rmc:RMC client is close

If you would like me take a look at your network dump, you can send it to me in private.

Sounds good. Should I send the dumps over your email on your profile or over a Discord account?

Email is fine.

I found out what is happening. When PRUDP packets are sent quickly after each other, they are sent in a single UDP packet. Your parser probably does not take this into account.

That was it! Thank you so much!


moving pvpool responsibility to noobaa operator

Explain the changes

creating pvpool using operator's pvpool backing store and not by  statefulset
returning agent_config to the operator on new pvpool creation
fixes to kube_utils and pool_controllers to support new method

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

@jackyalbo Why do we need the semaphore or pool controllers after the move to the operator.
All this code should have been removed, or am I missing something

@nb-ohad I did this PR as first minimal step for moving the responsibility to the operator - didn't want too many changes in both sides.  I will later do a proper cleanup.

@jackyalbo
Ok but not removing the semaphore will retain all the bugs that we currently have with regard to the semaphore.

fixed to kubestore instead of kube_utils


Update files to use config variables for classes

Rather than having the models/repositories etc use the class name that's hardcoded, it should use the value from the configuration file. This would allow for models to be extended in the parent system and the dependencies/code will continue to operate as expected.

This has now been done.