Redesign school claims index

Context
This commit also includes the redirection of the user to the claims index after login and a link to the mentors index page on the claims index when there are no mentors, basically not allowing a claim to be created without a mentor.
Changes proposed in this pull request

Claims index table
New status tag for claim

Guidance to review
This is the prototype design https://itt-mentoring-beta-94341ca35a2c.herokuapp.com/organisations/049fc44e-6876-4884-8a77-d363173ce8d2/claims
Sign in as a normal user
Create a claim with a provider and a mentor
Go back to the claims index page
Create a claim without a provider or mentor
Go back to claims index page
Screenshots
https://github.com/DFE-Digital/itt-mentor-services/assets/11318084/587cd22b-ada4-44f1-91f0-f20ecf31eaff

 As a User, I should not be able to create a Claim unless I have added a Mentor


Moble Search Fix

Switched from webdriver.close to webdriver.quit in browser.py so it properly cleans up the session and allows mobile searches to begin. Tested on my end to ensure it works properly and starts mobile searches as expected. This fixes  issue #102

Unfixes issue #94

Yeah I just saw that. Do you still have that issue with everything updated and the new code @Contamiply ? I haven't run into that and it completes successfully on both of my computers.

I no longer have that issue ever since ive changed it from quit to close, but now I have the mobile issue too. I could use task kill like I did before but that also closes all my chrome procceses and I am too lazy to open them again.

Chrome is updated and the python dependencies are all updated to the specified versions in requirements.txt? I can't reproduce the issue so I'll have to dig more to see if anything is obviously wrong. I'm not sure what you're doing with chrome but you can set chrome to reopen your tabs to where you left off if that's all you need to open.

So the verdict on this is that it breaks the closing of chrome but fixes mobile issues for some?

Seems so. Does everyone have the mobile issue after the .close change or is that user dependent as well?

Seems as though that is user dependent as well - I very rarely have an issue where it will crash on some weird fluke. Usually at that point I just hit run again and let it do its thing and it finishes...unless you turn on full debug and watch the screen for the error it might be hard to catch.


Add debug logging

Coverage remained the same at 100.0% when pulling 897221a3eed13975b2735a9e3ca05215e8032871 on mmacai:add-debug into fa9e175d9757d15fca172068d4c2ddcedecb7665 on Accenture:develop.

Coverage remained the same at 100.0% when pulling 3a94cdfed2336ff89674d95843b2645ed90f77a1 on mmacai:add-debug into fa9e175d9757d15fca172068d4c2ddcedecb7665 on Accenture:develop.


Rename virtualbox VM names to predictable

Hi!
I suggest you to add this to make virtualbox vm names more predictable.
Now i suspect that virtualbox vm names are slightly random.
cfg.vm.provider "virtualbox" do |vb, override|
...
vb.name = "name.windomain.local"
...
end

Let me explain.
Here what i see now in Virtualbox Manager

Last VM  on screenshot is the same dc but after i changed name.
This change makes VM file path more predictable too.

Looks good, thank you


Improve failure message for ConfigurationExporterTest#testGeneratedConfigurationDocsAreUpToDate

What does this PR do?
Improves failure message for ConfigurationExporterTest#testGeneratedConfigurationDocsAreUpToDate
Checklist

[x] This is something else

[ ] I have updated CHANGELOG.asciidoc

:grey_exclamation: Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Reason: The PR is not allowed to run in the CI yet

Start Time: 2021-11-26T16:05:53.020+0000

Duration: 3 min 25 sec

Commit: d4c4c0a68028b0405421c35d7947012fa258d5e1

:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run benchmark tests : Run the benchmark test.

run compatibility tests : Run the JDK Compatibility test.

run integration tests : Run the APM-ITs.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)


Strange brotli errors

I am seeing some very strange errors when compressing buffers with the brotli plugin.
Initially I had an "Operation failed" error with one input that was exactly 32768 bytes long (I can provide the exact bytes if needed). Then I updated to the latest version and now it fails immediately for any input.
I tried pinning the error down to one specific change either in squash or brotli but it got more complicated:

Until quixdb/squash@a195cb4a586a505efa6083fc14bb2f4616dc2f85 / google/brotli@4dd9114 all tested versions fail on the 32k file.
Between quixdb/squash@a195cb4a586a505efa6083fc14bb2f4616dc2f85 / google/brotli@1bf1b0a and google/brotli@27688e6 all brotli versions fail at a different input with 23570 bytes.
quixdb/squash@037d4072dab59bf0baa0b8f104a9ddcb806f00b8 is completely broken (missing symbol)
From quixdb/squash@e1540102840c757cc0485881df80ddf29fbae941 / google/brotli@417107b3ddb04cfd4ddfa7592fed0eb886186281 on, it fails with any input

I am not sure if this is at all a squash bug or a bug in Brotli.

Can you provide details of your system (OS, architecture, compiler)?  Do the unit tests that come with squash fail?
I'm not seeing any issues with brotli right now either on my computers or on CI…

My system is an Ubuntu 14.04 LTS (Kernel: 3.13.0-76-generic, GCC: 4.8.4), my CPU is Intel(R) Core(TM) i7 CPU 960
I am having some trouble running the tests. make test returns No tests were found!!! and tests/test-squash fails with undefined symbol: squash_set_default_search_path

Interesting; Ubuntu 14.04 is pretty well tested (that's where most of our CI testing is done).

I am having some trouble running the tests. make test returns No tests were found!!! and tests/test-squash fails with undefined symbol: squash_set_default_search_path

Okay, something is very wrong.  squash_set_default_search_path is a pretty new function, so it sounds like the test executable is somehow linking against an older version of the library… Can you try recompiling with a fresh checkout?  The procedure should look something like this:
git clone https://github.com/quixdb/squash.git
cd squash
mkdir build
cd build
../autogen.sh
make -j
make test

If that doesn't work, is your OS 32 or 64 bit?

Ok, I rebuilt the whole lib and now all tests are passing. However it did not change any of the errors I reported.
BTW: My OS is 64 bit

Okay, can you please post some sample data which fails?  Is this with the squash CLI, or some custom code?  If the latter (and you can't reproduce with the CLI), would it also be possible to also post the code?  Basically, I just need a way to reproduce the error.

Note the squash CLI will use the intsalled plugins by default; if you still have broken plugins installed from a previous installation you should remove then (they are installed to $LIBDIR/squash), or set the SQUASH_PLUGINS environment variable to the path to the new plugins (e.g., from your build dir, SQUASH_PLUGINS=plugins ./utils/squash …)

Ok the CLI version seems to work with my input. I have to debug this further...

FWIW, if you can get a minimal test case put together I'd be happy to take a look.

Ok, I think I found the problem. I am compressing 1896 bytes of data using brotli-1. squash_codec_get_max_compressed_size returns 1901 but the actual output (with the cli tool) is 2010 bytes long.
You can get my test vectors from https://gist.github.com/dswd/96223edd33085862df7a

I haven't tried the others, but for the first vector I get 1900 bytes, both from a sample program and the CLI.
 However, at first glance, I am using brotli level 1 and I think you are using the default level.

Yes I just tested it and the error only appears with level 1, level 2 and above is fine.

Thanks for tracking this down.  I just pushed a fix, or possibly a workaround, depending on whether this behavior is intentional; I've asked at https://github.com/google/brotli/issues/274#issuecomment-192718467

Great thanks a lot. Please note my observations about the different brotli versions. It seems that this behavior got worse in recent versions.

I found the following in the brotli source:
const size_t max_out_size = 2 * bytes + 500;


feat: Add the ability to compare existing index definitions to the projected one for a type

Hi @slorello89,
This PR adds support to be able to have a VERY simple mechanism for the consumer of the library to determine if they need to drop/recreate a given index.
I foresee people using this method like this pseudocode Index drop/create is async I believe, so the consumer would need to do their own polling to wait for the index to properly delete first but:
var provider = new RedisConnectionProvider("redis://localhost:6379");
var definition = provider.Connection.GetIndexInfo(typeof(Customer));

if (definition.IndexDefinitionEquals(typeof(Customer)) == false)
{
    provider.Connection.DropIndex(typeof(Customer));
}

provider.Connection.CreateIndex(typeof(Customer));

Please forgive me that I've not raised an issue for this beforehand - I'm hoping this PR makes enough sense for us to discuss here.
Rather than making all the internals of RedisIndex visible, I've added an extension method that can be used to compare index definitions.
Happy to discuss implementation/naming/test coverage but figure it's easier to discuss something visual.
EDIT: First run of the action failed, looks like ubuntu-latest has moved on to use docker compose over docker-compose (v2) now.

Bump @slorello89. Hope all's OK - not seen any issue activity on this repo for 2 months and this PR has been outstanding for a bit - any chance we can give the Redis .NET repo some love? ❤
cc @benoitdion

I support this functionality, it is much needed. I really enjoy Redis OM for .NET so far and this would be a nice improvement imo.

@abbottdev - I think the only thing this really needs is to be able to answer this question directly from the connection. e.g. IsModelCurrent
provider.Connection.IsModelCurrent(typeof(Customer));

that way the library just takes care of everything for you without you having to fetch the index definition and then serialize the model yourself.
WDYT?

Maybe even a method that keeps the index up to date
provider.Connection.UpdateIndex(typeof(Customer));

using the pattern below:

var provider = new RedisConnectionProvider("redis://localhost:6379");
var definition = provider.Connection.GetIndexInfo(typeof(Customer));

if (definition.IndexDefinitionEquals(typeof(Customer)) == false)
{
    provider.Connection.DropIndex(typeof(Customer));
}

provider.Connection.CreateIndex(typeof(Customer));

@VagyokC4 my big hangup with that approach is that I'm concerned about any implicit index dropping. Perhaps I'm overly cautious, but the drop / create has some implicit downtime (as the index rebuilds), so I'd rather not hide the drop.

I thought I saw somewhere there was an update index which doesn't drop it but merely patches it?


Unexpected autocomplete suggestion in TextInput after update to v3.19.0

After update to 3.19.0 there is unexpected behaviour in TextInput and other components, using TextInput - in some cases browser started to provide unneeded autocomplete suggestion.
Preliminary cause of issue is because all input tags started to receive id (although random) by default

@amje @ValeraS
the issue is in this line https://github.com/gravity-ui/uikit/blob/de0a0d5fc1ba81e6b68f214351a1611d69c0cd09/src/components/TextInput/TextInput.tsx#L66
fix will be: const id = label ? originalId || innerId : originalId;


Backlog 36168 7 sep dbmenufix

:white_check_mark: Build finished in 8m 2s
Build command:
mvn clean verify -B -e -Daudit -amd -pl plugins/repositories/core

:exclamation: No tests found!
:information_source:  This is an automatic message

@amitkumar3hitachi one last comment. Please change the commit message. Commit messages must be in the format:
[BACKLOG-36168] some meaningful message that relates to the code changes being made


Consider annotations and compilation of .json files

Motivation
The shape of JSON stored in files can be validated using JSON schema. However, JSON schema cannot be used to ensure the stored JSON corresponds to a specific TypeScript interface.
Understandably, most JSON tends to be stored in databases. However, there are many cases where small documents are stored on the file system in order to permit quick editing. Two common examples are config and resource files. The JSON in these files is loaded at runtime and accessed by JavaScript code, thereby necessitating they correspond to specific TypeScript interfaces.
We have a small but significant use case for this scenario.
Proposal
Interface declarations file user.d.ts
declare module user {

     interface Address {
           house: number;
           street: string;
    }

    interface User  {
          name: string;
          address:Address;
          age: number;
    }
}

TypeScript JSON file user.json.ts
/// <reference path="user.d.ts" />
user.User: {            <----------- Annotation
   "name": "Joe Bloggs", /*Comment*/    // Permit comments
   "address": {
       "house": "1", // Error: Number expected
       street: "London"  // Error: Names must be quoted
    },
   "age": 55     // Error: Missing comma
    "id": 101 // Error: Invalid property
}

Compiled output (after errors are corrected): File user.json
{
   "name": "Joe Bloggs",
   "address": {
       "house": 1
       "street": "London"
    },
   "age": 55
}

This is an idea we've discussed before and I think it has a lot of merit. I would propose that the type name be specified as part of the /// <reference/> comment so that the core JSON syntax doesn't have to be extended. For example:
/// <reference path="user.d.ts" type="user.User"/>
{
    "name": "Joe Bloggs",
    ...
}

A number of JSON processing tools already tolerate comments (but definitely wouldn't tolerate actual identifiers before the object literal).

I should clarify... The thing I find useful here is the type checking and associated IDE services such as statement completion. I'm less sure about producing "corrected" output.

@ahejlsberg, no I meant "after the user has corrected the errors", not for the compiler to correct it.
If I understand you correctly, with the /// <reference/> option, just dropping that into an existing .json file would cause the compiler to include it in the compilation run.
This is cleaner than having to generate emit, but would require stripping out the /// <reference/> manually. Furthermore, one cannot simply require the file in node.
var json = require('user.json');  // SyntaxError: user.json:  Unexpected token /

I like the idea of adding the annotation in the reference, but would like user.json.ts to emit user.json.

Currently I use type assertion syntax to enforce type check on a object literal:
<user.User>{
   "name": "Joe Bloggs",
   "address": {
       "house": 1,
       "street": "London",
    },
   "age": 55
}

Is this what you want?

@duanyao, that is not valid syntax in a .json file.

After compilation, it is a valid json. You presented an example of user.json.ts, and was compiled to user.json. Did I miss something?

Are you suggesting that as an alternative syntax? It's not clear from the statement "Is that what you want?"
The problem with the type assertion syntax is that here we are attempting to annotate JSON - not coerce it into something.

No. I suggest that you write .json.ts files in TS (of cause), and compile them to .json files.

In that case there is nothing new being suggested.

Why need something new if existing tools can solve the problem? Or are there problems I overlooked?

A JSON file is not equivalent to a JavaScript file. There is no top-level variable. That's probably what you're missing.

Top-level variable is not the issue. I just tested tsc:
<user.User>{
   "name": "Joe Bloggs",
   "address": {
       "house": 1,
       "street": "London",
    },
   "age": 55
}

was compiled into
{
    "name": "Joe Bloggs",
    "address": {
        "house": 1,
        "street": "London"
    },
    "age": 55
};

You see, the only problem is the trailing ;. I think tsc can be improved to support a "json mode" to remove these artifact.

That's interesting. But still won't do. JSON is a subset of JavaScript and things that are permissible in JavaScript is not valid JSON.
<any>{
   'name': 'Joe Bloggs', // e.g. single quotes
   'bar': () => 10
}

These format issues should also be addressed by "json mode" of tsc. Data type issues like () => 10 can be caught by the type assertion or "json mode".

Like I said there is nothing new being suggested.

@mhegazy, @RyanCavanaugh any chance of adding this to the discussion list?

Hi, guys, is this on the radar?

We have not discussed this in a while. I believe we still need a detailed proposal of how this can be accomplished.
One other issue that seems more appealing is https://github.com/Microsoft/TypeScript/issues/7071, this requires no annotation, it just allows the compiler to load these files and infer their types. you would get an error if you were to assign the contents of the file to a variable of an unmatching type. obviously this only works for node, so that might not be what you are looking for.

@mhegazy, #7071 is a special case that unfortunately doesn't make things any better from our perspective.
The use-case I outlined is to do with loading JSON resources via HTTP and using them on a client application. We just want to ensure the types used in the client correspond to the JSON file stored on the server. I'd say this is a common requirement.
The only way I can think of to accomplish this at the moment is to have a new file type foo.json.ts and have the compiler emit foo.json as outlined above.

I think the use case here is well-handled by a few things now:

JSDoc type annotations in JS files
import of json "works" as "expected" now (with a smaller inference surface)
io-ts exists

Given lack of other feedback here, I don't see us proceeding anytime soon.


Include node_modules when externals are populated via plugins (for eg., esbuild-node-externals)

Kudos to the team which is behind this plugin and the packaging speeds it comes with! 🙇
I have a question (feature-request?) for you. When the below code is packaged, the final deployment does not include the externals (node modules). I think it is because the externals are populated by a plugin in this case esbuild-node-externals.
https://github.com/vamche/sls-esbuild-node-externals/blob/4cc8843ed9c486727d3c3e89dabc1904ae7521ef/plugins.js#L3-L6
const { nodeExternalsPlugin } = require('esbuild-node-externals');

// default export should be an array of plugins
module.exports = [nodeExternalsPlugin()];

https://github.com/vamche/sls-esbuild-node-externals/blob/4cc8843ed9c486727d3c3e89dabc1904ae7521ef/serverless.yml#L15-L17
  esbuild:
    plugins : ./plugins.js
    packager: npm

Sample repo: https://github.com/vamche/sls-esbuild-node-externals
Is there a way to include node_modules when externals are populated via plugins?

FYI, serverless-webpack does this with includeModules config and webpack-node-externals plugin.
I am happy to help/contribute to code if this feature can be introduced (if not already available). I would need some direction though. :)

Hi @vamche , thank you for the issue and the example. I cloned your repo and indeed the esbuild-node-externals plugin is excluding your dependencies from the bundle, but serverless-esbuild assumes that they are bundled.
I think it makes sense to implement a special case with the plugin, so if it is used we just get the list of modules determined by the plugin and put them to external to have packed.
If you really like to contribute, here is the direction: the plugin has exported function findDependencies you can reuse it to build a list of dependencies to be packed, and then the list must be used in our packExternalModules function which does the job. Should be an easy fix. Thank you 🙏

Thank you @floydspace, I will give it a shot.


Test3

Coverage decreased (-0.09%) to 97.704% when pulling e83471b421db7b067c87004420262a6ed0761572 on venkatai:test into b62815ab23d40683da993ca170a326cabd8799f6 on Zizzamia:master.


Device Request - Zemismart 24W Ceiling light - _TZ3210_mja6r5ix / TS0505B

Device Name: _TZ3210_mja6r5ix
Device Model: TS0505B
Device Description: Zemismart 24W Ceiling light (RGB CW WW)
Link to device image: https://i5.walmartimages.com/asr/f3ea021a-08c8-4a94-91a1-fb6a995a9b93.bcb2a47d5a7900da395b6fa36b4d63d2.jpeg?odnHeight=2000&odnWidth=2000&odnBg=FFFFFF
Device Interview:
"ids": {
"modelId": "TS0505B",
"manufacturerName": "_TZ3210_mja6r5ix"
},
"endpoints": {
"ieeeAddress": "a4:c1:38:ef:b6:2f:ae:5c",
"networkAddress": 49368,
"modelId": "TS0505B",
"manufacturerName": "_TZ3210_mja6r5ix",
"endpointDescriptors": [
{
"status": "SUCCESS",
"nwkAddrOfInterest": 49368,
"_reserved": 36,
"endpointId": 1,
"applicationProfileId": 260,
"applicationDeviceId": 269,
"applicationDeviceVersion": 0,
"_reserved1": 1,
"inputClusters": [
0,
3,
6280,
4,
5,
6,
8,
768,
10,
4096,
65518
],
"outputClusters": [
25,
4096,
65518
]
}
],
"deviceType": "router",
"receiveWhenIdle": true,
"swBuildId": "z.1.0",
"capabilities": {
"alternatePANCoordinator": false,
"deviceType": true,
"powerSourceMains": true,
"receiveWhenIdle": true,
"security": false,
"allocateAddress": true
},
"extendedEndpointDescriptors": {
"1": {
"clusters": {
"basic": {
"attributes": [
{
"acl": [
"readable"
],
"id": 0,
"name": "zclVersion",
"value": 3
},
{
"acl": [
"readable"
],
"id": 1,
"name": "appVersion",
"value": 112
},
{
"acl": [
"readable"
],
"id": 2,
"name": "stackVersion",
"value": 2
},
{
"acl": [
"readable"
],
"id": 3,
"name": "hwVersion",
"value": 0
},
{
"acl": [
"readable"
],
"id": 4,
"name": "manufacturerName",
"value": "_TZ3210_mja6r5ix"
},
{
"acl": [
"readable"
],
"id": 5,
"name": "modelId",
"value": "TS0505B"
},
{
"acl": [
"readable"
],
"id": 7,
"name": "powerSource",
"value": "unknown"
},
{
"acl": [
"readable",
"writable"
],
"id": 18,
"name": "deviceEnabled",
"value": true
},
{
"acl": [
"readable"
],
"id": 16384,
"name": "swBuildId",
"value": "z.1.0"
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"identify": {
"attributes": [
{
"acl": [
"readable",
"writable"
],
"id": 0,
"name": "identifyTime",
"value": 0
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"groups": {
"attributes": [
{
"acl": [
"readable"
],
"id": 0,
"name": "nameSupport",
"value": {
"type": "Buffer",
"data": [
0
]
}
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"scenes": {
"attributes": [
{
"acl": [
"readable"
],
"id": 0
},
{
"acl": [
"readable"
],
"id": 1
},
{
"acl": [
"readable"
],
"id": 2
},
{
"acl": [
"readable"
],
"id": 3
},
{
"acl": [
"readable"
],
"id": 4
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"onOff": {
"attributes": [
{
"acl": [
"readable",
"reportable"
],
"id": 0,
"name": "onOff",
"value": true,
"reportingConfiguration": {
"direction": "reported",
"attributeDataType": 0,
"minInterval": 0,
"maxInterval": 0,
"minChange": {
"result": null,
"length": 0
},
"status": "SUCCESS"
}
},
{
"acl": [
"readable"
],
"id": 16384
},
{
"acl": [
"readable",
"writable"
],
"id": 16385,
"name": "onTime",
"value": 0
},
{
"acl": [
"readable",
"writable"
],
"id": 16386,
"name": "offWaitTime",
"value": 0
},
{
"acl": [
"readable",
"writable"
],
"id": 16387
},
{
"acl": [
"reportable"
],
"id": 61440
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"levelControl": {
"attributes": [
{
"acl": [
"readable",
"reportable"
],
"id": 0,
"name": "currentLevel",
"value": 254,
"reportingConfiguration": {
"status": "NOT_FOUND",
"direction": "reported"
}
},
{
"acl": [
"readable",
"reportable"
],
"id": 61440,
"reportingConfiguration": {
"status": "NOT_FOUND",
"direction": "reported"
}
},
{
"acl": [
"readable"
],
"id": 1,
"name": "remainingTime",
"value": 0
},
{
"acl": [
"readable",
"writable"
],
"id": 17,
"name": "onLevel",
"value": 255
},
{
"acl": [
"readable",
"writable"
],
"id": 19,
"name": "offTransitionTime",
"value": 10
},
{
"acl": [
"readable",
"writable"
],
"id": 18,
"name": "onTransitionTime",
"value": 10
},
{
"acl": [
"readable",
"writable"
],
"id": 16384
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"colorControl": {
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"time": {
"attributes": [
{
"acl": [
"readable"
],
"id": 7
},
{
"acl": [
"readable"
],
"id": 65533,
"name": "clusterRevision",
"value": 1
}
],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"touchlink": {
"attributes": [],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
}
},
"bindings": {
"touchlink": {
"attributes": [],
"commandsGenerated": "UNSUP_GENERAL_COMMAND",
"commandsReceived": "UNSUP_GENERAL_COMMAND"
},
"ota": {}
}
}
}
}
Additional Comments:
Light can be controlled via Homey, but only colors can be choosen, Warm White / Cold White are not available, which makes it quite useless at the moment :)

Tagging along on this request, I've got a light bulb with the same manufacturer and model.
Adding details here as to not duplicate request.
Device Name: TZ3210_mja6r5ix
Device Model: TS0505B
Device Description: Tuya E27 RGB+WW+CW Color Changing Led Bulb
Link to device image: https://ae01.alicdn.com/kf/S2380ec04afdc4859a69040196dfa10a39/18W-15W-Zigbee-3-0-Led-Light-bulb-RGB-WW-CW-E27-Wifi-Tuya-Smart-Home.jpg_640x640.jpg.webp
Device Interview:

Details
{
  "ids": {
    "modelId": "TS0505B",
    "manufacturerName": "_TZ3210_mja6r5ix"
  },
  "endpoints": {
    "ieeeAddress": "a4:c1:38:76:31:1d:13:56",
    "networkAddress": 31826,
    "modelId": "TS0505B",
    "manufacturerName": "_TZ3210_mja6r5ix",
    "endpointDescriptors": [
      {
        "status": "SUCCESS",
        "nwkAddrOfInterest": 31826,
        "_reserved": 36,
        "endpointId": 1,
        "applicationProfileId": 260,
        "applicationDeviceId": 269,
        "applicationDeviceVersion": 0,
        "_reserved1": 1,
        "inputClusters": [
          0,
          3,
          6280,
          4,
          5,
          6,
          8,
          768,
          10,
          4096,
          65518
        ],
        "outputClusters": [
          25,
          4096,
          65518
        ]
      }
    ],
    "deviceType": "router",
    "receiveWhenIdle": true,
    "swBuildId": "z.1.0",
    "capabilities": {
      "alternatePANCoordinator": false,
      "deviceType": true,
      "powerSourceMains": true,
      "receiveWhenIdle": true,
      "security": false,
      "allocateAddress": true
    },
    "extendedEndpointDescriptors": {
      "1": {
        "clusters": {
          "basic": {
            "attributes": [
              {
                "acl": [
                  "readable"
                ],
                "id": 0,
                "name": "zclVersion",
                "value": 3
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 1,
                "name": "appVersion",
                "value": 112
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 2,
                "name": "stackVersion",
                "value": 2
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 3,
                "name": "hwVersion",
                "value": 0
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 4,
                "name": "manufacturerName",
                "value": "_TZ3210_mja6r5ix"
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 5,
                "name": "modelId",
                "value": "TS0505B"
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 7,
                "name": "powerSource",
                "value": "unknown"
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 18,
                "name": "deviceEnabled",
                "value": true
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 16384,
                "name": "swBuildId",
                "value": "z.1.0"
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "identify": {
            "attributes": [
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 0,
                "name": "identifyTime",
                "value": 0
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "groups": {
            "attributes": [
              {
                "acl": [
                  "readable"
                ],
                "id": 0,
                "name": "nameSupport",
                "value": {
                  "type": "Buffer",
                  "data": [
                    0
                  ]
                }
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "scenes": {
            "attributes": [
              {
                "acl": [
                  "readable"
                ],
                "id": 0
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 1
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 2
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 3
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 4
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "onOff": {
            "attributes": [
              {
                "acl": [
                  "readable",
                  "reportable"
                ],
                "id": 0,
                "name": "onOff",
                "value": false,
                "reportingConfiguration": {
                  "direction": "reported",
                  "attributeDataType": 0,
                  "minInterval": 0,
                  "maxInterval": 0,
                  "minChange": {
                    "result": null,
                    "length": 0
                  },
                  "status": "SUCCESS"
                }
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 16384
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 16385,
                "name": "onTime",
                "value": 0
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 16386,
                "name": "offWaitTime",
                "value": 0
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 16387
              },
              {
                "acl": [
                  "reportable"
                ],
                "id": 61440
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "levelControl": {
            "attributes": [
              {
                "acl": [
                  "readable",
                  "reportable"
                ],
                "id": 0,
                "name": "currentLevel",
                "value": 1,
                "reportingConfiguration": {
                  "status": "NOT_FOUND",
                  "direction": "reported"
                }
              },
              {
                "acl": [
                  "readable",
                  "reportable"
                ],
                "id": 61440,
                "reportingConfiguration": {
                  "status": "NOT_FOUND",
                  "direction": "reported"
                }
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 1,
                "name": "remainingTime",
                "value": 0
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 17,
                "name": "onLevel",
                "value": 255
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 19,
                "name": "offTransitionTime",
                "value": 10
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 18,
                "name": "onTransitionTime",
                "value": 10
              },
              {
                "acl": [
                  "readable",
                  "writable"
                ],
                "id": 16384
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "colorControl": {
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "time": {
            "attributes": [
              {
                "acl": [
                  "readable"
                ],
                "id": 7
              },
              {
                "acl": [
                  "readable"
                ],
                "id": 65533,
                "name": "clusterRevision",
                "value": 1
              }
            ],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "touchlink": {
            "attributes": [],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          }
        },
        "bindings": {
          "touchlink": {
            "attributes": [],
            "commandsGenerated": "UNSUP_GENERAL_COMMAND",
            "commandsReceived": "UNSUP_GENERAL_COMMAND"
          },
          "ota": {}
        }
      }
    }
  }
}

Additional Comments:
https://www.aliexpress.com/item/1005003097642075.html

Looking at another zigbee implementation of this model, it seems to be a little bit different compare to the other models with the same name.
https://github.com/Koenkk/zigbee-herdsman-converters/blob/ddde6508e81192bf0f6bfd72ffe71903073d68fc/src/devices/tuya.ts#L921-L937

This device model, _TZ3210_mja6r5ix, is used for several kind of light devices. I'll add it as standard RGB bulb. If this does not work for someone please let me know. - Added to next release.


Error unwrapping for retrier/

Hello, I've some feature request regarding the title
Basically, I propose we use errors.Is() from standard library to check for the blacklisted/whitelisted error types (the err == pass bit)
https://github.com/eapache/go-resiliency/blob/b98ce2825f7276cc41027348986a241ef5d62fd8/retrier/classifier.go#L40-L44
The reason is that currently we need to specify exacly what the error kind is, without checking if an error actually contains ("wraps") another error which are actually listed
Say we've implements a custom type to better handle specific error usecase
type HTTPError struct {
    error
    host string
    path string
}

func (h HTTPError) Error() string { // implements Go's `error` type
    return "failed on HTTP request to " + h.host + h.path + " reason: " + h.error.Error()
}

func (h HTTPError) Unwrap() error { // will be called from standard library error unwrapping
    return h.error
}

And that the error type wraps context.Canceled, which is defined as a blacklist

backoff := // ...
class := retrier.BlacklistClassifier{
	context.Canceled,
}
retry := retrier.New(backoff, class)

// ...

retry.Run(func() {
    // ...
    return HTTPError{error: context.Cancelled, host: "...", path: "..."}
})

The blacklist check would fail, because err == pass won't know what the "internal" value of the error actually is

Please consider this improvement. Would be really handy.

Sorry it took me a while to get to this. I am bumping the minimum Golang version so I can accept this PR. Previously of course you are welcome to use your own implementation of the Classifier interface with the rest of this package.


Example error

Hi -
I've tried to run your example statements, it gave
PLS-00306: wrong number or types of arguments in call to 'BASIC'
I think "p_from" and "p_to" are mandatory parameters, so I think you may change.
Regards,
Omar

Absolutely. I changed the example on the README file. Thanks.


terminal reconnection isn't working

create some terminals and type in them
reload the window or restart VS Code
🐛 they don't persist

Started happening in OSS on Wednesday July 5th

layoutInfo is undefined

it gets set to undefined

@meganrogge that's the return value (see [RPC Response])

yes, true. but the getTerminalLayoutInfo call returns undefined so it's getting reset somewhere. not via saveState

I'm on to the cause... The process is getting killed after detaching on reload, so reconnect breaks:
2023-07-07 09:19:10.623 [trace] [RPC Request] PtyService#setTerminalLayoutInfo({"workspaceId":"199b212e1c8d1b0f08747c4773181855","tabs":[{"isActive":false,"activePersistentProcessId":3,"terminals":[{"relativeSize":1,"terminal":3}]},{"isActive":true,"activePersistentProcessId":4,"terminals":[{"relativeSize":1,"terminal":4}]}]})
2023-07-07 09:19:10.624 [trace] [RPC Response] PtyService#setTerminalLayoutInfo undefined
2023-07-07 09:19:10.869 [trace] [RPC Request] PtyService#input(4, "b")
2023-07-07 09:19:10.869 [trace] node-pty.IPty#write b
2023-07-07 09:19:10.869 [trace] [RPC Response] PtyService#input undefined
2023-07-07 09:19:10.869 [trace] node-pty.IPty#onData b
2023-07-07 09:19:10.876 [trace] [RPC Event] PtyService#_onProcessData.fire({"id":4,"event":"b"})
2023-07-07 09:19:11.064 [trace] [RPC Request] PtyService#input(4, "c")
2023-07-07 09:19:11.064 [trace] node-pty.IPty#write c
2023-07-07 09:19:11.064 [trace] [RPC Response] PtyService#input undefined
2023-07-07 09:19:11.064 [trace] node-pty.IPty#onData c
2023-07-07 09:19:11.069 [trace] [RPC Event] PtyService#_onProcessData.fire({"id":4,"event":"c"})
2023-07-07 09:19:11.406 [info] Persistent process "1": The reconnection grace time of 1m has expired, shutting down pid "3210"
2023-07-07 09:19:11.407 [trace] TerminalProcess#shutdown
    at TerminalProcess.shutdown (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:377:68)
    at PersistentTerminalProcess.shutdown (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:762:42)
    at ProcessTimeRunOnceScheduler.runner (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:678:22)
    at ProcessTimeRunOnceScheduler.onInterval (/Users/daimms/dev/microsoft/vscode/out/vs/base/common/async.js:784:26)
    at listOnTimeout (node:internal/timers:559:17)
    at process.processTimers (node:internal/timers:502:7)
2023-07-07 09:19:11.407 [trace] node-pty.IPty#kill
2023-07-07 09:19:11.408 [trace] [RPC Event] PtyService#_onProcessExit.fire({"id":1,"event":0})
2023-07-07 09:19:11.408 [info] Persistent process "2": The reconnection grace time of 1m has expired, shutting down pid "3211"
2023-07-07 09:19:11.408 [trace] TerminalProcess#shutdown
    at TerminalProcess.shutdown (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:377:68)
    at PersistentTerminalProcess.shutdown (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:762:42)
    at ProcessTimeRunOnceScheduler.runner (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:678:22)
    at ProcessTimeRunOnceScheduler.onInterval (/Users/daimms/dev/microsoft/vscode/out/vs/base/common/async.js:784:26)
    at listOnTimeout (node:internal/timers:559:17)
    at process.processTimers (node:internal/timers:502:7)
2023-07-07 09:19:11.408 [trace] node-pty.IPty#kill
2023-07-07 09:19:11.409 [trace] [RPC Event] PtyService#_onProcessExit.fire({"id":2,"event":0})
2023-07-07 09:19:11.409 [trace] node-pty.IPty#onData
2023-07-07 09:19:11.409 [trace] node-pty.IPty#onData
2023-07-07 09:19:11.409 [trace] node-pty.IPty#onData

2023-07-07 09:19:11.409 [trace] node-pty.IPty#onData

2023-07-07 09:19:11.412 [trace] TerminalProcess#_queueProcessExit
    at TerminalProcess._queueProcessExit (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:314:77)
    at /Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:285:22
    at EventEmitter2.fire (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/eventEmitter2.js:41:22)
    at PipeSocket.<anonymous> (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/terminal.js:93:76)
    at PipeSocket.emit (node:events:513:28)
    at Terminal.emit (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/terminal.js:155:34)
    at PipeSocket.<anonymous> (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/unixTerminal.js:98:27)
    at Object.onceWrapper (node:events:628:26)
    at PipeSocket.emit (node:events:525:35)
    at Pipe.<anonymous> (node:net:757:14)
2023-07-07 09:19:11.412 [trace] TerminalProcess#_queueProcessExit
    at TerminalProcess._queueProcessExit (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:314:77)
    at /Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:285:22
    at EventEmitter2.fire (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/eventEmitter2.js:41:22)
    at PipeSocket.<anonymous> (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/terminal.js:93:76)
    at PipeSocket.emit (node:events:513:28)
    at Terminal.emit (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/terminal.js:155:34)
    at PipeSocket.<anonymous> (/Users/daimms/dev/microsoft/vscode/node_modules/node-pty/lib/unixTerminal.js:98:27)
    at Object.onceWrapper (node:events:628:26)
    at PipeSocket.emit (node:events:525:35)
    at Pipe.<anonymous> (node:net:757:14)
2023-07-07 09:19:54.405 [trace] [RPC Request] PtyService#detachFromProcess(3, false)
2023-07-07 09:19:54.406 [trace] TerminalProcess#shutdown
    at TerminalProcess.shutdown (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/terminalProcess.js:377:68)
    at PersistentTerminalProcess.shutdown (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:762:42)
    at PersistentTerminalProcess.detach (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:720:22)
    at PtyService.detachFromProcess (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:245:43)
    at descriptor.<computed> (/Users/daimms/dev/microsoft/vscode/out/vs/platform/terminal/node/ptyService.js:30:35)
    at Object.call (/Users/daimms/dev/microsoft/vscode/out/vs/base/parts/ipc/common/ipc.js:850:39)
    at ChannelServer.onPromise (/Users/daimms/dev/microsoft/vscode/out/vs/base/parts/ipc/common/ipc.js:287:35)
    at ChannelServer.onRawMessage (/Users/daimms/dev/microsoft/vscode/out/vs/base/parts/ipc/common/ipc.js:266:33)
    at UniqueContainer.value (/Users/daimms/dev/microsoft/vscode/out/vs/base/parts/ipc/common/ipc.js:217:73)
    at Emitter._deliver (/Users/daimms/dev/microsoft/vscode/out/vs/base/common/event.js:832:26)
    at Emitter._deliverQueue (/Users/daimms/dev/microsoft/vscode/out/vs/base/common/event.js:843:22)
    at Emitter.fire (/Users/daimms/dev/microsoft/vscode/out/vs/base/common/event.js:866:22)
    at MessagePortMain.fn (/Users/daimms/dev/microsoft/vscode/out/vs/base/common/event.js:413:44)
    at MessagePortMain.emit (node:events:513:28)
    at MessagePortMain._internalPort.emit (node:electron/js2c/utility_init:2:367)
2023-07-07 09:19:54.406 [trace] node-pty.IPty#kill
2023-07-07 09:19:54.406 [trace] [RPC Event] PtyService#_onProcessExit.fire({"id":3,"event":0})
2023-07-07 09:19:54.406 [trace] [RPC Response] PtyService#detachFromProcess undefined
2023-07-07 09:19:54.406 [trace] [RPC Request] PtyService#detachFromProcess(4, false)
2023-07-07 09:19:54.406 [trace] [RPC Response] PtyService#detachFromProcess undefined

Revive after restart is working for me.

Ah found it:
2023-07-07 09:28:56.963 [trace] [RPC Request] PtyService#getTerminalLayoutInfo({"workspaceId":"199b212e1c8d1b0f08747c4773181855"})
2023-07-07 09:28:56.964 [info] Expanding terminal instance, old id 199b212e1c8d1b0f08747c4773181855-3 -> new id undefined
2023-07-07 09:28:56.964 [warning] Couldn't get layout info, a terminal was probably disconnected Could not find pty on pty host
2023-07-07 09:28:56.964 [info] Reattach to wrong terminal debug info - layout info by id {"relativeSize":1,"terminal":3}
2023-07-07 09:28:56.964 [info] Reattach to wrong terminal debug info - _revivePtyIdMap []

One of the sanity checks I put in place that didn't actually help fix the reattach problem caused it

Parsed state is null so reconnect stops here: https://github.com/microsoft/vscode/blob/4b2f5dcf6aa977e5d7f21a659a7da1976bb1cb2d/src/vs/workbench/contrib/terminal/electron-sandbox/localTerminalBackend.ts#L320


Are there any instructions for this program?

Are there any instructions for this program?

There's documentation which covers most of the functionality.


Add 'dotnet new' templates for NoTargets and Traversal

Fixes #472
Add two new projects: Microsoft.Build.NoTargets.Templates and Microsoft.Build.Traversal.Templates that each publish a NuGet package that can be installed / used via dotnet new.
I wasn't quite sure how best to structure the project, so feedback is appreciated. Some things to consider:

The template projects currently live under their respective "feature" folders. It's also common to create a top-level "Templates" folder like this, however that breaks the package-per-feature pattern currently used in the repo.

Additionally, currently the templates folder is a child of the main project in the source tree, but they're more conceptually siblings. If there's a different arrangement that makes more sense to people let me know.

Currently the version for the template packages match the version of the SDK package. I think that makes sense, but if they should be versioned independently I can make that change as well.
The NoTargets template uses the .msbuildproj extension. If we should use .proj, or if there's a better extension to use please let me know.
I added several comments to help new users, but there may be so much that it's annoying for experienced users. Editing feedback welcome.
In order to keep the NoTargets template consistent with the README, I changed the TargetFrameworkVersion in the README from net462 to netstandard2.0, since that's compatible with both .NET Framework and .NET, and still supported (basically) everywhere.

Friendly ping here. I fully expect this review to take a couple of rounds to get the directory layout the way maintainers want, so any help / guidance is very appreciated. Thanks!

@AndyGerlicher has assumed ownership of this repo

Just FYI discussing this internally with the MSBuild team. I want to make sure this aligns with their vision before we make this change. Sorry for the delay!

Hi there! Friendly ping here. Any updates, or concerns / issues I can help address? Thanks!


V 6.3.0 cause flutter build error: CocoaPods could not find compatible versions for pod "Sentry"

Platform:

[ ]  Dart
[x]  Flutter Android or iOS
[x]  Flutter Web

IDE:

[x]  VSCode
[ ]  IntelliJ/AS
[x]  XCode
[ ]  Other, which one?

split-debug-info and obfuscate (Flutter Android or iOS) or CanvasKit (Flutter Web):

[x]  Enabled
[ ]  Disabled

Platform installed with:

[x] pub.dev
[ ] GitHub

Output of the command flutter doctor -v below:
[✓] Flutter (Channel stable, 2.10.0, on macOS 12.1 21C52 darwin-arm, locale en-US)
• Flutter version 2.10.0 at /Users/sallyxu/Devtools/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 5f105a6ca7 (7 days ago), 2022-02-01 14:15:42 -0800
• Engine revision 776efd2034
• Dart version 2.16.0
• DevTools version 2.9.2
[✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
• Android SDK at /Users/sallyxu/Library/Android/sdk
• Platform android-32, build-tools 32.0.0
• Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• CocoaPods version 1.11.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2020.3)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
[✓] VS Code (version 1.64.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.34.0
[✓] Connected device (1 available)
• Chrome (web) • chrome • web-javascript • Google Chrome 98.0.4758.80
[✓] HTTP Host Availability
• All required HTTP hosts are available
• No issues found!
The version of the SDK (See pubspec.lock):
4.x.x

I have the following issue:
After updated sentry_flutter to v 6.3.0,  ios build is broken. Getting the error:
CocoaPods could not find compatible versions for pod "Sentry".
I tried to delete the Pods and Podfile.lock, and run pod install --repo-update, then got
[!] Oh no, an error occurred.
have to reverse back to v 6.2.2.
Following is the output from flutter build ios command:
Building com.xxx.ttpkpi for device (ios-release)...
Automatically signing iOS for device deployment using specified development team in Xcode project:
2Q9HT7X674
Running pod install...                                             751ms
CocoaPods' output:
↳
Preparing
Analyzing dependencies

Inspecting targets to integrate
  Using `ARCHS` setting to build architectures of target `Pods-Runner`: (``)

Finding Podfile changes
  - Flutter
  - connectivity
  - geolocator_apple
  - google_maps_flutter
  - package_info_plus
  - sentry_flutter
  - shared_preferences_ios
  - url_launcher_ios

Fetching external sources
-> Fetching podspec for `Flutter` from `Flutter`
-> Fetching podspec for `connectivity` from `.symlinks/plugins/connectivity/ios`
-> Fetching podspec for `geolocator_apple` from `.symlinks/plugins/geolocator_apple/ios`
-> Fetching podspec for `google_maps_flutter` from `.symlinks/plugins/google_maps_flutter/ios`
-> Fetching podspec for `package_info_plus` from `.symlinks/plugins/package_info_plus/ios`
-> Fetching podspec for `sentry_flutter` from `.symlinks/plugins/sentry_flutter/ios`
-> Fetching podspec for `shared_preferences_ios` from `.symlinks/plugins/shared_preferences_ios/ios`
-> Fetching podspec for `url_launcher_ios` from `.symlinks/plugins/url_launcher_ios/ios`

Resolving dependencies of `Podfile`
  CDN: trunk Relative path: CocoaPods-version.yml exists! Returning local because checking is only
  performed in repo update
  CDN: trunk Relative path: all_pods_versions_c_7_9.txt exists! Returning local because checking is only
  performed in repo update
  CDN: trunk Relative path: Specs/c/7/9/Reachability/3.2/Reachability.podspec.json exists! Returning
  local because checking is only performed in repo update
  CDN: trunk Relative path: all_pods_versions_a_d_d.txt exists! Returning local because checking is only
  performed in repo update
  CDN: trunk Relative path: Specs/a/d/d/GoogleMaps/6.0.1/GoogleMaps.podspec.json exists! Returning local
  because checking is only performed in repo update
  CDN: trunk Relative path: all_pods_versions_a_b_d.txt exists! Returning local because checking is only
  performed in repo update
  CDN: trunk Relative path: Specs/a/b/d/Sentry/7.7.0/Sentry.podspec.json exists! Returning local because
  checking is only performed in repo update
  CDN: trunk Relative path: all_pods_versions_4_2_c.txt exists! Returning local because checking is only
  performed in repo update
  CDN: trunk Relative path: Specs/4/2/c/FlutterMacOS/2.5/FlutterMacOS.podspec.json exists! Returning
  local because checking is only performed in repo update
[!] CocoaPods could not find compatible versions for pod "Sentry":
  In snapshot (Podfile.lock):
    Sentry (= 7.5.4, ~> 7.5.1)

  In Podfile:
    sentry_flutter (from `.symlinks/plugins/sentry_flutter/ios`) was resolved to 0.0.1, which depends on
      Sentry (~> 7.9.0)

Specs satisfying the `Sentry (= 7.5.4, ~> 7.5.1), Sentry (~> 7.9.0)` dependency were found, but they
required a higher minimum deployment target.

/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolution.rb:317:in
`raise_error_unless_state'
/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolution.rb:299:in `block in
unwind_for_conflict'
/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolution.rb:297:in `tap'
/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolution.rb:297:in `unwind_for_conflict'
/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolution.rb:257:in `process_topmost_state'
/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolution.rb:182:in `resolve'
/Library/Ruby/Gems/2.6.0/gems/molinillo-0.8.0/lib/molinillo/resolver.rb:43:in `resolve'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/resolver.rb:94:in `resolve'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer/analyzer.rb:1078:in `block in
resolve_dependencies'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/user_interface.rb:64:in `section'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer/analyzer.rb:1076:in
`resolve_dependencies'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer/analyzer.rb:124:in `analyze'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer.rb:416:in `analyze'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer.rb:241:in `block in
resolve_dependencies'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/user_interface.rb:64:in `section'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer.rb:240:in `resolve_dependencies'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/installer.rb:161:in `install!'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/command/install.rb:52:in `run'
/Library/Ruby/Gems/2.6.0/gems/claide-1.0.3/lib/claide/command.rb:334:in `run'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/lib/cocoapods/command.rb:52:in `run'
/Library/Ruby/Gems/2.6.0/gems/cocoapods-1.11.2/bin/pod:55:in `<top (required)>'
/usr/local/bin/pod:23:in `load'
/usr/local/bin/pod:23:in `<main>'

Error running pod install
The description goes here ...
Steps to reproduce:

Step

Actual result:

Actual

Expected result:

Result

@Sally-Xu It's probably caching issues, try to flutter clean, flutter pub get, pod update, pod install and friends

Usually deleting the Podfile.lock helps.

I tried all of the above suggestions before I posted the issue.
delete Pods, Podfile.lock
flutter clean,
flutter pub get
pod install --repo-update
None of them solved the problem.
Now I have fix the version of sentry_flutter to 6.2.2.
sentry_flutter: 6.2.2

@Sally-Xu it is an environmental issue, please double check it, I've tried on a new project and upgraded from an old one, both worked well.

@marandaneto ,
Could you point me a direction to check?  I just tried to create a new flutter project without adding any code, just by adding same sentry_flutter package would have the same problem:
flutter create myapp
cd myapp
flutter pub add sentry_flutter
flutter build ios

Getting error on pod install step:
Error output from CocoaPods:
↳
[!] Automatically assigning platform `iOS` with version `9.0` on target `Runner` because no platform
was specified. Please specify a platform for this target in your Podfile. See
`https://guides.cocoapods.org/syntax/podfile.html#platform`.

Error running pod install
If I change the sentry_flutter version to 6.2.2, the error is gone, build went through without a problem.
My machine is a brand new (less than a month old) MacBook Pro, all the softwares are up-to-date.

Add something like this to the top of your Podfile:
# Uncomment this line to define a global platform for your project
platform :ios, '12.0'

@kuhnroyal
I did. same error.

@Sally-Xu that's not a Sentry's issue, sadly.
As I said, I've created an empty project and it worked out.

These steps suggested by @marandaneto worked for me, specifically pod update:
deleting the Podfile.lock then:
flutter clean
flutter pub get
pod update
pod install

had to run another flutter pub get from VSCode but works fine now.

I had the same issue... Moving to the new version 6.4.0 solved it!


feat(CI): local pre-commit hook setup

Related #66
Describe the changes you have made in this PR -
Installed Husky and CommitLint for local pre-commit linting, added .node_modules and .idea to .gitignore
-Any new local commits made will be run through commitlint
Screenshots of the changes (If any) -

I am attempting this as part of the Hacktoberfest, but they have started accepting PRs after the 1st of October.
So you can merge it as soon as possible

I am attempting this as part of the Hacktoberfest, but they have started accepting PRs after the 1st of October.
So you can merge it as soon as possible

Fine, then merging the PR :+1:

@all-contributors please add @flushthemoney for infra


Jasper 5x3 configuration for published WERs

In the paper Jason Li et al. Jasper: An End-to-End Convolutional Neural Acoustic Model, 2019 https://arxiv.org/pdf/1904.03288.pdf, Jasper 5x3 has the following results:

I am trying to reproduce some of the WERs but wonder how many GPUs are used for these results and what is the patch size per GPU?

These experiments were done on 8x V100 GPUs with batch_size_per_gpu=64.

Thanks a lot for your information. I tried with 4x V100 GPUs, also batch per gpu=64, 50 epochs, using batch norm and ReLU, but validation on dev-clean got WER = 7.09%. Was that too good?

This is reasonable WER.
For J-10x5, I got WER=5.7 for 50 epochs using NovoGrad with init LR=0.02 and weight decay 0.001.

These are numbers for J-10x5:
Train (epochs)       greedy WER, %
100  -  4.87
200 -  4.44
400 -  4.23
600 -  4.06

I see. I am curious what made the difference between the published WER=8.82 and my test WER=7.09% for the J-5x3. I will appreciated it if you can point it out. I used these:
"optimizer": "Momentum",
"optimizer_params": {
    "momentum": 0.90,
},
"lr_policy": poly_decay,
"lr_policy_params": {
    "learning_rate": 0.01,
    "min_lr": 1e-5,
    "power": 2.0,
},
"larc_params": {
    "larc_eta": 0.001,
},

"regularizer": tf.contrib.layers.l2_regularizer,
"regularizer_params": {
    'scale': 0.001
},

"dtype": "mixed",
"loss_scaling": "Backoff",

We did experiments with activations/normalizations using slightly older configuration of the Jasper (in comparison with the latest jasper10x5_LibriSpeech_nvgrad.py. Two most important differences are:

Optimizer (SGD with Momentum vs NovoGrad)
Augmentation (no augmentation vs speed perturbation)

The optimizer's parameters from your snippet look good. Have you applied any data augmentation?

No I did not apply data augmentation. Actually, I only modified the previous git version's jasper_10x3_8gpus_mp.py to remove the repetition of B1-B5, and then change num_gpus to 4 and num_epochs to 50. It took me 2 days to finish. Could you try that and see if the result is the same?

BTW, although I set num_gpus to 4, most of the time I saw only one gpu was actively used. The usage is like this:

I ran the command in tensorflow docker like this:
python run.py --config_file=example_configs/speech2text/jasper_5x3_4gpus_mp.py --mode=train_eval
Is there any parameters that I missed to use to enable 4 gpus?

The tests that were done in the paper were conducted on a slightly different version of Jasper. The changes from the provided config are as follows:

The 5x3 models were tested only with normal residual. (One needs to set Dense_Residual to False in the config)
The tests were conducted with python_speech_features which does not apply a windowing function to spectrograms
The output of the residual and convolutional ops were added prior to batch normalization; currently batch normalization is applied to each individually prior to adding
Regarding the number of gpus, we observed that WER tends to decrease with more training iterations so using less gpus will increase the number of required iterations, real world training time, but decrease WER.

In terms of GPU utilization, have you disabled Horovod as well? Horovod is meant to be used with mpi4py (mpiexec --allow-run-as-root --np <num_gpus> python run.py ... ). The num_gpus parameter has no effect if Horovod is enabled.

Thanks for your explanation. Yes I enabled Horovod in previous tests. Thanks a lot for your answers.

@inchpunch would mind sharing your jasper 5x3 config file?
Thanks!


Update query API to VS2017 RTW

@heaths,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Build vswhere 1.0.44-beta+gaba420c55b completed (commit https://github.com/Microsoft/vswhere/commit/009172a076 by @heaths)


Allow Option<Entity> to leverage niche optimization

Objective

Implements change described in #3022
Goal is to allow Entity to benefit from niche optimization, especially in the case of Option<Entity>
Ultimately to reduce overhead with certain data structures with empty slots (ie. grids)

Solution

Entity::generation was changed to NonZeroU32, over Entity::id because:

It reduces the amount of API breakage, being effectively an implementation detail
It's, probably, less error-prone than constantly offsetting Entity::id

Entities::meta changed to Option<NonZeroU32> to match

None indicates a slot that has used all generation values and can no longer be reused.

Entity::from_bits was modified to return a Option<Entity>
Tests updated (mostly just Entity::generation init with NonZeroU32::new)
Added tests for Entity::from_bits when the bits are invalid; Option<Entity>niche optimization; Entities::meta generation increment; Entities::meta generation overflow behaviour;

Edit: PS first PR here, be harsh but gentle :sweat_smile:
Edit2: Updated to reflect latest commit

Oops, just realized that I didn't post my reply, just need to rewrite it - one sec. Sorry if this is a little rambly/overly-specific, bit late here and I like running silly numbers.

Is permanently retiring an id actually behavior that people want? It seems more like it's a consequence of wrapping the generation and not wanting a branch in its increment logic.

The reason for not wrapping generation IDs is to prevent the entire class of errors caused by aliasing, which range from subtle to significant bugs and are difficult to test for, design around and reproduce. By retiring we trade that class of errors for a single well-defined error case that both already exists (spawn more entities than assignable IDs) and is significantly less likely to occur (both than aliasing and running out of entity IDs alone). This has the additional advantage of potentially allowing us to expose a handler to the developer, if that is something we're interested in (ie. save critical data, shutdown gracefully, perform some operation to free entity IDs).

I wouldn't expect people to spawn 4 billion entities 4 billion times each (they might tho), but aside from being less code to change, this part feels arbitrary. I think this PR should argue why retirement is a good idea for everyone, since it would essentially put a hardcoded time-limit on a World, which is strange when things like change ticks wraparound.

If there is actual demand for destroying and re-creating that many entities over the lifetime of the program, I'd suggest that the first step would be to re-evaluate the approach being taken. Other than standard suggestions like merging entities into groups (ie. a particle system instead of particle entities), I would suggest implementing a pool system. This pool system will better-fit the user's advanced use case as they can then attach (or not attach) any amount of metadata to identify aliasing (ie. a u128 generation), and force the developer to identify which systems care about aliasing. One other solution is sub-apps/sub-worlds, and recreating them if you exhaust IDs, which is a little more transparent and only requires the user to define and facilitate cross-world communication.
I also believe that the chance of encountering this error is so low that any such designs that could, would already need to be designing around limits such as the entity ID address space. Assuming a 4GHz processor incrementing the generation of each entity in-turn, once per tick and performing no other logic/work, it would take 140 years to do. To bring that down to something like a decade, you would need to permanently occupy all but 300k entities (ie. permanently take-up 4 billion, or 93.75% of the address space), at which point you would be much more likely to run out of entity IDs than generations and likely have thus design around it (and it would be a significant burden on system resources) - meaning that it wouldn't be unreasonable to also require one to design around a generation limit at that extreme.

Also, in the future, it's plausible that Bevy would reduce the number of generation bits to add something else.

Reducing the number of generation bits could definitely be of concern. Reducing it by even 8 bits in this engineered scenario would only leave us with a little over a half year of uptime. The engineered case is obviously not very useful for doing any work however, and if you're spending 95% of your clock cycles on actual work, that half year stretches out to just over a decade).
Though as you point out the current implementation panics immediately as soon as generations roll over (though only in debug), and that would occur in just barely over a second (1.07). So on the whole this should be an improvement. Maybe it's something that should be re-visted later when reducing the number of generation bits moves forward? Or is it something worth talking about now? Since this PR effectively allows developers the freedom to chew up generations at a higher rate, and switching back later to be more strict again would break that code and switching to allow wrapping would also break such code

The reason for not wrapping generation IDs is to prevent the entire class of errors caused by aliasing, which range from subtle to significant bugs and are difficult to test for, design around and reproduce.

I also believe that the chance of encountering this error is so low that any such designs that could, would already need to be designing around limits such as the entity ID address space.

You acknowledge that developers who could even encounter this would likely be designing around the limits of the address space already, so it seems a bit overkill to enforce it. If we allow the generation to wraparound, we could just emit a warning when the it resets to 1, e.g. "Dangling references to entity ID {id} may have become valid again."

[A] pool system will better-fit the user's advanced use case as they can then attach (or not attach) any amount of metadata to identify aliasing (ie. a u128 generation), and force the developer to identify which systems care about aliasing.

One other solution is sub-apps/sub-worlds, and recreating them if you exhaust IDs, which is a little more transparent and only requires the user to define and facilitate cross-world communication.

The solutions suggested seem more troublesome than just warning the user about possible aliasing, since they expect the user to have a clear understanding of aliasing anyway or lean on future work to overcome the inconvenience.
Sub-worlds—or any extension to bevy_ecs that would let users reserve a range of entities—are what I'm mostly concerned about, since they're more likely to be impacted by retirement. Recreating an exhausted World does not sound like a cheap operation—not that Bevy has a method (or idiom) to actually do so.
I think retirement is OK relative to the current panic behavior. It's just, given the rarity of apps that approach the limits of a 64-bit ID and our shared assumption that their developers would most likely be careful to avoid dangling entity references anyway, ID retirement seems more like a weirdly targeted obstacle than a general user safeguard.
Other ECS additions we're expecting may also reduce the likelihood of encountering entity aliasing bugs. For example, automatic cleanup of dangling references following despawn is one of the highlights of relations whenever they're brought up in discussion.

You acknowledge that developers who could even encounter this would likely be designing around the limits of the address space already, so it seems a bit overkill to enforce it. If we allow the generation to wraparound, we could just emit a warning when the it resets to 1, e.g. "Dangling references to entity ID {id} may have become valid again."

No, the error I'm talking about in those quotes is exhausting the address space without generation rollover. A bad alias from rollovers is either at least as unlikely as world retirement or orders of magnitude more likely to occur - particularly if we were to reduce the number of bits. Enough that I believe it should be avoided by default. Designing for and debugging aliasing would be a non-insignificant burden on developers since it's such a hard issue to reproduce.

The solutions suggested seem more troublesome than just warning the user about possible aliasing, since they expect the user to have a clear appreciation of aliasing anyway or lean on future work to overcome the inconvenience.

I expect them to have a clear appreciation of aliasing in that case, because to encounter that case they would already have had to have designed around other limitations of the entity address space. I do think some kind of graceful shutdown in the case of a panic could be nice though - but that's another design talk for a different part of the engine really.

Sub-worlds—or any extension to bevy_ecs that would let users reserve a range of entities—are what I'm mostly concerned about, since they're more likely to be impacted by retirement.

I definitely agree regarding reserved ranges, that's definitely a huge concern.
I'm just not convinced that making generations rollover by default make sense when you're already manipulating the address space. In that case we could make it opt-in since they're already making active choices regarding the address space, for example we could a mechanism flag a range or subworld as having rollover, and still provide aliasing safety by default.

Recreating an exhausted World does not sound like a cheap operation, nor does Bevy have a method (or idiom) for doing that.

Recreating an exhausted world is definitely an expensive operation. However to need to do something like that you would, with the current amount of bits, need to be doing some kind of multi-decade data science number crunching and providing efficient patterns of such fringe use cases of a game engine seems out of scope to me.

Other ECS additions we're expecting may also reduce the likelihood of encountering entity aliasing bugs. For example, automatic cleanup of dangling references following despawn is one of the highlights of relations whenever they're brought up in discussion.

That's definitely a step towards removing aliasing issues. I do have concerns about that regarding efficient access patterns and storage overhead, primarily in the case of grid or tree structures. But I expect after relations drop there will be plenty of efficient implementations of such structures.

I think retirement is better than the current debug panic / silent release behaviour.

Yeah, that definitely makes me uncomfortable, even though it makes sense it still gives me anxiety just thinking about it. I do wonder if going with panic in both debug/release for now might be the better option, since its more restrictive it might be an easier transition when it later opens it up in other specific ways.

our shared assumption that their developers would most likely be careful to avoid dangling entity references anyway, ID retirement seems more like a weirdly targeted obstacle than a general user safeguard.

The main reason I have concerns about it is because the likelihood of it occurring ranges all the way from as unlikely as retiring a world, to almost certain to occur. Conceivably it can be encountered by developers and players, without any strange access patterns, creating a very difficult situation to debug. Where-as the access patterns required to exhaust a world via retirement is a  very fringe use-case, and would require the user to already be making informed design decisions around the entity address space.
However, I can't actually quantify the risk of the average user running into aliasing issues. With 32-bit generations it's either as or more significant than exhausting the address space but ultimately insignificant for most use cases, but dropping to even 24-bits significantly increases the risk of it occurring and is approaching being almost certain at something extreme like 16-bits. In those cases we would be trading a single entity id for effectively an additional 32/24/16-bits worth of unaliased generations, which is why I generally prefer to make the trade.
But that's the thing, it's just the trade I like to make in my hobby ECSs. This isn't the hill I want to die on and I trust that the people working on bevy are much more versed in the design space than I am, as well as on the future direction of Bevy's ECS. So, I'm not against rolling rollover it into this PR if its the general consensus - or we could be more conservative and enforce panic at both debug and runtime - or as you say continue with the PR as-is.
PS. Hope I'm not coming off as rude, definitely none of this is meant with any kind of bad tone.

Given that this is motivated by "optimization", have we done any tests or measurements to show that we successfully optimized something? What have we "won" here?

Hmmm I guess this is pretty reasonable proof:
    #[test]
    fn entity_option_size_optimized() {
        assert_eq!(
            core::mem::size_of::<Option<Entity>>(),
            core::mem::size_of::<Entity>()
        );
    }

I agree that "retirement" is an extremely reasonable approach for now, especially given that the current "spawn and despawn a lot of entities frequently" behavior will put a lot of load on lower entity ids (because we favor reuse over extending into higher ids). This way, we can "keep going" after greedily exhausting the lower ids. Very, very nice!

I also appear to experience some regressions, see attached:
https://gist.github.com/notverymoe/9b11f94ab76bccee69ff44fde3551442
But with 50000_entities_table, if I'm reading it right, is faster than master by 15-22%:
Benchmarking world_query_iter/50000_entities_table
Benchmarking world_query_iter/50000_entities_table: Warming up for 500.00 ms
Benchmarking world_query_iter/50000_entities_table: Collecting 100 samples in estimated 4.1968 s (106k iterations)
Benchmarking world_query_iter/50000_entities_table: Analyzing
world_query_iter/50000_entities_table
                        time:   [38.267 us 39.290 us 40.300 us]
                        change: [-22.294% -19.576% -16.936%] (p = 0.00 < 0.05)

Though, world_query_get/50000_entities_table & world_get/50000_entities_table appears to have regressed by a similar amount:
Benchmarking world_get/50000_entities_table
Benchmarking world_get/50000_entities_table: Warming up for 500.00 ms
Benchmarking world_get/50000_entities_table: Collecting 100 samples in estimated 6.0048 s (10k iterations)
Benchmarking world_get/50000_entities_table: Analyzing
world_get/50000_entities_table
                        time:   [665.63 us 684.25 us 700.73 us]
                        change: [+18.658% +21.273% +24.270%] (p = 0.00 < 0.05)
                        Performance has regressed.
...
Benchmarking world_query_get/50000_entities_table
Benchmarking world_query_get/50000_entities_table: Warming up for 500.00 ms
Benchmarking world_query_get/50000_entities_table: Collecting 100 samples in estimated 5.6335 s (10k iterations)
Benchmarking world_query_get/50000_entities_table: Analyzing
world_query_get/50000_entities_table
                        time:   [531.75 us 545.30 us 559.13 us]
                        change: [+15.406% +18.433% +21.994%] (p = 0.00 < 0.05)
                        Performance has regressed.

busy_systems/01x_entities_12_systems & empty_systems/001_systems appears to have regressed 20-30%, and busy_systems/01x_entities_03_systems & empty_systems/010_systems improved by slightly less.
Will have a look into things when I make it home from work (~3-4 hours)

Just got home and ran the tests again:
https://gist.github.com/notverymoe/2e7379dc9fe44e38a53064195cddada4
Notable regressions:

for_each_par_iter/threads/4 ~45%
busy_systems/02x_entities_12_systems ~25%
busy_systems/02x_entities_15_systems ~18%
busy_systems/03x_entities_12_systems ~33%
busy_systems/04x_entities_06_systems ~42%
busy_systems/04x_entities_12_systems ~32%
busy_systems/04x_entities_15_systems ~18%
busy_systems/05x_entities_12_systems ~25%

Notable improvements:

sized_commands_0_bytes/4000_commands ~20%
sized_commands_12_bytes/4000_commands ~26%
for_each_par_iter/threads/1 ~57%
empty_systems/000_systems ~48%
empty_systems/005_systems ~20%
busy_systems/03x_entities_03_systems ~30%
busy_systems/03x_entities_06_systems ~25%
busy_systems/03x_entities_09_systems ~16%
busy_systems/02x_entities_03_systems ~30%
busy_systems/02x_entities_06_systems ~29%
busy_systems/05x_entities_03_systems ~30%

Most of the world_get/world_query/world_entity/etc. have not change <~1%, and those that do seem to be in the range of 4-6% regressions. world_query_iter/50000_entities_table in particular has a 0.25% change on my home system.
Bunch of weird variances in the busy_system stuff, not sure if that's normal/expected? Maybe I have something funky with my setups? Home running Arch, latest stable rust, 16GiB, AMD Ryzen 7 3700X 8-Core (16 thread). Work running Ubuntu 18.04, latest stable rust, 32GiB ram, I'll check CPU when I'm in tomorrow and update this. I'll also give this a test on Windows sometime tomorrow.
Unfortunately I probably won't get a chance to properly look into this tonight unfortunately, or tomorrow night. I'll try for Thursday and Saturday GMT-8 to look more into it. I would expect some minor slowdown, but surely not 30%.... Hmmm...

Yeah I do think some of these tests are flaky. I really wish we had my tests from #2062, which are generally much stabler + have been very useful when I was implementing Bevy ECS V2. But that got reverted because they iterate on ecs_bench_suite, which doesn't have clear licensing.
I'll give those a run and see what they say.

I'm seeing (repeatable) notable regressions in:

simple_iter/base (~16%), which is in-line with the table iterator regression
get_component/base (~10%), which does make sense, given that we've added complexity to entity location lookups.

And a repeatable notable improvements in frag_iter/base (~12%).

"fragmented iteration" performance is generally more important than "unfragmented iteration", so the tradeoff seems welcome. And notably, we aren't seeing a significant reduction in system iterator impls.
get_component regressions are understandable, but unfortunate.
I suspect that some of these benches would benefit from black_box.

To me this doesn't feel clear cut. I'm guessing that the get_component regression stems from the "retirement" code (due to extra branching) and not the "niching" code. If it does come from "niching", idk if the Option<Entity> memory size gains are worth it. get_component (which is actually a poorly named Query::get benchmark) is a very common api can and should be finely tuned to perfection. We are "fast" when compared to other rust archetypal ecs-es, but we are "slow" when compared to direct vec indexing ... this is an interesting related conversation.
If the regression does come from "retirement" ... we might just want to pay that price. That behavior is much better than our current approach.

Currently working through this. I merged in https://github.com/bevyengine/bevy/pull/4332 to get those tests back in, and I'm getting a consistent "change within noise thresholds for" or, occasionally, a marginal 1.5% improvement for get_component on stable. On nightly I'm getting some very inconsistent runs, ranging from a 2% to 10% regression for get_component/base and up to between ~10% regression or 2% improvement on get_component/system.
Looking into it on godbolt, the regression would make sense. It seems to output more assembly in the Option<NonZeroU32> version for the comparison, which I'd often see people talk about as "free" on various forums - not certain what's up there. Am pleasantly surprised that there's no heavy regression on spawn/destroy performance, which is where I was anticipating it.
By changing the EntityMeta structure to use raw u32s I can, almost, revert the logic in the Entities::get except it needs to convert the NonZeroU32 to a plain u32, which should be 'free' (std::mem::transmute doesn't improve times at the very least). This doesn't seem to offer any improvement on times however. Strangely, on nightly, I get a ~10% regression on the u32 meta versus the NonZeroU32 meta version.
Changing it this PR to alter Entity::id might have a lower performance impact, since we'd only be performing a bit-negation or a single subtract we know won't overflow (or we can just have an unusable 0th entry in the array and pay nothing). However we would be paying that cost on basically every operation involving entities and it wouldn't provide any changes in the generation overflow behaviour that appears to be desired.

Removing from the milestone, this is nice, but shouldn't block a release.

@notverymoe any chance you want to resolve merge conflicts on this and get it up to speed? If so I'll remove the Adopt-Me label and wrangle some final reviews.

Happy to give this an update, might take me a few days though, first day back at work - might be a little hectic.

Not a rush, thanks!

Just updated this, the merge was a bit ugly to try and resolve so i just recreated the PR line by line.
The hierarchy system required some reworking since it made use of placeholders, I'm not sure my solution is ideal.

I'm not sure what to do about the failed github check, it seems to have failed to clone the repo

Yep I'm rerunning CI for you :) In the future, just ping me or another maintainer and we can do that.

I'd like to see some fresh benchmarks, because this adds branches in numerous key entity-related functions.
The removal of PLACEHOLDER and the introduction of Parent::try_get is also generally bad UX IMO.

Definitely agree on both those last points, I wasn't entirely certain what the preferred solution there was. I hope to get some fixes for this up by the weekend

I want to also propose an alternative implementation here that might avoid the branches in Entities: use NonZeroU32 for the index not the generation. We can resolve this by padding the Entities internal Vec with an unused EntityMeta, which would require fewer branches and unwraps, and would not incur any cost when indexing into Entities. We lose our on entity zero, but we literally have 2^32 - 2 others, which is probably a worthwhile tradeoff.

For the purposes of the issue being address, niche-optimization, absolutely and I'd be keen to go forward with it since it'll likely be more performant than previous tests for the generation indicate. However, we do lose out on having consistent behavior for generation overflows - currently in development exhausting a generation will cause a panic, whilst in production it'll silently overflow.

I mean, generation overflow behavior might be better addressed with a proposal since subworlds complicate this quite a bit.

Hey there, I haven't forgot about this, I just kept remembering when I was busy and forgot again by the time I was free :sweat_smile:
I wrote up the reserved-id branch a little while ago, and missed posting about it:
https://github.com/nvm-contrib/bevy/tree/niche-optimization-for-entity-via-id
I tried running the benches, but I got similar inconsistent results like mentioned earlier in this issue. If I get a chance I'll give the benches another go soon.

Bumping to 0.12 milestone: feel free to remake this PR if you think that will be easier than fixing merge conflicts!

Starting a conversation on discord, because honestly I need to know what direction I should be pushing this.

First pass attempt #2 at Entity::index nicheing. Still some quirks and questions:
https://github.com/nvm-contrib/bevy/pull/1/files
If we decide in the discord discussion that index is the go-forward, I'll close this PR and open one for the index version on here
I'll have an update for the Generation version of this change sometime this weekend. Some notes/changes I'll be looking to add into here, after having talked with Joy/Joy2 on discord:

Generation -> NonZeroU32
We can make flags and relationships work with it
We can mitigate performance regressions for iteration/get by removing Option from the entity meta, at the cost of a slightly more expensive free operation.
Wrap generation for now, and table for discussion, since it'll be less problematic to swap from wrapping to not-wrapping than the other way around.
There are plenty of generations to go around
Already planned work will address theoretical subworld id sync concerns

This PR has been superseded:
https://github.com/bevyengine/bevy/pull/9907


ARM compatibility

Added an unused int32 to the Channel structure. This fixes execution on ARM platforms (specifically Raspberry Pi) caused by https://github.com/golang/go/issues/599. After cross-compiling for ARM using
GOOS=linux GOARM=6 GOARCH=arm go build github.com/barnybug/go-cast/cmd/cast

it appears that the call
requestId := int(atomic.AddInt64(&c.requestId, 1))

in Request() causes errors because the Channel structure isn't appropriately 64-bit aligned. The returned error looks like this:
pi@raspberrypi:~ $ ./cast --name chromecast media play 'http://<IP_ADDRESS>:8080/music/play.mp3"
Connecting to <IP_ADDRESS>:8009...
Connected
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0x152894]

goroutine 1 [running]:
panic(0x332b60, 0x10824008)
	/usr/local/go/src/runtime/panic.go:500 +0x33c
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.HandleAction.func1(0x108f8f30)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/app.go:478 +0x290
panic(0x332b60, 0x10824008)
	/usr/local/go/src/runtime/panic.go:458 +0x454
sync/atomic.addUint64(0x1090159c, 0x1, 0x0, 0x1, 0x0)
	/usr/local/go/src/sync/atomic/64bit_arm.go:31 +0x68
github.com/barnybug/go-cast/net.(*Channel).Request(0x10901580, 0x76ef1120, 0x10816440, 0x495920, 0x4aa058, 0x141130, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/net/channel.go:77 +0x60
github.com/barnybug/go-cast/controllers.(*ReceiverController).GetStatus(0x108c5700, 0x76ef1120, 0x10816440, 0x0, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/controllers/receiver.go:150 +0x58
github.com/barnybug/go-cast.(*Client).launchMediaApp(0x10816900, 0x76ef1120, 0x10816440, 0x0, 0x0, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/client.go:133 +0x48
github.com/barnybug/go-cast.(*Client).Media(0x10816900, 0x76ef1120, 0x10816440, 0x4, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/client.go:171 +0x54
main.runCommand(0x76ef1120, 0x10816440, 0x10816900, 0x36447a, 0x4, 0x1080a0e8, 0x1, 0x1)
	/workspace/src/github.com/barnybug/go-cast/cmd/cast/main.go:343 +0xc8
main.cliCommand(0x10878280)
	/workspace/src/github.com/barnybug/go-cast/cmd/cast/main.go:128 +0x244
reflect.Value.call(0x31c8a0, 0x395e40, 0x13, 0x36428e, 0x4, 0x108f8ef0, 0x1, 0x1, 0x0, 0x0, ...)
	/usr/local/go/src/reflect/value.go:434 +0xd68
reflect.Value.Call(0x31c8a0, 0x395e40, 0x13, 0x108f8ef0, 0x1, 0x1, 0x0, 0x0, 0x0)
	/usr/local/go/src/reflect/value.go:302 +0x84
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.HandleAction(0x31c8a0, 0x395e40, 0x10878280, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/app.go:487 +0x1d0
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.Command.Run(0x36447a, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3676e2, 0xf, 0x0, ...)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/command.go:191 +0xc10
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.(*App).RunAsSubcommand(0x108883c0, 0x10878140, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/app.go:361 +0xd18
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.Command.startApp(0x3647da, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3671f2, 0xe, 0x0, ...)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/command.go:278 +0x78c
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.Command.Run(0x3647da, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3671f2, 0xe, 0x0, ...)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/command.go:79 +0x54
github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli.(*App).Run(0x10888300, 0x1080a0c0, 0x6, 0x6, 0x0, 0x0)
	/workspace/src/github.com/barnybug/go-cast/vendor/github.com/codegangsta/cli/app.go:240 +0x798
main.main()
	/workspace/src/github.com/barnybug/go-cast/cmd/cast/main.go:116 +0x6a0

The addition of the unused int32 forces the alignment to adjust appropriately and prevents the runtime error.

Nice spot.

Thanks! Haven't written much go, so it was quite a frightening stack trace to encounter: was quite pleased to realize the solution was quite that simple (even if I'm honestly not sure WHY that works).
Great work with go-cast, by the way. Finally got it all wired and the Raspberry Pi tucked in the basement can now stream any MP3 from the NAS to the chromecast audio in the living room with ease. Now just need a suitable media library so I can index the music on the NAS and I should be good to go.