bug: Make sure to build avante

Describe the bug
Freshly installed avante.nvim, attempted the build="make BUILD_FROM_SOURCE=true" modification as well as  build="make", still getting the following after a successful open  + attempt to avante ask
E5108: Error executing lua: ...ocal/share/Kickstart/lazy/avante.nvim/lua/avante/llm.lua:61: Make sure to build avante (missing avante_templates)
stack traceback:
[C]: in function 'error'
...cal/share/Kickstart/lazy/avante.nvim/lua/avante/path.lua:75: in function 'get'
...ocal/share/Kickstart/lazy/avante.nvim/lua/avante/llm.lua:61: in function 'stream'
.../share/Kickstart/lazy/avante.nvim/lua/avante/sidebar.lua:1299: in function 'handle_submit'
.../share/Kickstart/lazy/avante.nvim/lua/avante/sidebar.lua:1353: in function <.../share/Kickstart/lazy/avante.nvim/lua/avante/sidebar.lua:1343>
To reproduce
install via lazy as directed in README.md:
.config/Kickstart/lua/custom/plugins:
return {
   "yetone/avante.nvim",
   event = "VeryLazy",
   lazy = false,
   version = false, -- set this if you want to always pull the latest change
   opts = {
   	-- add any opts here
   },
   -- if you want to build from source then do `make BUILD_FROM_SOURCE=true`
   build = "make",
   -- build = "powershell -ExecutionPolicy Bypass -File Build.ps1 -BuildFromSource false" -- for windows
   dependencies = {
   	"stevearc/dressing.nvim",
   	"nvim-lua/plenary.nvim",
   	"MunifTanjim/nui.nvim",
   	--- The below dependencies are optional,
   	"nvim-tree/nvim-web-devicons", -- or echasnovski/mini.icons
   	"zbirenbaum/copilot.lua", -- for providers='copilot'
   	{
   		-- support for image pasting
   		"HakonHarnes/img-clip.nvim",
   		event = "VeryLazy",
   		opts = {
   			-- recommended settings
   			default = {
   				embed_image_as_base64 = false,
   				prompt_for_file_name = false,
   				drag_and_drop = {
   					insert_mode = true,
   				},
   				-- required for Windows users
   				use_absolute_path = true,
   			},
   		},
   	},
   },
}

Expected behavior
Ask succeeds.
Environment
NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1713484068
Run "nvim -V1 -v" for more info
Repro
vim.env.LAZY_STDPATH = ".repro"
load(vim.fn.system("curl -s https://raw.githubusercontent.com/folke/lazy.nvim/main/bootstrap.lua"))()

require("lazy.minit").repro({
	spec = {
		-- add any other plugins here{
		"yetone/avante.nvim",
		event = "VeryLazy",
		lazy = false,
		version = false, -- set this if you want to always pull the latest change
		opts = {
			-- add any opts here
		},
		-- if you want to build from source then do `make BUILD_FROM_SOURCE=true`
		build = "make",
		-- build = "powershell -ExecutionPolicy Bypass -File Build.ps1 -BuildFromSource false" -- for windows
		dependencies = {
			"stevearc/dressing.nvim",
			"nvim-lua/plenary.nvim",
			"MunifTanjim/nui.nvim",
			--- The below dependencies are optional,
			"nvim-tree/nvim-web-devicons", -- or echasnovski/mini.icons
			"zbirenbaum/copilot.lua", -- for providers='copilot'
			{
				-- support for image pasting
				"HakonHarnes/img-clip.nvim",
				event = "VeryLazy",
				opts = {
					-- recommended settings
					default = {
						embed_image_as_base64 = false,
						prompt_for_file_name = false,
						drag_and_drop = {
							insert_mode = true,
						},
						-- required for Windows users
						use_absolute_path = true,
					},
				},
			},
		},
	},
})

note: i'm unable to use the default leader key \ or the remapped leader key <space> from my other configs to test if aa works. :AvanteAsk works normally in this case. I noticed a build step occur in this repro that doesn't seem to be occuring in my kickstart.nvim config.

I have the same issue. I resolved it by running:
:AvanteBuild
I'm not sure if this can resolved if we replace "make" with "AvanteBuild" as well
cc: @twilwa

:AvanteBuild doesn't provide any terminal feedback when I just run it with neovim open and the error persists, did you modify the default config at all?

For me the issue was in a wrong architecture (I use x86), so I had to :AvanteBuild source=true
But it also failed (silently)
After some digging I figured the problem was with outdated version of rustc
After upgrading rust :AvanteBuild source=true succeeded and the issue is now resolved for me

@twilwa you can also verify outside of neovim,
by checking out https://github.com/yetone/avante.nvim/ into a separate directory
and then typing:
make BUILD_FROM_SOURCE=true
If you don't have the correct versions of cargo and rustc, then the compilation will fail, and it might be easier to see why.

If you run make BUILD_FROM_SOURCE=true and then start neovim, after about one second, neovim exits without outputting any error messages.

@rodrigc The environment is as follows

OS : macOS sonoma
neovim: NVIM v0.11.0-dev-3783+g33464189b-Homebrew
/Users/nekowasabi/.config/nvim/plugged/avante.nvim (using vim-plug
rustc: rustc 1.80.1 (3f5fd8dd4 2024-08-06) (Homebrew)
cargo: cargo 1.80.1

@nekowasabi  That version of cargo and rustc look good.  If you ran the make in the directory /Users/nekowasabi/.config/nvim/plugged/avante.nvim, and the command worked, that should be enough. Are you sure cargo built something?  If it did, there is something else wrong.

Sorry, the specific reason was not found because theoretically everything should work fine if you have build = "make" in your lazy config. I also couldn't reproduce it locally using Lazy.nvim, but you can manually fix this issue with the following steps:

Go to the location of avante.nvim:
a. lazy.nvim
cd ~/.local/share/nvim/lazy/avante.nvim

b. packer.nvim
cd ~/.local/share/nvim/site/pack/packer/start/avante.nvim

c. vim-plug
cd ~/.config/nvim/plugged/avante.nvim

d. Others follow similarly

Manually execute the make command:
make

Check if the make command successfully downloaded all dependencies:
ls ./build/

Normally, there should be several files in the ./build/ directory:
ls ./build/
avante_repo_map.dylib  avante_templates.dylib  avante_tokenizers.dylib

@nekowasabi No worries, thanks for your detailed analysis.  Glad you got it to work.
The make step can fail for various reasons, and currently the plugin does not show the output of the build step, so it can be difficult to figure out what is going on.
I ran into these two problems:

cargo and rustc were not in my PATH, so the build step never worked
I put cargo in my PATH, but it was cargo/rustc version 1.75, which was too old, and the compilation failed

If avante placed the output of the build command in a popup window, that might help people figure things out.

What operating system are you? Because I suspect that the previously precompiled artifacts did not include darwin-x86_64, which caused this issue. I just fixed it: https://github.com/yetone/avante.nvim/pull/637

sw_vers
ProductName:		macOS
ProductVersion:		14.6.1
BuildVersion:		23G93

NVIM v0.10.1
Build type: Release
LuaJIT 2.1.1720049189

Yes, I am on darwin-x86_64 (Intel i9), so looks like https://github.com/yetone/avante.nvim/issues/637 addresses one issue I had.
My advice would be to put the make command in a popup window, so that errors can be seen more easily, but that is not a big deal.
I have started using avante.nvim on my project, and it is amazing!!  Thanks for it, @yetone !

I feel that this issue has been resolved. If the same problem occurs again, I will reopen this issue.

@rodrigc @yetone
It worked. Thank you!!

Maybe I'm doing something wrong, but although everything seems to be executed smoothly:
LUA_VERSION=luajit bash ./build.sh
+ mkdir -p /Users/daiego/.vim/plugged/avante.nvim/build
+ curl -L https://github.com/yetone/avante.nvim/releases/download/v0.0.8/avante_lib-darwin-aarch64-luajit.tar.gz
+ tar -zxv -C /Users/daiego/.vim/plugged/avante.nvim/build
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0 8556k    0 16384    0     0  15240      0  0:09:34  0:00:01  0:09:33 15240x avante_repo_map.dylib
x avante_templates.dylib
100 8556k  100 8556k    0     0  6635k      0  0:00:01  0:00:01 --:--:-- 38.9M

I still get the error around "avante_repo_map", I'm using an M1 chip (arm64), not sure if that's supported atm or not.
I also tried to build from source and although everything runs fine, as soon as I open neovim I see the error popup again.

@itsdaiego there have been a few changes to avante.nvim in the past 24 hours.  Have you updated?  What version do you have?
If you want to try building from source, what versions of cargo and rustc are in your PATH?
If your rust tools are OK, try building manually with:
make  BUILD_FROM_SOURCE=true in the advance.nvim directory mentioned here:
https://github.com/yetone/avante.nvim/issues/612#issuecomment-2375729928

You're right @rodrigc, I tried this afternoon and it wasn't working, updated it again just now and I can see everything running smoothly!
Apologies, and thank you :)

Rustc version

building from source:

Plug Update

Then I open neovim:

I feel like I'm forgetting to do something 😅 , but I cant figure out what it is

I beleive the latest update resolved this error for me, just as an extra data point if anyone finds it helpful.

Maybe I'm doing something wrong, but although everything seems to be executed smoothly:
LUA_VERSION=luajit bash ./build.sh
+ mkdir -p /Users/daiego/.vim/plugged/avante.nvim/build
+ curl -L https://github.com/yetone/avante.nvim/releases/download/v0.0.8/avante_lib-darwin-aarch64-luajit.tar.gz
+ tar -zxv -C /Users/daiego/.vim/plugged/avante.nvim/build
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0 8556k    0 16384    0     0  15240      0  0:09:34  0:00:01  0:09:33 15240x avante_repo_map.dylib
x avante_templates.dylib
100 8556k  100 8556k    0     0  6635k      0  0:00:01  0:00:01 --:--:-- 38.9M

I still get the error around "avante_repo_map", I'm using an M1 chip (arm64), not sure if that's supported atm or not.
I also tried to build from source and although everything runs fine, as soon as I open neovim I see the error popup again.

Working on my macbook air m1, but weirdly I get the same error as the op when running it inside a ubuntu image in parallels. It builds fine too, but even using make BUILD_FROM_SOURCE=true it still throws an error.
Any ideas? Previous comment said linux is working but maybe not ubuntu?

i'm on pop, which should be ubuntu-equivalent -- quick note that it functioned normally when i used the minimal config and built correctly, but when i received the error, it didn't appear to perform the build step at all, so perhaps your error is slightly different than mine

Just an  update, in my case was a setup mistake, I wasn't loading avante_lib (I use Plug), after loading both avante and  avante_lib I'm able to install it without any errors 👍🏽

@itsdaiego Can you elaborate, please? I think I'm doing this, but I don't quite understand how.

@retarpt  I literally load avante_lib then avante as follows:
lua << EOF
require('avante_lib').load()
require('avante').setup({
  provider = "claude", -- Recommend using Claude
  auto_suggestions_provider = "claude", -- Since auto-suggestions are a high-frequency operation and therefore expensive, it is recommended to specify an inexpensive provider or even a free provider: copilot
  claude = {
    endpoint = "https://api.anthropic.com",
    model = "claude-3-5-sonnet-20240620",
    temperature = 0,
    max_tokens = 4096,
  },
  behaviour = {
    auto_suggestions = false, -- Experimental stage
    auto_set_highlight_group = true,
    auto_set_keymaps = true,
    auto_apply_diff_after_generation = false,
    support_paste_from_clipboard = false,
  },
  mappings = {
    --- @class AvanteConflictMappings
    diff = {
      ours = "co",
      theirs = "ct",
      all_theirs = "ca",
      both = "cb",
      cursor = "cc",
      next = "]x",
      prev = "[x",
    },
    suggestion = {
      accept = "<M-l>",
      next = "<M-]>",
      prev = "<M-[>",
      dismiss = "<C-]>",
    },
    jump = {
      next = "]]",
      prev = "[[",
    },
    submit = {
      normal = "<CR>",
      insert = "<C-s>",
    },
    sidebar = {
      switch_windows = "<Tab>",
      reverse_switch_windows = "<S-Tab>",
    },
  },
  hints = { enabled = true },
  windows = {
    ---@type "right" | "left" | "top" | "bottom"
    position = "right", -- the position of the sidebar
    wrap = true, -- similar to vim.o.wrap
    width = 50, -- default % based on available width
    sidebar_header = {
      align = "center", -- left, center, right for title
      rounded = true,
    },
  },
  highlights = {
    ---@type AvanteConflictHighlights
    diff = {
      current = "DiffText",
      incoming = "DiffAdd",
    },
  },
  --- @class AvanteConflictUserConfig
  diff = {
    autojump = true,
    ---@type string | fun(): any
    list_opener = "copen",
  },
})
EOF

Hopefully that will work for you as well 🙏🏽
Make sure you are building the lib correctly though

Please include the required rust version in the install document.

@retarpt I literally load avante_lib then avante as follows:
lua << EOF
require('avante_lib').load()
require('avante').setup({
  provider = "claude", -- Recommend using Claude
  auto_suggestions_provider = "claude", -- Since auto-suggestions are a high-frequency operation and therefore expensive, it is recommended to specify an inexpensive provider or even a free provider: copilot
  claude = {
    endpoint = "https://api.anthropic.com",
    model = "claude-3-5-sonnet-20240620",
    temperature = 0,
    max_tokens = 4096,
  },
  behaviour = {
    auto_suggestions = false, -- Experimental stage
    auto_set_highlight_group = true,
    auto_set_keymaps = true,
    auto_apply_diff_after_generation = false,
    support_paste_from_clipboard = false,
  },
  mappings = {
    --- @class AvanteConflictMappings
    diff = {
      ours = "co",
      theirs = "ct",
      all_theirs = "ca",
      both = "cb",
      cursor = "cc",
      next = "]x",
      prev = "[x",
    },
    suggestion = {
      accept = "<M-l>",
      next = "<M-]>",
      prev = "<M-[>",
      dismiss = "<C-]>",
    },
    jump = {
      next = "]]",
      prev = "[[",
    },
    submit = {
      normal = "<CR>",
      insert = "<C-s>",
    },
    sidebar = {
      switch_windows = "<Tab>",
      reverse_switch_windows = "<S-Tab>",
    },
  },
  hints = { enabled = true },
  windows = {
    ---@type "right" | "left" | "top" | "bottom"
    position = "right", -- the position of the sidebar
    wrap = true, -- similar to vim.o.wrap
    width = 50, -- default % based on available width
    sidebar_header = {
      align = "center", -- left, center, right for title
      rounded = true,
    },
  },
  highlights = {
    ---@type AvanteConflictHighlights
    diff = {
      current = "DiffText",
      incoming = "DiffAdd",
    },
  },
  --- @class AvanteConflictUserConfig
  diff = {
    autojump = true,
    ---@type string | fun(): any
    list_opener = "copen",
  },
})
EOF

Hopefully that will work for you as well 🙏🏽
Make sure you are building the lib correctly though

I just want to add that this solution worked for me. Avante doesn't provide Packer installation instructions, and it's critical to add the avante_lib line before loading avante to avoid errors.


Fix Travis CI errors

Can be seen at https://travis-ci.org/IMGIITRoorkee/django-filemanager/. Most of them are linting errors as of now. #14 shall be closed with this too.

@rohithasrk Can I claim this?

Sure. @ErwinJunge.


run: do not block on greenlets after command exits

The stdout/stderr greenlets will not necessarily exit when the command does if
child processes are stuck in an uninterruptible sleep. For example, the fsx.sh
workunit spawns fsx processes that may be left behind in the D state after
/bin/timeout kills fsx.sh. These are connected to the stdout/stderr pipes which
prevent the greenlets from exiting normally.
Signed-off-by: Patrick Donnelly<EMAIL_ADDRESS>

Example: /ceph/teuthology-archive/pdonnell-2018-10-01_03:19:12-multimds-wip-pdonnell-testing-20181001.011252-distro-basic-smithi/3090388/teuthology.log

I don't see why the build is broken or how it could be caused by this PR.

I think its a flake8 issue,  line 148: s initialized but never used


Travis npm install

Coverage decreased (-0.2%) to 87.662% when pulling 2ab0352d24141c0985d31507eb5ac9268f6b2f8e on travis-npm-install into 05931de655c0ac8ebc8e6bb3635a8d246543deba on master.

Coverage decreased (-0.2%) to 87.662% when pulling bc9301688030b00137394ac03f15b48a0f9b05c9 on travis-npm-install into 05931de655c0ac8ebc8e6bb3635a8d246543deba on master.


fix: Correct the logic for tagged field defaults

This fixes an issue where the logic for tagged fields was not in sync with upstream. This is in the realm of undocumented features of the Kafka protocol, so there is no accurate documentation to reference. The closest to documentation that exists, is this section in a source code README.
There was a long discussion about this behavior on the Kafka dev mailing list.
This fixes #215.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (a4a78b1) to head (aa2b82c).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #216   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           14        15    +1
  Lines          854       911   +57
  Branches       119       125    +6
=========================================
+ Hits           854       911   +57

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


management track2 tests should not wait in playback mode

they should not use sleep() in tests.  Use delay() from test recorder instead
they should pass a test polling options to LRO operations

We should probably update code gen to add the test polling options and add comments on when to use it.

Example PR: https://github.com/Azure/azure-sdk-for-js/pull/18546
Tests with unnecessary waiting time in playback mode:

[ ] arm-compute
[ ] arm-network
[ ] arm-eventhub
[ ] arm-eventgrid
[ ] arm-databricks
[ ] arm-appplatform
[ ] arm-apimanagement
[ ] arm-containerservice

@jeremymeng I think this issue should have been resolved by PR #18700.  Could you help double confirm ?

@qiaozha yeah looks great! thanks for the fix!


Batch entites by type in federated queries

When Apollo-Federation executes federated query, it queries _enties field on Query type with a list of entities to resolve. Currently each of entities is resolved separately from rest, so if there's 100 entities queried, this will call __resolve_entity 100 times which in turn will run 100 database queries.
This PR replaces default _entities resolver with new one that batches loads per entity type. graphene-federation's __resolve_entity is replaced by __resolve_entities that is expected to return list of items.
FEDERATED_QUERY_MAX_ENTITIES setting is included that allows imposing hard limit on number of entities that may be queried for at single time. Defaults to 300, but can be disabled by setting FEDERATED_QUERY_MAX_ENTITIES=0 in env, or increasing accordingly.
Note: I will provide documentation for federation implementation in Saleor as a separate task.
Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[x] Removed API types, fields, or mutations (Removed ProductImage from federation's _Entity union cos it's python representation is same as ProductMedia)
[x] Documentation needs to be updated Separate PR

Pull Request Checklist

[x] Privileged queries and mutations are guarded by proper permission checks
[x] Database queries are optimized and the number of queries is constant
[x] Database migration files are up to date
[x] The changes are tested
[x] GraphQL schema and type definitions are up to date
[x] Changes are mentioned in the changelog

<!--

@L3str4nge entity resolver for federation quietly skips entities that don't define __entities_resolver. This is up to federation gateway that's level above to resolve and handle missing entities.

I'm sending this back to draft, I'll need to update this PR to include two more federated types that for some reason don't have any entity resolvers (PageType and Group). Should be quick fix for tomorrow.

Codecov Report

Merging #8273 (9065e4e) into main (731cf61) will decrease coverage by 0.00%.
The diff coverage is 93.01%.

:exclamation: Current head 9065e4e differs from pull request most recent head 23dc4f9. Consider uploading reports for the commit 23dc4f9 to get more accurate results

@@            Coverage Diff             @@
##             main    #8273      +/-   ##
==========================================
- Coverage   93.05%   93.05%   -0.01%
==========================================
  Files         489      489
  Lines       36666    36820     +154
  Branches     4104     4159      +55
==========================================
+ Hits        34121    34262     +141
- Misses       1721     1726       +5
- Partials      824      832       +8

Impacted Files
Coverage Δ

saleor/graphql/core/federation.py
84.31% <78.37%> (-15.69%)
:arrow_down:

saleor/graphql/product/types/products.py
95.40% <95.77%> (-0.18%)
:arrow_down:

saleor/graphql/account/types.py
97.16% <96.55%> (+1.00%)
:arrow_up:

saleor/graphql/app/types.py
100.00% <100.00%> (ø)

saleor/graphql/page/types.py
100.00% <100.00%> (ø)

saleor/settings.py
74.17% <100.00%> (+0.12%)
:arrow_up:

saleor/graphql/product/filters.py
90.00% <0.00%> (-0.77%)
:arrow_down:

saleor/webhook/payloads.py
95.73% <0.00%> (ø)

saleor/order/notifications.py
93.12% <0.00%> (ø)

saleor/plugins/user_email/constants.py
100.00% <0.00%> (ø)

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 731cf61...23dc4f9. Read the comment docs.

Could we add benchmarks for those new queries?


Fixes for homestore based replication to run correctly

Codecov Report
Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (1dc1e6a) 75.06% compared to head (be18e06) 75.15%.

Files
Patch %
Lines

src/proto/proto_service.cpp
61.11%
6 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #58      +/-   ##
==========================================
+ Coverage   75.06%   75.15%   +0.09%
==========================================
  Files          20       20
  Lines         794      801       +7
  Branches       91       90       -1
==========================================
+ Hits          596      602       +6
- Misses        136      137       +1
  Partials       62       62

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Rename new ruff rule output format to "pretty"

The new ruff rule` output format introduced in 551b810aebb86f396e4c45a8b6b4406a8cf8cf71 doesn't print Markdown but rather some rich text with escape sequences for colors and links, it's actually the "text" format that prints Markdown, so naming the new format "markdown" is very confusing. This commit therefore renames it to "pretty".
This isn't a breaking change since there hasn't been a release yet.

This is netter. I struggled with this a bit.


Cake Contrib

@erikvanbrakel would it be possible to migrate this project to the Cake Contrib org?

@erikvanbrakel what are your thoughts on this?  Thanks

@JeremyCade this has now been completed.  Huge thanks to @erikvanbrakel for allowing the transfer to happen.


Fix build failure due to empty scene paths

Overview
There are occasions where the EditorBuildSettings enumerates all scenes, regardless whether they are missing or not. In that case the scene is reported as enabled, though has an empty path. The build process later fails, as is cant' find the scene at "". Therefore better be safe than sorry and filter out scenes that have an empty path.

Looks great, thanks for contributing!

/azp run


Restructured platform monitoring docs/dashboards

@sanderson a couple comments below:

Someone might have a different opinion than me but I think we should use max() as the agg function in the derivative functions we use:
 a) I think seeing peak periods is more important than average periods
 b) Nathaniel once told me that using `mean()` in a derivative doesn't make much sense.  I'd need him to explain that though--I just trust him.

In the "HTTP Requests Per Minute" cell, it looks like only queryReq is being selected.  We'd want to ask for writeReq as well if we want "HTTP Requests" overall.  Perhaps even req as well, to show a total...or use a stacked line graph (either way works).

Thanks @samhld. I changed most of the queries to use max(). The "HTTP Requests Per Minute" cell has multiple queries:

Query requests
Write requests
Client errors
Server errors

@sanderson ohh, I missed that there were multiple queries :).  Carry on!


Coprocessor task terminated due to exceeding the deadline for query

Bug Report

What version of TiKV are you using?

[10:31:48]TiDB root:test> select type,version,git_hash  from information_schema.cluster_info;
+------+-------------+------------------------------------------+
| type | version     | git_hash                                 |
+------+-------------+------------------------------------------+
| tidb | 7.5.0-test  | db0d44cbda83be81128dcc9d02dc0b8d9108873a |
| pd   | 7.5.0-alpha | 4176c1daac69d00aec0512bd1e362b46fdebdd9a |
| tikv | 7.5.0-alpha | 9fb1ce63a079cd486f0fc4661ff28abb76d0e734 |

What operating system and CPU are you using?

Steps to reproduce

1. create a partition table with 200 partitions;
2. insert about 250000000 rows
3. execute SQL select count(1) from t

related to tidb pingcap/tidb#47942
What did you expect?
query success
What did happened?
tikv log
[2023/10/24 02:08:31.809 +00:00] [INFO] [endpoint.rs:575] ["the max gap of leader resolved-ts is large"] [last_resolve_attempt=None] [duration_to_last_update_safe_ts=10896ms] [min_memory_lock=None] [txn_num=0] [loc
k_num=0] [min_lock=None] [applied_index=13] [read_state="ReadState { idx: 11, ts:<PHONE_NUMBER>64960261 }"] [gap=39079ms] [region_id=4777745] [thread_id=0x5]
[2023/10/24 02:08:32.298 +00:00] [INFO] [tracker.rs:269] [slow-query] [perf_stats.internal_delete_skipped_count=0] [perf_stats.internal_key_skipped_count=136006302] [perf_stats.block_read_byte=2327560933] [perf_sta
ts.block_read_count=240904] [perf_stats.block_cache_hit_count=13081] [scan.range.first="Some(start:<PHONE_NUMBER>000F0EF65F69800000000000000200 end:<PHONE_NUMBER>000F0EF65F698000000000000002FB)"] [scan.ranges=160] [scan.
total=136006411] [scan.processed_size=7576750080] [scan.processed=132925440] [scan.is_desc=false] [tag=index] [table_id=986870] [txn_start_ts=445150200404115457] [total_suspend_time=227.053597ms] [total_process_tim
e=59.772170009s] [handler_build_time=87.327µs] [wait_time.snapshot=23.879µs] [wait_time.schedule=21.859µs] [wait_time=45.738µs] [total_lifetime=59.999362611s] [remote_host=ipv4:<IP_ADDRESS>:45296] [region_id=4690463
] [session_alias=] [connection_id=553656796] [thread_id=0x5]
[2023/10/24 02:08:32.298 +00:00] [WARN] [endpoint.rs:850] [error-response] [err="Coprocessor task terminated due to exceeding the deadline"] [thread_id=0x5]
[2023/10/24 02:08:32.298 +00:00] [WARN] [tracker.rs:449] ["query deadline exceeded"] [tag=index] [table_id=986870] [txn_start_ts=445150200404115457] [total_suspend_time=227.053597ms] [total_process_time=59.772170009s] [handler_build_time=87.327µs] [wait_time.snapshot=23.879µs] [wait_time.schedule=21.859µs] [wait_time=45.738µs] [total_lifetime=59.99943366s] [remote_host=ipv4:<IP_ADDRESS>:45296] [region_id=4690463] [session_alias=] [connection_id=553656796] [current_stage=Tracked] [thread_id=0x5]

This is not a bug. TiKV is clearly telling you that, the query takes too long time to finish. It is usually caused by a very large request, or too many requests in the queue.


feat(techdocs): filter fetched entity kinds and fields

Signed-off-by: Phil Kuang<EMAIL_ADDRESS>Hey, I just made a Pull Request!
Fetch only applicable entity info from the catalog to speed up load times

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Thinking about it... I'd prefer we didn't filter on kinds. While we have a certain core model, people are extending the model with custom kinds that we can't know of beforehand.
I know that User and Group are desirable to exclude, because there often are so many of them. But maybe it's still a great incremental improvement to do the fields slicing. Possibly even making it even more precise, just keeping a smaller slice of metadata?

Thinking about it... I'd prefer we didn't filter on kinds. While we have a certain core model, people are extending the model with custom kinds that we can't know of beforehand.
I know that User and Group are desirable to exclude, because there often are so many of them. But maybe it's still a great incremental improvement to do the fields slicing. Possibly even making it even more precise, just keeping a smaller slice of metadata?

sure, that makes sense, hopefully one day this will utilize an opt-out type filter so it'll support custom kinds
Filtering out unnecessary fields (such as spec.definition for APIs) already provides a big performance improvement :)

Approved on my end! The techdocs team will surely check it out tomorrow


5.1 Feedback: no way to navigate forward/back between screens

Overview
There is no way to navigate forward/back between screens. Clicking Protocols, Labware, or Devices always navigates to the root of each of those sections.
Current Behavior
There are several new navigation paths in 5.1 that move from one nested screen to another. For example, there is a link from Devices > [Robot Name] > Robot Settings > Networking to App Settings > Advanced. After clicking that link, the only way to get back to the immediately previous screen is to click on Devices and navigate all the way down the tree again.
This is not the only such example. Another common one might be clicking the protocol name on a protocol run page. Overall, there could be a lot of tedious UX navigation and frustration for users.
Expected Behavior
Commands in the View menu or keyboard commands (e.g. ⌘[ and ⌘] on Mac) should allow navigation between recently viewed screens.
Alternatively, the click behavior for the top-level nav items (Protocols, Labware, Devices, ⚙) could navigate to the most recently viewed screen within that section rather than always going to the root. Users could get to the root level via breadcrumbs.
Steps To Reproduce
Follow various navigational links within the app.
Operating system
Mac
Robot setup or anything else?
No response

@emilywools - would like your thoughts on this


[documentation] [url_launcher] readme code sample is broken

Steps to Reproduce

run flutter create issue
replace the contents of main.dart with this code
run flutter run -v

Logs
red screen

logs
[  +81 ms] Syncing files to device Pixel 3a...
[   +1 ms] <- reset
[        ] Compiling dart to kernel with 0 updated files
[   +2 ms] <- recompile package:issue/main.dart f17a5964-e946-447d-97f9-843806625829
[        ] <- f17a5964-e946-447d-97f9-843806625829
[   +3 ms] I/flutter (21218): ══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
[        ] I/flutter (21218): The following assertion was thrown attaching to the render tree:
[        ] I/flutter (21218): No MediaQuery widget ancestor found.
[        ] I/flutter (21218): Scaffold widgets require a MediaQuery widget ancestor.
[        ] I/flutter (21218): The specific widget that could not find a MediaQuery ancestor was:
[        ] I/flutter (21218):   Scaffold
[        ] I/flutter (21218): The ownership chain for the affected widget is: "Scaffold ← [root]"
[        ] I/flutter (21218): No MediaQuery ancestor could be found starting from the context that was passed to MediaQuery.of().
[        ] I/flutter (21218): This can happen because you have not added a WidgetsApp, CupertinoApp, or MaterialApp widget (those
[        ] I/flutter (21218): widgets introduce a MediaQuery), or it can happen if the context you use comes from a widget above
[        ] I/flutter (21218): those widgets.
[        ] I/flutter (21218):
[        ] I/flutter (21218): When the exception was thrown, this was the stack:
[        ] I/flutter (21218): #0      debugCheckHasMediaQuery.<anonymous closure> (package:flutter/src/widgets/debug.dart:218:7)
[        ] I/flutter (21218): #1      debugCheckHasMediaQuery (package:flutter/src/widgets/debug.dart:233:4)
[        ] I/flutter (21218): #2      MediaQuery.of (package:flutter/src/widgets/media_query.dart:820:12)
[        ] I/flutter (21218): #3      ScaffoldState.didChangeDependencies (package:flutter/src/material/scaffold.dart:2775:50)
[        ] I/flutter (21218): #4      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4842:11)
[        ] I/flutter (21218): #5      ComponentElement.mount (package:flutter/src/widgets/framework.dart:4658:5)
[        ] I/flutter (21218): #6      Element.inflateWidget (package:flutter/src/widgets/framework.dart:3624:14)
[        ] I/flutter (21218): #7      Element.updateChild (package:flutter/src/widgets/framework.dart:3389:18)
[        ] I/flutter (21218): #8      RenderObjectToWidgetElement._rebuild (package:flutter/src/widgets/binding.dart:1208:16)
[        ] I/flutter (21218): #9      RenderObjectToWidgetElement.mount (package:flutter/src/widgets/binding.dart:1179:5)
[        ] I/flutter (21218): #10     RenderObjectToWidgetAdapter.attachToRenderTree.<anonymous closure> (package:flutter/src/widgets/binding.dart:1121:18)
[        ] I/flutter (21218): #11     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2730:19)
[        ] I/flutter (21218): #12     RenderObjectToWidgetAdapter.attachToRenderTree (package:flutter/src/widgets/binding.dart:1120:13)
[        ] I/flutter (21218): #13     WidgetsBinding.attachRootWidget (package:flutter/src/widgets/binding.dart:960:7)
[        ] I/flutter (21218): #14     WidgetsBinding.scheduleAttachRootWidget.<anonymous closure> (package:flutter/src/widgets/binding.dart:941:7)
[        ] I/flutter (21218): (elided 11 frames from class _RawReceivePortImpl, class _Timer, dart:async, and dart:async-patch)
[        ] I/flutter (21218): ════════════════════════════════════════════════════════════════════════════════════════════════════
[ +105 ms] Updating files.
[        ] DevFS: Sync finished
[   +1 ms] Syncing files to device Pixel 3a... (completed in 119ms)
[        ] Synced 0.0MB.
[        ] <- accept
[  +16 ms] Connected to _flutterView/0x77fe1640a0.
[   +1 ms] Flutter run key commands.
[   +1 ms] r Hot reload. 🔥🔥🔥
[   +1 ms] R Hot restart.
[        ] h Repeat this help message.
[        ] d Detach (terminate "flutter run" but leave application running).
[        ] c Clear the screen
[        ] q Quit (terminate the application on the device).
[        ] An Observatory debugger and profiler on Pixel 3a is available at: http://<IP_ADDRESS>:44743/7uH27cQa-ao=/
[        ] Running with unsound null safety
[        ] For more information see https://dart.dev/null-safety/unsound-null-safety
[+6086 ms] DevFS: Deleting filesystem on the device (file:///data/user/0/com.example.issue/code_cache/issueWXCJPO/issue/)
[  +32 ms] DevFS: Deleted filesystem on the device (file:///data/user/0/com.example.issue/code_cache/issueWXCJPO/issue/)
[ +309 ms] Failure in ext.flutter.exit: ext.flutter.exit: (-32000) Bad state: The client closed with pending request "ext.flutter.exit".

[  +15 ms] Service protocol connection closed.
[        ] Application finished.
[   +3 ms] executing: /home/francesco/Android/Sdk/platform-tools/adb -s 965AY0WP5C forward --list
[   +8 ms] Exit code 0 from: /home/francesco/Android/Sdk/platform-tools/adb -s 965AY0WP5C forward --list
[        ] 965AY0WP5C tcp:33501 tcp:39853
[   +1 ms] executing: /home/francesco/Android/Sdk/platform-tools/adb -s 965AY0WP5C forward --remove tcp:33501
[   +8 ms] executing: /home/francesco/Android/Sdk/platform-tools/adb -s 965AY0WP5C forward --list
[   +5 ms] Exit code 0 from: /home/francesco/Android/Sdk/platform-tools/adb -s 965AY0WP5C forward --list
[   +1 ms] "flutter run" took 37,757ms.
[ +255 ms] ensureAnalyticsSent: 252ms
[   +4 ms] Running shutdown hooks
[   +1 ms] Shutdown hook priority 4
[   +1 ms] Shutdown hooks complete
[        ] exiting with code 0

doctor
[✓] Flutter (Channel master, 1.25.0-5.0.pre.108, on Linux, locale en_US.UTF-8)
    • Flutter version 1.25.0-5.0.pre.108 at /home/francesco/snap/flutter/common/flutter
    • Framework revision 995aa138cd (13 hours ago), 2020-12-08 09:18:02 +0530
    • Engine revision b0d31ae166
    • Dart version 2.12.0 (build 2.12.0-125.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /home/francesco/Android/Sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: /snap/android-studio/current/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Linux toolchain - develop for Linux desktop
    • clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
    • cmake version 3.10.2
    • ninja version 1.8.2
    • pkg-config version 0.29.1

[✓] Android Studio
    • Android Studio at /snap/android-studio/current/android-studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • android-studio-dir = /snap/android-studio/current/android-studio
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] Connected device (3 available)
    • Pixel 3a (mobile) • 965AY0WP5C • android-arm64  • Android 11 (API 30)
    • Linux (desktop)   • linux      • linux-x64      • Linux
    • Chrome (web)      • chrome     • web-javascript • Google Chrome 87.0.4280.66

• No issues found!

pubspec
name: issue
description: A new Flutter project.
version: 1.0.0+1

environment:
  sdk: ">=2.7.0 <3.0.0"

dependencies:
  flutter:
    sdk: flutter
  url_launcher: ^5.7.10

flutter:
  uses-material-design: true

Proposed Solution
please consider this PR

Verified and app shows red screen upon running the readme code sample:

flutter doctor -v

[✓] Flutter (Channel stable, 1.22.4, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-IN)
    • Flutter version 1.22.4 at /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 1aafb3a8b9 (4 weeks ago), 2020-11-13 09:59:28 -0800
    • Engine revision 2c956a31c0
    • Dart version 2.10.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    • CocoaPods version 1.9.3

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 46.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.51.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.16.0

[✓] Connected device (1 available)
    • sdk gphone x86 arm (mobile) • emulator-5554 • android-x86 • Android 11
      (API 30) (emulator)

• No issues found!

PR Merged.


DB Throwing error: "error returned from database: relation "subscriptions" does not exist"

Hi,
I tried to solve this issue, but could not pinpoint to the origin of the error.
The DB creation and migration went smooth, but maybe the "subscriptions" table was not created for some reason.
Here is the full error:
error: error returned from database: relation "subscriptions" does not exist
  --> src/routes/subscriptions.rs:13:11
   |
13 |       match sqlx::query!(
   |  ___________^
14 | |         r#"
15 | |     INSERT INTO subscriptions (id, email, name, subscribed_at)
16 | |     VALUES ($1, $2, $3, $4)
...  |
21 | |         Utc::now()
22 | |     )
   | |_____^
   |
   = note: this error originates in the macro `$crate::sqlx_macros::expand_query` (in Nightly builds, run with -Z macro-backtrace for more info)

Thanks

needed to kill the postgres docker. all test are OK


Can't launch VS Code. Check failed: (isolate->snapshot_blob()) == nullptr.

VSCode Version: 1.34
OS Version: Windows 7 Enterprise

Steps to Reproduce:

Close Visual Studio Code then try to relaunch it and nothing happens, launching from Command Line gives:

C:\Users\BMartin>code --verbose
#
# Fatal error in , line 0
# Check failed: (isolate->snapshot_blob()) == nullptr.
#
#
#
#FailureMessage Object:<PHONE_NUMBER>18F3C0

Only workaround I've found is to do a reinstall, and select the "Launch Visual Studio Code" option from the installer once it has completed. Even this workaround will only last until VS code is next closed, then I have to reinstall again.

Does this issue occur when all extensions are disabled?: Yes

VSCode Version: 1.34

Please upgrade to the latest version 1.35. Can you still reproduce the problem?

Have upgraded to 1.35 and the problem still occurs

I'm seeing this same issue with the latest VSCode (1.37). I rolled back to an earlier version and the problem went away.

I'm on Windows 10 Enterprise, 64-bit

I have the same problem with 1.37.0. Rolling back to 1.36.1 solves it. The nightly insider also works.

1.37.0 started working now too. I use the portable zip version. I replace my previous installation each time I upgrade, keeping the data/ folder. After some back and forth with 1.36.1, insider, and 1.37.0, it started working. This time, I also started up the unzipped 1.37.0 before moving it to replace the old installation, and without the old data/ folder. It still works after replacing the old installation while keeping the data/ folder. I don't know which step solved it.


Add a dockerfile

Hey!
Here's a small change to add a Dockerfile to test locally in a clean and environment-agnostic manner.
To use, you just need to build and run it,
To build
docker build -t sondehub-tracker .

Thanks, looks good.


[BUG] Is there a data source for post race penalties?

Describe the issue:
Sprint results from 2023 Qatar are wrong when loading from F1 live timing API due to penalties applied after the finish. Not sure if there are any data sources to allow fastf1 to apply penalties outside of the official FIA documents or ergast.
Reproduce the code example:
qatar = fastf1.get_event(2023, 17)
sprint = qatar.get_sprint()
sprint.load()
print(sprint.results)

Error message:
No response

No, there are no other data sources for this data. And given that Ergast does not support the Sprint, it's currently impossible for FastF1 to provide the correct results.

@theOehrly Maybe worth adding sprint race support to #445?

It's not mentioned there, but I'm intending to support all session types, yes.


Update outdated MAX links

Update outdated MAX links
/cc @ptitzler @Tomcli

Thank you @ptitzler -- I had to fix the check-doc-links script to properly catch all defunct URLs

/lgtm


docs: Updated screenshots.

Fixes #1247
Changes: I have updated screenshots in readme.md.
I have used cloud.githubusercontent for saving images. I have used this because saving binary files in a repo unnecessarily bloats it. Using Images form cloud.githubusercontent will increase the size of repo.
Screenshots for the change:
Tracks

Schedule

Speakers

Sponsors

Locations

Speaker

Session List

Session

@the-dagger @mananwason
Please review the PR.

Add one with the maps too

Map

@mananwason
Done

@the-dagger
please review and merge

Thanks


Wrong links to JREs on webpage during release cycle

Browser: ALL

During the release week/days the webpage shown version often does not match the real link version for JREs (JDKs are released by Adoptium sometimes 24 hours+ before JRE):

This should be resolved now, thanks for rasing!


Support react-router-dom 5.0.1+

As per this release post v5 is backwards compatible (and should actually have been a minor version bump).

Thanks!


Clear DMA interrupt status

cyw43 seems to be causing a dma channel interrupt to go off if the same channels are reused after cyw43 is deinitialised. Achnowledging them seems to fix it.
Fixes #1156

yes - I'll have a look at adding that

I added dma_channel_cleanup


Achievement board

Achievement board that tracks player mined blocks and gives rewards based on completion

This will be review by the support team. Visit the live support page to initiate a chat with an agent for further assistance via live chat HERE  LIVE CHAT SUPPORT Regards


Cache failures cause job to fail / ServiceUnavailable error during cache download

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: CacheBeta
Environment

Server - Azure Pipelines or TFS on-premises?

If using TFS on-premises, provide the version:

If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

Agent - Hosted or Private:

If using Hosted agent, provide agent queue name:

If using private agent, provide the OS of the machine running the agent and the agent version:

Issue Description
When we get a failure on the pipeline which has happened quite a lot recently, it should use save that the cache was not restored (and not try to save it again).
Task logs
Information, Downloaded 789.2 MB out of 817.4 MB (97%).
Information, Response exception thrown: Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown.
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.GetDedupUrlsRawAsync(ISet`1 dedupIds, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.GetDedupGettersAsync(ISet`1 dedupIds, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClient.<>c__DisplayClass26_1.<<DownloadToFileAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.NonSwallowingActionBlock.<>c__DisplayClass5_0`1.<<CreateNonSwallowingFunc>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClient.DownloadToFileAsync(DedupNode node, String fullPath, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClientWithDataport.DownloadToFileAsync(IDedupDataPort dataport, DedupNode node, String fullPath, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClientWithDataport.DownloadToFileAsync(IDedupDataPort dataport, DedupIdentifier dedupId, String fullPath, UInt64 fileSize, GetDedupAsyncFunc dedupFetcher, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupManifestArtifactClient.<>c__DisplayClass24_0.<<DownloadAsyncWithManifestPath>b__6>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.NonSwallowingActionBlock.<>c__DisplayClass5_0`1.<<CreateNonSwallowingFunc>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.TargetBlockExtensions.SendAllAndCompleteAsync[T1,T2](ITargetBlock`1 targetBlock, IEnumerable`1 inputs, ITargetBlock`1 finalBlock, CancellationToken token)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupManifestArtifactClient.DownloadAsyncWithManifestPath(DownloadDedupManifestArtifactOptions downloadOptions, IEnumerable`1 minimatcherFuncs, Boolean downloadManifestReferences, ISet`1 excludedPaths, CancellationToken cancellationToken)
 Status code: ServiceUnavailable
 Inner exception:
 Source: Microsoft.VisualStudio.Services.WebApi
##[error]Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown.

I'm hoping the workaround of using continueOnError: true on the task will solve these issues.
If you think that's the general solution, I recommend adding it to the examples in the docs.

Thanks for reporting. We need to investigate why you are getting a ServiceUnavailable error.
Also linking to #10907 since we should also consider replicating the step's continueOnError value (default is false) to the post job (save) step.

It was a one off error affecting 2 builds during the same time period. I can give you more details if it would help?

Earlier in the logs you should see a line like this:
 DedupManifestArtifactClient will correlate http requests with X-TFS-Session ${GUID}

If you can share your session GUID we can look directly at the telemetry.

Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session a39b617b-566f-4c42-a5d2-ad03bb0994be
It happened a lot today at various times.
It also happened once on a build pipeline artifact (same error)

@willsmythe @johnterickson any progress on why it happens?
I've switched to 7z compressing node_modules before caching which obv. reduces the number of files massively and takes the payload from 800mb to 200mb and that seems to have stopped the errors occurring. I had one build that hadn't yet rebased on the new pipeline and that still failed, heres the logs
Information, There is a cache hit.
Manifest ID is: BC113A87408DC5914B7FBB298FB03D247100CED15F235AA00DD1CE4987EFEC6D02
Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session 0972f88e-2037-49b2-898e-2bc81a9835a0
Warning, [https://mzgvsblobprodsu6weus1.blob.core.windows.net/dbd2f735ff1e434ce8826b9e3578951332/DFECC24D14563F7BBA5EAF0A0EB703F3F39630A5980BB909809999E1464BD0ED01?sv=2017-04-17&sr=b&sig=MOgTrs7N3FqYMnhTZ26AhaYKJAxsoeNWhfEiOKP6oZc%3D&spr=https&se=2019-07-24T12%3A52%3A28Z&sp=r&rscl=x-e2eid-09b8dfe2-0b6c4786-a2400471-8a1890b7-session-0972f88e-203749b2-898e2bc8-1a9835a0] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'.
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Information, Could not initialize dataport.
Information, Downloaded 0.0 MB out of 814.1 MB (0%).
Information, Downloaded 108.7 MB out of 814.1 MB (13%).
Information, Downloaded 148.3 MB out of 814.1 MB (18%).
Information, Downloaded 175.0 MB out of 814.1 MB (21%).
Information, Downloaded 274.3 MB out of 814.1 MB (34%).
Warning, [https://kx9vsblobprodsu6weus66.blob.core.windows.net/dbd2f735ff1e434ce8826b9e3578951332/920506ED66C13868153A59AD50D5943AEE9CE0BBDA4429E6C37F0D15712C69C701?sv=2017-04-17&sr=b&sig=bVAdA1al0f2mTyAOun5ERmEk1iCnWRcIxulLYb3UsUc%3D&spr=https&se=2019-07-24T13%3A34%3A17Z&sp=r&rscl=x-e2eid-e1ef265b-1e34448c-9cbedea3-45f1289d-session-0972f88e-203749b2-898e2bc8-1a9835a0] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'.
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Information, Downloaded 324.8 MB out of 814.1 MB (40%).
Information, Downloaded 364.2 MB out of 814.1 MB (45%).
Warning, [https://41hvsblobprodsu6weus29.blob.core.windows.net/dbd2f735ff1e434ce8826b9e3578951332/BB828D6465B9328A3490487A7742D2D0560E75BD755B8512726356405AE69C7701?sv=2017-04-17&sr=b&sig=0mgoccTgDIdOGPXJCIikVJrLhea4M6kKIES8TqDBxjE%3D&spr=https&se=2019-07-24T12%3A43%3A57Z&sp=r&rscl=x-e2eid-053c6644-2b284f37-808bbc5e-f154124b-session-0972f88e-203749b2-898e2bc8-1a9835a0] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'.
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Warning, [https://s83vsblobprodsu6weus72.blob.core.windows.net/dbd2f735ff1e434ce8826b9e3578951332/5D5CEBB3CDAE0E76030789B47CD7E4391185069CD086D03DED6849260F1BB2D601?sv=2017-04-17&sr=b&sig=rUkzqgaKXHtoAVTS1oFS3WqYZJfdFreBFg0AThwMMls%3D&spr=https&se=2019-07-24T13%3A21%3A52Z&sp=r&rscl=x-e2eid-053c6644-2b284f37-808bbc5e-f154124b-session-0972f88e-203749b2-898e2bc8-1a9835a0] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'.
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Warning, [https://ijgvsblobprodsu6weus39.blob.core.windows.net/dbd2f735ff1e434ce8826b9e3578951332/239E65EE14E012943CC24EE31FBAD1E6F512F728FF7A383ECEABE1BD3B42CF5801?sv=2017-04-17&sr=b&sig=LwfrLqZNy9uvAyXcZrvibk5gG1jAqrCnZZH3tn85JsE%3D&spr=https&se=2019-07-24T13%3A08%3A21Z&sp=r&rscl=x-e2eid-2844cb8a-d4c2409a-be695e41-70d89284-session-0972f88e-203749b2-898e2bc8-1a9835a0] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'.
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Warning, [https://24uvsblobprodsu6weus18.blob.core.windows.net/dbd2f735ff1e434ce8826b9e3578951332/451A213992BC50FF8D33C605797ABF368C4F43BA42AB6EAA581D29198C011A4601?sv=2017-04-17&sr=b&sig=ippRRRd5LLiaHwkXcJdcDdB6FwgpiILgtFNpDrif55Y%3D&spr=https&se=2019-07-24T13%3A16%3A12Z&sp=r&rscl=x-e2eid-b27b710a-a651458b-974b30f2-3722943b-session-0972f88e-203749b2-898e2bc8-1a9835a0] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'.
   at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
   at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Information, Downloaded 381.4 MB out of 814.1 MB (47%).
Information, Downloaded 453.4 MB out of 814.1 MB (56%).
Information, Downloaded 477.4 MB out of 814.1 MB (59%).
Information, Downloaded 535.2 MB out of 814.1 MB (66%).
Information, Downloaded 630.7 MB out of 814.1 MB (77%).
Information, Downloaded 656.0 MB out of 814.1 MB (81%).
Information, Downloaded 748.2 MB out of 814.1 MB (92%).
Information, Downloaded 769.3 MB out of 814.1 MB (95%).
Information, Downloaded 778.7 MB out of 814.1 MB (96%).
Information, Response exception thrown: Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown.
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpMethod method, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.GetDedupUrlsRawAsync(ISet`1 dedupIds, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.GetDedupGettersAsync(ISet`1 dedupIds, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClient.<>c__DisplayClass26_1.<<DownloadToFileAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.NonSwallowingActionBlock.<>c__DisplayClass5_0`1.<<CreateNonSwallowingFunc>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClient.DownloadToFileAsync(DedupNode node, String fullPath, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClientWithDataport.DownloadToFileAsync(IDedupDataPort dataport, DedupNode node, String fullPath, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreClientWithDataport.DownloadToFileAsync(IDedupDataPort dataport, DedupIdentifier dedupId, String fullPath, UInt64 fileSize, GetDedupAsyncFunc dedupFetcher, Uri proxyUri, EdgeCache edgeCache, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupManifestArtifactClient.<>c__DisplayClass24_0.<<DownloadAsyncWithManifestPath>b__6>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.NonSwallowingActionBlock.<>c__DisplayClass5_0`1.<<CreateNonSwallowingFunc>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.TargetBlockExtensions.SendAllAndCompleteAsync[T1,T2](ITargetBlock`1 targetBlock, IEnumerable`1 inputs, ITargetBlock`1 finalBlock, CancellationToken token)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupManifestArtifactClient.DownloadAsyncWithManifestPath(DownloadDedupManifestArtifactOptions downloadOptions, IEnumerable`1 minimatcherFuncs, Boolean downloadManifestReferences, ISet`1 excludedPaths, CancellationToken cancellationToken)
 Status code: ServiceUnavailable
 Inner exception:
 Source: Microsoft.VisualStudio.Services.WebApi
##[error]Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown.

@lukeapage We've increased the capacity of the scale unit you were seeing this on. You should stop seeing this in an hour or two.

Capacity increased and throttled no longer observed. Do reactivate if you see this error again.

@johnterickson this issue was also about defaulting continue on error to true for cache tasks, do you want to reopen for that or is it handled elsewhere/ you don’t want to do it?

@willsmythe to confirm, you are support of cache steps being continueOnError?

Seeing this too on azure hosted agents (system independent)
https://dev.azure.com/renovatebot/renovate/_build/results?buildId=1157
##[section]Starting: Cache Yarn packages
==============================================================================
Task         : Cache (Beta)
Description  : Cache files between runs
Version      : 0.1.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key: yarn_cache|Linux|/home/vsts/work/1/renovate/yarn.lock
 - yarn_cache                           [string]
 - Linux                                [string]
 - /home/vsts/work/1/renovate/yarn.lock [file] --> F0AD86C2B0B74702E6F53D53D79AA09D29B71D38FF030FE378DF2DCDDCF680B5
Resolved to: yarn_cache|Linux|j+ut0SUhVacRYAzw3wqdvJ7lxDA9k5e2XFbaMEIu1sA=
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 775be811-ed32-41dc-9145-69d576cba85d
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `yarn_cache|Linux|j+ut0SUhVacRYAzw3wqdvJ7lxDA9k5e2XFbaMEIu1sA=`
Information, There is a cache hit: `yarn_cache|Linux|j+ut0SUhVacRYAzw3wqdvJ7lxDA9k5e2XFbaMEIu1sA=`
Information, Used scope: 1;6ed21661-b561-4777-8106-ca95b548903d;refs/heads/master;renovatebot/renovate
Entry found at fingerprint: `yarn_cache|Linux|j+ut0SUhVacRYAzw3wqdvJ7lxDA9k5e2XFbaMEIu1sA=`
Information, ArtifactHttpRetryMessageHandler.SendAsync: https://vsblobprodcus3.vsblob.visualstudio.com/A8ee6462d-ab82-4728-954c-a00ca9af968e/_apis/dedup/chunks/80E6E5C2AD755C018635D5FDD543065F10A351DF61528117707AFB936B0D757001 attempt 1/6 failed with StatusCode RedirectMethod, IsRetryableResponse False
Information, ArtifactHttpRetryMessageHandler.SendAsync: https://vsblobprodcus3.vsblob.visualstudio.com/A8ee6462d-ab82-4728-954c-a00ca9af968e/_apis/dedup/nodes/ECCBC8D9FA383C7041BE02D0D1221866B6E5CC600072B496959EEFE5B00C9C6002 attempt 1/6 failed with StatusCode RedirectMethod, IsRetryableResponse False
Information, Expected size to be downloaded: 401.2 MB
Information, Downloaded 0.0 MB out of 401.2 MB (0%).
Information, Downloaded 37.4 MB out of 401.2 MB (9%).
Information, Downloaded 73.9 MB out of 401.2 MB (18%).
Information, Downloaded 127.3 MB out of 401.2 MB (32%).
Information, Downloaded 184.9 MB out of 401.2 MB (46%).
Information, Downloaded 266.6 MB out of 401.2 MB (66%).
Information, Downloaded 341.5 MB out of 401.2 MB (85%).
Warning, [https://st8vsblobprodcus339.blob.core.windows.net/db8ee6462dab824728954ca00ca9af968e/5FA2790B529DAFD4F9577CA083E07CFF2C1300C353610FB2C4763A72183B333A01?sv=2017-04-17&sr=b&sig=vPtybVnRWgS5Ve5fKuyOwn69LFWKk4w7p1piPi9bIeg%3D&spr=https&se=2019-10-16T17%3A22%3A25Z&sp=r&rscl=x-e2eid-bbac4036-d1f94a89-9d0a8f52-1e858346-session-775be811-ed3241dc-914569d5-76cba85d] Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: SSL connect error
   at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
   at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
Information, ApplicationInsightsTelemetrySender correlated 3 events with X-TFS-Session 775be811-ed32-41dc-9145-69d576cba85d
##[error]A task was canceled.
##[section]Finishing: Cache Yarn packages


Potential issue with multiple versions of @aws-cdk/cdk

It seems like we might have an issue when multiple versions of the @aws-cdk/cdk are by different constructs in the closure. Even if the two versions are API-compatible, it appears TypeScript will still not allow them to use interchangbly:
Here's the error:
Types of property 'parent' are incompatible.
  Type 'import("node_modules/@aws-cdk/cdk/lib/core/construct").Construct' is not assignable to type 'import("node_modules/some-module/node_modules/@aws-cdk/cdk/lib/core/construct").Construct'.
    Types have separate declarations of a private property '_children'.

We should also examine how this behavior impacts jsii languages (related to awslabs/jsii#61)

I imagine it's just the typechecker that's affected, so shouldn't have any impact on jsii.
But this is a supposition :).

@rix0rrr you are right, but the current implementation of the jsii-runtime has a limitation (https://github.com/awslabs/jsii/issues/61) that won't allow multiple versions of the same module to be loaded into the runtime, so in a sense those are related.

This is the issue: https://github.com/Microsoft/TypeScript/issues/8346

Here's a repro:
repro.zip

Another discussion on the topic: https://github.com/Microsoft/TypeScript/issues/7755
Basically, private members do not participate in structural typing intentionally in order to protect from overriding private implementation.
As much as I can see the purist reasoning behind it (and it seems like the TypeScript team doesn't intend to change things), we need a solution that will allow multiple versions of @aws-cdk/cdk to work together. This is especially tricky around base classes like Construct and Resource.
To be pragmatic about this, I'd like to propose that for these base classes, we will not use private members, but rather just use _ prefix to indicate those members are private. This is not an uncommon convention in JavaScript and jsii will hide these member from native languages.
@rix0rrr @RomainMuller I wonder what you guys think about this.

But seriously.
I'd like to see automation around this, and preferably I wouldn't have to defile my code to work around these kinds of issues. I would like to see automatic translation of private xxx() to public _xxx() at the compilation phase. Either by postprocessing on the generated .js and .d.ts files, or by writing a compiler plugin.

I agree. It’s a huge limitation of private members in typescript, which I am not 100% sure they realize. I’ll chime in on one of these conversations to present our use case and ask for guidance.
In the meantime, we could either block the usage of private members at the Jsii level (tell you to use underscore and explain why) or just solve this locally for cdk.Construct and other core base classes, so that we could start versioning modules separately.

I share @rix0rrr's philosophical issue with this. It is uncanny to me that I cannot use privates as pure implementation details. However, Javascript is what it is, and from that perspective I guess we have to preserve usability of the CDK.

This is actually a Typescript thing. Whether you use the CDK or not, if you wanted to use TypeScript to vend a class library, you wouldn't want to use "private" properties. I know, it's weird.

I don't like it, but a limitation of the language is a limitation. Based on the issue MS/Typescript rep adding interfaces for all types seems like a major headache. I'm not sure how my OCD will handle the _ everywhere, but improving functionality and practicality right now to get real use cases to me is more important than a few semantics.
I do agree with @rix0rrr that maybe in the future some logic and automation around this would be useful, though for now, your proposal seems fair.

The implication is that we will have to define an interface for every exported class in every module and somehow enforce that only those interfaces are used to reference these classes... doesn’t seem realistic

@huijbers proposed that the only reason typescript actually exposes private properties in type declarations is to allow objects to reference private properties of other objects of the same type. This was also mentioned in one of the typescript discussions (https://github.com/Microsoft/TypeScript/issues/7755#issuecomment-204167721), another reason was to ensure that derived classes do not override the private property by accident.
@huijbers proposed to modify the typescrip  compiler and add a mode which won’t private propert declarations, and perhaps can even enforce accessing private properties that are not via “this”.
What do you guys think? Maybe we can propose this as a contribution to typescript?

Can you provide a link to that proposal? I can see it being valuable for some use cases.
A related question would be, what are private members being used for, and is there a way to eliminate them altogether, perhaps by making more types immutable?

In a conversation with @rix0rrr, he suggested that perhaps the correct modeling for dependencies that are exposed outside of a module's API is to use peerDependencies instead of normal dependencies, and I tend to agree.
For jsii code (CDK, AWS Construct Library, etc), we could enforce this at the jsii level: if a module's public API includes types from external modules, jsii can require that those are defined as peerDependencies instead of normal dependencies.
The problem is that we cannot enforce this for modules written by 3rd parties. We can recommend, guide and also make sure our templates are using peerDependencies for things like @aws-cdk/cdk (which by definition is exposed in the API), but we can't really enforce it as far as I know.


Features to support "singleton" entity component

Sometimes there is a component which, at any given moment, can be applied to one, and only one, entity.  I called this a "singleton" although that's more of an object-oriented term and I'm not sure is right here, but let's go with it for a minute.
An example might be which entity has focus (is being controlled by the player).  Only one entity at any given time can have focus.  If one entity is given focus the existing entity which had focus needs to lose it.
I can attach an "empty" component to an entity which functions as a boolean flag, and this makes it easy to query which entities have focus, or do not have focus.  But when I have a component which should function as a singleton, I'd additionally like:

A simple way to get which entity has focus
A simple way to switch which entity has focus, that will add the component to one entity and take it from another

If there's an attribute to say a component should behave in this way, then I think setting one entity to have "isFocused" to true should clear any others.  Instead of having it be a boolean member of every entity, the generated code should be a property accessor and setting it would change which entity is focused.
Of course one can already do stuff like this outside of Entitas but then you can't use the component as a filter in a GroupMatch.  So I think it would be great if Entitas could help here.

is that what Unique attribute for?
https://github.com/sschmid/Entitas-CSharp/wiki/Attributes#unique-entities

Yes, perfect.  Thanks.  Sorry I missed that.
I think I misinterpreted what was being marked as "Unique" because the title "Unique Entities" was misleading for me.  It is actually the component which is unique, not the entity.  Had I read it thoroughly I would have picked up on this but maybe the title of the section would be better as "Unique Components"

Actually, I may have spoken too soon.  I think your title of "Unique Entities" may have actually been the correct title and what I'm looking for is slightly different.
There is a difference between what Unique currently does, and what I'd prefer the attribute to do.  After applying [Unique], here's the generated code:
public partial class GameContext {

    public GameEntity playerControlledEntity { get { return GetGroup(GameMatcher.PlayerControlled).GetSingleEntity(); } }

    public bool isPlayerControlled {
        get { return playerControlledEntity != null; }
        set {
            var entity = playerControlledEntity;
            if (value != (entity != null)) {
                if (value) {
                    CreateEntity().isPlayerControlled = true;
                } else {
                    entity.Destroy();
                }
            }
        }
    }
}

It appears that if I set a second entity to isPlayerControlled = true, that it destroys the first entity.  But that is not what I want.  I just want it to set isPlayerControled = false on that entity.
I want isPlayerControlled to act such that only one can be true, and setting a second one to true is not a "programmer mistake" but is a valid way of just setting it for the one and clearing the others, much like you'd set the current selection of a radio button.
Of course I know of other ways to do this, but it would be easier to be in the framework.

Of course I know of other ways to do this

Yeah, this could be a good feature.
Share your way to do this, and maybe it gets into the next version.

It appears that if I set a second entity to isPlayerControlled = true, that it destroys the first entity. But that is not what I want. I just want it to set isPlayerControled = false on that entity.

It's not how it works actually. Just for future readers - there are two interfaces generated for Unique components, one for context and another for entity; Flag component is a special case for generated code.

For flag component context.playerControlledEntity returns GetGroup(GameMatcher.PlayerControlled).GetSingleEntity();, if there is more than 1 singleEntity throws exception.
Calling context.isPlayerControlled = true; if singleEntity is not available, creates entity and sets component to true.
Calling context.isPlayerControlled = false; if singleEntity is available it destroys it

Just use a PrimaryEntityIndex and set a unique component on the context to a given index. For example, EntityA has a primary index of "A", EntityB has a primary index of "B", and the context has a Unique string component containing either "A" or "B". Then your systems look up the context's index and use context.GetEntityWith[YourIndexName](context.[yourUniqueComponent]) to grab the entity.

Thanks for the suggestion, but this is not exactly solving the problem.  I may have explained things poorly.
I have a component, PlayerControlled, which acts as a flag on several different types of entities (vehicles, characters, etc.)  At any given time, the player can only control one thing, so if this component is added to any entity, it should be removed from any other.
I can, of course, enforce this myself by, whenever I want to set a new entity as player controlled, getting the old player controlled entity, removing the PlayerControlled component, and then adding the PlayerControlled component to the new entity.
public partial class GameEntity {
   public bool PlayerControlled {
      get {
         return isPlayerControlled;
      }
      set {
         if (Contexts.sharedInstance.game.playerControlledEntity != null)
            Contexts.sharedInstance.game.playerControlledEntity.isPlayerControlled = false;

         isPlayerControlled = true;
      }
   }
}

That's what I'm trying to do.  This code here is used by calling "entity.PlayerControlled = true".  But I wish there was a way that I didn't need to override it, I could just get this behavior when calling "entity.isPlayerControlled = true".  It seems like I wouldn't be the only one who wants to do something like this.

Entitas itself has no concept of single entities or components. However, as we know the code generator introduces single entities by extending the partial context with helper methods. That does not help you in the example above, because you actually want to flag specific existing entities with the PlayerControllerComponent. So you're already doing it right, by manually deselecting entities before assigning to a new one
public partial class GameEntity
{
   public bool PlayerControlled
   {
      get {
         return isPlayerControlled;
      }
      set {
         var entity = Contexts.sharedInstance.game.playerControlledEntity;
         if (entity != null)
            entity.isPlayerControlled = false;

         isPlayerControlled = true;
      }
   }
}


net: bug fix: don't use hard coded path to container dir

Containers now migrate thru state directories so hard
coded path was broken

LGTM

lgtm


Fix missing es-abstract 1.14.0

Solve error:
https://registry.yarnpkg.com/es-abstract/-/es-abstract-1.14.0.tgz: Request failed "404 Not Found"

I think this repo might need a full overhaul 🤔  — Apollo Client 3 is out and it simplified a lot of this stuff


Can't open user menu on iPad

Tapping the avatar on iPad takes you to a user's profile, and does not open the user nav items.
https://d.pr/v/dOJTal

@bpotterbaum confirm that we are holding off on this until global nav update?
the other option was to make the avatar always open up the menu and have a "Profile" link in the menu.

I would still like to get this update in before the global nav project.

lets go with the option to  make the avatar always open up the menu and have a "Profile" link in the menu.

I changed the avatar from a link to a span, but... If the user is logged in there is already a link to "Lists" (which is just /profile/:username), should we rename it, duplicate it with another link saying profile, or leave it as is?
    {{#if user.connect}}

       Lists

    {{/if}}

@bpotterbaum ^^

Uh, I don't understand the question?