Feat/dynamic-slice-and-mask

Overview

added Khovratovich & Peikert BIP32DerivationType enum
added deriveChildNodePublic method
added trunc256MinusGBits method
implemented trunc256MinusGBits in deriveChildNodePublic & deriveChildNodePrivate
updated method signatures for passing BIP32DerivationType to deriveKey, keyGen, signAlgoTransaction, signData, & ECDH
deriveKey return now includes the childChainCode
added cryptoCoreEd25519Add to SodiumHelper
added tests for trunc256MinusGBits & deriveChildNodePublic with Peikert derivation type

Related Issues:

https://algorandfoundation.atlassian.net/browse/ENG-275
https://algorandfoundation.atlassian.net/browse/ENG-301

All Tests are currently passing
Building for debugging...
[10/10] Linking bip32-ed25519-swiftPackageTests
Build complete! (1.74s)
Test Suite 'All tests' started at 2024-05-22 12:14:53.245.
Test Suite 'bip32-ed25519-swiftPackageTests.xctest' started at 2024-05-22 12:14:53.247.
Test Suite 'Bip32Ed25519Tests' started at 2024-05-22 12:14:53.247.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testAuthReqSigning]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testAuthReqSigning]' passed (0.008 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveChildNodePrivate]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveChildNodePrivate]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveChildNodePublic]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveChildNodePublic]' passed (0.008 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveChildNodePublicHardenedChange]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveChildNodePublicHardenedChange]' passed (0.007 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveHardened]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveHardened]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveKey]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveKey]' passed (0.003 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveNonHardened]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testDeriveNonHardened]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testECDH]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testECDH]' passed (0.009 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testFromSeed]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testFromSeed]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testGetBIP44PathFromContext]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testGetBIP44PathFromContext]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testHarden]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testHarden]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testInitializationWithSeeds]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testInitializationWithSeeds]' passed (0.042 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testKeyGeneration]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testKeyGeneration]' passed (0.005 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testMsgPackToSwift]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testMsgPackToSwift]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testPrefixError]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testPrefixError]' passed (0.003 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testSchema]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testSchema]' passed (0.003 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testTrunc256MinusGBits]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testTrunc256MinusGBits]' passed (0.002 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testValidateDataAuthRequest]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testValidateDataAuthRequest]' passed (0.012 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testValidateDataMsgSchema]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testValidateDataMsgSchema]' passed (0.003 seconds).
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testVerifyAlgoTx]' started.
Test Case '-[bip32_ed25519_swiftTests.Bip32Ed25519Tests testVerifyAlgoTx]' passed (0.003 seconds).
Test Suite 'Bip32Ed25519Tests' passed at 2024-05-22 12:14:53.371.
	Executed 20 tests, with 0 failures (0 unexpected) in 0.124 (0.124) seconds
Test Suite 'DataExtensionTests' started at 2024-05-22 12:14:53.371.
Test Case '-[bip32_ed25519_swiftTests.DataExtensionTests testHexStringInit]' started.
Test Case '-[bip32_ed25519_swiftTests.DataExtensionTests testHexStringInit]' passed (0.000 seconds).
Test Suite 'DataExtensionTests' passed at 2024-05-22 12:14:53.371.
	Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'bip32-ed25519-swiftPackageTests.xctest' passed at 2024-05-22 12:14:53.371.
	Executed 21 tests, with 0 failures (0 unexpected) in 0.124 (0.124) seconds
Test Suite 'All tests' passed at 2024-05-22 12:14:53.371.
	Executed 21 tests, with 0 failures (0 unexpected) in 0.124 (0.126) seconds

I realize that the ticket for ENG-301 did not include this information =) but to complete that ticket you should implement a function called something like "derivePublicly" that accepts 1) the extended public key (64 bytes, 32 bytes PK/address, 32 bytes chain code), 2) a change value, 3) a key index value.
Essentially it is what you've already implemented in your test, it's just something we'd package up and offer as an API to developers pulling in this package. So an outside counterparty (e.g. a customer of our user) gives our user a 64 byte xPK and with our lib the user can derive all the addresses of their counterparty.
To facilitate this, we should also consider adding a function that returns the extended public key at the account-level. So one way to call keyGen is to return the 32 byte address only at the key-index level. Another way would be to call keyGen with a shorter BIP44 path vector that ends at account-level and also returns the 64 byte xPK.
Perhaps we could replace the "isPrivate" flag with an enum that is 0: private, 1: PK, 2: xPK. And then depending on this enum we return derived, derived scalar multed, and derived scalar mulled + extended (like you wrote it).
Any ideas on what is the cleanest way? We'll need to coordinate with Bruno as well so the APIs are the same.
So essentially something like this:

deriveKey (takes in BIP44 path and does for loop, also has the 3 setting enum)
deriveDescendentAddress (takes in full BIp44 path, calls deriveKey, passes in enum PK - returns 32 byte PK address at the key index level)
getAccountPK - (takes in BIP44 path until account level, calls deriveKey, passes in enum xPK - returns 64 byte xPK at account level
rawSign - (same as current, calls deriveKey with full BIP44 path and enum private, gets 64 byte raw (32 byte scalar, 32 byte kR)
getOtherDescendantAddress - (takes in an xPK + change & key-index with the expectation that it is an account-level, calls deriveChildNodePublic twice with the given change & key-index)

Ideally with better names.
deriveKey is a for loop

@ehanoc

I realize that the ticket for ENG-301 did not include this information =) but to complete that ticket you should implement a function called something like "derivePublicly" that accepts 1) the extended public key (64 bytes, 32 bytes PK/address, 32 bytes chain code), 2) a change value, 3) a key index value.
Essentially it is what you've already implemented in your test, it's just something we'd package up and offer as an API to developers pulling in this package. So an outside counterparty (e.g. a customer of our user) gives our user a 64 byte xPK and with our lib the user can derive all the addresses of their counterparty.
To facilitate this, we should also consider adding a function that returns the extended public key at the account-level. So one way to call keyGen is to return the 32 byte address only at the key-index level. Another way would be to call keyGen with a shorter BIP44 path vector that ends at account-level and also returns the 64 byte xPK.
Perhaps we could replace the "isPrivate" flag with an enum that is 0: private, 1: PK, 2: xPK. And then depending on this enum we return derived, derived scalar multed, and derived scalar mulled + extended (like you wrote it).
Any ideas on what is the cleanest way? We'll need to coordinate with Bruno as well so the APIs are the same and Ill implement the Kotlin stuff accordingly.
So essentially something like this:

deriveKey (takes in BIP44 path and does for loop, also has the 3 setting enum)
deriveDescendentAddress (takes in full BIp44 path, calls deriveKey, passes in enum PK - returns 32 byte PK address at the key index level)
getAccountPK - (takes in BIP44 path until account level, calls deriveKey, passes in enum xPK - returns 64 byte xPK at account level
rawSign - (same as current, calls deriveKey with full BIP44 path and enum private, gets 64 byte raw (32 byte scalar, 32 byte kR)
"getOtherDescendantAddress"/"derivePublicly" - (takes in an xPK + change & key-index with the expectation that it is an account-level, calls deriveChildNodePublic twice with the given change & key-index)

Ideally with better names.

Okay so we are not doing this. Instead:

Make the deriveKey, deriveChildNodePrivate and deriveChildNodePublic functions public.
Add a README section explaining the difference between a normal 32 byte public key and a 64 byte extended public key, and how to do what you did in the test to 1) use derivedKey in isPrivate=false to extract the xPK at the account-level (or change-level), 2) given someone else's xPK, derive child key addresses using deriveChildNodePublic.


Schedules Direct: Enable Runtime/Duration Data

Schedules Direct has duration time for movies in its metadata (in SageTV lingo, this is referred to as runtime). This was deliberately disabled in the SageTV Schedules Direct implementation due to a future API change that would change the location of this meta data.
EnterNoEscape asked me to create an issue for this, and he will add it when he gets some time.
See this thread for more details: https://forums.sagetv.com/forums/showthread.php?p=604338&postcount=6

Do you have a screenshot of what this looks like when it's working? I made what I believe are the correct changes, but I can't seem to find an example of a movie that shows anything other than the airing duration.

Nevermind. I believe I triggered it. I guess I just wasn't patient enough.

This has been added in pull #288


Fix the code example in the document of fs-api

What changes are proposed in this pull request?
Fix the code example in the document of fs-api
Why are the changes needed?
The code example is incorrect
Does this PR introduce any user facing changes?
n/a

alluxio-bot, merge this please


.pkl model cannot resolve relative imports

Pickled model imports ML.src.features module after deserialization. This module cannot be found from WebBackend. Deserialized model also imports 'sklearn'.
Possible solution:

serialize the model entirely so it does not require additional imports
create a proper module structure so all the references can be found.

closed by #6


💄 Hide navigation during printing

Printing an article will now result in a clean, text only document.

Thanks! This is a good improvement. I'll merge it in now and it will be part of the next release.


Manage the issues and PRs

@BetaCat0 @GopherJ can you help manage the issues and review the PRs in this repo?

Sure. I'd like to :)

@hsluoyz Could you give me rights on casbin org? I only have rights on casbin-rs currently


Dev server crashes on thrown exception in async function

Describe the bug
When an error is thrown in an async function the dev server crashes with the static adapter enabled.
With no adapter, the dev server works just fine.
The static build seems to work just fine as well.
Reproduction

Create new sveltekit skeleton project
Add code similar to the solution from https://svelte.dev/tutorial/await-blocks to index.svelte:

<button on:click={handleClick}> generate random number </button>

{#await promise}

...waiting

{:then number}

The number is {number}

{:catch error}

{error.message}

{/await}

Execute npm run dev and open browser: everything works fine
Install static adapter and add it in svelte.config.js
Execute npm run dev and open browser: dev server crashes with error ⚡
Execute npm run build and open build on webserver: everthing seems to work fine
Execute npm run preview after build: preview server crashes with error as well ⚡

Logs
PS C:\Temp\sveltekit> npm run dev

><EMAIL_ADDRESS>dev
> svelte-kit dev

Pre-bundling dependencies:
  svelte/store
  svelte
  svelte/animate
  svelte/easing
  svelte/internal
  (...and 2 more)
(this will be run only when your dependencies or config have changed)
  vite v2.5.10 dev server running at:

  > Local: http://localhost:3000/
  > Network: use `--host` to expose

  SvelteKit v1.0.0-next.176

  network: not exposed
  network: not exposed
  network: not exposed
  network: not exposed
  local:   http://localhost:3000
  network: not exposed
  network: not exposed
  network: not exposed

  Use --host to expose server to other devices on this network

/src/routes/index.svelte:17
                throw new Error(no.toString());
                      ^

Error: 0.3690497328470541
    at getRandomNumber (/src/routes/index.svelte:17:9)

System Info
System:
    OS: Windows 10 10.0.19042
    CPU: (16) x64 AMD Ryzen 7 3700X 8-Core Processor
    Memory: 23.05 GB / 31.95 GB
  Binaries:
    Node: 16.4.1 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.4 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 7.19.1 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: 94.0.4606.61
    Edge: Spartan (44.19041.1023.0), Chromium (94.0.992.31)
  npmPackages:
    @sveltejs/adapter-static: ^1.0.0-next.20 => 1.0.0-next.20
    @sveltejs/kit: next => 1.0.0-next.176
    svelte: ^3.42.6 => 3.43.0

Severity
annoyance
Additional Information
No response

Execute npm run dev and open browser: everything works fine
Install static adapter and add it in svelte.config.js
Execute npm run dev and open browser: dev server crashes with error ⚡

This series of steps seems implausible to me. It's extremely unlikely that installing any adapter would affect dev mode. Rather, I expect what's happening is that because your example depends on Math.random it's triggered only sometimes and whether it was triggered or not was completely independent of whether the static adapter was installed.

I'm closing this since the issue reporting guidelines, which are very explicit about the requirement for a reproduction repo, were not followed

@benmccann You were right, Math.random made the error occour only sometimes. The error does not seem to be with the static adapter.
I made a repository and narrowed the bug down: https://github.com/toolongformore/sveltekit-adapter-static-bug (sorry for forgetting to include one in the original report)
The problem arises, when a variable is initialized with an error-throwing promise; an error thrown during runtime poses no problem:

And here's the error log:
(node:729478) UnhandledPromiseRejectionWarning: Error: error
    at eval (/src/routes/index.svelte:47:11)
    at Generator.next (<anonymous>)
    at eval (/src/routes/index.svelte:40:67)
    at new Promise (<anonymous>)
    at __awaiter (/src/routes/index.svelte:17:10)
    at getRandomNumber (/src/routes/index.svelte:45:10)
    at eval (/src/routes/index.svelte:57:16)
    at Object.$$render (/node_modules/svelte/internal/index.js:1684:22)
    at Object.default (/.svelte-kit/dev/generated/root.svelte:57:127)
    at eval (/.svelte-kit/dev/components/layout.svelte:8:41)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:729478) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:729478) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Like it says, you should wrap the code in the promise with a try-catch or add a {:catch error} to your {#await} block
Perhaps SvelteKit could use this: https://nodejs.org/api/process.html#process_event_unhandledrejection

Like it says, you should wrap the code in the promise with a try-catch or add a {:catch error} to your {#await} block

The given example in the repository has a {:catch error} block, this should not be the problem:
{#await promise}

...waiting

{:then number}

The number is {number}

{:catch error}

{error.message}

{/await}

Interestingly if I assign the promise in onMount, everything is fine as well. There only seems to be a problem, when the rejecting promise is assigned at declaration:
// crashes:
let promise = getRandomNumber(true);
// works:
let promise;
onMount(()=> { promise = getRandomNumber(true); });

For a component like
{#await Promise.reject('foo')}
	a
{:then}
	b
{:catch}
	c
{/await}

the compiler in SSR mode currently produces
/* App.svelte generated by Svelte v3.43.0 */
import { create_ssr_component, is_promise } from "svelte/internal";

const App = create_ssr_component(($$result, $$props, $$bindings, slots) => {
	return `${(function (__value) {
		if (is_promise(__value)) return `
	a
`;

		return (function () {
			return `
	b
`;
		})(__value);
	})(Promise.reject('foo'))}`;
});

export default App;

This crashes Node 16 because of the unhandled rejected promise.
If we wanted the compiled SSR code to always suppress this (by appeasing Node by making it see the promise as handled) we could make this change in the compiler:
diff --git a/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts b/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts
index 6a62f872b..c8b0eeb38 100644
--- a/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts
+++ b/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts
@@ -13,7 +13,10 @@ export default function(node: AwaitBlock, renderer: Renderer, options: RenderOpt

 	renderer.add_expression(x`
 		function(__value) {
-			if (@is_promise(__value)) return ${pending};
+			if (@is_promise(__value)) {
+				__value.catch(() => {});
+				return ${pending};
+			}
 			return (function(${node.then_node ? node.then_node : ''}) { return ${then}; }(__value));
 		}(${node.expression.node})
 	`);

Or, it might be a good idea to only add this __value.catch(() => {}); when there's a catch on the {#await}. I'm not sure.

If we wanted to do that,
diff --git a/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts b/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts
index 6a62f872b..ce51cb429 100644
--- a/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts
+++ b/src/compiler/compile/render_ssr/handlers/AwaitBlock.ts
@@ -1,6 +1,6 @@
 import Renderer, { RenderOptions } from '../Renderer';
 import AwaitBlock from '../../nodes/AwaitBlock';
-import { x } from 'code-red';
+import { b, x } from 'code-red';

 export default function(node: AwaitBlock, renderer: Renderer, options: RenderOptions) {
 	renderer.push();
@@ -13,7 +13,10 @@ export default function(node: AwaitBlock, renderer: Renderer, options: RenderOpt

 	renderer.add_expression(x`
 		function(__value) {
-			if (@is_promise(__value)) return ${pending};
+			if (@is_promise(__value)) {
+				${node.catch.start !== null && b`__value.catch(() => {});`}
+				return ${pending};
+			}
 			return (function(${node.then_node ? node.then_node : ''}) { return ${then}; }(__value));
 		}(${node.expression.node})
 	`);

would only suppress the rejected promise if there was a catch block, and would otherwise let it crash the process if it weren't handled some other way like with process.on.

I've created https://github.com/sveltejs/svelte/issues/6789 for the Svelte side of this. We can continue the discussion here about how to deal with other types of unhandled rejections, and whether that ought to be the framework's job or the user's job.

If the framework were to try and handle it, it would have to be in coordination with adapters, since process.on only makes sense in Node. In order for handleError to capture these errors we'd need to expose it as server.handleError or something (need to think about how this would work with scoped hooks/middleware — perhaps handleError can only be implemented in the root hook/middleware), and we wouldn't be able to associate it with an event

Do we have any sort of an idea of what other serverless hosting environments do when there's an unhandled rejection, and whether they let user code hook into that?

I have no idea, honestly

I'm facing this now, and it's supremely annoying - any update, @Rich-Harris, @Conduitry?


Implement multithread preprocess

Add flag: workers. Use multiprocess to parallelize Tokenizer.binarize.

Can you provide some numerical measurements of timing for binarization of wmt en-de on 1 worker, etc?

On my macbook with 1 core, 4 processors.

worker count
run time(s)
run time with out build dictionary
speed up
speed up (without build dictionary)

1
1734
1522
1
1

2
1235
1023
1.40405
1.48778

4
909
697
1.90759
2.18364

8
815
603
2.12761
2.52405

I am running experiments on dev server with more workers, and try to figure out if it is necessary to parallelize building dictionary.

As discussed offline, let's test performance on more cores say 20-40. And let's check if we need to parallelize dictionary construction.

I have parallelized dictionary build, and here is the new speed up on fb dev server, who have 24 cores.

worker_cnt
time
speedup

1
2263
1

12
591
3.8291

16
550
4.11455

24
517
4.37718

32
527
4.29412

48
532
4.25376

Thanks for posting this PR; it'd be a great addition! It looks like in the proposed approach all the workers  simultaneously attempt to scan the entire input file. You might also consider benchmarking an approach where the parent process (and only the parent) reads the file and deals groups of lines to the workers through shared memory.

@bouvrie Python's shared memory can only share fix length array with basic types. To share strings is not supported by default. Implementing producer-consumer model will also have overhead. Also I have recored the time for each part of the program, on WMT dataset, reading the whole file will only take very few seconds while binarizing and merging worker result will take hundreds of seconds. So I think it won't be worthy to optimize the file reading part.

@paramecinm Thanks, fair point on the file loading. I took a closer look at Tokenizer.tokenize and with the default tokenize_line it looks like most of the time is possibly spent in everything except the tokenizing (dict lookups, publishing to the replaced consumer). Perhaps this is the source of the contention/bottleneck in the many-thread setting?

@bouvrie Because the python shared memory in multiprocess doesn't support share complex structures, and using Python's data sharing api like Pipe to pass data is even slower than write data to file and read them in. So I choose to write down the partial result of each process and read them to merge in main process. The write and merge part takes a large amount of time. To further optimize the performance, I can only think of use the much powerful multithread in C. It will be great if you can find some better approach!

I wonder if we will have a better performance if master instead of waiting for workers to finish binarization will consume their output as they produce it. It can be done through piping. Right now, pickle.load() seems to kill a lot of performance. Also, it should save memory, as you won't need to keep everything in ram.

This is implemented via the --workers option to preprocess.py


Focus editor when using Find Widget

After typing in the search field, pressing enter, cmd + g, or cmd + shift +g should focus the current editor that is being searched.
Currently, the Find Widget keeps the focus, so the user is required to put the focus back in the editor before they can update any text.
With this change, users can always put the focus back in the editor with cmd + f.

I think the expected functionality of the Find Panel is not described in the User's Guide. So you can't tell if a particular Find Panel behavior is wrong or not.
The requirement to write basic rules on how the Find Panel working I entered here: Find Panel in the Guide. So far, I have no response whether this is a good idea or not. Also, some behaviors seem like a bug to me, but without Find panel definition I can't say for sure. I think they (vscode programmers) will recommend you using ESC :smile: which is not what I except like best practice of using Find panel.
In your case (and in ideal world) I would expect:

ENTER => Finding next occurrence, Find panel has still focus. (working)
SHIFT + ENTER => Finding previous occurrence, Find panel has still focus. (working)
CTRL + ENTER => Silngle occurrence confirmation, document get focusing, Find panel is closed. (not working, it's more intuitive to me than using ESC)
ALT + ENTER => All confirmation. All occurences are selected, cursor is multiplied, document has focus, Find panel is closed. (partially works, Find panel is not closed)

Anyway, I think it is first necessary to agree on the Find panel behavior and write these requirements into the user's guide.

Clear, you need option to close the Find panel after searching (replace actual ENTER and ESC sequence only with ENTER). And I would like to describe the behavior of the Find panel, because now I do not know if the correct behavior of the Find Panel is to find and close it (like you would), and the current behavior is wrong or it is as designed.

Coming from a Textmate 2 and Sublime background, I definitely find it counter-intuitive that the focus is not on the editor after doing a search. When I search for something, then I generally want to interact with the found string, either copying it, deleting it, changing it, or whatever, but the place where I most definitely don't need to do any more work is in the search field. So having to hit ESC after searching, is adding a couple of hundreds of extra key hits per day to my workflow.

I created a request to discuss the functionality of the Find Panel Find panel documentation is missing. @berliner please, contribute also to that request, if you have a better view of how to design the entire Find Panel functionality...
For me, it is intuitive (Notepad ++ in Windows, Geany in Linux) that pressing Enter will find the next occurrence of the search term, but the panel will not close because I do not expect to find the one occurrence that I search for immediately. Therefore, it is more natural for me to go through the individual occurrences with Enter. The use of ESC to confirm selection is significantly non-intuitive. Since using ESC, I would expect to close the panel and return the original selection (not the last item found). I would expect something like CTRL + ENTER to confirm selected occurrence (like in ALT + ENTER for all occurrences is being used).

@atiris Not sure that I will find the time to work on the proposal you linked too.
But some quick remarks anyway. In most editors you can use a different shortcut to loop through all found occurrences of a string, on a Mac it's usually CMD-g (and SHIFT-CMD-g for reverse loops). I have set this up in my vscode too for the shortcut editor.action.nextMatchFindAction, so this gives me your intended behaviour (looking for further occurrences because the one you're looking for is probably not the first one found). The main difference is that you can use that shortcut in the editor and the cursor automatically jumps to the next found match, enabling you to directly interact with the match.
If you look at bit at behaviours of search fields all over the web, you'll probably find that keeping the focus in the search bar is rather the exception than the rule. I don't mind this behaviour in word processors or excel-type programs, but code editors should focus on minimising key strokes.
The current behaviour feels in sync with how this is often implemented on Linux (no idea about Windows), but coming from an environment with a stronger emphasis on UX and workflows (as the Mac has been, though arguably in decline, bot of those points my personal view, let's not start a discussion about this ;)) it just doesn't seem like the best solution. TextMate really does an amazing job at this, if you get a chance, try it out for a little while.

Thanks all for your contribution. I understand the ask for focusing the editor after searching but I don't think we are going to make the change considering how many users will be affected by that. With https://github.com/microsoft/vscode/pull/63176 merged and another shortcut CMD+G, you can search selections without losing focus and if you use the Find Widget to type in new queries, once the focus goes into the editor, you can use CMD+G to navigate in the editor.

What's the more common thing that users do next after searching using the find widget? That's what should drive this decision.
What does keeping the user focused on the find widget really offer? Let's say a user doesn't find what they're searching for and wants to search for something else. Does leaving them focused on the find widget save them any time? They need to clear out their previous search by either deleting their previous search with backspace, selecting all text then typing, or entering the find widget hotkey, which selects all the search text anyway.
It just feels clunky right now. If this isn't going to be addressed simply because people are used to its clunky behavior, an option for those who prefer that the editor is focused upon searching would be greatly appreciated.

@rebornix Seeing some of the features that were added in 1.40.0, I'm having a hard time understanding why improvements to the find widget are considered out-of-scope.
You introduced an option to set the background color of the active activity bar indicator. I use the find widget constantly.
Please consider re-opening this issue, as I don't believe that it should be marked as out-of-scope.

I finally found a solution for giving the focus back to the editor after search (Ctrl+F): just map the Enter/F3 keys to the "workbench.action.focusLastEditorGroup" command, which will occur after the default mappings for "editor.action.nextMatchFindAction".
Add these mappings to your keybindings.json (edit it with command "Open Keyboards Shortcuts (JSON)":
{
    "key": "enter",
    "command": "workbench.action.focusLastEditorGroup",
    "when": "editorFocus && findInputFocussed"
},
{
    "key": "f3",
    "command": "workbench.action.focusLastEditorGroup",
    "when": "editorFocus && findInputFocussed"
}

now we can use workbench.action.focusActiveEditorGroup


pre-trained model

Hello, author. I have submitted a request to obtain permission to download the pre-trained model for my research purposes. I kindly request your approval for this permission. Thank you.

Hello!
Thank you for your interest in our work. You can download the pre-trained model.

老师不好意思，我仍然没有下载链接中的预训练模型的谷歌云盘的权限

不好意思，之前的链接分享有点问题。您现在再尝试一下，应该可以下载了。


cleanup: break cyclic dependencies

Break the gcs/project.py file in two, and move gcs/iam.py to
testbench/servers/.  This breaks the cycle between the testbench/
and gcs/ modules. I am still planning to reorganize the code to make
it harder to create such cycles, and to make the structure easier to
grok (or at least to explain without blushing).
Part of the work for #95

Ah this is starting to feel a bit more elegant, I like it.


Pass objects instead of dicts to service methods

Ticket
Issue #52
Changes

Add UserPatchParamsSchema and UserPostParamsSchema, which both apply validations from UserSchema
Pass UserPatchParams to user_patch
Pass User to user_create
Add PatchParamsBase to handle persisting original JSON input in a PATCH request
Added tests for various user_patch scenarios

Context for reviewers
Based on discussion here and here.
The meat of the change comes from UserPatchParamsSchema and UserPostParamsSchema. Each enforces validations from the inherited UserSchema, but produces different objects via the post_load function. This does a couple things:

Input is validated against the same UserSchema regardless of if the POST or PATCH endpoints are being hit
validation rules do not need to be redefined, since the schemas inherit from UserSchema
The UserPatchParamsSchema's post_load function creates a UserPatchParams object, that persists the original JSON payload, for use in user_patch

Testing
N/A

Closing this, since this work was addressed in @lorenyu 's https://github.com/navapbc/template-application-flask/pull/57!


pillsegmentedcontrol mask label shouldn't be accessible

Platforms Impacted

[x] iOS
[ ] macOS

Description of changes
Mask labels are rendered on top of the pill buttons to show smooth animation transition. However, it shouldn't VO accessible because user should directly interact with the pill button.
Verification

Before
After

Pull request checklist
This PR has considered:

[ ] Light and Dark appearances
[x] VoiceOver and Keyboard Accessibility
[ ] Internationalization and Right to Left layouts
[ ] Different resolutions (1x, 2x, 3x)
[ ] Size classes and window sizes (iPhone vs iPad, notched devices, multitasking, different window sizes, etc)

Microsoft Reviewers: Open in CodeFlow

Does pillMaskedLabelsContainerView.isUserInteractionEnabled = false either not apply to children, or not disable something as an accessible element? I would guess not if we have to make this change.

Does pillMaskedLabelsContainerView.isUserInteractionEnabled = false either not apply to children, or not disable something as an accessible element? I would guess not if we have to make this change.
@huwilkes great equation but no. think of situation like static label or image drawn in UI. Users can't interact with it because it is just a label but VO can still readout what it is to convey the information.


ActionSheetIOS does not work on iPad

ActionSheetIOS doesn't work in any of the iPad simulators or actual devices. No error, no console messages, just nothing...
Am I missing something here? Are you suppose to use something else for iPads? I don't see anything in the docs indicating otherwise.

you need to provide more info/some code for us to help you.
Make sure you pass anchor in options for it to work on ipad.
Refer: https://github.com/facebook/react-native/blob/62e8ddc20561a39c3c839ab9f83c95493df117c0/Libraries/ActionSheetIOS/RCTActionSheetManager.m#L130-L139
I think anchor is not documented in the docs. cc @paramaggarwal

Nothing out of the ordinary:
  showActionSheet() {
    var BUTTONS = [
      'Option 1',
      'Option 2',
      'Cancel',
    ];
    ActionSheetIOS.showActionSheetWithOptions({
      options: BUTTONS,
      cancelButtonIndex: 2,
    },
    (buttonIndex) => {

      switch (buttonIndex) {
        case 0:
          console.log("Option 1");
          break
        case 1:
          console.log("Option 2");
          break
      }

    });
  },

  render() {
    return (
      <TouchableOpacity onPress={() => this.showActionSheet()}>
        <Text>Touch Me</Text>
      </TouchableOpacity>
    )
  }

In this case for example, what would I pass to anchor?

I think this is a bug in showActionSheetWithOptions. This bug was fixed in showShareActionSheetWithOptions by passing the anchor.
anchor is passed like this:
anchor: React.findNodeHandle(this.refs.share), // Where you want the share popup to point to on iPad
@paramaggarwal  want to give it a go?

@chirag04 Good point, I should document anchor, but this is not related to that. This bug does not affect UIActionSheet as far as I'm aware.
@coderdave from Apple's docs:
Important: UIActionSheet is deprecated in iOS 8. (Note that UIActionSheetDelegate is also deprecated.) To create and manage action sheets in iOS 8 and later, instead use UIAlertController with a preferredStyle of UIAlertControllerStyleActionSheet.

ref: https://developer.apple.com/library/prerelease/ios/documentation/UIKit/Reference/UIActionSheet_Class/index.html

It's deprecated, but not removed. I suspect the issue is that action sheets need to be displayed inside a UIPopoverController on iPad, which is different from how they are used on iPhone. We probably don't have the correct code path implemented for iPad.

If action sheets are deprecated and it's recommended to use UIAlertControllerStyleActionSheet, how would that be implemented in react native? Or has that not been done yet? Also, I guess I would have to target certain iOS versions to use the correct action sheet implementation.

@coderdave it looks like I was wrong (not usual) - UIActionSheet does indeed seem to have stopped working when building with the iOS 9 SDK.
To implement UIAlertControllerStyleActionSheet we'll need to provide an alternative code path in the native module. For now, I think it's simplest if we stick with the same JS API and just change the native implementation.
As you guessed, we'll need to detect the OS version and provide different implementations, but fortunately that's quite easy to do in Objective-C. I'm working on it now.

Should be fixed in a859d9e9315c2ff7


delete media_kit.jar when verification failed

I found that when the download fails, there is also a default-arm64-v8a.jar file, but the size of this file is 0 kb. According to the judgment in build.gradle it is considered as having a file and will undergo an MD5 verification.

Perhaps it can be directly deleted after the verification fails. After all, this is just a cache file.

This is an annoying issue and still can't get past this

Still not working
Build file 'user/.pub-cache/hosted/pub.dev/media_kit_libs_android_video-1.3.6/android/build.gradle' line: 102

* What went wrong:
A problem occurred evaluating project ':media_kit_libs_android_video'.
> media_kit: arm64-v8a JAR verification failed.

https://github.com/media-kit/media-kit/pull/656 this pr fixes it, but sadly its not merged yet


Copy only modified compoenets using Windows machine file copy

From @vinothkumarb4u in https://github.com/MicrosoftDocs/vsts-docs/issues/3340
I have an requirement to copy only modified file from repository to target folder. is there any option to do this using windows machine file copy
ex
Source repo
a1.txt
b1.txt - Modified.
c1.txt
Target folder
Should have only b1.txt

Hi,  need more information on this issue regarding your scenario. do you mean that only the modified files should be copied over to the destination ( this means that files that are already existing in the destination remain that way ) OR the destination should have only the modified files ( i.e. the destination contains only the files deemed "modified" and no other files ) ? Currently windows machine file copy tries to synchronize the source and destination in the following way: let's say the source has a1.txt, b1.txt and c1.txt and destination also has the same set of files. Now if the file b1.txt is modified, then when the windows machine file copy task will execute, it will copy only b1.txt. Now the destination will contain a1.txt, b1.txt ( modified ), and c1.txt . Does this fit your scenario ?


ITAB_DUPLICATE_KEY Shortdump while importing OpenSAP offline repo

A shortdump occurs when this OpenSAP offline repo is imported to a local package. No problem with Z-Package.
ZIP file from here:
https://open.sap.com/courses/wtc1/items/5poy1IKePw8Tzv1rBc66cz

Looks like the issue is that local packages have no object directory entry

Also applies for those created in SE80. Plan to fix by trying the package table if tadir fails and the package is local

Installing something like https://github.com/abapGit-tests/DTEL to a local package works without problems

Will have a look.
Have a fix on my home box already, but want to find out why this doesn't have the issue.
Does your local package have a TADIR entry?

no, no local packages have TADIR entries

Funny, works for me too.
Afraid I forgot to update abapgit before trying (was just a few days behind)

I think you can close this unless @christianguenter2  has objections

Hm, still doesn't work for me.

Very weird.
I deleted and recreated the package and now is dumping again with the course file, while it works with DTEL
PR will follow


Improve shadow

Instead of going from center to every direction, it will be better to makes shadow going from corners to corners.

The shadow is placed in a matrix.


Add a new validation rule to allow case insensitive unique value

For example, in some cases, If we want to check whether email or username is unique, it's better to conver the string to lowercase or upppercase before validation. Otherwise if we have username registered as admin and Admin, it's quite confusing.

is common - see github username validation
github doesn't care about the case of the username, but validates it unique.

but nevermind


Philips dimmer not recognised as switch.

Philips dimmer switch isn't being shown in the webgui.
When looking with VNC, I can see the device being detected, but marked with a ? as it doesn't know what kind of device it is.
Philips modelnumber:<PHONE_NUMBER>
Issue appears in 2.11.00 and 2.11.05 (latest)
21:13:05:338 unknown node 0x0017880109A49E97 (0x28DE), lqi: 255
21:13:05:338 0xA0C6 nwk changed to 0x28DE
21:13:05:342 device announce 0x0017880109A49E97 (0x28DE) mac capabilities 0x80
21:13:05:342 set fast probe address to 0x0017880109A49E97 (0x28DE)
21:13:05:343 FP indication 0x0000 / 0x0013 (0x0017880109A49E97 / 0x28DE)
21:13:05:343                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:05:343 device announce 0x0017880109A49E97 (0x28DE) mac capabilities 0x80
21:13:05:348 CTRL skip polling while permit join is set
21:13:05:362 FP indication 0x0000 / 0x0013 (0x0017880109A49E97 / 0x28DE)
21:13:05:362                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:05:362 device announce 0x0017880109A49E97 (0x28DE) mac capabilities 0x80
21:13:05:362 FP indication 0x0000 / 0x0013 (0x0017880109A49E97 / 0x28DE)
21:13:05:363                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:05:363 device announce 0x0017880109A49E97 (0x28DE) mac capabilities 0x80
21:13:05:428 CTRL skip polling while permit join is set
21:13:05:635 CTRL skip polling while permit join is set
21:13:05:937 CTRL skip polling while permit join is set
21:13:05:940 CTRL skip polling while permit join is set
21:13:06:228 CTRL skip polling while permit join is set
21:13:06:288 [1] get node descriptor for 0x0017880109a49e97
21:13:06:534 CTRL skip polling while permit join is set
21:13:06:536 CTRL skip polling while permit join is set
21:13:06:833 CTRL skip polling while permit join is set
21:13:06:850 FP indication 0x0000 / 0x8002 (0x0017880109A49E97 / 0x28DE)
21:13:06:850                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:06:851 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) cluster 0x8002
21:13:06:851 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) clear timeout on cluster 0x8002
21:13:06:856 [2] get active endpoints for 0x0017880109a49e97
21:13:07:133 CTRL skip polling while permit join is set
21:13:07:142 CTRL skip polling while permit join is set
21:13:07:144 CTRL skip polling while permit join is set
21:13:07:158 FP indication 0x0000 / 0x8005 (0x0017880109A49E97 / 0x28DE)
21:13:07:158                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:07:158 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) cluster 0x8005
21:13:07:159 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) clear timeout on cluster 0x8005
21:13:07:164 [3] get simple descriptor 0x01 for 0x0017880109a49e97
21:13:07:435 CTRL skip polling while permit join is set
21:13:07:438 FP indication 0x0000 / 0x8005 (0x0017880109A49E97 / 0x28DE)
21:13:07:438                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:07:438 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) cluster 0x8005
21:13:07:444 wait response fastEnddeviceProbe() 0x0017880109A49E97, elapsed 279 ms
21:13:07:732 CTRL skip polling while permit join is set
21:13:07:748 CTRL skip polling while permit join is set
21:13:08:037 CTRL skip polling while permit join is set
21:13:08:061 FP indication 0x0000 / 0x8004 (0x0017880109A49E97 / 0x28DE)
21:13:08:061                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:08:062 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) cluster 0x8004
21:13:08:062 ZDP indication search sensors 0x0017880109A49E97 (0x28DE) clear timeout on cluster 0x8004
21:13:08:068 [4.1] Get manufacturer code
21:13:08:068 [4.2] get basic cluster attr 0x0004 for 0x0017880109a49e97
21:13:08:340 CTRL skip polling while permit join is set
21:13:08:353 FP indication 0x0104 / 0x0000 (0x0017880109A49E97 / 0x28DE)
21:13:08:353                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:08:353 Clear fast probe timeout for cluster 0x0000, 0x0017880109A49E97
21:13:08:359 [4.1] Get model ID
21:13:08:359 [4.2] get basic cluster attr 0x0005 for 0x0017880109a49e97
21:13:08:641 CTRL skip polling while permit join is set
21:13:08:652 SensorNode 2: Switch 2 added
21:13:08:654 FP indication 0x0104 / 0x0000 (0x0017880109A49E97 / 0x28DE)
21:13:08:655                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:08:655 Clear fast probe timeout for cluster 0x0000, 0x0017880109A49E97
21:13:08:657 discard double entry in binding queue (size: 2) for for 0x0017880109A49E97, cluster 0x0001
21:13:08:658 discard double entry in binding queue (size: 2) for for 0x0017880109A49E97, cluster 0xFC00
21:13:08:758 [4.1] Get sw build id
21:13:08:758 [4.2] get basic cluster attr 0x4000 for 0x0017880109a49e97
21:13:08:943 MAC poll fastEnddeviceProbe() 0x0017880109A49E97
21:13:08:944 wait response fastEnddeviceProbe() 0x0017880109A49E97, elapsed 185 ms
21:13:08:944 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0006
21:13:08:945 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0008
21:13:08:945 CTRL skip polling while permit join is set
21:13:08:953 MAC poll fastEnddeviceProbe() 0x0017880109A49E97
21:13:08:954 wait response fastEnddeviceProbe() 0x0017880109A49E97, elapsed 195 ms
21:13:08:954 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0006
21:13:08:954 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0008
21:13:08:955 CTRL skip polling while permit join is set
21:13:08:962 MAC poll fastEnddeviceProbe() 0x0017880109A49E97
21:13:08:963 wait response fastEnddeviceProbe() 0x0017880109A49E97, elapsed 204 ms
21:13:08:963 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0006
21:13:08:963 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0008
21:13:08:964 CTRL skip polling while permit join is set
21:13:09:014 FP indication 0x0104 / 0x0000 (0x0017880109A49E97 / 0x28DE)
21:13:09:015                       ...     (0x0017880109A49E97 / 0x28DE)
21:13:09:015 Clear fast probe timeout for cluster 0x0000, 0x0017880109A49E97
21:13:09:021 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0x0001
21:13:09:021 discard double entry in binding queue (size: 4) for for 0x0017880109A49E97, cluster 0xFC00

Providing additional screenshot from VNC.


getInitialLink() fired multiple times

The getInitialLink() continues to give a valid dynamic link after the first retrieval. This leads to a loop where a user is continuously forced into a page associated with the deep link

I have the same issue

ATTENTION: getInitialLink/getInitialUri should be handled ONLY ONCE in your app's lifetime, since it is not meant to change throughout your app's life.

This behavior is standard for the package, that is, you will receive the same cold start link every time. Until you close the application, the method will always return the same value.

How to clear initialUri value?


feat: Make the og:title tag match the  tag

Fixes #123.
I considered updating the title/og:title in root.tsx as well, however I assumed they were different by design. Let me know if the og:title should be set to the same value as title.

This looks good! Merged 👍


Overhaul of RepeatLast to be more accurate and useful

Yet another PR related to #161.
The extra accuracy is seen, e.g., when a command is picked from Help screen or Dashboard and now this very command is repeated, while previously the Help or Dashboard commands were.

I've just added some more tests with the aim of safeguarding (or proving unsound) a possible simplification by moving slastPlay from each level of the stack to a single copy at the toplevel of the state record. Surprisingly many of the tests fail. @rszczers if you would attempt investigating, at your leisure, please remember you can use the mock to print all steps of the uwinding of the macro (just given an empty list of steps as the expected outcome and it will print what it got instead).

Good news. No proofs yet, but I did a small computation on a side and found out that's most probably only the mock's fault. It stops the computation in case when slastPlay is empty (here), but that's precisely when the real code pops the buffer and continues until each slistPlay is empty (at least I hold strong belief it does so...). Tomorrow I'll investigate further.

You are right. Adding the missing case to mock fixed "RepeatLast test 26". However, some others are failing and also, the mock and the actual code now look quite differently. I wonder if we could put the bit from UI.hs into FrameM.promptGetKey, similarly as in the mock. Also, factor out and deduplicate this part in the mock.

I wonder if we could put the bit from UI.hs into FrameM.promptGetKey, similarly as in the mock.

Nah, it's tempting but then we would pop the buffer even before handling any keystroke. humanCommand is basically a wrapper around a loop that:

gets the keystroke through promptGetKey,
handles the key through cmdSemInCxtOfKM demonic incantation function,
then it pops the unnecessary buffer from sactionBuffer...
...and checks if we want to exit the loop.

So putting 3 into 1 is rather not an option. Hang on for a while, I'm trying to rewrite simplified version of humanCommand in test.hs.

That's what we have the tests for. :) Except that they fail and that our mock differs from the main code. :D

Still working on it. New mocks will be just like the real code, except the bad parts :-D
By the way, best way to prototype a feature: implement ad-hoc solution failing miserably, repeat, do loads of test cases to find out what you were on to, discover you failed to mock it, fit mocks to pass those test cases and then go back to fix an actual implementation. Is this real life? :-D

This is luxury. In real life you fail miserably, patch, work around the patch, then learn to live with it.

Crazy days. I think I've managed to rewrite the whole thing fixing the problem with trailing empty buffers. Embarrassingly, tests still glow red, starting from case 31, just like before.

Well done. Would you like to add anything else? If not I will merge once CI limps to completion.

 Now I understand why Lisp and Scheme are so proud of their macros.

I thought maybe we should split test.hs into two modules, but that's a detail.

Indeed, the mock could be separate. Your call.

Ok, I'll do this in a moment.

Looks great.

CI will take forever, so let me just merge right now. Thanks a lot!

BTW, I've just done some whitespace cleanup, etc., and pushed ASAP to master, because such trivial changes cause merge conflicts easily.

Sure, thanks and sorry for the mess. I''ll try to remember those conventions.

Not really. Just a few are in stylish-haskell: imports and aligning :: in record types (the only aligning I routinly do). The rest is a pretty standard mix of old Haskell conventions (apart of not aligning as much as people usually do).

BTW, I tried "I wonder if we could put the <> from UI.hs into FrameM.promptGetKey, similarly as in the mock." and all tests break. I also tried to move it to another place, the same failure. I'm convinced, even though I don't quite understand the causes.

BTW, I've just tried "simplification by moving slastPlay from each level of the stack to a single copy at the toplevel of the state record". It works, is simpler in some ways, but half the tests fail, because the semantics is different. I've added two tests that show the semantic difference and that, in fact, your version is more useful to the player, even if more messy semantically (not compositional; not referentially transparent' or rather, even less than mine). So it stays as is.

That's curious, didn't think about this like that. The key insight was that each macro runs an instance of the player on it's own, like player's ghost car in racing games that you can't collide with. From the beginning I wanted this to reassemble vim macros, but in a way that predefined macros can't mix up in-game macro. That's different from vim, because in vim we can bind “in-game” macro to whatever key we want, thus having a separate buffer for each and manage them manually. Here we had only one buffer that we couldn't avoid to share with predefined macros, and that's a good thing since that's a game, not a full-blown text editor. I think that difference is specifically why can't have referential transparency here.

This could work out (I mean referential transparency) if we had a map of unequally generated names for each macro, even the in-game ones, and a way to dynamically bind those names to keystrokes. This kinda reminds me lisp macros. Definitely it's worth checking out.

Interesting. I will slowly get to understand what you say. Right now, I grokked "each macro runs an instance of the player on it's own, like player's ghost car in racing games that you can't collide with". That's pretty cool.
Also, I've just pushed tests that depends on in-game macros, as opposed to named macros, not pushing a new buffer into the stack. I wonder what other consequence of this decision are. Instinctively I'd want repeatHumanTransition 1 == macroHumanTransition, but I don't understand the consequences yet. Perhaps its irrelevant or perhaps it's actually good that the player can create looping in-game macros so easily.

You mean checking if in-game macro does the same thing as running the same macro as predefined one? This seems valid and we could check this automatically.

Yes. If we want that property, that's finally a good candidate for QuickCheck. :)
However, I've just, at this moment, pushed tests showing that's not the case currently. And that's also what I wonder about in the comment above.

I'd rename ActionBuffer to KeyMacroFrame
smacroBuffer to keyMacroBuffer
slastPlay to keyPending
slastAction to keyLast
sactionPending to skeyMacroStack
What do you think?

Sure, they fit well. I moved them from the top-level simply to provide a proof of concept and left them like that not having a better idea. I'm always having a hard when it comes to thinking up a proper name, commit message etc. :-)

Perhaps its irrelevant or perhaps it's actually good that the player can create looping in-game macros so easily.
It took me a while to get what you're on to. So, right, in-game macros have slightly different semantics than predefined ones. Repeating in-game macro means that we just paste its contents to player's keyLast, so it also changes keyLast. When repeating a predefined macro, we push a new buffer on a stack and proceed there, locally. I think we could unify those behaviors by changing V semantics, i.e. by pushing a new buffer each time we repeat in-game macro.

by pushing a new buffer each time we repeat in-game macro.

God forbid, this also changes behavior of predefined macros. We'd get a spectacular memory leak whenever dealing with those recursively repeating macros, and that's quite hard to track down from player's perspective.

I've pushed the rename and also the change that reflects in types that the macro frame stack is never empty. Refactoring with such a test suite is a walk in the park. :)

It took me a while to get what you're on to.

I meant the looping macro tests that I've just added. Adn they don't use named macros, which was quite surprising. I'm all for unifying the semantics. I just have no clue right now what would change in practice. Tests that show the difference (in addition to the last few I added) would be very welcome.

God forbid

Oh, so you figured it out. Great. Could you illustrate that by tests, including speculation what their result would be in the alternative semantics?

Out of interest I've made that change
~/r/LambdaHack$ git diff
diff --git a/engine-src/Game/LambdaHack/Client/UI/HandleHumanLocalM.hs b/engine-src/Game/LambdaHack/Client/UI/HandleHumanLocalM.hs
index d57b7ad..5921326 100644
--- a/engine-src/Game/LambdaHack/Client/UI/HandleHumanLocalM.hs
+++ b/engine-src/Game/LambdaHack/Client/UI/HandleHumanLocalM.hs
@@ -714,13 +714,14 @@ selectWithPointerHuman = do
 -- at terrain change or when walking over items.
 repeatHuman :: MonadClientUI m => Int -> m ()
 repeatHuman n = modifySession $ \sess ->
-  sess {smacroFrame = repeatHumanTransition n (smacroFrame sess)}
+  sess {smacroFrame = repeatHumanTransition n (smacroFrame sess)
+       , smacroStack = (smacroFrame sess) : (smacroStack sess)}

 repeatHumanTransition :: Int -> KeyMacroFrame -> KeyMacroFrame
 repeatHumanTransition n macroFrame =
   let macro = KeyMacro . concat . replicate n . unKeyMacro . fromRight mempty
               $ keyMacroBuffer macroFrame
-  in macroFrame {keyPending = macro <> keyPending macroFrame}
+  in emptyMacroFrame {keyPending = macro}

 -- * RepeatLast

and it's not that bad in terms of failing tests:
    RepeatLast test 14:                                             FAIL
      test/Spec.hs:133:
      expected: "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
       but got: "xx"
   RepeatLast test 34:                                             FAIL
      test/Spec.hs:228:
      expected: "xyyzxyyzxyyxyyxyyzxyyzxyyxyy"
       but got: "xyyzxyyzxyyxyyzxyyzxyy"
    RepeatLast test 35:                                             FAIL
      test/Spec.hs:233:
      expected: "xyyzxyyzxyyxyyzxyyzxyyzxyyzxyy"
       but got: "xyyzxyyzxyyxyyzxyyzxyy"
    RepeatLast test 36:                                             FAIL
      test/Spec.hs:238:
      expected: "xyyzxyyzxyyxyyzxyyzxyyxyyzxyyzxyyxyyzxyyzxyy"
       but got: "xyyzxyyzxyyxyyzxyyzxyy"
    RepeatLast test 39:                                             FAIL
      test/Spec.hs:251:
      expected: [[(Right "","'xv'V","")],[(Left "","xv'V","")],[(Left "x","v'V","x")],[(Left "x","x'V","x")],[(Left "xx","'V","x")],[(Right "xx","V","x")],[(Right "xx","xx","V")],[(Right "xx","x","x")],[(Right "xx","","x")]]
       but got: [[(Right "","'xv'V","")],[(Left "","xv'V","")],[(Left "x","v'V","x")],[(Left "x","x'V","x")],[(Left "xx","'V","x")],[(Right "xx","V","x")],[(Right "","xx","")],[(Right "","x","x")],[(Right "","","x")]]
    RepeatLast test 40:                                             FAIL
      test/Spec.hs:262:
      expected: "xyxyy"
       but got: "xyxy"
    RepeatLast test 42:                                             FAIL
      test/Spec.hs:270:
      expected: "xyxyxy"
       but got: "xyxy"
    RepeatLast test 43:                                             FAIL
      test/Spec.hs:274:
      expected: "Macro looped"
       but got: ""
    RepeatLast test 44:                                             FAIL
      test/Spec.hs:277:
      expected: "Macro looped"
       but got: ""

The results for 14, 40 and 42 seem wrong, but I can't see how they were computed. Perhaps v should pop buffers from the stack as well or perhaps v is to blame in some other way?

That's trace from case 14. Something strange happens when we push macro onto the stack, line 4.

]

Isn't this a problem with dropping player's buffer?

 My bad. I forgot to the same change in in the mock:
``
diff --git a/test/SessionUIMock.hs b/test/SessionUIMock.hs
index 0712859..e693469 100644
--- a/test/SessionUIMock.hs
+++ b/test/SessionUIMock.hs
@@ -108,7 +108,8 @@ cmdSemanticsMock km = \case
return $ Left Nothing
HumanCmd.Repeat n -> do
modify $ \sess ->

 sess {smacroFrame = repeatHumanTransition n (smacroFrame sess) }

 sess {smacroFrame = repeatHumanTransition n (smacroFrame sess)

  , smacroStack = (smacroFrame sess) : (smacroStack sess)}

return $ Left Nothing
HumanCmd.RepeatLast n -> do
modify $ \sess ->
``

My fault, if I made the whole change in repeatHumanTransition, as I should have, mock would not type check and I'd spot the error.

The changes that I now see are actually for the best. But I feel there's not enough tests. I particular I'm now totally lost when is it that v repeats the whole previous macro and when only the last part of it (never? but previously it did; are the other tests not still showing that?).

BTW, thanks to what we have done, I will be able to simplify the macros define in game content. I see there are some fake commands added there just because macros didn't quite work.

I'm now totally lost when is it that v repeats the whole previous macro and when only the last part of it

Previously this was a case only when we did V and v right after it (like in case 40), since V pasted macro into a current buffer. Now we've got rid of it, so this should never happen. Yet, cases 43 and 44 reveal we've got a problem with repeating a macro when recording one.
After all, I consider this much more natural.

I think the easiest way out is to handle V as noop when recording (i.e. when buffer is Left)

Thank you. After what you said I have more confidence in the current behaviour. Actually, I relatively like the current results (after unifying repeatHumanTransition 1 and  macroHumanTransition, BTW, would you like to code that, or should I?) from 43 and 44:
     , testCase "RepeatLast test 43" $
         snd (last (unwindMacrosAcc coinput (stringToKeyMacro "'xyV'V")))
         @?= "xyxy"
     , testCase "RepeatLast test 44" $
         snd (last (unwindMacrosAcc coinput (stringToKeyMacro "'xyV'v")))
         @?= "xyxy"

In the first case, first V repeats a nonexistent buffer, so it has no effect and the second V repeats the overall results of that. In the second case, V is again inert initially, but v repeats not the results of the V, but literally the V command, which now acts in the context where a macro is defined, so it does something. I'm resigned to v being working on syntax, so the 'late binding' behaviour is fine. I think the only way to do this better would be to save whole sequences of commands instead of a single keypress, to be repeated by v, but how long a sequence that should be is also arbitrary. Anyway, i think it works fine as is.

Feel free to unify them. Otherwise it may take me some time.

In the first case, first V repeats a nonexistent buffer, so it has no effect and the second V repeats the overall results of that.

This sounds reasonable. I'm not quite sure yet how this is going to work when player presses V when recording in-game macro. It just ends recording, right?

OK, I will code that. My guess, from the code and test results, is that V would be ignored initially (not produce nor end anything), but resurrected by a v pressed afterwards. Yes, I just tested in-game and that's what's happening, exactly as in tests. :)

It turns out, I can't simplify Input.hs using our new macro machinery. After I looked closed and realized how much the current definitions do, e.g., automatically provide documentation for commands in menus and the help screen, I don't think they are too convoluted.


ShaderGLES2 Errors

Godot version:
2a10e30
OS/device including version:
Win10 64-bit
OpenGL ES 2.0 Renderer: GeForce 9600 GT/PCIe/SSE2
OpenGL ES 2.0 Batching: ON
Issue description:
Was tabbed back into work in the editor and saw these errors in the console that I haven't seen before, and not sure what caused them.

EDIT - Log
OpenGL ES 2.0 Renderer: GeForce 9600 GT/PCIe/SSE2
OpenGL ES 2.0 Batching: ON
	OPTIONS
	max_join_item_commands 16
	colored_vertex_format_threshold 0.25
	batch_buffer_size 16384
	light_scissor_area_threshold 1
	debug_flash False
	diagnose_frame False

**ERROR**: Condition "!cc" is true. Returned: 0
   At: drivers\gles2\shader_gles2.cpp:214:ShaderGLES2::get_current_version() - Condition "!cc" is true. Returned: 0
**ERROR**: Condition "!version" is true. Returned: false
   At: drivers\gles2\shader_gles2.cpp:88:ShaderGLES2::bind() - Condition "!version" is true. Returned: false
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1
**ERROR**: Condition "!version" is true. Returned: -1
   At: C:\Godot\drivers\gles2\shader_gles2.h:267:ShaderGLES2::_get_uniform() - Condition "!version" is true. Returned: -1

Steps to reproduce:
Currently not known.  Will post back if I can isolate it better.
Minimal reproduction project:
N/A

Will investigate when we get a MRP, and looking through to see what could cause this.
@clayjohn could this possibly be leaving conditional CanvasShaderGLES2::USE_TEXTURE_RECT set (or vice versa)? I'm not very familiar with how the shader versioning works, I'll look into it.
If this is the batching, and one of the conditionals, it should be easy to fix once we have a repeatable project.

It's proving elusive to catch.  I'm not sure about the console distinctions, so I don't think I can give you a good answer.  My knowledge on Godot runs thin here.
It was running outside of an IDE at that time.  I think it was being thrown by the tools/editor version of Godot, as the app wasn't running.
Any ideas on what I might probe at to try to reproduce it?

Ah it does sound like it was occurring just with the editor rather than with a running project. Those bugs are far more tricky to repeat and diagnose! :grin:
Did you have a particular project loaded at the time? Even running the editor it will be displaying the preview tabs of the project and that can help pin down what might be causing it. Any hints you might have as to what you had on screen (was it a 2d project, 3d etc) might be a clue.
If you can't repeat it don't worry too much, it is very helpful that you have pinpointed the lines where an error were occurring and I can do some tests and see if I can recreate the error artificially. I'll also familiarise myself with the shader versioning.
At this stage we can't say for sure it was due to the batching (it could have been a rare bug that occurs anyway) but I'll assume it is the batching for now until I can see otherwise.

Yeah, just throwing it out there, in case it is relevant or helpful for what you're working on.
It was a 2D project in GLES2.  I opened it, tabbed away to look at something else for a minute, when I returned it noticed those in the console.
I'll take another look and see if I can coax it out.

Ghost problems.  👻
Tried many things.  I also went back to the log, and there were also some later warnings about items that had gone missing from an instanced scene.  (It cleaned itself up.)
So I tried to replicate that to see if it maybe influenced something during startup, but nothing I'm trying has reproduced it yet.
A momentary lapse in the OS providing the version information at startup?
What would happen if it failed to get the version in a release build?
	if (!version->ok) { //broken, unable to bind (do not throw error, you saw it before already when it failed compilation).
		glUseProgram(0);
		return false;
	}

Ghost problems.
Oh I'm all too familiar with these 'heisenbugs'! :grinning:

So I tried to replicate that to see if it maybe influenced something during startup, but nothing I'm trying has reproduced it yet.

It is okay, don't worry too much, sometimes these things are hard to pin down until someone later finds a repeatable version. It could be anything, alt-tabbing, a race condition with background import of files etc. I'll try and have a systematic look through once I have tracked down another bug I'm working on, I'm not familiar enough with the shader versioning to guess what might be happening.

What would happen if it failed to get the version in a release build?

Nothing too scary, I suspect it would just not draw the items using that shader (and perhaps only on a single frame, blink and you miss it!).

Just to quote from clayjohn on this, he knows much more than me about this:

the shader bug just means a version failed to compile. We need an MRP to do anything. The issue could be with the users shader code, and they unknowingly fixed it and only ever saw one error.

Thanks for the follow up.  😸 The project doesn't yet have any custom shaders.  Only one or two codeless godot canvas material type shaders (made via inspector), where I set the blend mode to add for a glowing effect on some sprites.
Wondering if I should close this.  It doesn't seem like it would be much of an issue, and nothing to work on without pinpointing the shader involved.
Perhaps a bit off topic.  Are there any ways to retroactively debug these kinds of errors right after they happen?  I'm not too experienced with C++ debugging and profiling, but it seems like I would have to have a breakpoint set in advance, and some of the lines with error macros I can't conditionally breakpoint in VS, because they rely the direct return values of functions.  I would like to avoid the mess and time loss of writing in hacky lines and recompiling to do the typical debugging.

The best way to debug an error like this is to reproduce it. If you can figure out what happened in the project that led to this error appearing then we can figure out what caused it.

Perhaps a bit off topic. Are there any ways to retroactively debug these kinds of errors right after they happen? I'm not too experienced with C++ debugging and profiling, but it seems like I would have to have a breakpoint set in advance, and some of the lines with error macros I can't conditionally breakpoint in VS, because they rely the direct return values of functions. I would like to avoid the mess and time loss of writing in hacky lines and recompiling to do the typical debugging.

Well Godot tends to use its own error macros which fail gracefully. This is good in the sense that most errors won't stop a game / the IDE from running (for end users), however for debugging in development at the time it is usually better to have something like an assert which halts the program and allows you to debug with a stack trace, so you can find out what caused a condition immediately. This is especially important in 'heisenbugs', that rarely occur, or are unpredictable, which can be quite common in rendering.
There actually is an argument for having a #define or setting that allows a development build to replace things like ERR_FAIL with asserts that will halt immediately, allowing us to catch such bugs quicker. Perhaps there is such a macro already. I'll try and ask on IRC later.

@lawnjelly What is this argument that forces assertion?

@avencherus I think @lawnjelly said we should add a SCons flag to abort on errors. There isn't one currently.

Note that it is possible to set a GDB breakpoint on error_macros.cpp:82 or so -- that way it would stop right after printing each error, allowing one to use the bt and frame commands to examine the state of the program.
Running through rr might be a simpler way to go back to heisenbugs, however.

@bojidar-bg Thanks for this info, I haven't used GDB, but will have it bookmarked for later.  And I was wondering a good place to break for error macros.
I will close this issue, as I haven't seen it since.  I'll re-open if there is a way to reproduce it or it happens again and I catch it in a debugger.


change yarn lint

Description
A clear and concise description of the features you're adding in this pull request.
Change npm run to yarn as asked. Issue probably coming from IDE (vscode)
Metadata

Fixes #[Link to Issue]

doesn't seem to be achieving expectations, please read docs about husky!

Thank you, but it does perfectly, as the expectation never mentioned that I should fix configuration problems on your side.
You can try 'git config core.hooksPath ".husky" ', but do not expect lines of code to fix a problem of git/ide configuration.

We do, actually.
Thanks for your interest in contributing!
I'm closing this PR now as there's no proposed roadmap to address the issue described on the task.
To be clear on the expectations:

repo should lint:fix every time someone does git commit, this is totally unrelated to the IDE chosen, should also work with console.

My interest is pointless if you do not try to understand what I'm saying. I already showed you that it worked.
Thank you for the waste of time, the unprofessional attitude, and the clear lack of consideration (next time, redact correctly your bounty, thanks).


FirebaseAuth startup crash in memechat

Steps to Reproduce
Run the latest memechat app on Android.
Crash in the logs:
AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.yourcompany.memechat/com.yourcompany.memechat.MainActivity}: java.lang.IllegalStateException: Default FirebaseApp is not initialized in this process com.yourcompany.memechat. Make sure to call FirebaseApp.initializeApp(Context) first.
This is looks like either a misconfiguration or a FirebaseAuth startup race condition. I'm investigating.
/cc @emshack @efortuna

Fixed in efortuna/memechat#14

@collinjackson
This issue has been moved to https://github.com/FirebaseExtended/flutterfire/issues/1011. Any further collaboration will be done there.


Minor: [Python] Usage of build.sh should mention about doc

What is the purpose of the change
This PR fixes a minor issue that the usage of build.sh doesn't mention about doc option even though it supports the option.
Verifying this change
Now ./build.sh shows the usage with the doc option.
$ ./build.sh
Usage: ./build.sh {clean|dist|doc|interop-data-generate|interop-data-test|lint|test}

Documentation
No new features added.

Thank you, @sarutak !


Robocopy Broken

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
PowerShell 7.4.0 Preview 1
Can no longer use robocopy.exe from powershell.  As robocopy.exe returns status code other than 0 for successful copies.
Repro:

robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\

Expect: Runs without creating an error object.
Actual: Creates Error Object
NativeCommandExitException: Program "Robocopy.exe" ended with non-zero exit code: 1.
Issue: This will break all of our existing automation.  We use robocopy everywhere. And our scripts all use ErrorAction = Stop
Expected behavior
robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\

# Files are copied. No error is raised when all files are copied.

Actual behavior
robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\
 # An error is raised even though robocopy detected no errors.
 # 1 indicates all files copied.

Error details
PS > Get-Error

Exception             :
    Type        : System.Management.Automation.NativeCommandExitException
    Path        : C:\Windows\system32\Robocopy.exe
    ExitCode    : 1
    ProcessId   : 17460
    ErrorRecord :
        Exception             :
            Type    : System.Management.Automation.ParentContainsErrorRecordException
            Message : Program "Robocopy.exe" ended with non-zero exit code: 1.
            HResult : -2146233087
        CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordException
        FullyQualifiedErrorId : ProgramExitedWithNonZeroCode
    Message     : Program "Robocopy.exe" ended with non-zero exit code: 1.
    HResult     : -2146233087
TargetObject          : C:\Windows\system32\Robocopy.exe
CategoryInfo          : NotSpecified: (C:\Windows\system32\Robocopy.exe:String) [], NativeCommandExitException
FullyQualifiedErrorId : ProgramExitedWithNonZeroCode

Environment data
$PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.0-preview.1
PSEdition                      Core
GitCommitId                    7.4.0-preview.1
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Visuals
Steps.zip

What is in robocopy log?

I know you're aware of at least some of this, but let me summarize:
By convention, CLIs use nonzero exit codes to signal error conditions, and that is what the - experimental, at this point - PSNativeCommandUseErrorActionPreference feature and its $PSNativeCommandUseErrorActionPreference preference variable rely on.
Robocopy does not follow this convention and uses nonzero codes as success error codes too (which communicate additional information) - see https://ss64.com/nt/robocopy-exit.html
Robocopy is a prominent example of such an "unconvential" CLI, but there's no telling how many there are, so it's impossible to anticipate all possible CLIs that require special treatment (let alone maintain tables of what nonzero codes, specifically, indicate success), so I don't think maintaining a list of exceptions is practical.
Pragmatically speaking:

I'm assuming - perhaps @SteveL-MSFT  can confirm - that setting  $PSNativeCommandUseErrorActionPreference to $true will be opt-in if and when the experimental PSNativeCommandUseErrorActionPreference feature will be made official. Somewhat curiously, in the 7.4 preview versions it is opt-out.

Thus, in stable versions, only users who explicitly set $PSNativeCommandUseErrorActionPreference = $true will be affected (for the preview versions, you can simple disable the experimental feature or run $PSNativeCommandUseErrorActionPreference = $false)

If the feature is deliberately enabled, a - somewhat cumbersome ad-hoc workaround is to call via a child scope:
& { $PSNativeCommandUseErrorActionPreference = $false; robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ }

Note that while 2>$null would silence PowerShell's error, it would also silence the CLI's standard error output, which is undesirable. More importantly, 2>$null wouldn't prevent a script-terminating error if $ErrorActionPreference = 'Stop' were in effect.
(Tangentially related: #18368)

@mklement0 Thank you for the explanation.  A another way to prevent the failure would be to shell out to cmd.exe and redirect stderr to stdout. I had to do that 5 to 10 years ago.  When scripting in TFS. Ironically there was a tfs exe which wrote things it thought were very important to stderr.
To do that with robocopy will be very complicated if we need to detect real errors.

@RichardMeadowsTC, shelling out to cmd /c will only help you if place an explicit & exit 0 at the end - by contrast, it is fine not to redirect stderr to stdout, as it is only the exit code that matters.
It is not uncommon for CLIs to write anything that's not data to stderr, which may or may not be error messages.
However, If you shell out to cmd /c, you'll lose Robocopy's exit code, which you need if you want to robustly detect failure .
# OK, but you Robocopy's exit code is lost.
cmd /c 'robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ & exit 0'

I would need to run some prototypes.  I am not so sure about the cmd.exe not returning the exit code of the process which was ran.  I do know powershell.exe tends to return 0 only.  Anyway it is all overly complicated and would require a special wrapper for EXEs.

I am not so sure about the cmd.exe not returning the exit code of the process which was run

It does - that's precisely why you need & exit 0 to prevent the nonzero Robocopy exit code from being passed through to PowerShell and again triggering the PowerShell-side error message (but, as noted, & exit 0 comes at the expense of losing the specific Robocopy exit code).
A simple wrapper function that avoids the PowerShell error while preserving the exit code:
function Invoke-WithExitCodeIgnored {
  param([Parameter(Mandatory, Position=0)] [scriptblock] $ScriptBlock)
  $PSNativeCommandUseErrorActionPreference = $false
  & $ScriptBlock
}

# Sample call
Invoke-WithExitCodeIgnored { robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ }
# Now you can examine $LASTEXITCODE

There is no need for Exit 0 in cmd script.  Only ever needed that when running ps1 script from cmd scripts. Not the other way around.  How well last exit code is sent from cmd.exe I need to check.

I've explained why & exit 0 (a) is necessary  if you were to use  cmd /c in order to avoid the problem at hand and (b) that doing so would be suboptimal, because Robocopy's specific exit code would be lost in the process - and, by implication, (c) why this attempt at a workaround is best avoided.  Finally, (d) I've provided  a workaround that is generally viable.
If you have specific feedback about these points, please provide it.

I don't think you understand.  I working on my tuck today. Will get back to this in the morning.  But there a thing we can fix right now.  The exit code just set it to what robocopy sets.  But I think that is not even necessary. As I have never seen that needed in cmd scripts.  Only seen that needed in powershell scripts.
All that said, I think what you do have for properly working solution is very good.

A simple wrapper function that avoids the PowerShell error while preserving the exit code:
function Invoke-WithExitCodeIgnored {
  param([Parameter(Mandatory, Position=0)] [scriptblock] $ScriptBlock)
  $PSNativeCommandUseErrorActionPreference = $false
  & $ScriptBlock
}

# Sample call
Invoke-WithExitCodeIgnored { robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ }
# Now you can examine $LASTEXITCODE

What about a built-in variable to complement $PSNativeCommandUseErrorActionPreference, such as:
$ZEROEXITCODE
The variable resets to zero after a native command is executed.
Powershell calculates $LASTEXITCODE just prior to processing any errors.
Ex:
$ZEROEXITCODE = 1
robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\
# $LASTEXITCODE = %ERRORLEVEL% - $ZEROEXITCODE
$LASTEXITCODE
0

This would maintain the benefit of $PSNativeCommandUseErrorActionPreference = $true

All we have to do is set $PSNativeCommandUseErrorActionPreference = $false
Thankfully, it isn't raising error when the exe simply writes to stderr.  Like TFS Workflows were doing a few years ago.
I made an EXE to verify all this.  And sample scripts. https://github.com/RichardMeadowsTC/PowerShellCmdInvoke
This is a side note.  The cmd.exe will return current error level.  This is different than powershell which has always tried to hide script result from app which invokes powershell.exe or pwsh.exe.  I can guess they aren't changing it to prevent existing code from breaking.  But here clearly lots of code is going to be broken.  Much more than was broken with Pwsh.exe version 7.3.

All we have to do is set $PSNativeCommandUseErrorActionPreference = $false

Then set it back to $true after running the command. If not, why have the feature at all?
I think that's fine as a workaround solution, but it's not ideal.

The reason for not setting it back is that it never should have been enabled. Old scripts that we have are well tested and check $LASTEXITCODE where necessary.

Please see the following proposal for a possible solution:

#18961

I don't like the idea of an "ignore list" for the experimental feature, but robocopy isn't going to change and there are also other known tools that don't follow the non-zero exit is error convention.  choice.exe is one where it actually makes sense, but currently would also cause a problem.

For tools that don't follow the non-zero exit is error convention, there is the option to set (by default or otherwise) $PSNativeCommandUseErrorActionPreference = $false and continue to use $LASTEXITCODE for error handling. The other option is to employ some method for 'tweaking' exit codes so as to prevent 'false positives' from triggering Powershell's error detection, thereby maintaining consistency of error handling.
I would suggest that a good reason for favoring the second option is that eventually, all CLI executables that still have at least moderate usage, are going to be 'consumed' by Crescendo-based modules, and the authors of those modules can handle whatever 'tweaks' are required to produce the desired response. So I say make $PSNativeCommandUseErrorActionPreference = $true the default setting, and provide a mechanism for altering non-zero, non-error exit codes, that most Powershell users will rarely if ever need to use.

@SteveL-MSFT, please confirm that $PSNativeCommandUseErrorActionPreference will default to $false in release versions - not doing so would be a massively breaking change, not least because an effective $ErrorActionPreference = 'Stop' setting would then start breaking scripts with external-program calls that happen to return non-zero exit codes (which are currently ignored).
Proceeding on that assumption:

Setting $PSNativeCommandUseErrorActionPreference = $true is therefore a deliberate act and the user should be assumed to understand the consequences: any non-zero exit code reported reports a non-terminating error; as such, it can be controlled via $ErrorActionPreference, which, however, is just as cumbersome as temporarily setting $PSNativeCommandUseErrorActionPreference = $true

What's more, the error shouldn't statement-terminating, not non-terminating, as argued in #18368
Doing so would enable a try / catch solution - see below.
Those not setting $PSNativeCommandUseErrorActionPreference = $true could continue to happily ignore exit codes or, preferably, always test $LASTEXITCODE afterwards, CLI-appropriately.

Building exception list for specific CLIs along with the specific non-zero exit codes that constitute success would mean playing an everlasting game of catch-up, as new non-conformant CLIs are discovered; it would lead to the same mess as with $PSNativeCommandArgumentPassing's Windows mode - see https://github.com/PowerShell/PowerShell/issues/18554#issuecomment-1314547572

The only proper place to implement exceptions would be in - invariably _CLI-_specific - Crescendo cmdlets, though it looks like there's currently no mechanism to define success exit codes - so that may be worth adding (the JSON schema could support defining success exit codes, and the generated code could test for these).

Not everyone will or should be expected to be using Crescendo modules (if available at all), so the best, exception-free solution is to make per-command overrides of $PSNativeCommandUseErrorActionPreference as simple as possible:

A try / catch approach is reasonable concise, but requires fixing #18368
# !! Does NOT work as of v7.4.0-preview.1
# Ignore the error, and examine $LASTEXITCODE afterwards.
try { robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ } catch { }

The proposed Invoke-NativeCommand - see #18991 - could help too; assuming inc as its built-in alias:
 # !! No such cmdlet yet, as of v7.4.0-preview.1
 # Ignore the error, and examine $LASTEXITCODE afterwards.
 inc -IgnoreExitCode { robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ }

Arguably, analogous to bash's set -e, no error should be triggered for external-program calls in the non-final link of a pipeline chain, i.e with && and || - this is currently not the case. If this worked, a concise - albeit slightly obscure - solution would  be:
# !! Does NOT work as of v7.4.0-preview.1
# Ignore the error, and examine $LASTEXITCODE afterwards.
# @() is an effective no-op.
robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ || @()

The following assumes $PSNativeCommandUseErrorActionPreference = $true is the default.
Assume "$PSHOME\ZeroErrorCodes.json" store the minimum magnitude non-error exit codes (with sign), for CLI executables that do not follow the non-zero exit code implies error convention. Example content:
{
  "negExitcodes.exe": -1,
  "Robocopy.exe": 1,
  "choice.exe": 0
}

At Powershell startup:
$ZeroErrorCode = 0
$ZeroErrorCodeObj = Get-Content "$PSHOME\ZeroErrorCodes.json" | ConvertFrom-Json

When user inputs a native command, Powershell does this processing:
# get command line into $cmdline, then get exe name
$executable = (Get-Command $cmdline.Split()[0]).Name
if ($null -eq $ZeroErrorCodeObj.$executable) {
    # run command line
    $exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCode)
    $ZeroErrorCode = 0
    # process error based on $exitcode
} elseif ($ZeroErrorCodeObj.$executable -eq 0) {
    $PSNativeCommandUseErrorActionPreference = $false
    # run command line
    $PSNativeCommandUseErrorActionPreference = $true
    # user can interpret outcome by examining $LASTEXITCODE
} else {
    # run command line
    $exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCodeObj.$executable)
    # process error based on $exitcode
}

Consider these scenarios.
User runs Robocopy which copies zero files without error:
$exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCodeObj.$executable)
$exitcode = [math]::Max(0, 0 - 1) # zero

User runs Robocopy which copies files without error:
$exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCodeObj.$executable)
$exitcode = [math]::Max(0, 1 - 1) # zero

User runs Robocopy which returns error:
$exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCodeObj.$executable)
$exitcode = [math]::Max(0, 2 - 1) # one. PS error triggered.

CLI whose json value is zero:
$PSNativeCommandUseErrorActionPreference = $false
choice /C YN /M "Do you like this proposal"
$PSNativeCommandUseErrorActionPreference = $true
# exitcode = 0, $LASTEXITCODE = 1|2

An exe not in json file:
unknown.exe arg1 arg2 # $LASTEXITCODE = %ERRORLEVEL% = 1
$exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCode)
$ZeroErrorCode = 0
# PS detects error

Same but user sets ZeroErrorCode immediately prior to execution:
$ZeroErrorCode = 1
unknown.exe arg1 arg2 # $LASTEXITCODE = %ERRORLEVEL% = 1
$exitcode = [math]::Max(0, $LASTEXITCODE - $ZeroErrorCode)
$ZeroErrorCode = 0
# PS does NOT detect error

Discussion in the team we agree that this feature is working as expected and we don't want to special case robocopy.  Instead, I've opened a doc bug to ensure we have a good example for cases like robocopy to turn it off and on.

To offer an alternative, let me add solutions that already work / will work with $PSNativeCommandUseErrorActionPreference set to $false.
In other words: for general per-native-program-call error handling.
# With a conformant CLI
# In a script, use `exit` instead of `throw`
cmd /c 'echo hi && exit 5' || $(throw $LASTEXITCODE)

# With special handling for Robocopy
# Only treat exit codes > 3 as errors.
robocopy .\PowerShell-7.4.0-preview.1-win-x64\ .\temp\ || $(if ($LASTEXITCODE -gt 3) { throw $LASTEXITCODE })

Note the unfortunate need to enclose even throw and exit in $(...), otherwise they'll be treated as (usually non-existent command names) - see #10967 for background.

Discussion in the team we agree that this feature is working as expected and we don't want to special case robocopy. Instead, I've opened a doc bug to ensure we have a good example for cases like robocopy to turn it off and on.

Hang on... I didn't see a response to mklement's request for confirmation that we would still have existing behavior in release versions. Your statement, @SteveL-MSFT almost sounds like "no, we are turning this on, and doc'ing it better". If that is the case, please allow me to respectfully scream "bloody murder". I've already gotten bug reports from users of my modules for robocopy, and git, and I can't even remember what else. I assumed this was so clearly such a massively breaking change that it surely wouldn't stand...

@jazzdelightsme I believe it was mentioned in a separate issue that $PSNativeCommandUseErrorActionPreference is default to $false and only set to $true in 7.4 previews to get feedback.

Glad to hear it, @SteveL-MSFT, but it's somewhat disheartening that it took three separate pleas to get you to address this aspect.

@SteveL-MSFT Is there an explanation for the fact that the NativeCommandExitException disappears if the native command's output is discarded using 2>&1 | Out-Null? (I'm testing with 7.4.0-preview.4)

@sba923, the current behavior is a bit counter-intuitive:

A 2> redirection also swallows the NativeCommandExitException, because the latter is implemented as a non-terminating error and is therefore treated the same as the external program's stderr output.

That a non-terminating error is emitted is also problematic for other reasons, as discussed in #18368, and if it were converted to a statement-terminating one, it would no longer be swallowed.

However, if you have $ErrorActionPreference = 'Stop' in effect, the script aborts (as intended) and the NativeCommandException surfaces (but not the native program's stderr output, as intended).

@sba923, the current behavior is a bit counter-intuitive:

A 2> redirection also swallows the NativeCommandExitException, because the latter is implemented as a non-terminating error and is therefore treated the same as the external program's stderr output.

That a non-terminating error is emitted is also problematic for other reasons, as discussed in Native error handling: $PSNativeCommandUseErrorActionPreference = $true unexpectedly causes non-terminating, not statement-terminating errors #18368, and if it were converted to a statement-terminating one, it would no longer be swallowed.

However, if you have $ErrorActionPreference = 'Stop' in effect, the script aborts (as intended) and the NativeCommandException surfaces (but not the native program's stderr output, as intended).

This is indeed far from intuitive...


A'x falls back on generic mul

Currently, A'x where A::Toeplitz falls back onto a generic and slow method. By defining
function LinearAlgebra.Adjoint(A::Toeplitz{T}) where T
    At = Toeplitz(A.vr, A.vc)
end

the problem is mitigated and the optimized method for mul! is used. Still, if A'x occurs in a loop, the initialization of the Toeplitz object occurs each iteration, which is suboptimal since it performs some allocations and plans an FFT etc. Ideas on how to solve the problem in general is appreciated.
In the meantime, would you be up for a PR that adds the definition above?

Sounds great! Though you should never overload a type constructor (Adjoint) to return a different type, unless you really have good reasons: it is confusing for users. Instead, overload the function adjoint.
Note if a user is worried about A' being expansive, they can move that out of the loop.
Down the line,  this package may move to a trait-based setup using ArrayLayouts.jl, (as BandedMatrices.jl and BlockArrays.jl have). This would allow Adjoint(Toeplitz(A.vr, A.vc)) to conform to a ToeplitzLayout interface. Though that's a ways off from materialising.


FYI: Is dependabot-gitlab (unofficial "fork") okay?

https://gitlab.com/dependabot-gitlab/dependabot/
They definitely have a mention or two about this project but they do not respect your LICENSE. Just FYI.

I'm the maintainer of that project and it is not a fork, it's an orchestrator that uses dependabot-core.
I have been previously approached by dependabot team from GitHub, hence the disclaimer and as far as I know, it does respect the license, since it is not a commercial software and it mentions the dependabot-core in the LICENSE
https://gitlab.com/dependabot-gitlab/dependabot/-/blob/master/LICENSE#L17

Good to know it's all good and straight =)
For reference, I was referring to this line in the LICENSE

Ensure everyone who gets a copy of this software from you, in
source code or any other form, gets the text of this license
and the contributor and source code lines above.


Support for dynamic links

Add the possibility to change links into PDF document using JavaScript.

Is this still an issue or is it obsolete due to the new linlkHandlers feature?


Fix CFTimeZone crashes on Windows

CFTimeZone on Windows requires mappings between the Windows and Olson timezone identifiers. We unfortunately lost those mappings in the build for the re-core. This PR adds those mappings back which resolves crashes when calling into CFTimeZone. We add them back as literals in code rather than as plist resource files to make it easily compatible with both the CMake and SwiftPM builds. I've confirmed that when running tests on Windows the tests no longer crash with CFTimeZone issues

@swift-ci please test

@swift-ci please test

@swift-ci please test Windows platform

I'm okay with the approach, but I think that we should update the script that generates the mapping.

Ah I didn't realize there was a script that generates the mapping - do you know where that lives?

@swift-ci please test

@swift-ci please test Windows platform

I'm okay with the approach, but I think that we should update the script that generates the mapping.

Ah I didn't realize there was a script that generates the mapping - do you know where that lives?

I could've sworn that I checked that in >.< ... maybe I didn't - it was some XSLT horror. I might've just shoved it into the commit message :(. For now, lets go ahead and get this merged.

Do we want this in release/6.0 too if it's crashing the tests?

Do we want this in release/6.0 too if it's crashing the tests?

Yeah, this causes crashes in things that call down to some CFTimeZone functions (like DateFormatter) so I'll cherry-pick this back to release/6.0 too