Select WASM Interpreter to support in Stage 2

Our main goal for Stage 2 is to generate a ZK circuit (ZKASM program) for a real WASM interpreter. This circuit should take a batch of independent WASM programs as an input and return a batch of their effects as an output.
We need to investigate what would make a WASM interpreter a good fit for this purpose and select concrete candidates that we will try to support.
Concretely, we need to understand:

[ ] Which WASM features should the interpreter support
[ ] Do we need no_std to achieve good binary size and performance?
[ ] Does the interpreter need to have a good execution performance?

In the prior work (https://github.com/near/wasmtime/issues/28) we looked into:

wain (https://github.com/rhysd/wain) - written in Rust
wasmi (https://github.com/paritytech/wasmi/) - written in Rust
I think it's worth redoing the evaluation in the prior work.

I tried to compile wasmi) interpreter using our benchmark framework: https://github.com/akashin/bench/blob/wasmi_interpreter/src/main.rs
The results so far:

std version generated 293k lines of WAT; 726KB of WASM
no_std version generated 231k lines of WAT; 573KB of WASM (I needed to implement global allocator for this to work, but there are only 4 calls to it in the interpreter)

The no_std version contains:

A small number of f32 and f64 instructions
1200 functions
3 global variables
44k global data slots
183 indirect calls
WASM tables and br_table instruction
select instruction

It does not contain:

memory.fill and memory.grow instructions
popcnt instruction
float sqrt, ceil, floor, etc.

I'll investigate next what is necessary to compile that version to ZKASM

I've compiled WASMI to ZKASM in https://github.com/near/wasmtime/pull/202. The resulting ZKASM file is 443k lines (from 231k lines of WASM). This seems manageable. Most importantly, we care about execution time of WASM programs on this interpreter rather than the size of the interpreter itself.
So far, there are only 5 missing instructions:

Select
BrTable
CallInd
fpromote
fdemote

In total, the first 3 are called 1500 times: https://gist.github.com/akashin/88683dd08106ebdf0912172629365cae
The next steps are:

Quickly check how hard would it be to implement those in case they are low-hanging fruits and we can get a working interpreter now
Compile std-version of wasmi
Select and compile another WASM interpreter for comparison

I'm working on std version support and it currently fails when executing the WAT file with the following error:
Err = Err(
    Error {
        context: "error while executing at wasm backtrace:\n    0: 0x5b3a7 - <unknown>!<wasm function 1125>\n    1: 0xc661 - <unknown>!<wasm function 148>\n    2: 0x66dc - <unknown>!<wasm function 34>\n    3: 0x71fd - <unknown>!<wasm function 53>",
        source: Error {
            context: "memory fault at wasm address 0x100000058 in linear memory of size 0x20000",
            source: MemoryOutOfBounds,
        },
    },
)
thread 'test_zkasm::tests::run_wat' panicked at cranelift/filetests/src/test_zkasm.rs:280:25:

The error is quite descriptive, but I can't find a place where we set the memory size to 0x20000.
From out code, the size of the heap should be 0x1_0000_0000: https://github.com/near/wasmtime/blob/41bb3255a24ba3b33708a9fdbdc0803ab9b12c78/cranelift/wasm/src/environ/zkasm.rs#L348
Will continue debugging this.

Some progress, this is not related to the ZKASM environment memory size, it's about running the WAT file using the Wasmtime engine. I was able to change the error message by setting
        let mut config = Config::default();
        config.static_memory_maximum_size(0);
        let engine = Engine::new(&config)?;

in https://github.com/near/wasmtime/blob/41bb3255a24ba3b33708a9fdbdc0803ab9b12c78/cranelift/filetests/src/test_zkasm.rs#L247
but it still fails with
Err = Err(
    Error {
        context: "error while executing at wasm backtrace:\n    0: 0x5b3a7 - <unknown>!<wasm function 1125>\n    1: 0xc661 - <unknown>!<wasm function 148>\n    2: 0x66dc - <unknown>!<wasm function 34>\n    3: 0x71fd - <unknown>!<wasm function 53>",
        source: MemoryOutOfBounds,
    },
)

I'm not that surprised, as the memory access at 0x100000058 is too large anyway for static memory, unless this is a virtual memory address.

I was able to confirm this error both with wasmtime and with wasm-interp:
Error from wasm-interp:
error initializing module: out of bounds memory access: access at<PHONE_NUMBER>+8 >= max value 131072

Error from wasmtime:
Error: failed to run main module `compiled-wasm`

Caused by:
    0: failed to instantiate "compiled-wasm"
    1: error while executing at wasm backtrace:
           0: 0x5b3f6 - <unknown>!<wasm function 1126>
           1: 0xce50 - <unknown>!<wasm function 160>
           2: 0x66c2 - <unknown>!<wasm function 33>
           3: 0x71e8 - <unknown>!<wasm function 53>
           4: 0x71be - <unknown>!<wasm function 52>
    2: memory fault at wasm address 0x100000058 in linear memory of size 0x20000
    3: wasm trap: out of bounds memory access

So this has to do with the way we compile Rust to WASM. Will start tweaking the settings to fix this.

I was able to fix the problem by increasing the stack size:
[target.wasm32-unknown-unknown]
rustflags = [
  # "-C", "link-args=-z stack-size=2048",
  "-C", "link-args=-z stack-size=4096",
]

This should allow us to add WASMI as a benchmark to our codebase.

I've updated the benchmark https://github.com/near/wasmtime/pull/202/commits/cb687a384bdc9b2584b3047829a30943d12cd5e1 and now it fails with:
thread 'test_zkasm::tests::run_benchmarks' panicked at /Users/akashin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/regalloc2-0.9.3/src/ion/process.rs:1174:13:
assertion failed: lowest_cost_split_conflict_cost.is_some() ||
    lowest_cost_evict_conflict_cost.is_some()

This is a part of https://github.com/bytecodealliance/regalloc2, concretely this place: https://github.com/bytecodealliance/regalloc2/blob/ea86b8740f427a4e20f71e80727ff18985ae8f11/src/ion/process.rs#L1199
Looks like we'll need more detailed logs to debug this further.

I've also recently discovered a new tool wasm-stats and ran it on the resulting WASMI interpreter: https://gist.github.com/akashin/352793b47dc8ae1778261e4fc5312617
Here is the breakdown of top20 instructions:
Opcode counts:
local.get: 71492
i32.const: 48239
i32.add: 23429
local.set: 16986
end: 16427
i32.load: 15149
block: 13453
local.tee: 11128
i32.store: 10937
br_if: 10574
call: 9188
br: 7052
i32.eqz: 3272
i64.store: 3067
i32.load8_u: 2439
i64.load: 2373
i64.const: 2141
global.set: 1933
i32.shl: 1786
unreachable: 1628

It's also interesting to see the breakdown of operations with immediates:
Opcode counts with immediates:
i32.add: 23429
end: 16427
block: 13453
local.get 2: 12373
local.get 0: 12105
local.get 1: 12009
local.get 3: 9115
local.get 4: 7663
br_if 0: 6577
i32.load 2, 0: 5833

I've updated the benchmark cb687a3 and now it fails with:
thread 'test_zkasm::tests::run_benchmarks' panicked at /Users/akashin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/regalloc2-0.9.3/src/ion/process.rs:1174:13:
assertion failed: lowest_cost_split_conflict_cost.is_some() ||
    lowest_cost_evict_conflict_cost.is_some()

This is a part of https://github.com/bytecodealliance/regalloc2, concretely this place: https://github.com/bytecodealliance/regalloc2/blob/ea86b8740f427a4e20f71e80727ff18985ae8f11/src/ion/process.rs#L1199
Looks like we'll need more detailed logs to debug this further.

I was able to debug this by enabling trace-log feature on regalloc2 crate and studying the logs just before the crash:
[2024-02-19T11:28:31Z TRACE regalloc2::ion::process] process_bundle: bundle LiveBundleIndex(9514) hint PReg(hw = 63, class = Int, index = 63)
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement] compute_requirement: LiveBundleIndex(9514)
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement]  -> LR LiveRangeIndex(12493): CodeRange { from: progpoint3119-post, to: progpoint3123-pre }
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement]   -> use Use { operand: Def: v23180f reg, pos: progpoint3119-post, slot: 1, weight: 35828 }
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement]      -> req Register
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement]   -> use Use { operand: Use@Late: v23180f reg, pos: progpoint3122-post, slot: 1, weight: 35703 }
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement]      -> req Register
[2024-02-19T11:28:31Z TRACE regalloc2::ion::requirement]  -> final: Register
[2024-02-19T11:28:31Z TRACE regalloc2::ion::process] attempt 1, req Register
[2024-02-19T11:28:31Z TRACE regalloc2::ion::process]  -> lowest cost evict: set None, cost None
[2024-02-19T11:28:31Z TRACE regalloc2::ion::process]  -> lowest cost split: cost None, point progpoint0-pre, reg PReg(hw = 63, class = Int, index = 63)

Concretely, the name of the register involved is v23180f which is a floating point register (see suffix f) which we have disabled in zkasm backend. By adding 1 floating point register, I get an error further down, which totally makes sense: not implemented: Float reg saved in prologue.
I've added naive implementation for floating points and the code now fails at:
thread 'test_zkasm::tests::run_benchmarks' panicked at cranelift/codegen/src/machinst/lower.rs:766:21:
should be implemented in ISLE: inst = `v10 = symbol_value.i64 gv0`, type = `Some(types::I64)`
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I'm not sure what that error means, will investigate further.

Fixed the error above by defining the ISLE rule for symbol_value, a few errors later, got to another register allocation problem:
[2024-02-19T13:09:04Z ERROR cranelift_codegen::machinst::compile] Register allocation error for vcode
    VCode {
      Entry block: 0
      v194 := v192
      v195 := v193
      v196 := v204
      v197 := v196
    Block 0:
        (original IR block: block0)
        (successor: Block 1)
        (instruction range: 0 .. 7)
      Inst 0: args v192=A v193=B
      Inst 1: SUB v200,v199,v198
      Inst 2: srl v201,v194,v198
      Inst 3: sll v202,v194,v200
      Inst 4: select_reg v203,0,v202##condition=(v198 eq 0)
      Inst 5: OR v204,v201,v203
      Inst 6: j label1
    Block 1:
        (original IR block: block1)
        (instruction range: 7 .. 8)
      Inst 7:   :JMP(RR) v197=A
    }

    Error: SSA(VReg(vreg = 199, class = Int), Inst(1))
    CLIF for error:
    function u0:1430(i64, i64) -> i64 fast {
                                    block0(v0: i64, v1: i64):
    @8ce4a                              v3 = ireduce.i32 v0
    @8ce4d                              v4 = ireduce.i32 v1
    @8ce4e                              v5 = rotr v3, v4
    @8ce4f                              v6 = uextend.i64 v5
                                        v2 -> v6
    @8ce50                              jump block1

                                    block1:
    @8ce50                              return v2
    }

thread 'test_zkasm::tests::run_benchmarks' panicked at cranelift/codegen/src/machinst/compile.rs:76:14:
register allocation: SSA(VReg(vreg = 199, class = Int), Inst(1))

Maybe it has to do something with select_reg instruction?

I'm working through enabling necessary Float register support: https://github.com/near/wasmtime/pull/202/commits/6c1eb613f3ba15fef2446f1b14da841bd6f50c48
Right now it fails with another register allocation issue:
thread 'test_zkasm::tests::run_benchmarks' panicked at cranelift/codegen/src/machinst/compile.rs:76:14:
register allocation: SSA(VReg(vreg = 945, class = Float), Inst(407))

From the context, I see that Instruction 407 refers to "fsw" which corresponds to Storing f32:
Inst 404: AND v611,v609,v610
Inst 405: load_sym v608,userextname0+0
Inst 406: ADD v607,v557,v556
Inst 407: fsw v205,4(v558)
Inst 408: auipc v606,0; ld v606,12(v606); j 8; .4byte 0x3f
Inst 409: load_sym v605,userextname0+0
Inst 410: ADD v604,v563,v562

For some reason, vreg 945 is not defined, even though it is ought to be.

I fixed the previous problem by defining fpromote and fdemote. The next error is:
thread 'test_zkasm::tests::run_benchmarks' panicked at cranelift/codegen/src/machinst/compile.rs:76:14:
register allocation: SSA(VReg(vreg = 23180, class = Float), Inst(3123))

v7016 = load.f32 little heap v7015
v7017 = fpromote.f64 v7016
v7018 = uextend.i64 v92
v7019 = symbol_value.i64 gv1
v7020 = iadd v7019, v7018
store little heap v7017, v7020

Inst 3119: MLOAD f32 v23182,4(v6522)
Inst 3120: 0 => v23181  ;; LoadConst64
Inst 3121: load_sym v23179,userextname0+0
Inst 3122: ADD v23178,v6528,v6527
Inst 3123: MSTORE f64 v6526,0(v6529)

I was able to narrow down the problem to a simple CLIF function:
test compile precise-output
target zkasm-unknown-unknown

function u0:1(i32 vmctx) fast {
    block0(v0: i32):
        v1 = iconst.i64 4
        v2 = load.f32 little heap v1
        v3 = fpromote.f64 v2
        store little heap v3, v1
        jump block1

    block1:
        return
}

Now it fails with:
 ERROR cranelift_codegen::machinst::compile > Register allocation error for vcode
VCode {
  Entry block: 0
  v193 := v199
  v194 := v198
  v195 := v196
Block 0:
    (original IR block: block0)
    (successor: Block 1)
    (instruction range: 0 .. 6)
  Inst 0: args v192=A
  Inst 1: 4 => v199  ;; LoadConst64
  Inst 2: MLOAD f32 v198,0(v193)
  Inst 3: 0 => v197  ;; LoadConst64
  Inst 4: MSTORE f64 v195,0(v193)
  Inst 5: j label1
Block 1:
    (original IR block: block1)
    (instruction range: 6 .. 7)
  Inst 6:   :JMP(RR)
}

Error: SSA(VReg(vreg = 196, class = Float), Inst(4))
CLIF for error:
function u0:1(i32 vmctx) fast {
block0(v0: i32):
    v1 = iconst.i64 4
    v2 = load.f32 little heap v1  ; v1 = 4
    v3 = fpromote.f64 v2
    store little heap v3, v1  ; v1 = 4
    jump block1

block1:
    return
}

thread 'worker #2' panicked at cranelift/codegen/src/machinst/compile.rs:76:14:
register allocation: SSA(VReg(vreg = 196, class = Float), Inst(4))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

on this line in regalloc2: https://github.com/bytecodealliance/regalloc2/blob/ea86b8740f427a4e20f71e80727ff18985ae8f11/src/ssa.rs#L69


Consistently treat let properties as immutable within the type checker

There are a number of places where references let properties are treated as lvalues by the type checker, and we rely on Definite Initialization to detect and diagnose inout uses of these properties. This includes:

let instance properties of a type within an initializer.
Local let properties that aren't initialized at point of declaration.

The result of modeling these as lvalues in the type checker is that we could end up selecting a mutating function for a let base, which would then get diagnosed in Definite Initialization. If there was an alternative non-mutating overload that is otherwise not preferred, it will essentially get ignored.
Rework our handling of let properties so that they are treated as rvalues uniformly except when they are being assigned to. This moves the inout/mutating checking up into the type checker, allowing overload resolution to choose appropriately.
Fixes rdar://127258363, a source compatibility issue with AsyncIteratorProtocol.next() that uncovered this issue.

@swift-ci please test

@swift-ci please test source compatibility

https://github.com/apple/swift-syntax/pull/2653
@swift-ci please test

https://github.com/apple/swift-syntax/pull/2653
@swift-ci please test source compatibility

@swift-ci please smoke test

@swift-ci please test source compatibility

@swift-ci please smoke test

@swift-ci please test source compatibility

@swift-ci please clean smoke test macOS

@swift-ci please clean test source compatibility

@swift-ci please test source compatibility


Newly created application/bot in

What went wrong?
traceback (most recent call last):
File "/root/MusicBot/musicbot/bot.py", line 1074, in run
raise exceptions.HelpfulError(
musicbot.exceptions.HelpfulError: ('Bot cannot login, bad credentials.', 'Fix your token in the options file.  Remember that each field should be on their own line.')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/MusicBot/musicbot/bot.py", line 1082, in run
self._cleanup()
File "/root/MusicBot/musicbot/bot.py", line 1057, in _cleanup
pending = asyncio.all_tasks()
File "/usr/lib/python3.8/asyncio/tasks.py", line 44, in all_tasks
The new discord bot tokens are much longer, so they can allow more bots in their system.
"the length of the discord tokens has changes over the time because of the decoded timestamp in the Base64 format in the token."
Example:
MTA0ODM0MTA0MTk2ODg1NzEyOA.GDoWzA.rn4-oX7-rYJB_FTG-vBkrWzq3Hyn1TUY1214512
(I changed some of the characters at the end, and this was previously before I generated a new token anyway)
Is there a way to make it authenticate with the longer/newer discord bot tokens?
All of my old apps/bots created in discord developer work, but this new one with new token I am using will not work.
How do we reproduce this?
Make a new application and add a bot in discord developer and use one of the newer tokens.
Which version of the bot are you using?
release-260419
Which operating system are you using?
Ubuntu 18.04 or higher
Just checking...

[X] I have confirmed that my issue has not already been reported.
[X] I am using Python 3.8 or higher.
[X] I have updated my Python dependencies.

You can close this, it was an issue with writing to the file options.ini, I just noticed the token was much longer than any of the older bot/apps in discord development and thought that may have been it.


Progressive migration not working with Next.js v14.2.5, keep getting the error "Error: No QueryClient set, use QueryClientProvider to set one"

Hi all, I've been trying to do progressive migration as mentioned here in the doc: https://portal.thirdweb.com/typescript/v5/migrate#progressive-migration. However it doesn't seem to work in Next.js. It seems I can only use one SDK, either v4 or v5 and can't use both together.
-> I tried installing and using the QueryClientProvider from @tanstack/react-query to avoid getting the QueryClientProvider error but the issue still persists.
(package.json file)
{
"name": "amazingapp",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@tanstack/react-query": "^5.51.21",
"@thirdweb-dev/react": "^4.9.4",
"@thirdweb-dev/sdk": "^4.0.99",
"ethers": "^5.7.2",
"next": "14.2.5",
"react": "^18",
"react-dom": "^18",
"thirdweb": "^5.43.2"
},
"devDependencies": {
"@types/node": "^20",
"@types/react": "^18",
"@types/react-dom": "^18",
"eslint": "^8",
"eslint-config-next": "14.2.5",
"postcss": "^8",
"tailwindcss": "^3.4.1",
"typescript": "^5"
}
}
This is my app structure:
(layout.tsx file)
`import { Inter } from "next/font/google";
import "./globals.css";
import { ThirdwebProviderV4 } from "./ThirdwebProviderv4";
import { ThirdwebProvider } from "thirdweb/react";
const inter = Inter({ subsets: ["latin"] });
export const metadata = {
title: "Create Next App",
description: "Generated by create next app",
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
< html lang="en" >
< body className={inter.className} >
< ThirdwebProviderV4 clientId="clientId" >
< ThirdwebProvider>{children}
</ ThirdwebProviderV4 >
</ body >
</ html >
);
}`
(ThirdwebProviderV4.tsx file)
`"use client";
import { ThirdwebProvider } from "@thirdweb-dev/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { useState } from "react";
export function ThirdwebProviderV4({
children,
clientId,
}: {
children: React.ReactNode;
clientId: string;
}) {
const [queryClient] = useState(() => new QueryClient());
return (
< QueryClientProvider client={queryClient} >
< ThirdwebProvider
clientId={clientId}
autoConnect={false}
autoConnectTimeout={100}
>
{children}
</ ThirdwebProvider>
</ QueryClientProvider>
);
}
`

I am able to use the v5's ConnectButton component in a page.
The v4's ConnectWallet button renders but doesn't connect to any wallet. When you click on any wallet inside the modal (for example, MetaMask) then it gives the error  "Error: No QueryClient set, use QueryClientProvider to set one". And if I don't use 'use client' then v4 ConnectWallet button will not even work.
Cannot use both ConnectWallet and ConnectButton components in a single file.

Hi @codingfarhan thanks a lot for reporting this issue with such details!
I was able to replicate the issue, and I have prepared this repo that might be what you are looking for ^
https://github.com/kien-ngo/progressive-migration
So the trick here is, if you want to use 2 react-query versions at the same time, you can install the react-query v5 with an alias:
npm install react-query-v5:npm:@tanstack/react-query@latest

Then your package.json will look like this:
"dependencies": {
  "@tanstack/react-query": "^4",
  "@thirdweb-dev/react": "^4.9.4",
  "@thirdweb-dev/sdk": "^4.0.99",
  "ethers": "5.7.2",
  "next": "14.2.5",
  "react": "^18",
  "react-dom": "^18",
  "react-query-v5": "npm:@tanstack/react-query@latest",
  "thirdweb": "^5.44.1"
},

Thanks a lot for answering @kien-ngo ! It looks like it's working perfectly :)
Btw just out of curiosity, is there a way to use both v4 and v5 together without installing extra dependencies like tanstack or react-query-v5? Or is this the only way to make them work together?
Cheers.

I'm afraid you'll need both! But the point of the migration is you'll eventually get rid of the outdated one ^

Right, makes sense,


clients/egui: remove eframe as lib dependency

fixes https://github.com/lockbook/lockbook/issues/3131


fix(messagelistener.ts): generate events for custom domain

Generate events when fullscript client is initialized with a custom domain

LGTM!

LGTM!!


Adding Java instructions for MSI in functions

@yevster : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@yevster : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Great - thanks!
#sign-off

Only Microsoft employees can sign off on PRs. Your PR will be routed for review.


Unable to modify Role permissions

I am already working on this. It is because all the roles are created with all permission given. Once this is done then modification of permission is not possible. Changing the code and will update the changes in a while.


Bug: redirect_to redirecting via HTTP instead of HTTPS

I am experiencing the same error as was solved in Turbolinks-classic here, yet I am using TL 5.0.1.
Was the fix they included in TL-classic also included in TL 5? Or is this an outstanding issue with TL 5?
The only way I got it working was to disable TL on that 1 link, so it certainly was a TL issue.

Hi,
I also found this bug in 5.0.1.

I am also experiencing this problem with turbolinks 5.0.1 regardless of redirecting to a _url or _path generated method.  And the workaround was as described above to add:
data: { turbolinks: false }
to the call to link_to(...)

I am experiencing this bug on 5.1.1 as well

I've not been able to reproduce this. Here is an app with a link which redirects to another path: https://ancient-castle-17652.herokuapp.com. The redirect works as expected and is all done over HTTPS.
Are you still experiencing this, and if so, could you add some more details about your case (preferably with a live demo)? Thanks.


Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'map')

Description
Hello,
I am a brand new TypeSense user. I am trying to move my existing Docusaurus website from Algolia --> TypeSense. I followed the steps from the documentation, and I cannot get it working.
My Docusaurus site is published to: https://isaacscript.github.io/
When attempting to type anything into the search, I get the following error in the JavaScript console:
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'map')
    at t.value (37702.a034fb89.js:2:3949)
    at t.value (37702.a034fb89.js:2:8672)
    at 90888.529c7e8f.js:1:56429
    at p (37702.a034fb89.js:2:105125)
    at Generator.<anonymous> (37702.a034fb89.js:2:106462)
    at Generator.next (37702.a034fb89.js:2:105488)
    at jt (90888.529c7e8f.js:1:55241)
    at i (90888.529c7e8f.js:1:56585)

More Info
I am self-hosting TypeSense. I can see what the version is from the log file:
I20230208 07:07:05.562175 480505 typesense_server_utils.cpp:357] Starting Typesense 0.24.0

My "typesense-server.ini" looks like this:
; Typesense Configuration

[server]

api-address = <IP_ADDRESS>
api-port = 8108
data-dir = /var/lib/typesense
api-key = 9AiKF0AGD145wC9fc4NafATcCam89XT7v2NxGf2ymFucN7b6
log-dir = /var/log/typesense

And the config in my "docusaurus.config.js" file is this:
      typesense: {
        typesenseCollectionName: "isaacscript",
        typesenseServerConfig: {
          nodes: [
            {
              host: "isaacracing.net",
              port: 8108,
              protocol: "https",
            },
          ],
          apiKey: "9AiKF0AGD145wC9fc4NafATcCam89XT7v2NxGf2ymFucN7b6",
        },

        typesenseSearchParameters: {},

        contextualSearch: true,
      },

For reference, the "docusaurus.config.js" file is located here:
https://github.com/IsaacScript/isaacscript/blob/main/packages/docs/docusaurus.config.js
Furthermore, I am using the following version of the plugin, which is located in my "package.json" file:
    "docusaurus-theme-search-typesense": "^0.9.0",

For reference, the "package.json" file is located here:
https://github.com/IsaacScript/isaacscript/blob/main/package.json

Turns out this was because I was incorrectly specifying the name of the collection in my Docusaurus settings. I propose that the runtime error should be changed to something more appropriate, like:
Failed to query the remote collection of "foo". Make sure that the configured collection name matches what is actually present on the remote Typesense server.


Relisten not playing Phish on Sonos

I've had Sonos for over a year and never had any issues playing Phish via the Relisten music service on my Sonos app, until recently. Every other band on Relisten will play thru my Sonos system, except for Phish. Any track selected will not play with error "Unable to play (song title)-the connection with Relisten was lost. If I select any other band GD, JRAD, etc everything plays fine. Thought I would post here to see if there's some bug. I can play Phish tracks via Relisten on my mobile device Chrome browser as well as on my laptop with no issue, only problem is trying to play on my Sonos speakers thru the Sonos app. I have Sonos support looking into it as well.

This is due to the fact that Phish.in began enforcing SSL (https addresses on all requests) and the site currently uses CloudFlare as a CDN.  Sonos apparently does not support CloudFlare's SSL setup.  Even using a self-signed cert on the Phish.in server itself does not solve the issue.
I'm asking folks to contact Sonos and encourage them to support CloudFlare's (as I understand it, pretty common) SSL scheme.

This should now be fixed due to efforts by @alecgorge on the Relisten side of things.  🎉


Completed Binance IOTA transfer not reflected to Firefly wallet for 45 mins. TX ID: 1dc8e3e473cf7892b90a60f1a2c63eabeaceee7a136151a5ce42a8daf8e0085a

TX ID : 1dc8e3e473cf7892b90a60f1a2c63eabeaceee7a136151a5ce42a8daf8e0085a
WAllet address : iota1qryg79ly5duhr4qp52x9edf73ej9hlx8m3xcxe6su5nawzgzf75p6ng7uun

Have you already synced your wallet? Do you use a ledger device?

ı dont know how to syncronize and no I didnt use a ledger device. just downloaded firefly for windows and did the setup

I have the same issue, i have sent iota from indodax exchange to firefly wallet, and about 3 hours ago, i didnt receive to my firefly wallet, i did it twice to make sure the trx but it isnt not received yet.
TX id 9e6feaa5fe14df86f4f186a442d76c822fef15e36d834381c8463b56ad1e4ba5
wallet address: atoi1qre9jzfjffpnrq6tfm8wdvkvtrvfh0mhv2jeruh79u4cf20xgfe4qver78p
Please help how to solve this issue?

@Oflezer sir, is your problem solved? Please share thx

@kanggarya You have used a developer profile. Therefore your address starts with atoi... The mainnet addresses all start with iota...
If the withdrawal was completed by your exchange try to create a new profile in Firelfy and DO NOT use developer profile. Then restore your profile with the recovery phrase

@MiKeF05 thx sir, my problems solve, i unchecked developer and update my profile

@Oflezer

@Oflezer can you please join our Discord and ask for support in the #help channel?


Ability to mock providing QR codes to emulator

Is your feature request related to a problem? Please describe.
Example: I have a lot of userflows that depend on users scanning QR codes with their devices.
(sometimes QR codes are animated, like, several frames, to transmit more data).
Describe the solution you'd like
It would be awesome if I could provide detox an image with QR code, and this image could be like, in front of a camera of emulator, so next time my app launches camera and looks for QR - it will scan this exact QR.
Describe alternatives you've considered

Additional context

I am not familiar with any OS-standard ways of scanning QRs. So this cannot really be achieved in Detox. You should have this mocking as part of your test logic, and perhaps you can open sources it so that people get ideas how to do it in their apps. But I just don't see how this can be done in a general-purpose way.


Update AdvisedRequest.java

systemMessage should come first index

Thanks for your PR! We have an issue tracking how to improve the way system messages are handled in ChatClient: https://github.com/spring-projects/spring-ai/issues/873 I would suggest having a conversation there before making any change to make sure all use cases that have been identified are covered.


Complete 0.3 enclosed build

What has to be done
This is a collector issue - for all tasks under enclosure build.
This encompasses the building of two complete prototypes, one in Maryland and one in California.
The sub-issues for each build are as follows:

Component
MD build ticket
CA build ticket

back panel
#785
#786

filter panel
#790
#791

display panel
#788
#789

pneumatics
#781
#782

main enclosure
#783
#784

final integration
#792
#793

How do you know it has to be done
Design review of beta build is completed, all lesson learned identified and issues/actions identified (in Git)
#787
Other
Tasks to be identified to support enclosure build.
20200708-Respiraworks V3 Enclosure Package (1).zip

Enclosure ordered - July 24th with PegaPrecision.  Enclosure manufacturing to be completed August 21.

All constituent tickets are closed. This umbrella issue is therefore also done.


Lazier automatic casts when used in macros

It would be nice to be able to do this (carc.in):
enum Foo
  One
  Two
end

class HashWrapper
  property hash = {} of String => Foo

  forward_missing_to @hash
end

hw = HashWrapper.new
hw.hash["x"] = :One # This works
hw["y"] = :Two      # This doesn't

In the second line, :Two doesn't get cast because it's passed to a macro, not directly to a method that takes a Foo. Would it be possible and wise to defer automatic casts until after macros are expanded? The downside I can think of is that macro authors can no longer trust their type checks when they look for SymbolLiterals, but that doesn't seem too common.

This doesn't have anything specific with to do with calling the macro forward_missing_to.
I don't this can be solved.
I'd recommend explicitly declaring the forwarded methods anyway. This clearly describes the exposed API and works with responds_to?.

Thanks, you're right. The thing I'm asking for is already the current behavior. My bad.


Selenium/WebDriver (WIP: DO NOT MERGE)

This PR SHOULD NOT BE MERGED. It is a raw commit from the state of my tree at the end of the spike.

This PR documents the work I did investigating how we may integrate Selenium/Webdriver into our testing as outlined in #144. A brief summary is below followed by a list specific parts of the PR I think are of utility and ways we may wish to use them.
Summary conclusion
I propose that we write a stand alone end-to-end test suite which is given a URL and tests the instance of the application at that URL. This will be done as an anonymous user for the moment. The test suite can be run nightly against the test deployment on CircleCI. (This should be done in the media-deploy repo's CircleCI config.) While we're doing that, doing a nightly deploy to test would be a sensible thing to do as well.
Although it is tempting to make end-to-end testing part of the tox test suite, I think the mechanism which is being used in this PR is brittle and prone to failure. Instead, I think it would be better to have a stand-alone end-to-end test suite which can be pointed at a deployed instance or a local development instance brought up via ./compose.sh (development|production) up.
Detail
Of most interest is the new testutils application which provides a decorator for unit tests allowing them to be passed a selenium Webdriver instance for each browser we are interested in. Browserstack credentials are specified via the TESTUTILS_WEBDRIVER_COMMAND_EXECUTOR environment variable. (See the Browserstack docs for how to find this URL.)
In addition, a new container is added to the tox docker-compose configuration in order to allow Browserstack browsers to talk to live servers run via tox.
As a matter of plumbing, some work was done to parameterise the production Dockerfile to modify it slightly when running the test suite. Care needs to be taken here because part of the reason we want to run the tests in the production container is that we are also testing the actual container we are deploying. Having the tox container differ from production also has implications for CircleCI build times.
Things of use
For reasons outlined above, much of the plumbing work is probably of little utility. Of greater utility is the code in the testutils module. This can be split out into a separate end-to-end test suite which can be run against a deployed instance.
Problems
Testing a deployed instance has its own problems re authentication. Ideally we'd want to be able to test authenticated user flows as well but that has a problem in that robot accounts are currently difficult/impossible to obtain for Raven. To that end, I think we need to add a different authentication mechanism, perhaps a username/password form, which can authenticate non CRSID users. We can then add bot accounts to the deployment for testing. This has a knock-on effect for getting information on users from lookup.
At a minimum, however, there are no blockers on us writing a minimal "does it render" test for each page reachable by anonymous users.

Totally agree with the conclusion. Closing the PR then but keeping the branch for future reference.


feat: graphql api support with codgen

GraphQL apis available with codegen.

Codecov Report

Merging #6 (923aa84) into main (b3416f8) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 923aa84 differs from pull request most recent head 4eaa0ab. Consider uploading reports for the commit 4eaa0ab to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@           Coverage Diff           @@
##             main       #6   +/-   ##
=======================================
  Coverage   90.80%   90.80%
=======================================
  Files           1        1
  Lines          87       87
  Branches        5        5
=======================================
  Hits           79       79
  Misses          8        8

Flag
Coverage Δ

integration
90.80% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Fix 368 - Lacheck reports errors from input{} files.

This PR adds additional check to lacheck linter to exclude any warnings
related to sourced files via latex \input{} command.
Closes: #368

Added some tests to lacheck linter. It actually had no tests at all.
I tried to get the absolute paths of the files reported by lacheck but was more difficult that expected. Once we change current paths lacheck will report all inputted files as "Cannot open".

Cheers! :beers:


Access the Service via ingress and traefik

I'm trying to get the service running via Ingress and traefik but nothing seems too work. Q1 is it even supportet Q2 If not, will it be?

Same. I've got access to the site, but no page fully loads. I'm guessing 4326 is hard-coded somewhere, because all pages just sit at the loading dots. I've attached my stack for reference.
version: "3.7"

services:
  web:
    image: itzg/rcon:latest
    volumes:
    - database:/opt/rcon-web-admin/db
    environment:
    - RWA_PASSWORD=MyPass
    deploy:
      labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.http.routers.rcon.rule=Host(`rcon.my.domain`)
      - traefik.http.routers.rcon.tls=true
      - traefik.http.routers.rcon.tls.certresolver=le
      - traefik.http.services.rcon.loadbalancer.server.port=4326
      restart_policy:
        condition: on-failure
        delay: 60s
        max_attempts: 3
        window: 1200s
      placement:
        constraints:
          - node.role==manager
    networks:
      - traefik-public

volumes:
  database:

networks:
  traefik-public:
    external: true
    name: traefik-public

I ran into this issue too and found a solution.
My setup might be not the same as yours (no ingress), but basically it should be the same problem.
The thing is that the service is listening on two ports: 4326 for the web interface and 4327 for web socket connection.
You have to tell traefik somehow to send http requests to 4326 (what you already might have done) and the ws (websocket) requests to 4327.
Luykily there is already the possibility to configure the urls for the websocket  as you can see from the readme:
RWA_WEBSOCKET_URL and RWA_WEBSOCKET_URL_SSL
So one (maybe not the best, at least working for me) solution is

to change the RWA_WEBSOCKET_URL_SSL (or RWA_WEBSOCKET_URL if you'r using http instead of https) and add a sub path e.g.
RWA_WEBSOCKET_URL_SSL=https://rcon.my.domain/wss (or whatever you domain is)

Add a second traefik rule (besides the default http/s rule) to proxy everything with path /wss  to port 4327
Example:

      - "traefik.http.routers.mcwebwss.rule=Host(`rcon.my.domain`) && Path(`/wss`)"
      - "traefik.http.routers.mcwebwss.entrypoints=<entrypoint>"
      - "traefik.http.routers.mcwebwss.tls.certresolver=<certresolver>"
      - "traefik.http.routers.mcwebwss.service=mcwebwss"
      - "traefik.http.services.mcwebwss.loadbalancer.server.port=4327"

(<entrypoint> and <certresolver> are placehoulders, they are custom settings)
As you have two routers now for one container, you have to tell which router belongs to which service:
 - "traefik.http.routers.mcwebwss.service=mcwebwss"
Hope it works for you as well.

As a note, Websockets expect their URLs to start with wss:// instead of https:// and that was causing some problems for me.


LINK-1809 |  Solve snyk warnings

Description
Bump axios to 1.6.2
Closes
LINK-1809

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6ce5714) 99.61% compared to head (1941cc7) 99.61%.
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #117   +/-   ##
=======================================
  Coverage   99.61%   99.61%
=======================================
  Files         188      188
  Lines        3631     3631
  Branches      595      595
=======================================
  Hits         3617     3617
  Misses         13       13
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add jump links to the application

Add jump links to the header for navigating to Hackathon, Conferences & Start up events section.
Add it to header-component.

Hello @geekyd i'll like to handle this

@danidee10 Sure go ahead!


Digest analysed-drive-statistics

Leverage Disk stats by processing "GET /storage-systems/{system-id}/analysed-drive-statistics"

https://github.com/plz/E-Series-Graphite-Grafana/commit/413c78bcd263c08f34afaf067da8e6c8dca12e87
Enabled the collection of these metrics.
Still have to create a dashboard and make a flag to opt out

Pushed an initial version to show these metrics: https://github.com/plz/E-Series-Graphite-Grafana/commit/9ecb8cda84150d27e99ddd82a16613fc3d1c97bf


refactor: minor code refactoring

Related issues
N/A
Short description and why it's useful
Since the minimum Node.js release supported is v8+, it definitely makes sense to use Array.includes() to check for existence.
Screenshots of visual changes before/after (if there are any)
N/A
Which environment this relates to
Check your case. In case of any doubts please read about Release Cycle

[ ] Test version (https://test.storefrontcloud.io) - this is a new feature or improvement for Vue Storefront. I've created branch from develop branch and want to merge it back to develop
[x] RC version (https://next.storefrontcloud.io) - this is a stabilisation fix for Release Candidate of Vue Storefront. I've created branch from release branch and want to merge it back to release
[ ] Stable version (https://demo.storefrontcloud.io) - this is an important fix for current stable version. I've created branch from hotfix or master branch and want to merge it back to hotfix

Upgrade Notes and Changelog

[x] No upgrade steps required (100% backward compatibility and no breaking changes)
[ ] I've updated the Upgrade notes and Changelog on how to port existing VS sites with this new feature

Contribution and currently important rules acceptance

[x] I read and followed contribution rules

@jamesgeorge007 we will not support node v8+ anymore starting from release 1.11 as node 8 becoming end of life

@andrzejewsky But still Array.includes() is the way to go, indexOf()  is sort of obselete.


Stream support domain name.

Hello, thank you for developing this project.
Now, I have a required function.
I may often use Stream as a proxy.
However, I use docker deployment, and only open ports 80 and 443.
After the container is started, it is not so easy to modify the exposed port, but the current Stream can only listen other ports, so once you need to add Stream, you need to modify and restart the container.
Can the Stream support domain name forwarding?

Routing streams via domain names is generally not possible with Nginx. Because Nginx needs to know what the target domain is to route it to the correct server. The HTTP protocol does contain the domain name, which is why it is easy to do this for websites. But streams are on a lower network level, where the information about the target domain is simply not there, because that depends on the protocol. There is a feature called SNI, but it really depends on what data is being transmitted via the stream host. If it is HTTPS, it's theoretically possible (see https://github.com/jc21/nginx-proxy-manager/issues/853#issuecomment-941035257), but most protocols do not disclose the target domain names.
So this is first and foremost a limitation of how networking works and of nginx, since npm is only an interface to generate nginx configs, we can only do what nginx actually supports.


Enable default inclusion of views in MetaData object (SQL database)

Feature description
Please enable the MetaData.reflect function in dlt to include views by default when accessing database resources. Currently, views are not included by default, requiring additional steps to configure, leading to potential oversight.

Yes, I run dlt in production.
Use case
When extracting and loading views from a SQL database, users need to manually enable view inclusion by setting MetaData.reflect(bind=engine, views=True) method. This additional step can be overlooked and is not initially intuitive.
Proposed solution
Modify the default behavior of the MetaData object to include views by default by setting views=True in the MetaData.reflect method. This would streamline processes for users who manage databases and want to load views.
Related issues
No response

To implement that correctly:

check that sql_table can extract a view by writing a test even without setting any flags in sql alchemy
add a flag to sql_database to include views in reflection (default: false)
write tests that check if the flag works


[TC-466] Delivery-service tenancy based access control - statidnsentry

We have recently added "tenancy" to the project.
With tenancy, every resource have a tenant, where resource can be a delivery-service, a server (future) and even a user.
We are now starting to enforce access-control based on the resource tenancy. A user can manage a resource only if the resource is under the user tenancy.
This JIRA deals with another step of "delivery-service as a resource" - enforcing access control on static-dns-entry, according to their Delivery service.
We still need to finalize the specification of this access control, and it would not be included in the first phase of tenancy introduction, assuming that the application hide from the user the ability to deal with entries of DSes which are not under his tenancy.
Author: Nir Sopher
JIRA Link: https://issues.apache.org/jira/browse/TC-466

PR #2609 provides the ability to CRUD static dns entries for a delivery service so i "hope" it checks tenancy.


Status bar doesn't update when layer selection changes

🐛 Bug

The status bar at the bottom of the window shows the value of the pixel under the hovering cursor in the current image layer. As the cursor is moved around in the viewer, this value changes. When selecting a different image layer, the value should change to be the pixel value in this layer, but it doesn't. It only updates once the cursor is moved.
To Reproduce
Steps to reproduce the behavior:

Open two different image layers.
Click on one layer to select it.
Move the mouse to hover over a pixel in the viewer.
Press the up or down key to select the other layer.
The status bar fails to update when the layer selection is changed. It doesn't update until the cursor is moved.

Expected behavior

When the selected image layer is changed, the status bar should update and display the new pixel value. It seems there is a missing callback (to ViewerModel's _update_status_bar_from_cursor?) that would update the status bar when the LayerList selection changes.
Environment
napari: 0.4.17rc8
Platform: Windows-10-10.0.19045-SP0
Python: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 05:59:45) [MSC v.1929 64 bit (AMD64)]
Qt: 5.15.6
PySide2: 5.15.6
NumPy: 1.23.4
SciPy: 1.9.3
Dask: 2022.10.2
VisPy: 0.11.0
magicgui: 0.6.0
superqt: 0.3.8
in-n-out: 0.1.5
app-model: 0.1.0
npe2: 0.6.1
OpenGL:

GL version: 4.6.0 - Build <IP_ADDRESS>65
MAX_TEXTURE_SIZE: 16384

Screens:

screen 1: resolution 1920x1080, scale 1.0
screen 2: resolution 1920x1200, scale 1.0

Additional context

First issue, hope for many more to come. 😃 Napari is great, amazing work!

This function
https://github.com/napari/napari/blob/95356e45ef7573e6e5f48abde0639388a0cea007/napari/components/viewer_model.py#L460
It should be not only connected to mouse move event:
https://github.com/napari/napari/blob/95356e45ef7573e6e5f48abde0639388a0cea007/napari/components/viewer_model.py#L232-L234
But also in the event of a change selection of layers.
Like here:
https://github.com/napari/napari/blob/95356e45ef7573e6e5f48abde0639388a0cea007/napari/components/viewer_model.py#L239
or called at the end of this function
https://github.com/napari/napari/blob/95356e45ef7573e6e5f48abde0639388a0cea007/napari/components/viewer_model.py#L405
If you make PR I will be happy to review it.


Updates for the 2018-07-01 version of the .Net SDK for Azure Media Se…

Updates for the 2018-07-01 version of the .Net SDK for Azure Media Services

Description

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] Please add REST spec PR link to the SDK PR
[x] I have read the contribution guidelines.
[x] The pull request does not introduce breaking changes.

General Guidelines

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[x] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[x] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[x] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

I must have not edited the template correctly. I don't see the link to the PR for the swaggers that generated this code.
That PR is https://github.com/Azure/azure-rest-api-specs/pull/4483


feat: alert notification via websocket

This PR adds support for notifying clients such as pyro-platform instances of new alerts via websocket. Previously the platform had to periodically ping the API to ask if new alerts were sent.

Websockets have many differences and subtleties with respect to HTTP routes. Without being able to test the setup locally, due to #217, #231, the development is quite hard.
The feature is implemented in a simple setup consisting of an API passing messages received via HTTP to websocket clients and a dash application (code) that receives those messages via websocket. Demo below.
The alternatives I see to move forward:

Accept the PR basically as it is with the websocket route unprotected, meaning any one can listen to the alerts (but not posting, like before). This allows implementing the feature in pyro-platform as soon as it is deployed
Wait for #217, #231 to be able to test the setup locally
Port the security implementation to the test-API above, which can be long and maybe never exactly like this API so back to 1. or 2.

Any suggestions? @frgfm ?

Before diving into a review, and iterating on this, I want to address something here:

if in our stack, there is something that is critical to the whole workflow, it's the API. So we have to be quite careful about that
now here, I understand that the websocket could help the platform reduce its ping on the API. If that's the case, we need to investigate. But then, I suggest making sure that could work, by running the API & the platform locally to make sure that works
if that doesn't, let's avoid adding a new feature :)


Carbon Console Deprication

Is your feature request related to a problem? Please describe.
All the features available in the carbon console need to be catered by the new portals in order to deprecate the carbon console.
Describe the solution you would prefer
This issue tracks the remaining features we need to onboard to the new portals.

OIDC scope management
Adaptive Fn library
Application-based permission management
XACML policy management UI + REST API
Workflow Management UI + REST API
Consent Purpose Management
Tenant Management
Handling browser back button and bookmarked login page (currently registry involved)
Force Password Reset
Resident SP
Resident IDP config

REST API improvements

Capability to update userstore domain name in /userstores API
API to retrieve server realm Info in UserAdmin Service

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Fix hadoop-mapreduce-client-core dependency.

Description
In https://github.com/apache/incubator-pinot/pull/6716, we upgraded Avro version to 1.9.2. This PR fixes the scope of hadoop-mapreduce-client-core dependency in pinot-plugins/pinot-input-format/pinot-parquet/pom.xml.
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

[ ] Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

[ ] Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

[ ] Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes
If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release.
If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text
Documentation
If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document

@npawar @jackjlli Please review.


Testing Guide

TODO

[ ] Add to wiki.  Link to it from CONTRIBUTING.md.
[ ] Table of Contents
[ ] Resize images

Testing Guide
Our development culture has a commitment to testing.  Cesium is used in diverse use cases on a wide array of platforms so it is important for it to be well tested.
As of Cesium 1.15, Cesium has over 7,000 tests with 93% code coverage.  Cesium has almost as much test code (94K lines) as engine code (98K).  We are unaware of any other project of this size and lifetime and with this many contributors that has similar stats.
All new code should have 100% code coverage and should pass all tests.  Always run the tests before opening a pull request.
Running the Tests
The Cesium tests are written in JavaScript and use Jasmine, a behavior-driven testing framework.  Jasmine calls an individual test, e.g., a function with one or more assertions, a spec (however, the Cesium team usually still say "test"), and a group of related tests, e.g., all the tests for Cartesian3, a suite.  Jasmine also calls an assertion, an expectation.
When running Cesium locally, browse to http://localhost:8080/ and there are several test options:
Run all tests (Run with WebGL validation)
Runs all the tests.  As of Cesium 1.15, on a decent laptop, they run in about a minute in Chrome.  It is important that the tests run quickly so we run them often.
When all the tests pass, the page looks like:

When one or more tests fail, the page looks like:

In this case, the number of failing tests is listed at the top, and details on each failure is listed below, including the expected and actual value of the failed expectation and the call stack.  The top several functions of the call stack are inside Jasmine and can be ignored.  Above, the file and line of interest for the first failing test starts with an @:
@http://localhost:8080/Specs/Renderer/FramebufferSpec.js:637:13

Click on the failed test to rerun just that test.  This is useful to save time when fixing an issue by avoiding rerunning all the tests.  Always rerun all the tests before opening a pull request.
The link to Run with WebGL validation passes a query parameter to the tests to enable extra low-level WebGL validation such as calling gl.getError() after each WebGL call.  We use this when doing the monthly Cesium release and when making changes to Cesium's renderer.
Select a test to run
This option loads the test page without running any tests.

We can then use the browsers' built-in search to find a test or suite and run only it.  For example, below just the tests for Cartesian3 were run.

This uses a query parameter to select the test/suite to run so refreshing the page will run just that test/suite again.
Often when developing, it is useful to run only one suite, to save time, instead of all the tests, and then run all the tests before opening a pull request.
Run only WebGL tests
Suites can have a category associated with them.  This option runs all tests in the WebGL category, which includes all tests that use WebGL (basically anything that requires creating a Viewer, CesiumWidget, Scene, or Context).
Run only non-WebGL tests
Likewise, this option runs all tests not in the WebGL category.
Perhaps surprising, this is the bulk of Cesium tests, which include math and geometry tests, imagery provider tests, data source tests, etc.
These tests run quickly (for example, 15 seconds compared to 60) and are very reliable across systems since they do not rely on the underlying WebGL implementation, which can vary based on the browser, OS, driver, and GPU.
Run all tests against combined file (Run all tests against combined file with debug code removed)
Most test options load Cesium using the individual source files in the Source directory, which is great for debugging.
However, many users build apps using the built Cesium.js in Build/Cesium (which is created, for example, by running npm run combine).  This option runs the tests using this instead of individual Cesium source files.
The Run all tests against combined file with debug code removed is the same except it is for use with the release version of the built Cesium.js (which is created, for example, by running npm run combineRelease).  The release version has DeveloperError exceptions optimized out so this test option makes toThrowDeveloperError always pass.
See the Contributor's Guide for all the Cesium build options.
Run all tests with code coverage (build 'instrumentForCoverage' first.)
JSCoverage is used for code coverage.  It is especially important to have outstanding code coverage since JavaScript doesn't have a compiler and linker to catch early errors.
To run code coverage, first create a build of Cesium that is instrumented for coverage by running npm run instrumentForCoverage.  Currently, this is Windows only.
Then use this test option to run the tests with code coverage.  Click on the Summary tab to see the total code coverage and coverage for each individual source files.

Click on a file to see line-by-line coverage for just that file.  For example, here is AssociativeArray:

In the left margin, green indicates a line that was executed and red indicates a line that was not.  Many lines, like comments and semicolons, are not colored since they are not executable.
For the contains function above:

AssociativeArray.prototype.contains = function(key) { is executed once when Cesium is loaded to assign the contains function to the AssociativeArray's prototype.
The if statement and return statement are executed 3,425 times.
The throw statement is not executed, which indicates that test coverage should be improved here.  We strive to test all error conditions.

When writing tests do not confuse 100% code coverage with 100% tested.  For example, it is possible to have 100% code coverage without having any expectations.  Also consider the following code:
if (a) {
    // Code block a.1
} else {
    // Code block a.2
}

if (b) {
    // Code block b.1
} else {
    // Code block b.2
}

It is possible to have 100% code coverage with two tests: one test where a and b are both true, and another when both are false; however, this only takes into account the case when // Code block a.1 and // Code block b.1 run together or when // Code block a.2 and // Code block b.2 run.  There could be an issue when, for example, // Code block a.1 and // Code block b.2 run together.
The number of linearly independent paths (four in this case) is called the cyclomatic complexity.  Be mindful of this when writing tests.  On one extreme, 100% code coverage is the least amount of testing, on the other extreme is covering the cyclomatic complexity, which quickly becomes unreasonable.  Use your knowledge of the implementation to devise the best strategy.
Testing Previous Versions of Cesium
Sometimes it is useful to see if an issue exists in a previous version of Cesium.  The tests for all versions of Cesium back to b15 (April 2013) are hosted on the Cesium website via the downloads page.  Use the "Documentation, Sandcastle, tests, etc." links.
testfailure Label for Issues
Despite our best efforts, sometimes tests fail.  This is often due to a new browser, OS, or driver bug that breaks a test that previously passed.  If this indicates a bug in Cesium, we strive to quickly fix it.  Likewise, if it indicates that Cesium needs to workaround the issue (for example, as we did for Safari 9), we also strive to quickly fix it.
If a test failure is likely due to a browser, OS, or driver bug, or a poorly written test, and the failure does not impact actual Cesium apps, we sometimes submit an issue with the testfailure label to fix it at a later time.  A great way to contribute to Cesium is to help fix these issues.
Writing Tests
We love to write tests.  We often write them as we write engine code (meaning Cesium itself) - or if the engine code is experimental, we make a second pass and write tests before opening a pull request.  Sometimes we do both; we write tests right away for the new code we expect to be stable, and we wait to write tests for the code in flux.
Directory Organization
Tests are located in the Specs directory (recall, Jasmine calls a test a "spec"), which has a directory structure that mirrors the Source directory.  For example, all the tests for files in Source/Core are in Specs/Core.  Likewise, all the tests for Source/Core/Cartesian3.js are in Specs/Core/Cartesian3Spec.js.  The filenames are the same except for the Spec suffix.  Each spec file corresponds to at least one suite (sometimes suites are nested inside).
Bottom-Up Unit Testing
The Cesium tests are largely unit tests because they test individual units, e.g., functions or classes.  The simplest units are tested individually, and then units built upon other units are also tested.  This allows us to build Cesium on well-tested foundations and to quickly narrow down issues.
For example, a BoundingSphere is composed of a Cartesian3 that defines its center and a number that defines its radius.  Even though tests for BoundingSphere implicitly test parts of Cartesian3, there are separate tests that explicitly test Cartesian3 as a unit so anything that relies on Cartesian3 knows it is already tested.
Often, we also test private units individually for the same reason.  For example, ShaderCache is a private class in Cesium used by primitives, but it is still individually tested in ShaderCacheSpec.js
Sometimes classes or functions are even designed with a separation specifically to enable more precise testing.  For example, see getStringFromTypedArray and getStringFromTypedArraySpec.js.
Test Code is Code
Tests are written in JavaScript using Jasmine.  It is important to realize that the tests themselves are code, just like Cesium.  As such, the test code is held to the same standards as the engine code; it should be well organized, cohesive, loosely coupled, fast, and go through peer review.
Testing Basics
Cartesian3Spec.js contains the tests for Cartesian3, which is a class representing a 3D point or vector with x, y, and z properties, and typical functions like adding two Cartesian3 objects.
Here is a stripped down version of the tests:
/*global defineSuite*/
defineSuite([
        'Core/Cartesian3'
    ], function(
        Cartesian3) {
    "use strict";

    it('construct with default values', function() {
        var cartesian = new Cartesian3();
        expect(cartesian.x).toEqual(0.0);
        expect(cartesian.y).toEqual(0.0);
        expect(cartesian.z).toEqual(0.0);
    });
});

defineSuite identifies this file as a test suite, and include modules just like define is used in engine code.  The modules are listed in alphabetical order as usual, except the module being tested is listed first.
Using Jasmine, each test is defined by calling it and passing a string that describes the test and a function that is the test.
This test constructs a default Cartesian3 object and then expects that the x, y, and z properties are zero (their default) using Jasmine's expect and toEqual functions.
Tests should have at least one expect call, but they may also have several as long as the test is cohesive.  A test should test one behavior; if a test grows too complicated, it is hard to debug when it fails.  To test one function may only require one test with one expect, or it may require multiple tests, each with multiple expect statements.  It depends on context.  Experience, peer review, and the existing tests will help guide you.
The above test does not require creating a Viewer widget or even a WebGL context; the only part of Cesium it uses is Cartesian3 and anything it depends on.

To learn the ins and outs of Jasmine, take 15 minutes to go through their examples.  We will not cover all the details in this guide.

Comparing Floating-Point Values
We often can't rely on an exact floating-point comparison.  In this case, use toEqualEpsilon instead of toEqual to compare within a tolerance.
it('angleBetween works for acute angles', function() {
    var x = new Cartesian3(0.0, 1.0, 0.0);
    var y = new Cartesian3(1.0, 1.0, 0.0);
    expect(Cartesian3.angleBetween(x, y)).toEqualEpsilon(CesiumMath.PI_OVER_FOUR, CesiumMath.EPSILON14);
    expect(Cartesian3.angleBetween(y, x)).toEqualEpsilon(CesiumMath.PI_OVER_FOUR, CesiumMath.EPSILON14);
});

toEqualEpsilon is a custom Jasmine matcher that the Cesium tests add, see Specs/addDefaultMatchers.js for all the custom matchers.  In general, all test utility functions are in files in the Specs root directory.
Testing Exceptions
In addition to testing success cases, we also test all failure cases.  The custom machters, toThrowDeveloperError and toThrowRuntimeError, can be used to expect an exception to be thrown.
it('fromDegrees throws with no latitude', function() {
    expect(function() {
        Cartesian3.fromDegrees(0.0);
    }).toThrowDeveloperError();
});

Above, Cartesian3.fromDegrees is expected to throw a DeveloperError because it expects longitude and latitude arguments, and only longitude is provided.
Tips:

When testing for exceptions, only put code that is expected to trigger the exception inside the function passed to expect() in case setup code unintentionally throws an exception.
To verify the right exception is thrown, it is often useful to comment out the expect call when first running the test, for example:

it('fromDegrees throws with no latitude', function() {
//    expect(function() {
        Cartesian3.fromDegrees(0.0);
//    }).toThrowDeveloperError();
});

Before and After Tests and Suites
The Jasmine functions beforeAll and afterAll are used to run a function before and after, respectively, all the tests in a suite.  Likewise, beforeEach and afterEach run a function before and after each test is ran.  For example, here is a common pattern from DebugModelMatrixPrimitiveSpec.js:
var scene;

beforeAll(function() {
    scene = createScene();
});

afterAll(function() {
    scene.destroyForSpecs();
});

afterEach(function() {
    scene.primitives.removeAll();
});

Above, scene is scoped at the suite-level so all tests in the file have access to it.  Before the suite is ran, beforeAll is used to assign to scene (see below), and after the suite is ran, afterAll is used to destroy the scene.  Using afterEach, after each test is ran, all the primitives are removed from the scene.  scene is typically used in a test like this:
it('renders', function() {
    var p = scene.primitives.add(new DebugModelMatrixPrimitive());
    expect(scene.renderForSpecs()).not.toEqual([0, 0, 0, 255]);
});

The test knows scene will be defined and does not need to worry about cleaning up the scene because afterEach and afterAll take care of it.
Rendering Tests
Unlike the Cartesian3 tests we first saw, many tests need to construct the main Cesium Viewer widget or one of its major components.  Low-level renderer tests construct just Context (which, itself, has a canvas and WebGL context), and primitive tests construct a Scene (which contains a Context).
As shown above, these tests use Cesium test utility functions, createViewer, createScene, or createContext.  These functions honor query parameters passed to the tests (for example, like enabling WebGL validation) and add extra test functions to the returned object.
For example, createScene creates a 1x1 pixel canvas with a Cesium Scene and adds renderForSpecs and pickForSpecs to the returned Scene object:
it('renders', function() {
    var p = scene.primitives.add(new DebugModelMatrixPrimitive());
    expect(scene.renderForSpecs()).not.toEqual([0, 0, 0, 255]);
});

it('does not render when show is false', function() {
    scene.primitives.add(new DebugModelMatrixPrimitive({
        show : false
    }));
    expect(scene.renderForSpecs()).toEqual([0, 0, 0, 255]);
});

it('is picked', function() {
    var p = scene.primitives.add(new DebugModelMatrixPrimitive({
        id : 'id'
    }));

    var pick = scene.pickForSpecs();
    expect(pick.primitive).toEqual(p);
    expect(pick.id).toEqual('id');
});

In the first test, renderForSpecs initializes the frame, renders the scene into the 1x1 canvas, and then returns the RGBA value of the rendered pixel.  Like most rendering tests, this uses a coarse-grained expectation to check that the pixel is not the default value of black.  Although an expectation this coarse-grained may not catch all subtle errors, it is reliable across platforms, and we rarely have bugs a more fine-grained test would have caught, especially with some manual testing (see below).
In the second test, renderForSpecs is used again, but this time it is to verify that the pixel value is the same as the default background color since the primitive's show property is false.

Most Cesium apps do not render the scene directly; instead, the Viewer object's default render loop renders the scene implicit to the user.  The tests are an exception; most tests explicitly render the scene.

In the final test, pickForSpecs executes a Scene.pick for the one-pixel canvas.  A typical follow-up expectation verifies the primitive of interest was picked and its id is the expected value.
GLSL
GLSL is the shading language used by WebGL to run small graphics programs in parallel on the GPU.  Under-the-hood, Cesium contains a library of GLSL identifiers and functions.  These are unit tested by writing a simple fragment shader that outputs white if the test passes.  For example, here is an excerpt from BuiltinFunctionsSpec.js;
var context;

beforeAll(function() {
    context = createContext();
});

afterAll(function() {
    context.destroyForSpecs();
});

it('has czm_tranpose (2x2)', function() {
    var fs =
        'void main() { ' +
        '  mat2 m = mat2(1.0, 2.0, 3.0, 4.0); ' +
        '  mat2 mt = mat2(1.0, 3.0, 2.0, 4.0); ' +
        '  gl_FragColor = vec4(czm_transpose(m) == mt); ' +
        '}';

    context.verifyDrawForSpecs(fs);
});

createContext returns a Context object with a test function, verifyDrawForSpecs, that renders a point to the 1x1 canvas and verifies the pixel value is white, e.g.,
expect(context.readPixels()).toEqual([255, 255, 255, 255]);

In the test above, the expectation is implicit in the GLSL string for the fragment shader, fs, which assigns white to gl_FragColor if czm_transpose correctly transposes the matrix.
Spies
It can be useful to expect if a function was called and inspect information about the function call like the arguments passed to it.  Jasmine spies are used for this.
Here is an excerpt from TweenCollectionSpec.js:
it('add() adds with a duration of zero', function() {
    var complete = jasmine.createSpy('complete');

    var tweens = new TweenCollection();
    tweens.add({
        startObject : {},
        stopObject : {},
        duration : 0.0,
        complete : complete
    });

    expect(tweens.length).toEqual(0);
    expect(complete).toHaveBeenCalled();
});

Tweens are used for animation.  This test creates a spy with jasmine.createSpy to verify that a tween calls the provided complete function when a tween finishes animating using toHaveBeenCalled(), which is immediately in this case given duration is 0.0.
Spies can also provide more information about the function call (or calls).  Here is an excerpt from GeocoderViewModelSpec.js:
it('Zooms to longitude, latitude, height', function() {
    var viewModel = new GeocoderViewModel({
        scene : scene
    });

    spyOn(Camera.prototype, 'flyTo');

    viewModel.searchText = ' 1.0, 2.0, 3.0 ';
    viewModel.search();
    expect(Camera.prototype.flyTo).toHaveBeenCalled();
    expect(Camera.prototype.flyTo.calls.mostRecent().args[0].destination).toEqual(Cartesian3.fromDegrees(1.0, 2.0, 3.0));

    viewModel.searchText = '1.0   2.0   3.0';
    viewModel.search();
    expect(Camera.prototype.flyTo.calls.mostRecent().args[0].destination).toEqual(Cartesian3.fromDegrees(1.0, 2.0, 3.0));
});

Here, spyOn is used to replace Camera.flyTo (prototype function on instances) with a spy.  When the Geocoder is used to search for a location, the test expects that Camera.flyTo was called with the right arguments.
Spies can also be used on non-prototype functions.  Here is an excerpt from ModelSpec.js:
it('Applies the right render state', function() {
    spyOn(RenderState, 'fromCache').and.callThrough();

    return loadModelJson(texturedBoxModel.gltf).then(function(model) {
        var rs = {
            frontFace : WebGLConstants.CCW,
            cull : {
                enabled : true,
                face : WebGLConstants.BACK
            },
            // ...
        };

        expect(RenderState.fromCache).toHaveBeenCalledWith(rs);
        primitives.remove(model);
    });
});

This test verifies that a glTF model uses the expected render state.  First, a spy is added to RenderState.fromCache.  Since we want the spy to collect information but still call the original function, and.callThrough() is used.  Once the model is loaded, toHaveBeenCalledWith is used to expect that RenderState.fromCache was called with the expected arguments.
For more examples of what you can do with spies, see the Jasmine examples.
Beware of too tightly coupling a test with an implementation; it makes engine code hard to refactor and results in specific narrow tests.  Given that we are usually white box testing (where we know the implementation details, as opposed to black box testing), we need to resist the urge to let too many implementation details leak into a test.  In particular, reach into private members (whose names start with _) sparingly.
Test Data and Services
Sometimes, a test requires sample data, like a CZML file or glTF model, or a service.  When possible, we try to procedurally create data or mock a response in the test instead of reading a local file or making an external request.  For example, loadArrayBufferSpec.js uses a spy to simulate an XHR response.
When external data can't be avoided, prefer storing a small file in a subdirectory of Specs/Data.  Avoid bloating the repo with an unnecessarily large file.  Update LICENSE.md if the data requires a license or attribution.  Include a README file when useful, for example, see Specs/Data/Models/Box-Textured-Custom.
Make external requests that assume the tests are being used with an Internet connection very sparingly.  We anticipate being able to run the tests offline.
Promises
Tests can return promises.  Here is an excerpt from ModelSpec.js:
var texturedBoxUrl = './Data/Models/Box-Textured/CesiumTexturedBoxTest.gltf';
var texturedBoxModel;

var cesiumAirUrl = './Data/Models/CesiumAir/Cesium_Air.gltf';
var cesiumAirModel;

beforeAll(function() {
    var modelPromises = [];
    modelPromises.push(loadModel(texturedBoxUrl).then(function(model) {
        texturedBoxModel = model;
    }));
    modelPromises.push(loadModel(cesiumAirUrl).then(function(model) {
        cesiumAirModel = model;
    }));

    return when.all(modelPromises);
});

Given a model's url, loadModel (detailed below) returns a promise that resolves when a model is loaded.  Here, beforeAll is used to ensure that two models, stored in suite-scoped variables, texturedBoxModel and cesiumAirModel, are loaded before any tests are ran.
An implementation of loadModel is:
function loadModelJson(gltf) {
    var model = primitives.add(new Model());

    return pollToPromise(function() {
        // Render scene to progressively load the model
        scene.renderForSpecs();
        return model.ready;
    }, { timeout: 10000 }).then(function() {
        return model;
    });
}

Since loading a model requires asynchronous requests and creating WebGL resources that may be spread over several frames, Cesium's pollToPromise is used to return a promise that resolves when the model is ready, which occurs by rendering the scene in an implicit loop (hence the name "poll") until model.ready is true or the timeout is reached.
pollToPromise is used in many places where a test needs to wait for an asynchronous event before testing its expectations.  Here is an excerpt from BillboardCollectionSpec.js:
it('can create a billboard using a URL', function() {
    var b = billboards.add({
        image : './Data/Images/Green.png'
    });
    expect(b.ready).toEqual(false);

    return pollToPromise(function() {
        return b.ready;
    }).then(function() {
        expect(scene.renderForSpecs()).toEqual([0, 255, 0, 255]);
    });
});

Here a billboard is loaded using a url to an image.  Internally, Billboard makes an asynchronous request for the image and then sets its ready property to true.  The function passed to pollToPromise just returns the value of ready, it does not need to render the scene to progressively complete the request like Model.  Finally, the resolve function (passed to then) verifies that the billboard is green.
Mocks
To isolate testing, mock objects can be used to simulate real objects.  Here is an excerpt from SceneSpec.js;
function MockPrimitive(command) {
    this.update = function(frameState) {
        frameState.commandList.push(command);
    };
    this.destroy = function() {
    };
}

it('debugCommandFilter filters commands', function() {
    var c = new DrawCommand({
        pass : Pass.OPAQUE
    });
    c.execute = function() {};
    spyOn(c, 'execute');

    scene.primitives.add(new MockPrimitive(c));

    scene.debugCommandFilter = function(command) {
        return command !== c;   // Do not execute command
    };

    scene.renderForSpecs();
    expect(c.execute).not.toHaveBeenCalled();
});

This test verifies that debugCommandFilter can be used to filter the commands executed when the scene is rendered.  Here, the function passed to debugCommandFilter explicitly filters out the command, c.  In order to ask the scene to execute the command in the first place, a mock object, MockPrimitive, is used to return the command when the scene is rendered.
This test is more cohensive and easier to debug than if it were written using a real primitive, which brings along all of its extra behavior and does not provide direct access to its commands.
Pragmatic Advice
Start With a Similar (Small) Test
Since I wrote the very first Cesium test, I have not written a suite - or even individual test - from scratch.  I suspect no one does.

The first 73 Cesium tests from March 2011.
Instead, start with a similar suite or test, copy it, strip it down to the minimum you need, and then start adding your specific code.  For example, if you are adding a new math type, start with Cartesian3Spec.js or Matrix4Spec.js.  If you are adding a new primitive, start with DebugModelMatrixPrimitiveSpec.js.
Manual Testing
Sometimes running the unit tests is all that is needed to verify new code.  However, we often also manually run Cesium to see the effects of new code.  Sometimes it is as simple as running Cesium Viewer before opening a pull request perhaps because we just added a new function to Cartesian3.  Other times, it is as involved as going through each example in Sandcastle and testing the different options because, for example, we refactored the renderer for WebGL 2.  Most often, there is a middle ground, for example, we added a new feature to Model so we run the Sandcastle examples that create 3D Models.
Debugger-Aided Incremental Improvements
Two decades ago, I didn't know what unit tests and code coverage were.  I wrote a little bit of code, put breakpoints in all the new code paths, and then ran the app in the debugger.  When a breakpoint hit, I stepped through the code, looked at the variable values, and verified that the code was doing what I intended.  I did this until everything worked and all the breakpoints were hit.  Then, I repeated: I added a bit more new code, new breakpoints, and incrementally tested.
Even today, with modern tools and experience, I never code for hours straight without testing and I rarely write new code without seeing it execute in the debugger.  Debuggers are not a reactive tool for when a bug is found, they are a proactive tool for gaining insight and avoiding surprises.
Try this approach for yourself!
Resources
See Section 4.4 of Getting Serious with JavaScript by Cesium contributors Matthew Amato and Kevin Ring in WebGL Insights for a more deep but less broad presentation of Cesium testing.

This is ready.  Please provide tech comments by end of Wednesday so this can go through copyediting.  If you want to make a small change or add a section, just do it.

Last call for tech feedback.

This is a good write up. My one overall comment is that it's wierd that you are using the first person singular voice (unless this is meant to be a blog post first?). If it is going to eventually live on the wiki, then it is ultimately a commuity document that will be updated and edited by everyone and it's odd to have personal anecdotes in it.
Other feedback:

Not sure if it's worth mentioning, but defineSuite is also a custom Cesium function that wraps Jasmine define calls and provides the category capability that's not part of core Jasmine. (categories are also not part of Jasmine).  All of this functionality is part of Specs/spec-main.js
The number one rule of unit tests is that it should be able to run standalone and does not depend on previous tests or what order they are run in.  Unfortunately, we have to break this rule in Cesium in any WebGL tests (because as you know some browsers can't handle the number of WebGL Contexts that get created).  It might be good mention that here.
The fact that you can return promises from a test is another Cesium-specific Jasmine modification. Jasmine instead uses a done callback function that gets passed into test functions.  We actually replace the it function with our own version in spec-main.js.
We should probably include an example of testing a promise that is supposed to be rejected. Here's one example Though after looking at this we should add a toBeRejected helper.
Maybe mention that we usually choose as high an epsilon as we can when comparing floating point values and potential reasons why you may need to use a low/high epsiloon.  If this is intended for beginners, we might want to also explain (or link to) why exact double comparison is unreliable.
Is it worth having a small paragraph about what we would like to improve about our unit tests, or is that outside the scope of the guide (again, not sure if this will be a blog post or not).

Though after looking at this we should add a toBeRejected helper.

I'll add the example in the meantime, but perhaps submit an issue for toBeRejected.

Thanks for the feedback.  Updated.  This will be part of the contributor guides, not a blog post.  I put the first person paragraphs in block quotes.
Anyone else have feedback?


Removed Joken.Helpers

And added get_current_time to the Joken module #65

Taking the opportunity we are moving this function and that people will need to refactor existing code, I think we could remove the "get_" part of the function name. So we would end up with current_time. This is more in line with timex, erlang and current date/time libraries. Some use now but current_time seems better as it currently does not clash with names in other libraries.
Other than that, moving it to Joken makes total sense as people will probably import Joken most of the times.

That's a good point. I will change it

Nice that you liked the idea! This is good!

I guess we can merge this too! What do you think @bryanjos ?

merged!


samples: coop_synchronization: new synchronization sample which works without a systick

New cooperative thread switching sample app which can run without a systick timer; handy for testing arch_swap &| arch_switch.
Based off of the original synchronization sample app
A simple application that demonstrates even more basic sanity of the kernel.
Two threads (A and B) take turns printing a greeting message to the console.
They use yields and semaphores to control the whose turn it is to print.
The rate is only controlled via busy waits. This demonstrates kernel communication only.
This is a good example to test new platforms that may not have a working sys timer.
It's also a very simplistic demo allowing users to follow the execution trace in
a no-timer (i.e. nonexistent, disabled, deferred) scenario and experience predictability.
To that end, sample should ideally be built with no interrupt-driven-UART to avoid spurious
IRQ fires which may trigger unpredictable context switch. However, some platforms do not
support polling api for UART. For those, only the regular variant is built.
This sample is originally what I used to develop the microblaze port's arch_swap() (coming soon).
Builds for all platforms except s32z270dc2_rtu0_r52 and  s32z270dc2_rtu1_r52.
From what I tried, practically none of the samples build for these platforms.

Did the close/open PR to hopefully get CI moving on this.

Thanks @galak I'll get on with the compliance reports.

@andyross Thank you, they're all great reviews. I'll get on with all of them ASAP.

@alpsayin just FYI:
I've fixed several issues related to CONFIG_SYS_CLOCK_EXISTS=n configuration recently:
https://github.com/zephyrproject-rtos/zephyr/pull/59704
https://github.com/zephyrproject-rtos/zephyr/pull/59704
So on ARC this sample now works without issues (at least on non-SMP platforms)

@alpsayin just FYI:
I've fixed several issues related to CONFIG_SYS_CLOCK_EXISTS=n configuration recently: #59704 #59384
So on ARC this sample now works without issues (at least on non-SMP platforms)

Thanks for the info @evgeniy-paltsev,  works perfectly on ARC platforms as far as I can tell but if I set CONFIG_SYS_CLOCK_EXIST=n, then I get get the below error on qemu_x86_64.
So many other platforms also fail to build as well. So I think I'll keep it as is for now ¯_(ツ)_/¯
/home/asayin/zephyr_sdks/microblaze_devel/zephyr-sdk-0.16.1-4-g8113984/x86_64-zephyr-elf/bin/../lib/gcc/x86_64-zephyr-elf/12.2.0/../../../../x86_64-zephyr-elf/bin/ld.bfd: zephyr/arch/arch/x86/core/libarch__x86__core.a(cpu.c.obj): in function `k_busy_wait':
/home/asayin/zephyr_sdks/microblaze_devel/zephyr/build/zephyr/include/generated/syscalls/kernel.h:132: undefined reference to `z_impl_k_busy_wait'

@nashif @evgeniy-paltsev your re-reviews required. Alternatively, if you think this is not as useful, I can just dismiss it.
It has fulfilled its purpose for me already (also this changed to something much different than what I used for "proving grounds").


Should not show completion when properties of type literals/interfaces span different lines

This doesn't happen today as far as I can tell. The issue can probably be closed.


Updates to influx pkg and subcommands need doc updates

Describe the issue here.
The influx pkg commands had some updates today that will need to be updated in the docs.
relevant issues from influxdb:

https://github.com/influxdata/influxdb/pull/16767
https://github.com/influxdata/influxdb/pull/16765

Relevant URLs

Provide relevant URLs
https://v2.docs.influxdata.com/v2.0/reference/cli/influx/pkg/
https://v2.docs.influxdata.com/v2.0/reference/cli/influx/pkg/summary/
https://v2.docs.influxdata.com/v2.0/reference/cli/influx/pkg/validate/

What products and version are you using?
OSS v2 master

Thanks @jsteenb2!


GH-44844: [CI] Uninstall pkg-config entirely on verification and java-jars macOS jobs

Rationale for this change
Jobs are failing on nightlies.
What changes are included in this PR?
Remove pkg-config entirely as we've done on other CI jobs.
Are these changes tested?
Yes via archery
Are there any user-facing changes?
No

GitHub Issue: #44844

@github-actions crossbow submit java-jars verify-rc-source-macos-

@github-actions crossbow submit java-jars

@github-actions crossbow submit java-jars verify-rc-source-ruby-macos-amd64

@github-actions crossbow submit verify-rc-source-ruby-macos-amd64

@github-actions crossbow submit java-jars verify-rc-source-macos-

@kou the verify-rc-source-ruby-macos-amd64 job is also related to pkg-config when installing meson but I haven't seen other failures on amd64 so I am not entirely sure what is going on on this one.

Meson doesn't find pkgconf by default. So we need to specify it explicitly. See also:

https://github.com/apache/arrow/pull/44798#issuecomment-2489968836
https://github.com/apache/arrow/pull/44798/commits/afeb15e27b670f352ced1db11a3df62568f83d77

@github-actions crossbow submit verify-rc-source-ruby-macos-*

Thanks @kou


Call check_proof_attribute for contract harnesses

Kani enforces that [kani::proof] attribute is not applied to generic functions. We do not currently enforce this restriction on contract harnesses. When the compiler searches for harnesses to verify, it only looks at monomorphized functions. Thus, currently a user can write this code:
#[kani::requires(true)]
fn foo() {}

#[kani::proof_for_contract(foo)]
fn check_foo<T>() {
    foo()
}

and get
"No proof harnesses (functions with #[kani::proof]) were found to verify."
In the case where a user is running many harnesses, they may not notice that Kani skipped the harness. For example, we currently have this harness in the standard library, which doesn't actually run. (PR to fix is here). After this PR merges, the code snippet above would instead error with:
error: the '#[kani::proof_for_contract]' attribute cannot be applied to generic functions
 --> src/lib.rs:4:1
  |
4 | #[kani::proof_for_contract(foo)]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in the attribute macro `kani::proof_for_contract` (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to 1 previous error

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

Note that we should merge https://github.com/model-checking/verify-rust-std/pull/86 first to avoid breaking the standard library.

Note that we should merge model-checking/verify-rust-std#86 first to avoid breaking the standard library. (I would appreciate if someone would "Request changes" on this PR to avoid accidental auto-merge; I can't do it myself).

Merging this shouldn't break the std repository, right? BTW, you could leave it as draft for now too since you don't think this is ready to be merged.

Note that we should merge model-checking/verify-rust-std#86 first to avoid breaking the standard library. (I would appreciate if someone would "Request changes" on this PR to avoid accidental auto-merge; I can't do it myself).

Merging this shouldn't break the std repository, right? BTW, you could leave it as draft for now too since you don't think this is ready to be merged.

It will break our harnesses in the std repository--see the new error messages in the logs:
2024-09-17T20:23:16.3424994Z    Compiling core v0.0.0 (/home/runner/work/kani/kani/verify-rust-std/library/core)
2024-09-17T20:23:26.2257417Z error: the '#[kani::proof_for_contract]' attribute cannot be applied to generic functions
2024-09-17T20:23:26.2258407Z    --> /home/runner/work/kani/kani/verify-rust-std/library/core/src/ptr/unique.rs:287:5
2024-09-17T20:23:26.2258952Z     |
2024-09-17T20:23:26.2259233Z 287 |     #[kani::proof_for_contract(Unique::cast<U>)]
2024-09-17T20:23:26.2259645Z     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-17T20:23:26.2259971Z     |
2024-09-17T20:23:26.2260750Z     = note: this error originates in the attribute macro `kani::proof_for_contract` (in Nightly builds, run with -Z macro-backtrace for more info)
2024-09-17T20:23:26.2261394Z
2024-09-17T20:23:26.4192473Z error: aborting due to 1 previous error

Marking as draft for now.


Updated notebooks incorporating feedback

Signed-off-by: Pari Ram<EMAIL_ADDRESS>The updated notebooks are here:

03_custom.ipynb
06_multobj.ipynb

@kiran-kate @hirzel Here is the PR with the updated notebooks incorporating the feedback.
I am still working on the disparate impact multi-obj example and will add that if I get to make it work successfully.

@kiran-kate @hirzel Here is the PR with the updated notebooks incorporating the feedback.
I am still working on the disparate impact multi-obj example and will add that if I get to make it work successfully.

@rithram The PR passes all the tests, should I merge it?

@rithram The PR passes all the tests, should I merge it?
@kiran-kate Yes please. thank you. I'll work on the other changes and put it in a separate PR.


Investigate default setup.py of UP

For the moment it's not executing the setup.py of https://github.com/classner/up/. Instead a custom requirements.txt file is used. Check if we can proceed with executing setup.py during build phase and leave the requirements.txt file behind.
The setup.py conflicted during build, therefor i was ignored. Didn't investigate further at that time

Not needed. Moved all the required python libs to this projects requirements file. Since I've pinned all the versions of libs. It's safer.


Input does not work on android (Pico 4)

Hello and thanks for your great work!
I'm using godot 4.2 and trying to run the game from godot-xr-template
when I launch the application built for Android, I see a splash screen, I can turn my head with tracking, but all the controller keys do not work and I cannot go to the next step.
but if I run virtual desktop and the application runs through steam vr, everything works fine. (Piсo controller keys work fine)

Hello! If you are still interested, I posted a -possible- solution in a more recent issue: https://github.com/GodotVR/godot-xr-template/issues/33


Update/extend pikaday api support

Pikaday API specifies minDate and maxDate options with dynamic setters as well. We needed support for these in our application, so I've added them here. See the observers for dynamically setting these values.
Also adds ability to disable the datepicker.

+1

@markstuart Sorry for the late response, thanks for your contribution! Can you please add tests that cover the new functionality?

Feel free to add tests and reopen the pull request, I'm closing it for the time being due to inactivity.


Adding more hover effects in GetHelp Section

The buttons of the GetHelp section don't show any change when we hover on them. I think it should be there to make the website more attractive. @ghulamyazdani I would like to work on it. Pls assign it to me.

@ghulamyazdani pls give a reply

Go ahead with the changes

Go ahead with the changes


Uncompilable code is generated for Spring class using Optional

To Reproduce
Try to generate tests for the following method from open-source project spring-boot-testing-main:
public Double getOrderPriceById(long id) throws Exception {
        var order = orderRepository.findById(id);
        if (order == null) {
            throw new Exception("Order id is incorrect");
        }

        return order.get().getPrice();
    }

Method findById has the following signature Optional<T> findById(ID id);
Expected behavior
Generated tests may be compiled successfully.
Actual behavior
Two of three generated tests starts with the following line, where optional variable was never declared.
(when(orderRepositoryMock.findById(any()))).thenReturn(optional);

Additional context
Yuri recommends to ensure that we call setupInstrumentation properly and to find problem around it.

Blocked by https://github.com/UnitTestBot/UTBotJava/issues/2092


TLS Config checking

currently we only dump the registry
recreating to an enumeration of current settings only would reduce the payload we generate

item was  applied to .netConfig checking
further redefining Schannel Reg dumps in future updates