Performance of calling methods from jitclass

Using Numba 0.29.0.
Calling methods of a jitclass is considerably slower than calling functions.
The following code is a small benchmark which shows calling methods is at best ~2.5 times slower than calling an analogous function.
The generic function names are to be understood in the manner that, e.g. method_function(x, inner_calls, outer_calls) calls a method outer_calls times which in turn calls a function inner_calls times.
For each benchmarked function timeit.repeat is called with an increased number of inner_calls (100 multiplied by 10 for each subsequent timeit.repeat call) to show that the increasing runtimes are indeed mostly dependent on the number of inner class (and not the outer calls).
Also notice that the type of the parameter x seems to determine whether do_something(x) and function(x) get inlined. If x is primitive a type like an int calls to function appear to get eliminated. The same does not seem to hold for Class.method, though.
from timeit import repeat

from numba import njit, jitclass

@njit
def do_something(x):
    pass

@jitclass([])
class Class:
    def __init__(self):
        pass

    def call_method(self, x, inner_calls):
        for _ in range(inner_calls):
            self.method(x)

    def call_function(self, x, inner_calls):
        for _ in range(inner_calls):
            function(x)

    def method(self, x):
        do_something(x)

@njit
def call_function(x, inner_calls):
    for _ in range(inner_calls):
        function(x)

@njit
def call_method(x, inner_calls, c):
    for _ in range(inner_calls):
        c.method(x)

@njit
def function(x):
    do_something(x)

@njit
def function_function(x, inner_calls, outer_calls):
    for _ in range(outer_calls):
        call_function(x, inner_calls)

@njit
def function_method(x, inner_calls, outer_calls):
    c = Class()
    for _ in range(outer_calls):
        call_method(x, inner_calls, c)

@njit
def method_function(x, inner_calls, outer_calls):
    c = Class()
    for _ in range(outer_calls):
        c.call_function(x, inner_calls)

@njit
def method_method(x, inner_calls, outer_calls):
    c = Class()
    for _ in range(outer_calls):
        c.call_method(x, inner_calls)

outer_calls = 10
inner_calls = 100
steps = 4
step_scale = 10
funcs = (function_function, method_function, method_method, function_method)
for x in (0, [0]):
    print(type(x))
    for func in funcs:
        func(x, 1, 1)
        ts = [min(repeat(lambda: func(x, step_scale**i * inner_calls, outer_calls),
                         number=1, repeat=5))
              for i in range(steps)]
        print(' {:<18}'.format(func.__name__), *map('{:.6f}'.format, ts))

output:
<class 'int'>
 function_function  0.000003 0.000003 0.000003 0.000003
 method_function    0.000003 0.000003 0.000003 0.000004
 method_method      0.000023 0.000198 0.001939 0.019516
 function_method    0.000026 0.000228 0.002048 0.022774
<class 'list'>
 function_function  0.000019 0.000153 0.001662 0.016331
 method_function    0.000021 0.000168 0.001659 0.016271
 method_method      0.000050 0.000456 0.004194 0.044940
 function_method    0.000047 0.000443 0.004078 0.042478

The biggest issue for myself is that method to method calls are slow. My use case is some kind of encoder which I model as a jitclass. To encode an input sequence, a method encode will call some other method for each element of the input. Thus, for long input sequences, there will be many calls from encode to the other method, which is rather slow due to the present issue.
Of course it would be nice if the other issues, i.e. inlining methods as well as inlining of do_something(x) / function(x) when x has a non-primitive type, are a being looked upon at some time. I for one will be glad if method to method calls could somehow be sped up.

+1. i'm also interested in faster methods and inlining with more complex objects. Will this be possible with upcoming type inference changes?

@sklam @seibert I looked into this a bit an found a couple of things
Accessing the method itself inside of a loop is slowed down because of the fact that the getattr numba IR ultimately lowers to an Assign operation inside the scope of a loop which means that every loop iteration is triggered an incref followed by a subsequent decref. This can be mostly alleviated by assigning the method itself to a variable. For example:
In [4]: @jitclass([('total', float64), ('count', int64)])
   ...: class Sum:
   ...:     def __init__(self):
   ...:         self.total = 0.0
   ...:         self.count = 0
   ...:
   ...:     def step(self, value):
   ...:         if not np.isnan(value):
   ...:             self.total += value
   ...:             self.count += 1
   ...:
   ...:     def finalize(self):
   ...:         return self.total if self.count else np.nan
   ...:

In [5]: @njit(float64(float64[:]))
   ...: def dont_do_this(values):
   ...:     agg = Sum()
   ...:     for value in values:
   ...:         agg.step(value)
   ...:     return agg.finalize()
   ...:
   ...: @njit(float64(float64[:]))
   ...: def do_this(values):
   ...:     agg = Sum()
   ...:     step = agg.step
   ...:     for value in values:
   ...:         step(value)
   ...:     return agg.finalize()
   ...:
   ...: @njit(float64(float64[:]))
   ...: def fast_sum(values):
   ...:     total = 0.0
   ...:     count = 0
   ...:     for value in values:
   ...:         if not np.isnan(value):
   ...:             total += value
   ...:             count += 1
   ...:     return total if count else np.nan
   ...:
   ...:

In [6]: values = np.random.randn(int(1e6))

In [7]: timeit dont_do_this(values)
29 ms ± 34.6 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [8]: timeit do_this(values)
16.4 ms ± 16.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [9]: timeit fast_sum(values)
1.71 ms ± 1.75 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [10]: nanvalues = np.random.rand(int(1e6))

In [11]: nanvalues[nanvalues > 0.5] = np.nan

In [12]: timeit dont_do_this(nanvalues)
32.1 ms ± 159 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [13]: timeit do_this(nanvalues)
16.8 ms ± 17.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [14]: timeit fast_sum(nanvalues)
1.72 ms ± 9.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [15]: timeit np.nansum(nanvalues)
6.45 ms ± 13.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

This still leaves about 9x on the table, for reasons that are unknown to me but that may not be related to method calls.
Is there a similar issue with accessing the fields on an instance of a class (refcounting up and down on every access)?

This is interesting!  We have a couple of heuristics before lowering to eliminate unnecessary incref/decref pairs to avoid similar performance issues, but apparently we need some more.

It looks like this would need to analyze the entire CFG, since the accounting is happening across basic blocks and nrtopt.py is optimizing within a single basic block.

@seibert Is it possible to avoid generating this kind of code entirely for instance member access?

Confirmed fixed by @alendit in gh-3531. Thanks!

@mbargull The performance problem that I show above wasn't fixed by #3531. Should I open a new issue?

I'm looking into this...

Can confirm the results reported by @cpcloud

values = np.random.randn(int(1e6))
%timeit dont_do_this(values)
%timeit do_this(values)
%timeit fast_sum(values)
--
61.7 ms ± 632 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
33.2 ms ± 1.23 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
3.18 ms ± 138 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

and
nanvalues = np.random.rand(int(1e6))
nanvalues[nanvalues > 0.5] = np.nan
%timeit dont_do_this(nanvalues)
%timeit do_this(nanvalues)
%timeit fast_sum(nanvalues)
%timeit np.nansum(nanvalues)
--
62.5 ms ± 100 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
34.3 ms ± 76.6 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
3.31 ms ± 257 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
21.4 ms ± 3.88 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

If the atomic part of refcounting is disabled then the following is obtained.

values = np.random.randn(int(1e6))
%timeit dont_do_this(values)
%timeit do_this(values)
%timeit fast_sum(values)
--
9.04 ms ± 113 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
6.24 ms ± 37.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
3.06 ms ± 6.19 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

and
nanvalues = np.random.rand(int(1e6))
nanvalues[nanvalues > 0.5] = np.nan
%timeit dont_do_this(nanvalues)
%timeit do_this(nanvalues)
%timeit fast_sum(nanvalues)
%timeit np.nansum(nanvalues)
--
11.6 ms ± 74 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
9.21 ms ± 208 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
3.09 ms ± 10.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
16.2 ms ± 769 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

which gets the factor down to 2-3x for the method based calls and I think strongly hints where most of the problem lies.

Looking at the assembly...
dont_do_this variant:
.LBB0_5:
	.loc	1 24 1
	movsd	(%r12), %xmm0
	.loc	1 25 1 is_stmt 1
	movsd	%xmm0, -88(%rbp)
	movq	%rbx, %rdi
	movabsq	$NRT_incref, %rax
	callq	*%rax
	movq	$0, -64(%rbp)
	leaq	-64(%rbp), %rdi
	movq	%r15, %rsi
	movq	%rbx, %rdx
	movq	%r13, %rcx
	movsd	-88(%rbp), %xmm0
	movabsq	$_ZN8__main__3Sum8step$243EN8instance8jitclass55Sum$237fcfca31c1d0$3ctotal$3afloat64$2ccount$3aint64$3eEd, %rax
	callq	*%rax
	cmpl	$-2, %eax
	je	.LBB0_4
.Ltmp11:
	testl	%eax, %eax
	jne	.LBB0_7
.Ltmp12:
.LBB0_4:
	.loc	1 24 1
	incq	%r14
	.loc	1 25 1
	movq	%rbx, %rdi
	movabsq	$NRT_decref, %rax
	callq	*%rax
	.loc	1 24 1
	addq	32(%rbp), %r12
	cmpq	24(%rbp), %r14
	jl	.LBB0_5

vs the do_this variant:
.LBB0_5:
	.loc	1 32 1
	movsd	(%r12), %xmm0
	.loc	1 33 1 is_stmt 1
	movq	$0, -64(%rbp)
	leaq	-64(%rbp), %rdi
	movq	%r15, %rsi
	movq	%r13, %rdx
	movq	%rbx, %rcx
	movabsq	$_ZN8__main__3Sum8step$243EN8instance8jitclass55Sum$237fdf83eaf518$3ctotal$3afloat64$2ccount$3aint64$3eEd, %rax
	callq	*%rax
	cmpl	$-2, %eax
	je	.LBB0_4
.Ltmp11:
	testl	%eax, %eax
	jne	.LBB0_7
.Ltmp12:
.LBB0_4:
	.loc	1 32 1
	incq	%r14
	addq	32(%rbp), %r12
	cmpq	24(%rbp), %r14
	jl	.LBB0_5

there's clearly an extra inc/dec ref pair in the loop in the dont_do_this case, also the Sum.step() function failed to inline in either case.
The assembly for Sum.step(), I've added comments with what's going on, essentially there's an incref on self, if a condition is true, some trivial maths, and then a decref on self. This would be running as a function call in the tight loop above.
_ZN8__main__3Sum8step$243EN8instance8jitclass55Sum$237f7b517a1be0$3ctotal$3afloat64$2ccount$3aint64$3eEd:
.Lfunc_begin6:
        .loc    1 12 0
        .cfi_startproc
        pushq   %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset %rbp, -16
        movq    %rsp, %rbp
        .cfi_def_cfa_register %rbp
        pushq   %r15
        pushq   %r14
        pushq   %rbx
        pushq   %rax
        .cfi_offset %rbx, -40
        .cfi_offset %r14, -32
        .cfi_offset %r15, -24
.Ltmp22:
        movsd   %xmm0, -32(%rbp)
.Ltmp23:
        movq    %rcx, %r15
        movq    %rdx, %rbx             ; %rbx = %rdx
        movq    %rdi, %r14
.Ltmp24:
        .loc    1 13 1 prologue_end
        movabsq $NRT_incref, %rax      ; move incref function to %rax
        movq    %rdx, %rdi             ; move %rdx (self) to %rdi as first arg
        callq   *%rax                  ; incref called with self as arg
        movsd   -32(%rbp), %xmm0       ; move `value` to xmm register
        ucomisd %xmm0, %xmm0           ; isnan check via unordered comparison
        jp      .LBB6_2                ; jump if parity (unordered)
.Ltmp25:                               ; This is the True branch of `if not np.isnan(value):`,
        .loc    1 14 1
        addsd   (%r15), %xmm0          ; self.total += value
        movsd   %xmm0, (%r15)          ; store result to total
        .loc    1 15 1
        incq    8(%r15)                ; self.count += 1
        movabsq $NRT_decref, %rax      ; move decref function to %rax
        movq    %rbx, %rdi             ; move %rbx to %rdi, (i.e. move self as first arg)
        callq   *%rax                  ; decref called %rbx==%rdx, is this a reference to self
        xorl    %ebx, %ebx
.Ltmp26:
.LBB6_2:                               ; This is the False branch of `if not np.isnan(value):`, it does nothing, just decrefs self and sets up for return
        movabsq $NRT_decref, %rax      ; Same pattern as above for doing a decref on self
        movq    %rbx, %rdi
        callq   *%rax
        movq    $0, (%r14)
        xorl    %eax, %eax
        addq    $8, %rsp
        popq    %rbx
.Ltmp27:
        popq    %r14
        popq    %r15
        popq    %rbp
        .cfi_def_cfa %rsp, 8
        retq
.Lfunc_end6:
        .size   _ZN8__main__3Sum8step$243EN8instance8jitclass55Sum$237f7b517a1be0$3ctotal$3afloat64$2ccount$3aint64$3eEd, .Lfunc_end6-_ZN8__main__3Sum8step$243EN8instance8jitclass55Sum$237f7b517a1be0$3ctotal$3afloat64$2ccount$3aint64$3eEd
        .cfi_endproc

The main loop, once inlined looks like:
        movabsq $NRT_decref, %r15
        .p2align        4, 0x90
.LBB0_2:
        movsd   (%r13), %xmm0
        movsd   %xmm0, -64(%rbp)
        movq    %r12, %rdi
        movabsq $NRT_incref, %rax
        callq   *%rax
        movsd   -64(%rbp), %xmm0
        ucomisd %xmm0, %xmm0
        movq    %r12, %rdi
        jp      .LBB0_4
        addsd   (%r14), %xmm0
        movsd   %xmm0, (%r14)
        incq    8(%r14)
        movq    %r12, %rdi
        callq   *%r15
        xorl    %edi, %edi
.LBB0_4:
        callq   *%r15
        addq    32(%rbp), %r13
        decq    %rbx
        jne     .LBB0_2

Redefining Sum as:
@jitclass([('total', float64), ('count', int64)])
class Sum:
    def __init__(self):
        self.total = 0.0
        self.count = 0

    def step(self, value):
        foo = 0
        if not np.isnan(value):
            foo = 1
        self.total += value
        self.count += foo

    def finalize(self):
        return self.total if self.count else np.nan

such that the references to self in Sum::step are in the same block yields but the level of work is about the same yields:
values = np.random.randn(int(1e6))
%timeit dont_do_this(values)
%timeit do_this(values)
%timeit fast_sum(values)
--
3.27 ms ± 40.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
1.57 ms ± 15.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
3.08 ms ± 4.36 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

and the assembly:
.LBB0_8:
        movsd   (%rsi), %xmm1
        xorl    %edx, %edx
        ucomisd %xmm1, %xmm1
        setnp   %dl
        addq    %rax, %rdx
        addsd   %xmm1, %xmm0
        movsd   %xmm0, (%rbx)
        movq    %rdx, 8(%rbx)
        movsd   (%r10,%rsi), %xmm1
        addq    $2, %rdi
        xorl    %eax, %eax
        ucomisd %xmm1, %xmm1
        setnp   %al
        addsd   %xmm1, %xmm0
        movsd   %xmm0, (%rbx)
        addq    %rdx, %rax
        movq    %rax, 8(%rbx)
        addq    %rcx, %rsi
        cmpq    %rdi, %r13
        jne     .LBB0_8
        testq   %r9, %r9
        je      .LBB0_5

the loop is unrolled and there's no inc/dec refs present.
I think that this confirms that the issue here really is the inability to prune reference counts across basic blocks, this is exacerbated in the case of jitclass whereby the prevalence of self leads to many more {inc, dec}refs being required.

@stuartarchibald Thanks for digging in. In the meantime, it looks like avoiding any control flow inside of methods might be a viable workaround.

@cpcloud no problem.

In the meantime, it looks like avoiding any control flow inside of methods might be a viable workaround.

I wasn't sure about suggesting that as it might lead to unnatural/obfuscated code :), but it should yield the performance as demonstrated in https://github.com/numba/numba/issues/2166#issuecomment-519089015. I think it will likely come down to branches containing references to self preventing the current reference count pruner from working. We have plans for a more advanced reference count pruning pass but will have to implement this as an LLVM level pass which is in itself quite involved.


Searching with a plus sign (+) does not work

Hi,
Because of the way the searching uses a regex if you try and search with a plus sign (+) it fails.
A workaround is to escape the plus sign with a backslash but that's not ideal.

Yeah, there are so many characters here that mess up the underlying regex . We should do an escapeRegex on the string passed to mongo. We basically stopped using the inbuilt search mostly but it's worth fixing so you don't see mongo server errors pop up when somebody types * into the search box.


[stable/spring-cloud-data-flow] Update to new SCDF version 2.5.0

What this PR does / why we need it:

update scdf version to 2.5.0
update skipper version to 2.4.0
minor cleanups

Signed-off-by: Chris Schaefer<EMAIL_ADDRESS>Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

/assign @sabbyanandan


[ja] Localize How To Contribute

Localize https://glossary.cncf.io/contribute/

I would like to work on it .

Hi @aryasoni98,
Thank you for your interest on Japanese localization task, but this task needs a native Japanese speaker skills like 「てにおは」「ですます」 and discuss in Japanese with approvers. Is this alright with you?
https://github.com/cncf/glossary/pull/2140#issuecomment-1610418343 helps you.

Closing as it is already localized in https://github.com/cncf/glossary/pull/2088


SSR Prerendering not working with Router 3.0 and Preact X

Recreate:

npm install -g preact-cli
preact create default my-project
cd my-project
Add a prerender-urls.json into rootdir

[
    {
        "url": "/",
        "title": "Home"
    },
]

rm -rf node_modules package-lock.json
npm uninstall preact preact-router --save && npm install preact@next preact-router@next
npm run build

Will throw this error:
matty@laptop: ~/Github/genesis/www/mh on master [+!?]
$ npm run build

><EMAIL_ADDRESS>build /Users/matty/Github/genesis/www/mh
> preact build

Build [==                  ] 10% (0.0s) building modules[Browserslist] Could not parse /Users/matty/Github/genesis/package.json. Ignoring it.
Build [=================== ] 95% (2.2s) emittingUnable to read file: /Users/matty/Github/genesis/www/mh/src/node_modules/preact/hooks/dist/hooks.module.js

TypeError: Cannot read property '__H' of undefined
method: null
at: /Users/matty/Github/genesis/www/mh/src/node_modules/preact/hooks/dist/hooks.module.js:1:257

Source code:

This could be caused by using DOM or Web APIs.
Pre-render runs in node and has no access to globals available in browsers.

Consider wrapping code producing error in: 'if (typeof window !== "undefined") { ... }'

Alternatively use 'preact build --no-prerender' to disable prerendering.

See https://github.com/developit/preact-cli#pre-rendering for further information.npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>build: `preact build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      /Users/matty/.npm/_logs/2019-04-26T18_23_42_875Z-debug.log

If you change the router to 2.6.1 and preact to latest ssr/prerender works just fine.
Any help?

https://github.com/developit/preact-render-to-string/releases/tag/5.0.2
fixed in this preact-render-to-string release, but needs to be updated in preact-cli

@kidqueb I installed<EMAIL_ADDRESS>and it seems to be getting further; however, now I'm getting a new error
matty@laptop: ~/Github/genesis/www/mh on logo [!]
$ npx preact build
Build [] 95% (2.7s) emitting
 ssr-bundle.60186.css ⏤  495 B (+495 B)
        ssr-bundle.js ⏤  17.8 kB (+17.8 kB)

Build [] 10% (0.0s) building(node:13069) DeprecationWarning: Tapable.apply is deprecated. Call apply on the plugin directly instead
Build [] 90% (1.2s) module assets processing✖ ERROR TypeError: Cannot read property 'hasErrors' of undefined
    at showStats (/Users/matty/Github/genesis/www/mh/node_modules/preact-cli/lib/lib/webpack/run-webpack.js:118:12)
    at compiler.run (/Users/matty/Github/genesis/www/mh/node_modules/preact-cli/lib/lib/webpack/run-webpack.js:106:4)
    at finalCallback (/Users/matty/Github/genesis/www/mh/node_modules/webpack/lib/Compiler.js:220:39)
    at onCompiled (/Users/matty/Github/genesis/www/mh/node_modules/webpack/lib/Compiler.js:228:20)
    at compilation.seal.err (/Users/matty/Github/genesis/www/mh/node_modules/webpack/lib/Compiler.js:626:23)
    at hooks.optimizeChunkAssets.callAsync.err (/Users/matty/Github/genesis/www/mh/node_modules/webpack/lib/Compilation.js:1317:14)
    at _promise1.then._err1 (eval at create (/Users/matty/Github/genesis/www/mh/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:22:1)


Removing repetitive work

Description of  the change
Describe more about the change.
We are taking the same ss of compiling and deploying process which is becoming a friction as it is the same process in any contract ,
So we can make a simple file to tell everyone of how to run the contract on Remix.
Domain

[x] Readme
[] Solidity
[] Python
[] JavaScript
[] Golang
[x] Other , feel free to mention below

Are you contributing under any open-source program ?
SSOC22

@Vikash-8090-Yadav let me know of your thoughts on this

Graet issue @legendarykamal  start working on this


Removing PDF Generation as it is not supported in reference builds

Docs Build status updates of commit 98db9c4:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:x:Error

Details

.openpublishing.publish.config.json

[Error-InvalidPublishConfigFile]

Publish service's config file: .openpublishing.publish.config.json is not valid. Details: System.ArgumentException: Invalid json string for type Microsoft.OpenPublishing.Build.DataContracts.GitModel.PublishConfig. ---> Newtonsoft.Json.JsonSerializationException: Error converting value "need_generate_intellisense" to type 'Microsoft.OpenPublishing.Build.DataContracts.GitModel.DependentRepository'. Path 'dependent_repositories[1]', line 249, position 30. ---> System.ArgumentException: Could not cast or convert from System.String to Microsoft.OpenPublishing.Build.DataContracts.GitModel.DependentRepository.
   at Newtonsoft.Json.Utilities.ConvertUtils.EnsureTypeAssignable(Object value, Type initialType, Type targetType)
   at Newtonsoft.Json.Utilities.ConvertUtils.ConvertOrCast(Object initialValue, CultureInfo culture, Type targetType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)
   --- End of inner exception stack trace ---
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at Microsoft.OpenPublishing.Build.Common.JsonUtility.FromJsonString[T](String value) in d:\a\1\s\Libraries\Common\JsonUtility.cs:line 95
   --- End of inner exception stack trace ---
   at Microsoft.OpenPublishing.Build.Common.JsonUtility.FromJsonString[T](String value) in d:\a\1\s\Libraries\Common\JsonUtility.cs:line 103
   at Microsoft.OpenPublishing.Build.WorkflowTasks.SyncGitSourceWithDownloadingPackage.CheckPublishConfig() in d:\a\1\s\Libraries\WorkflowTasks\SyncGitSourceWithDownloadingPackage.cs:line 68

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 9d64794:
:clock10: Full build: the average full build time is 17 min(s) 41 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 9d64794:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Can not find runtime target for framework 'DNXCore,Version=v5.0' and RID's 'win10-x64, win81-x64, win8-x64, win7-x64'. Possible causes

I use 'dotnet restore' in the following error：
log  : Restoring packages for D:\core\samples\dotnetbot\project.json...
error: Unable to resolve NETStandard.Library (>= 1.0.0-rc2-23704) for DNXCore,Version=v5.0.
info : Committing restore...
log  : Restore failed in 312ms.
Errors in D:\core\samples\dotnetbot\project.json
Unable to resolve NETStandard.Library (>= 1.0.0-rc2-23704) for DNXCore,Version=v5.0.
NuGet Config files used:
I use 'dotnet run' in the following error：
Can not find runtime target for framework 'DNXCore,Version=v5.0' and RID's 'win10-x64, win81-x64, win8-x64, win7-x64'. Possible causes:

Project is not restored or restore failed - run dotnet restore
Project is not targeting runable framework (netstandardapp* or net*)

How do I fix it

I do have same issue

Same issue here

One issue you have is there's errors in your restore.

maybe you choose the wrong NuGet Feeds, the default Feed don't have NETStandard.Library yet.
try
dnu restore -s https://myget.org/f/dotnet-core

or write a NuGet.Config file adding the myget feed into it.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
    <add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
    <add key="api.nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
  </packageSources>
</configuration>

After dotnet restore, i try to run dotnet run and get this error:
Can not find runtime target for framework 'DNXCore,Version=v5.0' and RID's 'win7-x64'. Possible causes:

Project is not restored or restore failed - run dotnet restore
Project is not targeting runable framework (netstandardapp* or net*)

My problem was solved: problem is when vmware or virtualpc network adapters are activated in system. Disabling helps.

Closing this old issue.  Please feel free to reactivate if there is still a problem.
Instructions are available at http://dot.net/core

hi,
are you update  to .NET Core 1.0 for Visual Studio?.
Please change your frameworks list in 'Project.json' by
"frameworks": {
"netcoreapp1.0": {
"dependencies": {
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.0"
}
},
"imports": "dnxcore50"
}
},
rebuild your application
Good luck


Update to Godot 3.5.3

ref: https://godotengine.org/article/maintenance-release-godot-3-5-3/
ref: https://blog.isosceles.com/the-webp-0day/

Done https://gitlab.com/fdroid/fdroiddata/-/commit/b22dad8f0b624fadb3fe46e46e1ec4a1589d9644 thanks

Thank you too, for updating the recipe.


Scraping multiple pages of Google, unexpected results

The problem
Selenide scrapes the first page of google results as expected. For second the scraped results show a mix of first and second page. When using non-headless mode, the results on the second page are actually different from first page in full.
Details
The results of the code example for first page:
http://www.cs.otago.ac.nz/cosc348/alignments/Lecture04_StringSearch.pdf
http://www.ajer.org/papers/v6(11)/ZA0611216222.pdf
http://care.diabetesjournals.org/content/diacare/suppl/2016/08/12/39.9.1643.DC1/DC152171SupplementaryData.pdf
http://stephanus.tlg.uci.edu/helppdf/text.search.pdf
http://dimacs.rutgers.edu/BMC/TeacherMaterials/SuffixTrees.pdf
https://link.springer.com/content/pdf/10.1007/3-540-51324-8_38.pdf
https://www.toddington.com/wp-content/uploads/Facebook-Search-Strings-2.pdf
https://www.eiseverywhere.com/file_uploads/8a9341f04ab274ca10d993444d2002c0_4.ODonoghueVince.pdf
https://www.loggly.com/wp-content/uploads/2017/08/Loggly-Search-Cheat-Sheet.pdf
https://www.bu.edu/tech/files/2016/06/BW-How-to-Use-Wildcards-in-Searches.pdf
the results for second page:
http://www.cs.otago.ac.nz/cosc348/alignments/Lecture04_StringSearch.pdf
http://www.ajer.org/papers/v6(11)/ZA0611216222.pdf
http://care.diabetesjournals.org/content/diacare/suppl/2016/08/12/39.9.1643.DC1/DC152171SupplementaryData.pdf
http://stephanus.tlg.uci.edu/helppdf/text.search.pdf
http://dimacs.rutgers.edu/BMC/TeacherMaterials/SuffixTrees.pdf
https://link.springer.com/content/pdf/10.1007/3-540-51324-8_38.pdf
https://www.toddington.com/wp-content/uploads/Facebook-Search-Strings-2.pdf
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/fundamentals/configuration/15_sy/fundamentals-15-sy-book/cf-cli-search.pdf
https://ieeexplore.ieee.org/iel5/4/22616/01052819.pdf
http://www.cs.princeton.edu/courses/archive/fall11/cos226/lectures/53SubstringSearch.pdf
Notice how the first 7/10 results are the same. It varies how many of them are the same.
Tell us about your environment

Selenide Version: 4.14.1
Chrome\Firefox\IE Version: Firefox 64.0
Browser Driver Version: selenide default
Selenium Version: selenide default
OS Version:  macos mojave 10.14

Code To Reproduce Issue [ Good To Have ]
a simple scala template to reproduce:

import com.codeborne.selenide.{CollectionCondition, Configuration, ElementsCollection, Selenide}
import com.codeborne.selenide.Selenide.{$, $$}
import org.openqa.selenium.By

object GoogleMultipage extends App {

  val includeNonEmpyLinks: String = "string-length(@href)>3"
  val excludeCacheLinks: String = "not(@class='fl')]"

  Configuration.headless=false

  Selenide.open("http://www.google.com")

  // set Google search bar to a search string and press enter
  $(By.name("q"))
      .setValue("""
                  |"search"+"string" -"negative" filetype:pdf
                """.stripMargin)
      .pressEnter()

  // get google result page 1 (10 results per page)
  val res1 = $$(By.xpath(s"//div[@class='srg']//div[@class='g']//div//div[@class='rc']//div[@class='r']//a[$includeNonEmpyLinks and $excludeCacheLinks"))
      .shouldBe(CollectionCondition.size(10))

  for (j <- 0 to 9) println(res1.get(j).attr("href"))

  // scroll down and go to page 2
  $(By.xpath(s"//a[@aria-label='Page 2']")).scrollTo().click(1,1)

  // get google result page 2 (10 results per page)
  val res2 = $$(By.xpath(s"//div[@class='srg']//div[@class='g']//div//div[@class='rc']//div[@class='r']//a[$includeNonEmpyLinks and $excludeCacheLinks"))
      .shouldBe(CollectionCondition.size(10))

  for (j <- 0 to 9) println(res2.get(j).attr("href"))

}

The thing is that after click you are changing from page1 to page2, but
starting the evaluate 10 result immediately. So 7 are gathered from page 1,
3 from page 2 in your case. How to avoid it? Basic idea: try to figure out
the moment when you are definitely on page 2. for example:
$(pageno).shouldHave(exactText("2"))
Or $(firsthit_from_page1).should(disappear)
Only after that - $$()...

Yes, it is solved, except that if you have a construct of 10 items with
some layout, which are replaced with 10 items of identical layout (but
different texts) it is impossible to tell arbitrary of 10 items you see are
old or new. So you need some anchor on inhalt that get changed - like page
number or concrete text of an item. In your example the anchor was - 10
results which is same every time

After the click to second page, added the following:
  $(By.xpath(s"//td[@class='cur']")).shouldBe(Condition.exactText("2"))

Google changes the page link to text and adds class cur.
After this continued with the $$(..
Did one round of manual testing with the same browser. This seems to have fixed it.
Thanks @vinogradoff ! Closing this.


[ObservableLookup] implementation

I need an ObservableLookup which implements ILookup<TKey, TValue> interface?
Can I contribute it or will you implement it?

I think ILookup is readonly so no observable.

Even though it is read-only our lookup class can be mutable. The main point is that it has to work like an ObservableDictionary storing ObservableCollections as values

Value changes not be monitoring other types either.

I have not gotten your point

This is what I implemented
https://github.com/Ecierge/Elmish.Uno/tree/eCierge/src/Elmish.ObservableLookup
from this as a source
https://github.com/xamarin/Xamarin.PropertyEditing/blob/972186e6ddd68566375da7bc336e566196f44307/Xamarin.PropertyEditing/ObservableLookup.cs

Can we port that to you?

What do you think?

From what I see, you are still adding a mutable type called IMutableLookup, right?

In my sample, I do. However, it is not mandatory

Any thoughts?

not positive.


iphoneh padding, flex for buttons

Added padding to iphoneh class.  Moved .carti higher up in the CSS. Created a button-container to hold iphone buttons so they could have flex display of row on larger screens and column on small screens. Adjust width of Add Cart button to better match the widths of the iphone buttons.
This is a first time using github to work on open source - forgive any mistakes along the way :)

Make sure to follow Hamim Reza & star the repo.


Add LeftEyeBlink/RightEyeBlink boolean parameters

It would be neat to have LeftEyeBlink and RightEyeBlink boolean parameters that indicate whether the eye is fully closed to allow for simple blink animations that use 1 parameter bit each instead of 8 as with the LeftEyeLid and RightEyeLid parameters.

This will be added in the next update. It will be named the same as the normal float params (LeftEyeLid and RightEyeLid) but will register true while someone is blinking beyond 0.85 and false when the eye is open.


Use script module instead of shell module to run verify-maas tasks.

Fixes #769
(cherry picked from commit 42311f617d6a1e48277f379b501f54966c8946d2)
Signed-off-by: Matthew Thode<EMAIL_ADDRESS>

:+1: straightforward backport

travis timed out connecting to pypi

recheck


Issue 102

Pull Request Review
Checklist

[x] Issue: https://github.com/IBM/kubernetes-driver/issues/102
[x] List of related PRs : N.A.
[x] Project builds without any errors.
[x] Unit Tests - all pass.
[x] Ran manual functional “smoke” test (does product as a whole still work?) :  Yes, QA has validated the dev image and everything is working fine. They have uploaded the test results in IBM internal user stories.
[x] User Story meets the acceptance criteria - and passes. acceptance criteria should be understood at outset.
[x]  Description of the changes:

Enable Logging for External Request & Response. Drivers will be updated to log specific messages with details of requests & responses they send and receive from external systems. Kubernetes driver makes multiple API calls to a Kubernetes API server. Each call is a request/responses pair which the driver will log.

Hi @HariKrishna1-ibm,
Gentle Reminder! Can you please review it. Thanks.


5.4.0+1版本有问题

该版本在键盘弹出时会导致ScreenUtilInit的builder方法重复调用

Hello friend, do you have any better idea?

I changed it to use ScreenUtil.init instead of ScreenUtilInit

@fuyunmanbu it's recommanded to use ScreenUtil.init() unless you want to use the library to build MaterialApp's themedata. However, in the next few weeks, there will be an upgrade, especially in performance. So stay tuned and you're welcome to give any idea or PR. You can contact me via email if you want<EMAIL_ADDRESS>

@fuyunmanbu I've updated the library and the error is fixed. Go check it out.


Run shared subscriptions tests on mqttv4 and mqttv5

Make the shared subscription tests run on both mqttv4 and mqttv5...

btw didn't add a changelog to this as I consider this purely internal stuff.


merge package template from Kronos-Tools/npm-package-template

package.json

chore(devDependencies): add<EMAIL_ADDRESS>from template
chore(devDependencies): add<EMAIL_ADDRESS>from template

Codecov Report

Merging #146 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #146   +/-   ##
=======================================
  Coverage   61.53%   61.53%
=======================================
  Files           1        1
  Lines          13       13
=======================================
  Hits            8        8
  Misses          5        5

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29a6492...20bcdf9. Read the comment docs.


FID not matching on CIFAR10

I downloaded CIFAR10 500K checkpoint from readme, sampled it for 10k images and my FID points to > 200. I tested few different implementations of FID calculations and am comparing it to CIFAR10 50k training data. It's ridiculously high, nowhere near 3 and my models trained on CIFAR10 also show high FID after many iterations. What could be wrong in this case?

Even I am facing the same issue, computing FID with dims 64 for InceptionNet gives me FID of 1.52

I downloaded CIFAR10 500K checkpoint from readme, sampled it for 10k images and my FID points to > 200. I tested few different implementations of FID calculations and am comparing it to CIFAR10 50k training data. It's ridiculously high, nowhere near 3 and my models trained on CIFAR10 also show high FID after many iterations. What could be wrong in this case?

The FID I got was ridiculously high too.Have you figured out what the problem is?

I was able to get the FID of 4.41 for hybrid loss and 7.88 for VLB loss (3.19 and 11.47 as stated in paper) with dividing the samples with 255. so they are float type instead of uint8 and also the original dataset is saved  in JPEG format.


Crashes on run if config file has perm error..

node-debug was crashing for me when it tried to open the browser..
Changing the launcher() call to this seems to keep it from crashing, assuming tmpdir is accessible, and otherwise helpfully prints out the error.
var tmpdir = require( 'os' ).tmpdir();
launcher( require( 'path' ).join( tmpdir, '.config' ), function ( err, launch ) {
 if ( err ) {
  if ( cfg.verbose ) {
   console.log( 'Error:', err.message );
  }
 }

Hello, that's interesting, thx for the report, I will take a look on it shortly!


Segwit bech32

This pull request brings us up to date with the latest changes done to breadwallet-core, most of them related to Ethereum but the merge will show all the commits, also we deleted the Ethereum files regardless, including all our changes and fixes, it adds the following:

Updates our source code to the latest pushed by BRD.
Adds default bech32 to receive addresses
Adds checkpoints for faster syncing times
Adds tests for new cypher functions
Updates Java bindings

Please test, it's currently implemented on Android in the segwit-bech32 branch, legacy addresses are under Menu -> Settings -> Wallet Section -> Show Legacy Address.
TODO

[x] iOS Support

This has been implemented in iOS and Android branches pull requests added:
https://github.com/hodlwallet/hodl-wallet-android/pull/37
https://github.com/hodlwallet/hodl-wallet-ios/pull/58


Straighten filaments

Have you considered adding a button [Straighten], which would render the intensities along the filament in a new coordinate system where the filament would run straight long the x axis?

Hello @tischi
Do you mean, like "Plot Profile" in ImageJ?
Yes, I plan to work on this and + some additional basic measurements (length, ends distributions, etc) in the near future.

Hi @tischi ,
fixed in the lastest release 0.0.5.
So far only for a one pixel thickness, but more is coming.

Super cool! Thanks a lot!!


Update docs for chef-server-automate reporter

Fixes #241
@vjeffrey @kekaichinose, should be also cleanup any doc references to the integration with the original Compliance product?

@alexpop per #241, yeah, I would update this doc to clean up the reference:
https://github.com/chef-cookbooks/audit/blob/master/docs/supported_configuration.md
Thanks!

ya, +1 on the cleanup, the original compliance has been eol for a while now

Merging this now as https://github.com/chef-cookbooks/audit/pull/385 went in as well


您好，能够出一份详细的使用视频教程？

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

能说一个用途吗？现在的功能并不复杂

主要是想有个具体详细怎么做精准测试的流程哈

可以自己录啊

行吧，谢谢了


agc context deploy fail for regions in China

I am playing with amazon-genomics-cli but found agc context deploy does not work for an AWS account in a China region (cn-northwest-1).
agc context deploy myContext
2022-05-22T05:51:44-06:00 𝒊  Deploying context(s)
2022-05-22T05:51:52-06:00 ✘  Failed to deploy context 'myContext'. Below is the log for that deployment
2022-05-22T05:51:52-06:00 ✘   error="1 context deployment failures"
Error: an error occurred invoking 'context deploy'
with variables: {contexts:[] deployAll:false}
caused by: 1 context deployment failures
suggestion: To resolve failure 1, determine the cause of: operation error ECR: ListImages, https response error StatusCode: 400, RequestID: xxx, api error AccessDeniedException: User: arn:aws-cn:iam::<myaccountid>:user/<myusername> is not authorized to perform: ecr:ListImages on resource: arn:aws-cn:ecr:cn-northwest-1:680431765560:repository/aws/cromwell-mirror because no resource-based policy allows the ecr:ListImages action

It looks like that it tries to pull image from a mirror arn:aws-cn:ecr:cn-northwest-1:680431765560:repository/aws/cromwell-mirror in that region, but the mirror may not really exist.
Related: #141
AGC Version: 1.4.0

Hi Marilyn, currently the containers used by AGC are only distributed to the aws partition. We don't yet support additional partitions such as aws-cn or aws-us-gov.
One work around would be to change the region if your profile to a nearby AWS region. Alternatively, you could pull the container, retag it and then load it to an ECR repository in your account. Having done this you would then need to set environment variables to point to this image.

Thank you for the quick response! I will try to set up my own ECR mirror for cromwell.


Error on attempt to run tests in parallel mode with nodejs 15.11

Prerequisites

[x] Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
[x] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
[x] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
[x] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description
Noticed that on attempt to run mocha in parallel mode with nodejs 15.11 (also with 15.8) causes error.
Steps to Reproduce
You can clone my project (specific commit with with parallel enabled)- https://github.com/Xotabu4/pet-store-tests-READY/commit/ee6160b7fb36eb8f535f770271ee3665f3fbe915
it has package-lock.json file with all deps tree to reproduce correctly.
Just run:
npm cit

Expected behavior:
Parallel mode expected to be working
Actual behavior: [What actually happens]
➜  pet-store-tests-READY git:(7-configuration) ✗ npm run ttest

><EMAIL_ADDRESS>ttest
> mocha

TypeError [ERR_INVALID_ARG_TYPE]: The "options" argument must be of type object. Received an instance of Array
    at ChildProcess.target.send (node:internal/child_process:716:7)
    at Array.forEach (<anonymous>)
    at dispatchQueuedRequests (/mnt/c/GitHub/api-project-examples/pet-store-tests-READY/node_modules/workerpool/src/WorkerHandler.js:250:21)
    at ChildProcess.<anonymous> (/mnt/c/GitHub/api-project-examples/pet-store-tests-READY/node_modules/workerpool/src/WorkerHandler.js:209:7)
    at ChildProcess.emit (node:events:378:20)
    at ChildProcess.EventEmitter.emit (node:domain:470:12)
    at emit (node:internal/child_process:920:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

Reproduces how often: 100
Versions

The output of mocha --version and node node_modules/.bin/mocha --version: 8.2.1
The output of node --version: v15.11.0 and 15.8
Your operating system

name and version: WSL2 ubuntu 18 (Windows 10 pro)
architecture (32 or 64-bit): x64

Your shell (e.g., bash, zsh, PowerShell, cmd): zsh
Your browser and version (if running browser tests): api tests, no browser
Any third-party Mocha-related modules (and their versions): not related
Any code transpiler (e.g., TypeScript, CoffeeScript, Babel) being used (and its version): ts-node (9.1.1), typescript (4.1.3)

Works fine with
parallel: false in mocharc.js

@Xotabu4 Mocha v8.3.0 should be fine. Please try and revert back, thanks.

@Xotabu4 Mocha v8.3.0 should be fine. Please try and revert back, thanks.

Indeed, bumping to 8.3.1 solves the problem. Thanks!


Blestress fixes

@mkasenberg please review these changes

There is still crash in test 14 - I'll update a PR with fix

All crashes are resolved - the only problem left is silent restart on RX side during test 9 if test 7 was executed before - for now, skip test 7. Reason of this restart is yet unknown - there is no crash visible in debugger, no crash information is printed to console, HCI trace only shows lack of activity - basically, restart causes RTT to deattach.  All tests are working if run independently.

please rebase


Remove t.is and t.not from enhanced patterns

Somehow forgot to remove them before landing magic-assert. Needed to produce this:

instead of this:

I think this should've been 3.0.0 release. And https://github.com/avajs/ava/commit/154215df51fb1fbe9f5f5ca2f76e00b46d30c7d3 should list that as its minimum version (or at least 2.0.1).
Not quite sure what happens if somebody has AVA 0.18.0 with version 2.0.1 of this preset.

I didn't make it a major release on purpose, so that all new AVA 0.18.0 installations would pick up 2.0.1 of this preset (since semver covers it).

Yes, though does that work as expected without the corresponding change in https://github.com/avajs/ava/commit/154215df51fb1fbe9f5f5ca2f76e00b46d30c7d3?
I think generally we should bump the versions in AVA, e.g. with https://github.com/avajs/pretty-format/releases/tag/v1.0.2.

I think this should've been 3.0.0 release.

You could probably argue either way. It doesn't really break anything though. It only slightly changes the output, which in AVA is really a bug fix.

And avajs/ava@154215d should list that as its minimum version (or at least 2.0.1).

I agree. Generally if you need a new feature in the release, even if it's covered by semver, you should ensure it can never be lower. That could happen if the dependency tree has multiple versions of it.

Yes, though does that work as expected without the corresponding change in avajs/ava@154215d?

@novemberborn No, I don't think so. I think you were right initially, it should've been 3.0.0. Will know next time!


Get helper to fetch featured posts, else block not executing

I'm using the get helper to output a featured posts carousel. If there are no featured posts, blog cover image should be displayed instead. It was working fine with Ghost 0.11.x version, but with Ghost 1.0 the else block is not executing even though there are no posts marked as featured.
Here is my code:
{{#get "posts" filter="featured:true" limit="5" include="author,tags" as |featured|}}

        {{#foreach featured}}

                {{#if feature_image}}

		{{/if}}

		    {{#if tags}}

			    {{tags separator=""}}

		    {{/if}}

{{title}}

	{{/foreach}}

{{else}}

	{{#if @blog.cover_image}}

	{{/if}}

	    <h2<EMAIL_ADDRESS>	    <p<EMAIL_ADDRESS>

{{/get}}

Technical details:

Ghost Version: 1.0
Node Version: 6.11.1

Hi @astab, this behaviour deliberately changed between 0.11.x and 1.0.
The way this worked in 0.11.0 was confusing, because the {{else}} needed to belong to get instead of foreach, and yet it was the foreach helper that most people are used to and which had the else block documented. Following a discussion it was decided to change this.
Unfortunately, this has been missed in our changelog for 1.0.0 and other documentation. I'm in the process of updating that now and have added a new section to the docs.
Hopefully this gets you on your way!
Closing this as we're tracking out standing documentation tasks

Hi @ErisDS,
thank you for your reply. In my opinion, {{else}} should be the part of the {{get}} helper. It is very limited now. If I add any other elements inside {{get}} helper before {{#foreach}} or wrap the {{#foreach}} helper, these elements will be displayed even if there are no results. For instance, in this simple example the heading and empty ul tag will be displayed even if there are now featured posts.
{{#get "posts" filter="featured:true" limit="5"}}

Featured posts

      {{#foreach posts}}

          {{title}}

      {{/foreach}}

{{/get}}

@astab in this case you should be able to use an explicit {{#if}} conditional:
{{#get "posts" filter="featured:true" limit="5" as |featured_posts|}}
  {{#if featured_posts}}

Featured posts

        {{#foreach featured_posts}}

            {{title}}

        {{/foreach}}

  {{/if}}
{{/get}}

Thank you, @kevinansfield. That solves my problem.

@astab totally get where you're coming from. @kevinansfield's answer is correct and it is also  possible to do things like:
{{#foreach featured_posts}}
   {{#if @first}}... something here{{/if}}
   {{#if @last}}... something here {{/if}}
{{/foreach}}

So, whilst the old way this worked was more convenient in a certain sense, I don't think we have added any limitations here, only improved consistency as these same patterns are required when you're not using the {{#get}} helper.


Tokenize strings of only N-types of characters?

Hi, I'm looking at this for tokenizing biological sequences: protein, DNA, RNA. These have between 4-22 letters, generally. When I use the procedure, it only finds the base-letters as tokens. The vocabulary that is produced  consists of the base ascii characters
less vocab.txt
^A
^B
^C
^D
^E
^F
^G
^H

^K
^L
^M
^N
^O
^P
^Q
^R
^S
^T
^U
^V
^W
^X
^Y
^Z
^[
^\
^]
^^
^_

!
"
#
$
%
&
'
(
)
*
+
,
-
.
/
0
1
2
3
4
5
6
7
8
9
:
;
<
=
>
?
@
A
B
C
D
...

Even though none of those characters, except for ACGT were in the input text. The code I entered was this:
./getalltokens -charset utf8 -chunk-size 10000 -dataset my_data.txt -min-occur-chunk 2 -output vocab.vc -workers 2
./trainvocab -charset utf-8 -dataset my_data.txt -dir vocab_dir -dictionary vocab.vc -max-token-length 64 -vocab 4095
./exporttokens ./vocab_dir/794556_5737.zlib vocab -capcode -charset UTF-8 -txt

Can you advise?

Yes, there is a parameter during training that adds all base characters, which you need to disable.
For your purpose you want to ensure you use charset "binary". With getalltokens there is a parameter for -include-single-bytes, set that to false. Then on trainvocab you need to set the -no-reserve-256 flag.

Thanks for the help! Would you have a buy-me-a-coffee? I always appreciate those dealing with my silly mistakes.

It worked very well. Thank you again!


EndDevice cannot sleep before start commissioning (TZ-738)

Answers checklist.

[X] I have read the documentation ESP Zigbee SDK Programming Guide and tried the debugging tips, the issue is not addressed there.
[X] I have updated ESP Zigbee libs (esp-zboss-lib and esp-zigbee-lib) to the latest version, with corresponding IDF version, and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.2.1
esp-zigbee-lib version.
1.2.2
esp-zboss-lib version.
1.2.2
Espressif SoC revision.
ESP32-C6
What is the expected behavior?
Devices startes in factory reset mode with autostart=false, without starting commissioning and starts sleeping
What is the actual behavior?
Devices startes in factory reset mode with autostart=false, without starting commissioning, starts sleeping and immediately wakes up
Steps to reproduce.
I created esp_zb_sleepy_end_device example with one difference: I don't want to start searching network without button click
void esp_zb_app_signal_handler(esp_zb_app_signal_t *signal_struct)
{
    uint32_t *p_sg_p       = signal_struct->p_app_signal;
    esp_err_t err_status = signal_struct->esp_err_status;
    esp_zb_app_signal_type_t sig_type = *p_sg_p;

    ESP_LOGI(TAG, "ZDO signal: %s (0x%x), status: %s", esp_zb_zdo_signal_to_string(sig_type), sig_type,
                 esp_err_to_name(err_status));

    switch (sig_type)
    {
    case ESP_ZB_ZDO_SIGNAL_SKIP_STARTUP:
    //Notifies the application that ZBOSS framework (scheduler, buffer pool, etc.) has started, but no join/rejoin/formation/BDB initialization has been done yet.
        ESP_LOGI(TAG, "SKIP_STARTUP. Device started up in %sfactory-reset mode", esp_zb_bdb_is_factory_new() ? "" : "non ");
        esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_INITIALIZATION);
        ESP_LOGI(TAG, "Zigbee stack initialized");
        break;
    case ESP_ZB_ZDO_SIGNAL_DEFAULT_START:
        break;
    case ESP_ZB_BDB_SIGNAL_DEVICE_FIRST_START:
    //Device started for the first time after the NVRAM erase
        if(err_status == ESP_OK)
        {
            ESP_LOGI(TAG, "FIRST_START. Device started up in %sfactory-reset mode", esp_zb_bdb_is_factory_new() ? "" : "non ");
            if (esp_zb_bdb_is_factory_new()) {
                ESP_LOGI(TAG, "Start network steering");
                esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_NETWORK_STEERING);
            }
        }
        else
        {
            ESP_LOGE(TAG, "FIRST_START.  Device started up in %sfactory-reset mode with an error %d (%s)", esp_zb_bdb_is_factory_new() ? "" : "non ", err_status, esp_err_to_name(err_status));
        }
        break;
    case ESP_ZB_BDB_SIGNAL_DEVICE_REBOOT:
    //Device started using the NVRAM contents.
        if(err_status == ESP_OK)
        {
            ESP_LOGI(TAG, "Start network steering after reboot");
            esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_NETWORK_STEERING);
        }
        else
        {
            ESP_LOGE(TAG, "DEVICE_REBOOT. Device rebooted with an error %d (%s)", err_status, esp_err_to_name(err_status));
        }
        break;
    case ESP_ZB_BDB_SIGNAL_STEERING:
    //BDB network steering completed (Network steering only)
        if (err_status == ESP_OK)
        {
            steering_retry_count = 0;
            esp_zb_ieee_addr_t extended_pan_id;
            esp_zb_get_extended_pan_id(extended_pan_id);
            ESP_LOGI(TAG, "Joined network successfully (Extended PAN ID: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x, PAN ID: 0x%04hx, Channel:%d)",
                     extended_pan_id[7], extended_pan_id[6], extended_pan_id[5], extended_pan_id[4],
                     extended_pan_id[3], extended_pan_id[2], extended_pan_id[1], extended_pan_id[0],
                     esp_zb_get_pan_id(), esp_zb_get_current_channel());
        }
        else
        {
            ESP_LOGE(TAG, "Network steering was not successful (status: %s)", esp_err_to_name(err_status));
            if(steering_retry_count < steering_retry_count_max)
            {
                steering_retry_count++;
                esp_zb_scheduler_alarm((esp_zb_callback_t)bdb_start_top_level_commissioning_cb, ESP_ZB_BDB_MODE_NETWORK_STEERING, 1000);
            }
        }
        break;
    case ESP_ZB_COMMON_SIGNAL_CAN_SLEEP:
    //Notifies the application that the MCU can sleep for the specified period of time.
        ESP_LOGI(TAG, "Zigbee can sleep");
        int64_t t_before_us = esp_timer_get_time();
        esp_zb_sleep_now();
        int64_t t_after_us = esp_timer_get_time();
        const char* wakeup_reason;
        esp_sleep_wakeup_cause_t cause = esp_sleep_get_wakeup_cause();
        switch (cause) {
            case ESP_SLEEP_WAKEUP_TIMER:
                wakeup_reason = "timer";
                break;
            case ESP_SLEEP_WAKEUP_GPIO:
                wakeup_reason = "pin";
                break;
            case ESP_SLEEP_WAKEUP_UART:
                wakeup_reason = "uart";
                /* Hang-up for a while to switch and execuse the uart task
                 * Otherwise the chip may fall sleep again before running uart task */
                vTaskDelay(1);
                break;
            default:
                wakeup_reason = "other";
                break;
        }
        printf("Returned from light sleep, reason: %s (%d), t=%lld ms, slept for %lld ms\n",
                wakeup_reason, cause, t_after_us / 1000, (t_after_us - t_before_us) / 1000);
        break;
//other signals. doesn't matter
    default:
        ESP_LOGE(TAG, "No case for ZDO signal: %s (0x%x)", esp_zb_zdo_signal_to_string(sig_type), sig_type);
        break;
    }
}

and I receive infinite loop

I (608) OutDoorSensor: SKIP_STARTUP. Device started up in factory-reset mode
I (616) OutDoorSensor: Zigbee stack initialized
I (623) OutDoorSensor: ZDO signal: BDB Device Start (0x5), status: ESP_OK
I (629) OutDoorSensor: FIRST_START. Device started up in factory-reset mode
I (637) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (643) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=649 ms, slept for 0 ms
I (655) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (661) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=666 ms, slept for 0 ms
I (673) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (679) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=684 ms, slept for 0 ms
I (691) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (697) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=702 ms, slept for 0 ms
I (708) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (714) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=719 ms, slept for 0 ms
I (726) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
...

if I add
    case ESP_ZB_BDB_SIGNAL_DEVICE_FIRST_START:
    //Device started for the first time after the NVRAM erase
        if(err_status == ESP_OK)
        {
            ESP_LOGI(TAG, "FIRST_START. Device started up in %sfactory-reset mode", esp_zb_bdb_is_factory_new() ? "" : "non ");
            **if (esp_zb_bdb_is_factory_new()) {
                ESP_LOGI(TAG, "Start network steering");
                esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_NETWORK_STEERING);
            }**
        }
        else
        {
            ESP_LOGE(TAG, "FIRST_START.  Device started up in %sfactory-reset mode with an error %d (%s)", esp_zb_bdb_is_factory_new() ? "" : "non ", err_status, esp_err_to_name(err_status));
        }
        break;

everything goes right way
I (590) OutDoorSensor: SKIP_STARTUP. Device started up in factory-reset mode
I (598) OutDoorSensor: Zigbee stack initialized
I (605) OutDoorSensor: ZDO signal: BDB Device Start (0x5), status: ESP_OK
I (610) OutDoorSensor: FIRST_START. Device started up in factory-reset mode
I (618) OutDoorSensor: Start network steering
I (2826) OutDoorSensor: ZDO signal: BDB Steering (0xa), status: ESP_FAIL
E (2827) OutDoorSensor: Network steering was not successful (status: ESP_FAIL)
I (2831) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (2838) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: timer (4), t=3841 ms, slept for 998 ms
I (6110) OutDoorSensor: ZDO signal: BDB Steering (0xa), status: ESP_FAIL
E (6110) OutDoorSensor: Network steering was not successful (status: ESP_FAIL)
I (6114) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (6121) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: timer (4), t=7113 ms, slept for 998 ms
I (9393) OutDoorSensor: ZDO signal: BDB Steering (0xa), status: ESP_FAIL
E (9394) OutDoorSensor: Network steering was not successful (status: ESP_FAIL)
I (9398) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (9405) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: timer (4), t=9555 ms, slept for 168 ms
I (9640) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (9640) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: timer (4), t=10390 ms, slept for 783 ms
I (12686) OutDoorSensor: ZDO signal: BDB Steering (0xa), status: ESP_FAIL
E (12687) OutDoorSensor: Network steering was not successful (status: ESP_FAIL)
I (12691) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (12698) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: timer (4), t=13657 ms, slept for 998 ms
I (15969) OutDoorSensor: ZDO signal: BDB Steering (0xa), status: ESP_FAIL
E (15970) OutDoorSensor: Network steering was not successful (status: ESP_FAIL)
I (15974) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (15981) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: timer (4), t=16928 ms, slept for 998 ms
I (19253) OutDoorSensor: ZDO signal: BDB Steering (0xa), status: ESP_FAIL
E (19254) OutDoorSensor: Network steering was not successful (status: ESP_FAIL)
I (19258) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (19265) OutDoorSensor: Zigbee can sleep
[and sleeps]

I think allowing sleep before searching network for battery powered devices saving a little bit of battery and useful. What do you think? or it`s bad behavior?
More Information.
No response

@grebenyuk51 ，
If I have not miss you mean, I believe that the issue is not a bug. Could you attempt to follow the below code for a test?
static void zb_buttons_handler(switch_func_pair_t* button_func_pair)
{
    if (button_func_pair->func == SWITCH_ONOFF_TOGGLE_CONTROL) {
        /* Get the switch ieee address */
        ESP_LOGI(TAG, "Start network steering");
        esp_zb_lock_acquire(portMAX_DELAY);
        esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_NETWORK_STEERING);
        esp_zb_lock_release();
    }
}

void esp_zb_app_signal_handler(esp_zb_app_signal_t *signal_struct)
{
    uint32_t *p_sg_p       = signal_struct->p_app_signal;
    esp_err_t err_status = signal_struct->esp_err_status;
    esp_zb_app_signal_type_t sig_type = *p_sg_p;
    switch (sig_type) {
    case ESP_ZB_ZDO_SIGNAL_SKIP_STARTUP:
        ESP_LOGI(TAG, "Initialize Zigbee stack");
        esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_INITIALIZATION);
        break;
    case ESP_ZB_BDB_SIGNAL_DEVICE_FIRST_START:
    case ESP_ZB_BDB_SIGNAL_DEVICE_REBOOT:
        if (err_status == ESP_OK) {
            ESP_LOGI(TAG, "Deferred driver initialization %s", deferred_driver_init() ? "failed" : "successful");
            ESP_LOGI(TAG, "Device started up in %s factory-reset mode", esp_zb_bdb_is_factory_new() ? "" : "non");
            if (esp_zb_bdb_is_factory_new()) {
                ESP_LOGI(TAG, "Factory new");
            } else {
                ESP_LOGI(TAG, "Device rebooted");
            }
        } else {
            /* commissioning failed */
            ESP_LOGW(TAG, "Failed to initialize Zigbee stack (status: %d)", err_status);
        }
        break;
    case ESP_ZB_BDB_SIGNAL_STEERING:
        if (err_status == ESP_OK) {
            esp_zb_ieee_addr_t extended_pan_id;
            esp_zb_get_extended_pan_id(extended_pan_id);
            ESP_LOGI(TAG, "Joined network successfully (Extended PAN ID: %02x:%02x:%02x:%02x:%02x:%02x:%02x:%02x, PAN ID: 0x%04hx, Channel:%d, Short Address: 0x%04hx)",
                     extended_pan_id[7], extended_pan_id[6], extended_pan_id[5], extended_pan_id[4],
                     extended_pan_id[3], extended_pan_id[2], extended_pan_id[1], extended_pan_id[0],
                     esp_zb_get_pan_id(), esp_zb_get_current_channel(), esp_zb_get_short_address());
        } else {
            ESP_LOGI(TAG, "Network steering was not successful (status: %d)", err_status);
            esp_zb_scheduler_alarm((esp_zb_callback_t)bdb_start_top_level_commissioning_cb, ESP_ZB_BDB_MODE_NETWORK_STEERING, 1000);
        }
        break;
      case ESP_ZB_COMMON_SIGNAL_CAN_SLEEP:
          ESP_LOGI(TAG, "Zigbee can sleep");
          esp_zb_sleep_now();
          break;
    default:
        ESP_LOGI(TAG, "ZDO signal: %s (0x%x), status: %s", esp_zb_zdo_signal_to_string(sig_type), sig_type, esp_err_to_name(err_status));
        break;
    }
}

and this code leads to not sleep

I (626) OutDoorSensor: SKIP_STARTUP. Device started up in factory-reset mode
I (643) OutDoorSensor: Zigbee stack initialized
I (651) OutDoorSensor: ZDO signal: BDB Device Start (0x5), status: ESP_OK
I (656) OutDoorSensor: FIRST_START. Device started up in factory-reset mode
I (664) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (671) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=676 ms, slept for 0 ms
I (682) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (688) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=693 ms, slept for 0 ms
I (681) main_task: Returned from app_main()
I (700) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (711) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=716 ms, slept for 0 ms
I (723) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (729) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=734 ms, slept for 0 ms
I (739) HTU21D: Temperature: 21.76
I (740) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (750) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=755 ms, slept for 0 ms
I (762) OutDoorSensor: ZDO signal: Can Sleep (0x16), status: ESP_OK
I (768) OutDoorSensor: Zigbee can sleep
Returned from light sleep, reason: other (0), t=773 ms, slept for 0 ms
[...infinite...]

if add esp_zb_bdb_start_top_level_commissioning(ESP_ZB_BDB_MODE_NETWORK_STEERING); to case ESP_ZB_BDB_SIGNAL_DEVICE_FIRST_START: device will sleep

@grebenyuk51 You are correct. Once the Zigbee main loop is up and running without Zigbee commissioning, the end-device should also be able to enter sleep mode. Thank you very much for reporting the issue.
We have identified the root cause of the problem and have already fixed it in an internal MR. The next version of the esp-zigbee-sdk will include this fix.

thank you. I will close the issue after my check

with library version 1.3.1 device sleeps without Zigbee commissioning
thanks


ImageStatic at high zoom levels in Chrome

The static image example has a problem with Chrome at high zoom levels: after a certain zoom level it just disappears.
In the example, it happens at zoom 23, but I had the same thing happen in another map at a different zoom level. E.g. in this fiddle using a red/yellow 4x4 image with dataUrl, it happens at zoom 24.
I tried with two different versions of Chrome (46.0.2490.71 m and 48.0.2536.0 canary), with the same results (see screenshots below).
I have no problem zooming to max zoom with IE (11.0.9600.18015) or Firefox (41.0.1) or Firefox Developer Edition (43.0a2).

Zoom 22

Zoom 23
I know this is almost the opposite of another issue ... strange, but this is what happens to me

This issue, like #3712, has to do with the number ranges that are available for matrix transformations on a canvas. If these number ranges are exceeded, the transformation will fail, and no image will be rendered. Usually it does not make sense to go beyond zoom level 21 anyway, so we should change the example to use a maxZoom of 21.


feat: Allow additional routes for intra and private routing tables

Description

Allow additional routes to be added for the private and intra routing tables, like it is already available for the default table.
An example use case could be peering with some VPC and making it routed with the VPC managed by the module. This example assumes that requesting and accepting the peering request should be handled by a user. As well as adding default routes on the other (peered VPC) side.
Ensure no conflicts for additional routing rules with private routes defined for NAT gateway managed with this module.
In order to support both NAT gateway routes on the private route tables, and the ones provided via the new parameters for custom routes, use an aws_route resource instead of the one for managing route tables. That is, per the documentation:
NOTE on Route Tables and Routes: Terraform currently provides both
a standalone Route resource and a Route Table resource with routes
defined in-line. At this time you cannot use a Route Table with
in-line routes in conjunction with any Route resources. Doing so
will cause a conflict of rule settings and will overwrite rules.

To keep input values for custom default route table, private and intra routing rules consistent and interchangeble:

use the same aws_route approach for both private and intra cases;
alias (ipv6_)cidr_block and destination_(ipv6_)cidr_block so that both inputs can be used interchangebly for default route table and private/intra subnet route resources.

Motivation and Context

The use case is configuring all required routes within the VPC module, instead of adding custom
calls for aws_route. The latter is not always possible with terragrunt modules for example, which allows
only other modules there, and no custom resources.
(see https://github.com/gruntwork-io/terragrunt/issues/1187 and https://github.com/gruntwork-io/terragrunt/issues/1150)
Custom routes may be required for the private/intra VPC route tables to cover many cases, like
peering the managed VPC with another VPC (outside of this feature scope) and manage additional
routes on both sides. This change allows custom routes to be added for the managed VPC, for this
example. While adding the needed routes on the other side of the peering, and creating the peering would be
left for the user exercise.
Implements https://github.com/terraform-aws-modules/terraform-aws-vpc/issues/820
Breaking Changes

How Has This Been Tested?

[x] I have updated at least one of the examples/* to demonstrate and validate my change(s)
[x] I have tested and validated these changes using one or more of the provided examples/* projects (with my tEKS fork)

[x] I have executed pre-commit run -a on my pull request

I workarounded this in my teks fork by using another module to manage VPC peering and custom routes


black and ruff format

I reformatted the code using black and ruff

I tested my sbi_lens codes, everything works perfectly :)


how to change the default width of the tooltip ?

Hi !
I want to set a custom width to the tooltip as I am showing some progress bars and additional info inside it. I've tried "wMin" property as well as defining width in className of the children and inside the Tooltip.Popper props , but I'm unable to change it's width . How can I do so ?

Hi, the wMin worked for me as expected (see the screenshot below). Could you please open the sandbox and check what is difference from how you are trying to change the width?

Oh I got it correct now ! There was a bug f how tooltips were rendering , I fixed it ! Thanks..


added healthcheck route #11181

Hello.
This healthcheck route is required for using Ghost behind a cloud-service load balancer which can not manually set "X-Forwarded-Proto: https".
Issue: https://github.com/TryGhost/Ghost/issues/11181

[x] There's a clear use-case for this code change
[x] Commit message has a short title & references relevant issues
[x] The build will pass (run yarn test and yarn lint)

this is really usefull

This is exactly what we need

This is exactly what we need

You can work around this by allowing 3xx HTTP status codes as succesful with the AWS application load balancer since the maintainers apparently do not care whether their product is actually useful within cloud infrastructure.

I'm sorry to have neglected this PR for so long 😳
The thing that's missing for me here is reference material / precedent or a spec for what the URL should be. Why /health? Can we point to other platforms that have this? Is it the most common endpoint?
I'm happy to accept a PR of this nature but would just like to know I'm not gonna get another one right away suggesting we change it or add a second more popular route! 😬

/health a common endpoint for a generic healthcheck endpoint. So is /healthcheck. Unless you need something more specific, either is fine (for example, if your app takes 3 minutes to "warm up", you might want to separate the healthcheck in to multiple endpoints /health/warmup & /health/live)

Didn't mean to close this - I am looking for some specific reference material for what a healthcheck endpoint needs to do and common patterns for naming.
If it's just an endpoint that returns a 200 when the service is running, then why not ping / or /ghost/api/admin/site? I'm sure there is a reason!
Looking again, I'm also not sure how this PR mitigates the problem of not being able to set "X-Forwarded-Proto: https" 🤔

@ErisDS The health checks purpose is to primarily let a load balancer or monitoring to determine the health about the service.
Here is a good primer. https://testfully.io/blog/api-health-check-monitoring/

If it's just an endpoint that returns a 200 when the service is running, then why not ping / or /ghost/api/admin/site? I'm sure there is a reason!

Pings are not sufficient they only tell if the service is reachable. This is why there are typically two: one that monitors if reachable and then another that determines service readiness. Since it might take a bit for Ghost to contact a remote database or spin up other components. If we have a couple instances behind a load balancer that routes traffic between multiple nodes based on service health, as is common for clustering, then us admins need this functionality in order to route with confidence to our nodes.

I'm going to close this for now for a few reasons:

there's no clarity here in response to my questions (I was not asking what healthchecks are for)
we've restructured the codebase meaning this would need a rebase
The OP has (quite understandably) moved on

I'm more than happy to see a new PR that adds healthchecks to Ghost, with some description of why the proposed implementation is correct & serves the most use cases. My aim is to end up with one set of healthcheck tools that work for everyone, and not have to keep adding new routes that do a slightly different thing.

there's no clarity here in response to my questions (I was not asking what healthchecks are for)

Was my direct response to you previously insufficient? What sort of resource were you expecting?

we've restructured the codebase meaning this would need a rebase

Fair enough, but this languished for too long.
This issue is still worth pursuing.
Thank you for your time.

I agree this languished too long. I'm more than happy to support a new PR for this with a thorough solution that addresses all the concerns both in this thread and the OG issue #11181.
I'm looking for resources that demonstrate precedent for and a solution that covers:

The name and location of the endpoint
The exact response code and body
Handling proxies and HTTPS correctly
When in Ghost's lifecycle a response should be served

This PR added a route at /ghost/api/health/ that returns a 200 status code and the text "SUCCESS". The route is mounted at the end of the boot cycle (so will return 503s whilst Ghost is booting) and also has no handling for anything to do with proxies or https despite this being mentioned in the PR description.
The aim is to merge a single endpoint that will work for everyone and is more useful than the existing endpoint that can be used for healthchecks which lives at /ghost/api/admin/site and returns a 200 and a JSON blob if Ghost is up and running. ATM I don't see that the proposed code in this PR gives us anything over and above /ghost/api/admin/site so I don't see a reason to merge it.

@ErisDS According to ghost acces logs the /ghost/api/admin/site returns a 301 status code which is incompatible with many cloud load balancers. Many cloud load balancers require a 200 status code only for health checks
Access Logs from ghost v5.10.1:
[2022-08-20 06:30:22] INFO "GET /ghost/api/admin/site" 301 1ms

Access Logs from ghost v4.48.2:
[2022-08-20 06:17:29] INFO "GET /ghost/api/admin/site" 301 1ms

You can see requests to the endpoint /ghost/api/admin/site in both versions 4.x and 5.x are returning 301 HTTP status codes
This was tested in a production kubernetes environment via gke with the following health check configuration:
livenessProbe:
    httpGet:
        path: /ghost/api/admin/site
        port: http
    initialDelaySeconds: 5
    periodSeconds: 5
    timeoutSeconds: 1
readinessProbe:
    httpGet:
        path: /ghost/api/admin/site
        port: http
    initialDelaySeconds: 5
    periodSeconds: 5
    timeoutSeconds: 1

As a workaround in v4.x we extended the api router with a custom health check endpoint at   /ghost/api/admin/site/health, but that no longer works since the structure of the app has changed in v5.x. It's annoying to rebuild that functionality for each major version change.
Can you please re-evaluate adding a health check only endpoint that does not use redirects and returns only a 200 status code with an empty response body

@danmasta you're missing a trailing slash on your URLs, the requests should be made to GET /ghost/api/admin/site/

Hi @kevinansfield, I added the trailing slash and I'm still seeing a 301 status code:
[2022-08-20 23:10:39] INFO "GET /ghost/api/admin/site/" 301 1ms

When I exec into the container and test with curl it looks like it tries to redirect to the ghost url config:
curl http://localhost:2368/ghost/api/admin/site/
Moved Permanently. Redirecting to https://$GHOST_URL/ghost/api/admin/site/

When testing the same thing in a local docker container with default config it does work with no redirects:
curl http://localhost:2368/ghost/api/admin/site/
{"site":{"title":"Ghost","description":"Thoughts, stories and ideas","logo":null,"icon":null,"accent_color":"#FF1A75","url":"http://localhost:2368/","version":"5.10"}}

It seems like the issue is the /ghost/api/admin/site/ api endpoint is subject to ghost redirection logic, particularly in production with the url configuration set. We really need an endpoint that exists outside that redirect logic in order to be useful for health checks

Recommendation for anyone at the end of this thread:
Write a sidecar service (I did one in Go) that monitors the systemd status and exposes it via a separate HTTP port. You can use this to evaluate health.
So for example, my Go service just exposes an HTTP server which returns 200 if Ghost is running on the system or 5xx if not. You can throw that server on, say port 8000, and configure your LB to hit that endpoint/port for health checks.

@danmasta it looks like you're having configuration issues, which would have been better suited to the forum.
@samrap it sounds like you've found something that works for you, but without really explaining why the existing suggested workaround didn't work for you - which doesn't really help with building up a plan for what a healthcheck should look like in Ghost.
The bottom line is, I am pretty sure this PR doesn't do what anyone thinks it does - there's no one in this thread saying they've used this in production and it works as a healthcheck. I can't see that this new /ghost/api/health/ endpoint adds anything over and above what /ghost/api/admin/site/ already does.
On the flip side, I do acknowledge completely that a healthcheck route that has some functionality beyond /ghost/api/admin/site/ (e.g. doesn't enforce HTTPS) might be a valid addition to Ghost, but from this PR and thread I have zero clarify on what the actual requirements would be for such a route, nor what the standards are used in other tools.
I suggest that anyone interested in getting this over the line open a thread in the Contributing section of the Ghost forum with an outline of a spec and some reference material for why that spec is correct for collaboration - OR if you don't think it needs discussion, open a new PR with the same info and the solution.
For now, I'm going to lock this thread and again clarify that I would be very happy to merge a working, tested, useful healthcheck route into Ghost.


Added Stream support to ChangeFeedProcessor

Pull Request Template
Description
Added the ability to use a Stream delegate rather than a typed collection in order to provide support for custom serialization and use patterns where deserialization of the items is not required.
Type of change
Added set of classes needed to support the Stream delegate.
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] This change requires a documentation update

Closing issues
#865
Assignee
Richard Fuoco

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: fuocor sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

This PR is duplicating a lot of the Change Feed Processor files, which makes maintaining this code in the future very hard.
A better approach would probably be:

Make the base abstract Observer class already receive a Stream on ProcessChangesAsync
Add a new delegate that sends the stream
Have 2 implementations of the Observer, one that serializes like it currently happens in FeedProcessorCore, another that passes the Stream, each would receive a different delegate as part of their constructors
Have the ChangeFeedProcessorBuilder, based on which delegate type is used, create one Observer implementation or the other.

That would reduce the amount of files changed greatly.

@fuocor I wonder, if we fix the Custom Serializer support, would you still need Stream support? ref https://github.com/Azure/azure-cosmos-dotnet-v3/issues/865

@fuocor I wonder, if we fix the Custom Serializer support, would you still need Stream support? ref #865

Yes... I want to work with the batch stream

This PR is duplicating a lot of the Change Feed Processor files, which makes maintaining this code in the future very hard.
A better approach would probably be:

Make the base abstract Observer class already receive a Stream on ProcessChangesAsync
Add a new delegate that sends the stream
Have 2 implementations of the Observer, one that serializes like it currently happens in FeedProcessorCore, another that passes the Stream, each would receive a different delegate as part of their constructors
Have the ChangeFeedProcessorBuilder, based on which delegate type is used, create one Observer implementation or the other.

That would reduce the amount of files changed greatly.

I was looking at that possibility, but was trying not to be too intrusive on the existing code.
Now that I have my head around the existing code then I will update my code

/azp run

@fuocor it looks like the Unit Test project is not building, that will block the build pipeline until it's fixed

I fixed the broken unit tests. There seems to be a number of tests failing that are outside the scope of the Change Feed processing

Unit Tests  > ContractChanges seem to be failing due to the fact that a public API change is part of the PR. The test failure message indicates to run Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.Tests/testbaseline.cmd /update to update the public surface and add the update json file as part of the PR.

/azp run

/azp run

I'm not sure how to address  the failed tests which seem  to be unregulated the change feed processor

@fuocor seems they are transient failures, trying to rerun them atm

@fuocor it looks like TestWithStartTime_Beginning seems to be failing in all attempts, does it fail locally for you?

https://cosmos-db-sdk-public.visualstudio.com/cosmos-db-sdk-public/_build/results?buildId=15388&view=logs&jobId=2fc708b4-0f45-51b9-cfa1-ef653a9c354a
https://cosmos-db-sdk-public.visualstudio.com/cosmos-db-sdk-public/_build/results?buildId=15387&view=logs&jobId=7e849056-1f62-5d31-79c5-3a465e9ea4c9

I have a number of transient test failures when running off of the master branch or mine, none of which are related to the changes I made

I updated my local emulator and they seem to pass except for the one's testing with rate limiting.

Im still retrying to understand why is it failing on the CI pipeline, without a green there, it cannot be merged

@fuocor Finally got the tests to work correctly

Just wanted to say that I came in to post an issue/feature request for this exact feature (the ability to use custom deserialization for ChangeFeed pushes), and I'm waiting patiently for the PR to be merged into master. Thanks for the work, all of you!

@lisardggY The ability to use custom deserialization is already possible, you can implement a CosmosSerializer and it will be called when the SDK deserializes the Stream to push the changes to your delegate.
This PR is to expose the Stream in case you need to pass that Stream to another component for later deserialization.

@lisardggY The ability to use custom deserialization is already possible, you can implement a CosmosSerializer and it will be called when the SDK deserializes the Stream to push the changes to your delegate.
This PR is to expose the Stream in case you need to pass that Stream to another component for later deserialization.

Yes, but this is an all-or-nothing setting for any code using the CosmosClient.  We need different serialization options for different scenarios to deserialize polymorphic payloads inside documents, and I would rather have the CosmosClient be a shared, generic component rather than creating different ones for different scenarios in our code.

The conflicts came in code modified on your end, but remain superseded in my branch. I'm not sure how to go about resolving them

I merged this with the latest master and updated the change feed tests

Hi folks, we're closing in on a year since this was opened. I have been keeping it in sync with master and still believe there is value to using streams in the change feed in order to support things like block copying of values (which is what I use as part of a lambda architecture with data lakes)

/azp run

/azp run

@fuocor There seems to be some failures on the Unit tests (besides the contract change one), could you take a look?
Reference: https://cosmos-db-sdk-public.visualstudio.com/cosmos-db-sdk-public/_build/results?buildId=29987&view=logs&jobId=011e1ec8-6569-5e69-4f06-baf193d1351e&j=011e1ec8-6569-5e69-4f06-baf193d1351e&t=c5b2d381-db84-5070-406d-2daac9cef82b

Tests have been fixed

/azp run

/asp run

/azp run

@fuocor Tests kept failing on this PR (UTs mainly), I am cherry picking your work into #2331 as we are adding new delegate signatures

Closing as the changes were ported to #2331


Enabling type checking with Pyright

This PR enables pyright, and fixes all typing errors. If the current settings are too strict, feel free to add more exceptions to any rules.
closes #29

Big fan of type checking, thanks (-:


Ingest & sync with Hugging Face

utilise marketplace action to pull datasets directly from HF
cleanup + sync GHA
Testing and add to documentation

Resources:
https://huggingface.co/docs/hub/en/spaces-github-actions
https://discuss.huggingface.co/t/managing-spaces-with-github-actions/31338/4
Requires token based auth to connect GitHub with HF


install instructions need updating

Install-Module -Name PnP.PowerShell
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3b3240d3-6a7d-fc4b-62fd-d3a5375d8a4a
Version Independent ID: 561212df-4dab-92b7-b634-57d53d313c6f
Content: PnP PowerShell Overview
Content Source: sharepoint/docs-conceptual/sharepoint-pnp/sharepoint-pnp-cmdlets.md
Service: sharepoint-powershell
GitHub Login: @pamgreen-msft
Microsoft Alias: pamgreen

Hello @ishaisagi document has been already updated (the whole PnP documentation too) :)
Thank you!

@get-itips Thank you very much for the contribution and sharing this explanation. @ishaisagi Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


Add Support for Scala 2.13 and drop support for Scala 2.13

Resolves #1202 by adding support for 2.13. We're also forced to drop support for 2.11 here because a number of dependencies that are not cross built for both. Big thanks to @sergeykolbasov for figuring the weird SAM issue that caused the Mapper to not get resolved in a bunch of cases.
Couple things about this PR:
[] Introduces a bunch of warnings from Scalatest, I plan on cleaning these up in a separate PR
[] I'd still like to investigate cleaning up the sbt settings, I really like how Odin is does it.
[] Examples and docs cannot be cross built until Twitter Server is cross built for 2.13

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@77deee6). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master    #1204   +/-   ##
=========================================
  Coverage          ?   82.24%
=========================================
  Files             ?       54
  Lines             ?      991
  Branches          ?       42
=========================================
  Hits              ?      815
  Misses            ?      176
  Partials          ?        0

Impacted Files
Coverage Δ

core/src/main/scala/io/finch/Endpoint.scala
72.22% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77deee6...d5db5a3. Read the comment docs.

Hrm, not sure what's up with that coverage report. I might try to re-apply these changes directly to master instead of merging master into it.

Ok I've managed to fix the coverage report by reapplying my changes to master instead of having the merge commit. I also cleaned up a couple unneeded things. AND because Travis released Circe 0.13 between when I posted this PR and when I got home, I was able to use that instead of the milestone build.

I think I have all of the scalacOptions figured out. We were just missing the one in the 2.13 list.

LGTM
@vkostyukov what do you think?

Hi @rpless
twitter-server got released for 2.13
https://search.maven.org/artifact/com.twitter/twitter-server_2.13/20.3.0/jar
I guess we could finish this PR now and then merge it 😎

@sergeykolbasov I agree. I just pushed something to crossbuild examples and docs to 2.13. If it builds and you think its good I'll merge.

@sergeykolbasov I've fixed the version of Twitter Server and removed that parallelExecution directive. Build looks good.

LGTM 👍


Assign workflows to agents that match the most by labels

This make sure if there are org and global agents, that workflows get fetched by the org agent they are in first and only if not available will the global one fetch it.
This also let you control agent priority etc ...
before the selection of an agent was kinda at random ...

or is it a feature?

https://app.codecov.io/gh/woodpecker-ci/woodpecker/pull/4201/blob/server/grpc/filter.go
100% test coverage :D