proposal: reflect: add Value.Caller

reflect.Call allocates a []reflect.Value slice of length nout for the number of return arguments from the function. This is fine for a one-off call of a function, but when repeatedly invoking a function, reflect.Call allocations can quickly add up.
As an example, I have a helper package that sorts arbitrary types, and if a type has a func (t T) Less(other T) bool (i.e., Name == "Less", NumIn() == 1, NumOut() == 1, In(0) == T, Out(0) == bool), the package calls that function for sorting. This is done within sort.Slice, meaning Less is called as many times as sort.Slice needs to compare. For an ordered slice of 1000 elements, each sort.Slice allocates 25735 times.
The only four allocations in this benchmark are (tip @ 2cf85b1fb8)
  105.50MB 68.28% 68.28%   105.50MB 68.28%  reflect.Value.call /home/twmb/go/go.tip/src/reflect/value.go:565
      41MB 26.53% 94.81%       41MB 26.53%  reflect.methodReceiver /home/twmb/go/go.tip/src/reflect/value.go:862
    4.50MB  2.91% 97.72%     4.50MB  2.91%  reflect.Value.call /home/twmb/go/go.tip/src/reflect/value.go:609
    1.50MB  0.97% 98.70%     1.50MB  0.97%  runtime.allocm /home/twmb/go/go.tip/src/runtime/proc.go:1866

70% of the allocations are from allocating the output argument slice. If possible, I could provide the input slice and reuse it for all Calls, which would effectively eliminate this alloc.
I looked into line 862 and into the runtime, I'm not sure how to get rid of that allocation: the code takes the address of an unsafe pointer, and that allocates; if possible it would be quite nice to get rid of this alloc as well but this would likely require deeper wiring into the runtime (the point here is to have a method pointer). Line 609 has a TODO that could eliminate the allocation. runtime/proc.go:1866 looks to be unrelated.
reflect.Value.call can be:
func (v Value) call(op string, in, out []Value) []Value {

If out is non-nil, then it is used rather than allocating a new slice. If the input out is too small, the code will panic. Essentially, line 565 turns into:
ret = out
if ret == nil {
    ret = make([]Value, nout)
}
if len(ret) < nout { // optional check
    panic("short output slice")
}

Going forward would it make sense to write your package using generics?

I don't think so. This is doing deep sorting on struct fields (admittedly this is not a high priority package). If the field is a slice, I check the inner type for the method. With generics, even if I know the method has a Less function, I cannot type assert into Lesser[T] because T requires instantiation and I do not know the type.

If we add CallInto or something like it, does reflect.Call go down to zero allocations?
It would be good to know we were done with this particular issue, at least for Call.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

This seems like a duplicate of #43732.

If we add CallInto or something like it, does reflect.Call go down to zero allocations?

According to my comment https://github.com/golang/go/issues/43732#issuecomment-761704581, we can get it down to zero allocations.

Interesting, yes I think this is a duplicate of #43732. Does the CL linked take care of methods? I don't see a change in methodReceiver, which was the remaining allocation I ran into.
I also tried removing the allocation locally -- I got a similar sort of speedup: not as much as I hoped, but the allocation drop is somewhat nice regardless.

Does the CL linked take care of methods?

I didn't do anything special for method functions. I'll take a look later if something is needed there.

Indeed, this is a duplicate of #43732, but this one is in the active column, so I'll close that one.

It sounds like people basically agree about the signature (even in #43732): it takes two []Value and returns nothing.
But we need a name, and we need to figure out whether it handles variadic functions like Call or like CallSlice.
If it handles them like Call, doesn't that allocate? So probably it should handle them like CallSlice.
Maybe it would be CallSlices?
Other names welcome but CallInto and CallWith both sound like they are missing something.

What should we do about Call vs CallSlice? We probably don't want to add two new functions.
And if we do this, are we down to zero allocations, so we know we won't need to add another variant in the future?

The naming problem could be avoided by introducing a reflect.Caller type.
//Caller is a low-level type used to make
//zero-allocation function calls.
type Caller struct {}

I suspect that adding this type has an additional advantage, it leaves the door open to a NewCaller/Caller constructor of the  type being able to optimise repeat calls to the function it's calling.
echo := reflect.ValueOf(func(in string) string { return in }).Caller()

or
echo := reflect.NewCaller(reflect.ValueOf(func(in string) string { return in }))

then
results := make([]reflect.Value, 1)
args := []reflect.Value{
    reflect.ValueOf("hello"),
}

for i := 0; i < 100; i++ {
    echo.Call(args, results) //more efficient
}

What should we do about Call vs CallSlice?

reflect.Caller provides at least two options:

Have two methods on the reflect.Caller type, Call and CallSlice.
Have reflect.Caller.Call behave like CallSlice when it is constructed from a variadic function.

It can then be considered that reflect.Value.Call and reflect.Value.CallSlice are shorthand/abstractions around the use of the "low-level" Caller type.

And if we do this, are we down to zero allocations, so we know we won't need to add another variant in the future?

In my echo example above, it is my understanding that the string header returned from echo will escape to the heap when it is converted into a reflect.Value and placed in the results slice. Therefore 'zero-allocation' is only possible by providing pointers in the results slice for Call to write into. I think it should be the requirement of reflect.Caller's user to allocate all the results needed to call the function. Only reflect.Value.Call, reflect.Value.CallSlice and reflect.Caller's constructor should be allowed to allocate.
results := []reflect.Value{new(string)}
args := []reflect.Value{reflect.ValueOf("hello")}
for i := 0; i < 100; i++ {
    echo.Call(args, results) //zero-allocations.
}

Adding the ability to specify the output slice removes allocations for non-method function calls. For methods, there still is an allocation to create a method pointer. Since call immediately uses and then discards this method pointer, it would be good if this allocation could be removed, but I'm not sure how, and removing the allocation for output arguments does not remove the method pointer allocation.

It sounds like we are having trouble getting to an API that will actually guarantee zero allocations and that is simple enough to be a reasonable API change. Does anyone see how to thread this needle?

It still sounds like we don't have a final API to decide about,
one that will guarantee zero allocations and is simple enough to present to users.
Does anyone want to try to do that?

@twmb
Is this something that a reflect.Caller type could handle? If the reflect.Value is a method, allocate this method pointer and reuse it across sequential calls.

I think reflect.Caller is a nifty idea: the method pointer is allocated once, and function lookups are cached (and this work eliminated) across repeated invocation. I'm not sure what the code would look like but I'm :+1: on the idea.
Also, reflect.Caller could have one single method. Stealing some documentation from the current reflect.Value.Call,
// Call calls the function v with the input arguments in. For example, if len(in)
// == 3, v.Call(in) represents the Go call v(in[0], in[1], in[2]).  It returns the
// output results as Values. As in Go, each input argument must be assignable to
// the type of the function's corresponding input parameter. If v is a variadic
// function, Call creates the variadic slice parameter itself, copying in the
// corresponding values.
//
// The returned slice is reused across repeated invocations to call. If you wish
// to keep the argument slice itself, it must be copied.
func (c reflect.Caller) Call(args []reflect.Value) []reflect.Value

In this scheme, the returned argument slice can be allocated once on the first call, and reused across future invocations.
Alternatively, this type could have a second method. CallInto was questioned above, and I'm not so sure on CallWith myself (with what?), but perhaps CallWithReturns (still open to better names)?
In summary for new APIs,
func NewCaller(interface{}) reflect.Caller
func (v reflect.Value) Caller() reflect.Caller

type reflect.Caller struct { ... }

func (c reflect.Caller) Call(args []reflect.Value) []reflect.Value // reuses return slice across repeated invocations
func (c reflect.Caller) CallWithReturns(args, returns []reflect.Value)

I believe it should be possible to have zero allocation calls with this helper type.

Is Caller just a cache for allocated things? If so, maybe it shouldn't embed the receiver? That is, maybe it should be
var caller reflect.Caller
caller.Call(v, args)
because then you can do
caller.Call(otherV, otherArgs)
Does anyone want to try implementing this?

Sounds like we are waiting on someone to implement this to understand whether it satisfies the performance goals for the proposal.

Placed on hold.
— rsc for the proposal review group

I have taken a run at implementing this, which can be found here: https://github.com/nrwiersma/go/blob/caller/src/reflect/caller.go
and it seems it is possible to get it down to zero allocs.
This is basically just a copy of Value.Call that caches a bunch of the values, and I am not sure I fully understand everything that happens here. It seems that caching the output of funcLayout reduces the time taken by roughly half (~80ns of ~160ns) and from what I gather from its description should be safe. Without caching it the call time is basically unchanged while the allocs are removed.
I chose, at least for this PoC, not to allow the user to provide their own output slice, as to remove the alloc from L270 a fair amount of type checking was needed, and it was starting to get really ugly. For now the values are created on construction and reused.
My benchmark numbers are as follows:
name           time/op
Value_Call-8    159ns ± 1%
Caller_Call-8  76.1ns ± 0%

name           alloc/op
Value_Call-8    32.0B ± 0%
Caller_Call-8   0.00B

name           allocs/op
Value_Call-8     2.00 ± 0%
Caller_Call-8    0.00

While I was at it I noticed that callMethod is also allocating on methodReceiver and takes a long time on funcLayout, but this is likely for another issue.

Thanks. Taking off hold.

The implemented API seems to be:
package reflect

type Caller struct { ... }
func NewCaller(v Value) *Caller
func (c *Caller) Call(rcvr Value, in []Value) []Value

This lets NewCaller compute various things about v that can be reused across multiple calls to v.
I'm not sure whether it should be a method on Value instead of a top-level function,
and of course we'd want to deduplicate the code with reflect.Call
(presumably Value.Call would use NewCaller), but it looks reasonable.
@twmb, @dsnet, any thoughts?

(presumably Value.Call would use NewCaller)

NewCaller.Call is using CallSlice style input args, so this would be a change to the way that Value.Call behaves.

@nrwiersma thanks for the reminder about CallSlice. It seems like Caller should have both Call and CallSlice methods.
Then both Value.Call and Value.CallSlice would be wrappers.
@twmb, @dsnet any thoughts?

@rsc I have refactored as requested. Caller has moved into https://github.com/nrwiersma/go/blob/caller/src/reflect/value.go#L432 and implements both Call and CallSlice. It is now accessed via Value.Caller() and Value.Call and Value.CallSlice both now use Caller. Value.call has moved to Caller.call and I needed to add methodReceiverType to check the actual receiver type.
All tests are passing and my benchmarks look like this:
name                      time/op
Call-8                       121ns ± 7%
CallerCall-8                11.7ns ± 1%
CallMethod-8                 141ns ± 0%
CallerCallMethod-8          33.3ns ± 0%

name                      alloc/op
Call-8                        256B ± 0%
CallerCall-8                 0.00B
CallMethod-8                  256B ± 0%
CallerCallMethod-8           0.00B

name                      allocs/op
Call-8                        1.00 ± 0%
CallerCall-8                  0.00
CallMethod-8                  1.00 ± 0%
CallerCallMethod-8            0.00

+1 to the current API. How is this avoiding allocations on the return slice?

So the current API is:
type Caller struct { ... }
func (v Value) Caller() *Caller
func (c *Caller) Call(rcvr Value, in []Value) []Value
func (c *Caller) CallSlice(rcvr Value, in []Value) []Value

and the current Value.Call and Value.CallSlice become wrappers:
func (v Value) Call(in []Value) []Value {
	var rcvr Value
	if v.flag&flagMethod != 0 {
		rcvr = v
	}
	return v.Caller().Call(rcvr, in)
}

The detail about the receiver not being pre-bound already in the result of v.Caller() seems wrong.
v.Caller().Call(in) should be the same as v.Call(in) in all cases.
That means the Call and CallSlice methods should not have a rcvr Value argument.
So the API I'm suggesting would change to:
type Caller struct { ... }
func (v Value) Caller() *Caller
func (c *Caller) Call(in []Value) []Value
func (c *Caller) CallSlice(in []Value) []Value

and the wrapper would become
func (v Value) Call(in []Value) []Value {
	return v.Caller().Call(in)
}

Does that seem reasonable?

Fair. Will make the change. It does simplify things a little.

My concern with this API is that I don't know how I'm suppose to use it.
I looked at two use-cases to validate this:

use of reflect.Value.Call in "github.com/google/go-cmp"
hypothetical use of reflect.Value.Calue in "encoding/json" for #5901

(we no longer need reflect.Value.Call for this now that we have Go generics, but it's still worthwhile to consider as a case study)

In order for this to be useful, I'm responsible for pooling the reflect.Caller objects. How do I do that when I'm managing an unbounded number of discrete functions? For each function, I can't just have one pre-prepared reflect.Caller since reflect.Caller is not concurrent safe. I need a pool of them for each function. It does not scale to have a sync.Pool for each of these.

To deal with the concurrent case, the user would need to provide the output slice, or the output slice just has to allocate.
One option is the add CallWithOut(in, out []Value) and CallSliceWithOut(in, out []Value) (names can obviously be changed) which would then be concurrency safe while not changing the general non-concurrent use case which would still reuse the output slice. The other option is to always have to provide the output slice.

One option is the add CallWithOut(in, out []Value) and CallSliceWithOut(in, out []Value)...

That's essentially what's proposed in #43732 and implemented in https://golang.org/cl/284222.
Speaking of concurrency, I think that's also one of the dangers of the API of:
type Caller struct { ... }
func (v Value) Caller() *Caller
func (c *Caller) Call(in []Value) []Value
func (c *Caller) CallSlice(in []Value) []Value

It's not obvious that Caller.Call is not concurrent safe. It's a method that takes an input and returns an output and the functionality doesn't seem to have any reason why it mutates c. It seems like it serves a similar purpose as regexp.Regexp, where the receiver is some statically pre-computed state. Thus, this may be easy to accidentally hold wrong.

One benefit of this API that was not present in #43732 is that the Caller implementation can cache lots of information about v that doesn't have to be recomputed (like frame layout offsets).
@dsnet, what do you think about keeping the Caller type and that cacheability but also add the out slice as in https://github.com/golang/go/issues/49340#issuecomment-1347710586 as a parameter to make it concurrency-safe?
We would probably also need to add:
func (c *Caller) NumOut() int

so that the user of this code knows how big an out slice to allocate.
Thoughts?

@rsc To be clear, are you suggesting adding out slices to the Call and CallSlice methods, or adding CallWithOut and CallSliceWithOut functions?

I think the suggestion is to add an out parameter to the Call and CallSlice methods.  There's no reason for additional methods.

@dsnet, what do you think about keeping the Caller type and that cacheability but also add the out slice as in https://github.com/golang/go/issues/49340#issuecomment-1347710586 as a parameter to make it concurrency-safe?

Generally, that SGTM. So essentially:
func (c *Caller) Call(out, in []Value)
func (c *Caller) CallSlice(out, in []Value)

It's unfortunate that out and in are of the same type, so it will be easy to accidentally swap the order, but that will hopefully be trivially detected by any unit test.
What's the behavior if the provided out has elements that mismatch what the function actually outputs? I propose:

If the n-th output type is assignable to out[n].Type() and out[n].CanSet, then the output is directly stored using the equivalent of out[n].Set.
Otherwise, the implementation allocates a new value for out[n] and overwrites the old Value at out[n].
After calling Call, every out[n].CanSet reports true so that we can reuse resources between calls. Note that this goes contrary to the existing Value.Call method, which always returns Values that are not settable.

We would probably also need to add...

If we add NumOut, we probably want NumIn to be consistent.

The changes have been made. The API is now:
type Caller struct { ... }
func (v Value) Caller() *Caller
func (c *Caller) NumIn() int
func (c *Caller NumOut() int
func (c *Caller) Call(in, out []Value)
func (c *Caller) CallSlice(in, out []Value)

@dsnet There is no need add complexity by looking at existing values in the output slice, the output values from the function are set to the slice, regardless of what is in there.

There is no need add complexity by looking at existing values in the output slice, the output values from the function are set to the slice, regardless of what is in there.

Without doing that, I fail to see how this implementation can possibly be concurrency safe and be allocation free. The underlying storage for each output value must exist somewhere.

Fair point about the allocation. I had forgotten I had dealt with this in previous code. I have added the code back in that deals with reusing the value.
Instead of looking for flagAddr I am looking for flagIndir which seems to have the desired effect without breaking Value.Call which would not be good. The check is this:
s := out[i].ptr
if out[i].typ != tv || out[i].flag&flagIndir != flagIndir {
	s = unsafe_New(tv.common())
}

I think it should be out, in []Value to match the builtin copy function.

Fair, the API is now:
type Caller struct { ... }
func (v Value) Caller() *Caller
func (c *Caller) NumIn() int
func (c *Caller NumOut() int
func (c *Caller) Call(out, in []Value)
func (c *Caller) CallSlice(out, in []Value)

FuncOf corresponds to declaration and Call with application

Talked to @ianlancetaylor and others at proposal review. Let's go back to in, out. Otherwise this seems good. Any objections?

The current proposal is:
type Caller struct { ... }
func (v Value) Caller() *Caller
func (c *Caller) NumIn() int
func (c *Caller) NumOut() int
func (c *Caller) Call(in, out []Value)
func (c *Caller) CallSlice(in, out []Value)

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

I have run into an issue with reusing the output values. Initially I had used the following to reuse where possible:
s := out[i].ptr
if out[i].typ != tv || out[i].flag&flagIndir != flagIndir || s == nil {
	s = unsafe_New(tv.common())
	out[i] = Value{tv.common(), s, flagIndir | flag(tv.Kind())}
}

found here
This was to not break the current behaviour of call, but as it turns out, this allows for the following in the out:
var i int
out := []Value{ValueOf(i)}

This cases some serious weirdness, and it seems I have not way to detect this case, as the only flag set is flagIndir (outside of kind flags).
This could be resolved by checking out[i].CanSet() but then the newly created Value must have flagAddr which, as stated above, breaks the current implementation of Value.Call and Value.CallSlice and a test.
It is unclear to me how to proceed or if in fact I have made a mistake somewhere.

@nrwiersma What is the status of your work on this issue?  I see that we now have https://go.dev/cl/482375 to implement this.

@ianlancetaylor I ran into a road block which I don't know how to resolve without changing the behaviour of Value.Call and a test. Details are here: https://github.com/golang/go/issues/49340#issuecomment-1437299990.
I saw the implementation, and it seems it suffers the same issue I ran into, allowing
var i int
out := []Value{ValueOf(i)}

Sorry, I'm not sure I entirely understand what you mean.  Can you give a short example using Value.Caller where this causes a problem?  Thanks.

Sure. This is from a test I expected to fail, and it does, kind of.
fn := func() int {
	return 27
}

var i int
v := ValueOf(i)
out := []Value{v}

ValueOf(fn).Caller().Call([]Value{}, out)

if i != 27 {
	t.Errorf("unexpected result for output value; got %d, want 27", i)
}

The actual error you get is unexpected result for output value; got 27, want 27. v actually should be v := ValueOf(&i).Elem().
The issue here is that v is not addressable, and normally this should panic. But as call uses flags flagIndir | flag(tv.Kind()) and to make the output slice reusable, this is allowed and undetectable (I dont know if call made the output or if it was provided by the user). If I were to enforce flagAddr on the output values I ether cannot reuse output values created by call as the dont have this flag or I add flagAddr to the call created output values and I break the existing behaviour of Value.Call and a test (I believe TestFunc). That is my conundrum.

Sure. This is from a test I expected to fail, and it does, kind of.
fn := func() int {
	return 27
}

var i int
v := ValueOf(i)
out := []Value{v}

ValueOf(fn).Caller().Call([]Value{}, out)

if i != 27 {
	t.Errorf("unexpected result for output value; got %d, want 27", i)
}

The actual error you get is unexpected result for output value; got 27, want 27. v actually should be v := ValueOf(&i).Elem().
The issue here is that v is not addressable, and normally this should panic. But as call uses flags flagIndir | flag(tv.Kind()) and to make the output slice reusable, this is allowed and undetectable (I dont know if call made the output or if it was provided by the user). If I were to enforce flagAddr on the output values I ether cannot reuse output values created by call as the dont have this flag or I add flagAddr to the call created output values and I break the existing behaviour of Value.Call and a test (I believe TestFunc). That is my conundrum.

It output in my mac m1:
unexpected result for output value; got 0, want 27

I think this is reasonable.

@nrwiersma I see, thanks for the example.  I agree that the behavior is not ideal but I think it's OK.  It seems understandable.  And it doesn't seem like a likely mistake.

In order for this to be useful, I'm responsible for pooling the reflect.Caller objects. How do I do that when I'm managing an unbounded number of discrete functions? For each function, I can't just have one pre-prepared reflect.Caller since reflect.Caller is not concurrent safe. I need a pool of them for each function. It does not scale to have a sync.Pool for each of these.

In looking at the concrete CL, it seems that there is still a sync.Pool access internally on each invocation, which can lead to substantial cache contention (see #24479) and scalability issues, especially for Caller instances wrapping short and very efficient functions.
If the reflect.Caller has that sort of nontrivial buffer management, then I don't really see the point of it: it seems like most of its improvements could be made without an API change, by adding a sync.Map or two to store the pre-computed information keyed by rtype and methodIndex. (I suppose there is still an advantage to making the out slice an out-parameter, but it's not clear to me that that optimization is worth defining an entire receiver type.)
Moreover: once we promise that Caller is safe for concurrent use we cannot reverse that decision, but if we disallow concurrent use for now, we can easily revisit that decision and make it safe later. (Making a concurrency-safe API no longer so is a breaking change, but making a non-concurrency-safe API safe for concurrent use is strictly compatible.)
So I think it is important that the Call and CallSlice methods of reflect.Caller not be safe for concurrent use today.

What's the behavior if the provided out has elements that mismatch what the function actually outputs? I propose:
If the n-th output type is assignable to out[n].Type() and out[n].CanSet, then the output is directly stored using the equivalent of out[n].Set.

That seems too subtle to me: making a small change to a program that was previously allocating new values could cause it to silently switch to overwriting existing values (or vice-versa), the resulting aliasing bugs seem likely to be difficult to diagnose.
Perhaps instead the Caller could require that the out slice is always prepopulated with values of correct types for which CanSet is true, and provide a separate method method to allocate and initialize values as needed?
That method could look like:
// MakeOut returns a slice of Values suitable for passing as the out argument
// to Call or CallSlice.
func (*Caller) MakeOut() []Value

So I think it is important that the Call and CallSlice methods of reflect.Caller not be safe for concurrent use today.

This would mean that Value.Call and Value.CallSlice could not use Caller, meaning Caller would need to be a entirely new way of function calling, or the Value would need its own sync.Pool in order to use it.
Also it would make no sense to provide the out values to Call and CallSilce as the Caller should be able to entirely manage the output values.

This would mean that Value.Call and Value.CallSlice could not use Caller

Why is that a problem? Even in https://go.dev/cl/483255/6 a new Caller instance is being allocated for each call to Value.Call or Value.CallSlice and is not being shared across multiple concurrent calls.

Also it would make no sense to provide the out values to Call and [CallSlice] as the Caller should be able to entirely manage the output values.

That doesn't follow. It still seems useful to pass in the out slice to allow the user of Caller to avoid unnecessary allocations, especially if they are reusing the same allocated slice across multiple calls to different functions.

Fair enough. If this is the new direction, I don't have the time to completely restart this PR. In that case I would rather close it and let someone else have a go.

@bcmills Note that the sync.Pool is only accessed when there are no output arguments.
Personally I think the current behavior of the output slice is correct: it gets automatically initialized for you, and if you keep passing the same output slice, it doesn't do any additional allocations.  Yes, if you change the output slice in some way it may not behave as you expect.  Don't do that.  Or, we could panic if the element in the output slice is both initialized and the wrong type.


Aria2 Status is "disconnected", but background JSONRPC returns 200.

The status on webpage is always "disconnected", so I opened webconsole to check the request and response. The response from Aria2 seems alright. What should I check?
AriaNg is latest release. Aria2 is 1.35.
Response of JSONRPC: {"id":"QXJpYU5nXzE1OTkzNjg4MzNfMC42NzMyNDMzMTk4Mjg4MjA5","jsonrpc":"2.0","result":{"downloadSpeed":"0","numActive":"0","numStopped":"0","numStoppedTotal":"0","numWaiting":"0","uploadSpeed":"0"}}

I found the reason: https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS/Errors/CORSMissingAllowOrigin

Closing since this is not AriaNg issue.


Fix/infra modle golangci lint problem

Describe what this PR does / why we need it
make the infra modle pass the golangci-lint check
Does this pull request fix one issue?
none
Describe how you did it
use gofmt, goimorts to fmt code
remove unnecessary leading newline
make const naming conform to the standard
Describe how to verify it
cd infra
golangci-lint run  -c .golangci.yaml
Special notes for reviews
none

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Query: re-visit FunctionPreprocessingExpressionVisitor

During pre-process we add null checks to StartsWith and EndsWith. We should double check if we can do better there, in most cases those checks are not needed, perhaps we can be smarter in detecting the cases when they matter.

Currently, we don't have any ideas to do anything better. If we ever come up with a solution we can improve it. This is not causing any issue right now so closing it as we don't need to track it.


fix srcDir + filtering

PR https://github.com/broccolijs/broccoli-funnel/pull/120 broke the case when you have both srcDir and some filtering options like exclude. The included test demonstrates the bug. It crashes during processFile because the sourcePath doesn't exist.
This PR fixes the bug by restoring the correct relative paths as inputs to walkSync.

I have a similar issue from using srcDir, destDir, and include.
I tested your changes and they resolve the problem.
@rwjblue Can you review please?

Wow, fast! Appreciated.

There are other regressions that remain unfixed that stem from the same changes that introduced this bug, see https://github.com/broccolijs/broccoli-funnel/pull/136


Why running so long?

This was adressed in other issues before, please read them before comming up with new issues, to prevent overflow.
Anyhow the answer is:
This is adressed in the original paper of the the Mask RCNN. They trained for 2 days on a 8 GPU machine. Since parallelism does not speed up linear this may take this long. Even if it'd be linear it would be 16 days on a single GPU. This was done on the COCO dataset so other datasets may differ. Since you can increase the Batchsize much higher the more memory you have available (which would be 8 times higher with 8 GPUs) the CUDA-Stream applications will increase the heterogenous tasks, which will result in a shorter training time.
Like discussed in:  #31


ensure pxt-jacdac client services are implemented

Describe the bug
Some services are not yet implemented in pxt-jacdac (e.g. relay). Adding them as an extension causes a Network Request Failed error because pxt.json does not exist.
We need to make sure all client services are working and available in pxt-jacdac. If you click the "Add Blocks" button and one of the modules results in a Network Request Failed error, none of the packages are added.
Check:

[x] relay
[x] accelerometer
[x] button
[x] ambient light
[x] potentiometer
[x] temp
[x] humidity
[x] rotary encoder
[ ] probably more...

It seems that they were all there minus relay. Next, is testing them all with hardware + sim.

@tballmsft you can add your test programs as "test.ts" in the source

@pelikhan - what test programs are you referring to?  The programs that are going in the docs?

Lumping this into docs work.


MqDockerUp crashing frequently

I'm not quite sure why, but MqDockerUp keeps crashing. I keep noticing updates listed in Home Assistant that I have already installed, and when I check my docker containers on UnRAID, MqDockerUp has quit. The latest time was during some container updates. This is the end of the log when the container quit:
19.03.2024 13:04:38 [info]      Checking for removed containers...
19.03.2024 13:04:38 [info]      Checking for containers...
19.03.2024 13:04:38 [info]      Checking for image updates...
19.03.2024 13:04:38 [error]     MqDockerUp stopped due to an error
19.03.2024 13:04:38 [error]     object
19.03.2024 13:04:38 [error]     Error: (HTTP code 404) no such container - No such container: e119ba7d1906d6ea633d3c7d40353acf0dd2fe1cd744edc4538f05bd5633ba2b
    at /app/node_modules/docker-modem/lib/modem.js:382:17
    at getCause (/app/node_modules/docker-modem/lib/modem.js:417:7)
    at Modem.buildPayload (/app/node_modules/docker-modem/lib/modem.js:378:5)
    at IncomingMessage.<anonymous> (/app/node_modules/docker-modem/lib/modem.js:346:16)
    at IncomingMessage.emit (node:events:529:35)
    at IncomingMessage.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

Based on the log I'm assuming MqDockerUp was running update checks still when one of the containers was upgraded, and it panicked because a container it expected to be there wasn't (because it was updated). Any ideas about whether MqDockerUp can recover gracefully from this rather than dying?

Oh, and right after I restarted the container:
><EMAIL_ADDRESS>start
> ts-node src/index.ts

  __  __       ____             _             _   _
 |  \/  | __ _|  _ \  ___   ___| | _____ _ __| | | |_ __
 | |\/| |/ _` | | | |/ _ \ / __| |/ / _ \ '__| | | | '_ \
 | |  | | (_| | |_| | (_) | (__|   <  __/ |  | |_| | |_) |
 |_|  |_|\__, |____/ \___/ \___|_|\_\___|_|   \___/| .__/
            |_|                                    |_|
Starting MqDockerUp V1.8.0 ...

19.03.2024 13:13:33 [info]      Connected to the database.
19.03.2024 13:13:33 [info]      Database initialized successfully
19.03.2024 13:13:33 [error]     MQTT client connection error:  connack timeout
19.03.2024 13:13:33 [error]     MqDockerUp stopped due to an error
19.03.2024 13:13:33 [error]     object
19.03.2024 13:13:33 [error]     Error: connack timeout
    at Timeout._onTimeout (/app/node_modules/mqtt/src/lib/client.ts:853:23)
    at listOnTimeout (node:internal/timers:569:17)
    at processTimers (node:internal/timers:512:7)

Maybe some "retry" logic to be a bit more resilient?

Same issue here, although I haven't seen the "no such container" messages. Just continuous, every minute or so, "connack timeout" messages followed by the container restarting. Been working on it for a few days. Not sure when exactly it started.

Same here on my second machine... the first one works perfectly

I also had the same issue with 'no such' available.  Mine specifically was an esphome/esphome:latest image that was in a stopped container.  I've since switched to a fixed tag for esphome from ghcr.io so it's possible the old one doesn't exist anymore.
I'll add that mine was an image that hadn't been used in over 10 months that only recently started showing in MqDockerUp.  I'm not sure that I love the default behavior to check for all images on the machine.  I'd prefer to only check running container images or even only images that have been used in the last 4 weeks by default.

Same issue here. When i call an update, the container stop. To resume :
02.06.2024 05:25:46 [error] 	object
02.06.2024 05:25:46 [error] 	SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at IncomingMessage.<anonymous> (/app/src/services/DockerService.ts:23:28)
    at IncomingMessage.emit (node:events:517:28)
    at IncomingMessage.emit (node:domain:489:12)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at IncomingMessage.Readable.push (node:internal/streams/readable:278:10)
    at HTTPParser.parserOnBody (node:_http_common:131:24)
    at Socket.socketOnData (node:_http_client:541:22)
    at Socket.emit (node:events:517:28)

And after it tries to restart withe the same error in loop a fue times (2 times).
02.06.2024 05:26:02 [error] 	MQTT client connection error:  connack timeout
02.06.2024 05:26:02 [error] 	MqDockerUp stopped due to an error
02.06.2024 05:26:02 [error] 	object
02.06.2024 05:26:02 [error] 	Error: connack timeout
    at Timeout._onTimeout (/app/node_modules/mqtt/src/lib/client.ts:853:23)
    at listOnTimeout (node:internal/timers:569:17)
    at processTimers (node:internal/timers:512:7)

It finally succeed on 3rd restart with 1 error.
02.06.2024 05:26:30 [info] 	Connected to the database.
02.06.2024 05:26:30 [info] 	MQTT client successfully connected
02.06.2024 05:26:30 [info] 	Checking for removed containers...
02.06.2024 05:26:30 [info] 	Database initialized successfully
02.06.2024 05:26:30 [info] 	Checking for containers...
02.06.2024 05:26:31 [info] 	Checking for image updates...
02.06.2024 05:26:31 [info] 	Image nocturno/hyper-hdr:latest is up-to-date
02.06.2024 05:26:32 [info] 	Image linuxserver/overseerr:latest is up-to-date
02.06.2024 05:26:32 [info] 	New version available for image linuxserver/prowlarr:latest
02.06.2024 05:26:32 [info] 	Image linuxserver/radarr:latest is up-to-date
02.06.2024 05:26:33 [info] 	Image linuxserver/sonarr:latest is up-to-date
02.06.2024 05:26:33 [info] 	Image micrib/mqdockerup:latest is up-to-date
02.06.2024 05:26:33 [info] 	Image flaresolverr/flaresolverr:latest is up-to-date
02.06.2024 05:26:34 [info] 	Image adguard/adguardhome:latest is up-to-date
02.06.2024 05:26:34 [info] 	Image portainer/portainer-ce:latest is up-to-date
02.06.2024 05:26:36 [info] 	Image jlesage/jdownloader-2:latest is up-to-date
**Failed to check for new Dockerhub image digest: TypeError: Cannot read properties of undefined (reading 'split')
02.06.2024 05:26:56 [error] 	Cannot read properties of undefined (reading 'split')**
02.06.2024 05:26:56 [warn] 	Failed to find new digest for image markusmcnugen/qbittorrentvpn:latest
02.06.2024 05:26:56 [info] 	Image containrrr/watchtower:latest is up-to-date
02.06.2024 05:26:56 [info] 	Finished checking for image updates
02.06.2024 05:26:56 [info] 	Next check in 5 minutes

Docker sometimes crash during an update. Then, i have to manually update, restore and restart the container aimed by mqdockerup.

I'm not quite sure why, but MqDockerUp keeps crashing. I keep noticing updates listed in Home Assistant that I have already installed, and when I check my docker containers on UnRAID, MqDockerUp has quit. The latest time was during some container updates. This is the end of the log when the container quit:
19.03.2024 13:04:38 [info]      Checking for removed containers...
19.03.2024 13:04:38 [info]      Checking for containers...
19.03.2024 13:04:38 [info]      Checking for image updates...
19.03.2024 13:04:38 [error]     MqDockerUp stopped due to an error
19.03.2024 13:04:38 [error]     object
19.03.2024 13:04:38 [error]     Error: (HTTP code 404) no such container - No such container: e119ba7d1906d6ea633d3c7d40353acf0dd2fe1cd744edc4538f05bd5633ba2b
    at /app/node_modules/docker-modem/lib/modem.js:382:17
    at getCause (/app/node_modules/docker-modem/lib/modem.js:417:7)
    at Modem.buildPayload (/app/node_modules/docker-modem/lib/modem.js:378:5)
    at IncomingMessage.<anonymous> (/app/node_modules/docker-modem/lib/modem.js:346:16)
    at IncomingMessage.emit (node:events:529:35)
    at IncomingMessage.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

Based on the log I'm assuming MqDockerUp was running update checks still when one of the containers was upgraded, and it panicked because a container it expected to be there wasn't (because it was updated). Any ideas about whether MqDockerUp can recover gracefully from this rather than dying?

Did you find a solution? Because I still get the error when updating a container. And because MQdockerup just restarts, it doesn't update the container, and the container never starts

Sorry to say, but the problem is still here 🥲 in the latest version 1.12.1
><EMAIL_ADDRESS>start
> ts-node src/index.ts
  __  __       ____             _             _   _
 |  \/  | __ _|  _ \  ___   ___| | _____ _ __| | | |_ __
 | |\/| |/ _` | | | |/ _ \ / __| |/ / _ \ '__| | | | '_ \
 | |  | | (_| | |_| | (_) | (__|   <  __/ |  | |_| | |_) |
 |_|  |_|\__, |____/ \___/ \___|_|\_\___|_|   \___/| .__/
            |_|                                    |_|
Starting MqDockerUp V1.12.1 ...
25.10.2024 21:38:00 [info] 	Connected to the database.
25.10.2024 21:38:00 [info] 	MQTT client successfully connected
25.10.2024 21:38:00 [info] 	Checking for removed containers...
25.10.2024 21:38:00 [info] 	Database initialized successfully
25.10.2024 21:38:00 [info] 	Checking for containers...
25.10.2024 21:38:00 [info] 	Removed missing container vuetorrent from Home Assistant and database.
25.10.2024 21:38:00 [info] 	Removed missing container qbittorrent from Home Assistant and database.
25.10.2024 21:38:00 [info] 	Adding container vuetorrent to database
25.10.2024 21:38:00 [info] 	Adding container qbittorrent to database
25.10.2024 21:38:01 [info] 	Checking for image updates...
Failed to check for new Dockerhub image digest: AxiosError: Request failed with status code 404
25.10.2024 21:38:01 [error] 	qbittorrent-vuetorrent
25.10.2024 21:38:01 [error] 	Request failed with status code 404
25.10.2024 21:38:02 [info] 	Image linuxserver/qbittorrent:latest is up-to-date
25.10.2024 21:38:02 [info] 	Image frooodle/s-pdf:latest is up-to-date
25.10.2024 21:38:03 [info] 	Image healthchecks/healthchecks:latest is up-to-date
25.10.2024 21:38:03 [info] 	Image ghcr.io/linuxserver/grocy:latest is up-to-date
25.10.2024 21:38:03 [info] 	Image ghcr.io/linuxserver/piper:latest is up-to-date
25.10.2024 21:38:04 [info] 	Image ghcr.io/linuxserver/faster-whisper:latest is up-to-date
25.10.2024 21:38:04 [error] 	MqDockerUp stopped due to an error
25.10.2024 21:38:04 [error] 	object
25.10.2024 21:38:04 [error] 	Error: (HTTP code 404) no such image - No such image: ghcr.io/linuxserver/speedtest-tracker:latest
    at /app/node_modules/docker-modem/lib/modem.js:382:17
    at getCause (/app/node_modules/docker-modem/lib/modem.js:417:7)
    at Modem.buildPayload (/app/node_modules/docker-modem/lib/modem.js:378:5)
    at IncomingMessage.<anonymous> (/app/node_modules/docker-modem/lib/modem.js:346:16)
    at IncomingMessage.emit (node:events:529:35)
    at IncomingMessage.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

I'm afraid this is a different issue. The original was "no such container". This happened when you have multiple instances of mqdockerup and you send an update command to one, the other would look to update the container on the other machine where it doesn't exist.

oh gosh, you are right!
after some testing I found the problem (after the registry change a couple of images had fucked up)


Missing accounts doc after block #5327000

Doing a sync of mainnet with no filters, I ran into a problem sometime shortly after block #5327000 which resulted in an endless stream of the following error:
error 2018-10-24T20:03:23.292 thread-1  deserializer.cpp:28           get_abi_by_account   ] elasticsearch connection error, line 28, All hosts failed for request.
Running a tcpdump on the connection I am seeing these requests and error responses:
GET /accounts/_doc/6138663588472832000 HTTP/1.1
Host: <IP_ADDRESS>:5000
Authorization: Basic Og==
User-Agent: curl/7.56.0-DEV
Accept: */*
Content-Length: 0
Content-Type: application/x-www-form-urlencoded

20:15:37.459712 IP <IP_ADDRESS>.commplex-main > <IP_ADDRESS>.55398: Flags [P.], seq 1:172, ack 211, win 350, options [nop,nop,TS val<PHONE_NUMBER> ecr<PHONE_NUMBER>], length 171
E.... @.@..............f...kY@=?...^.......
bk)Nbk)KHTTP/1.1 404 Not Found
content-type: application/json; charset=UTF-8
content-length: 78

{"_index":"accounts","_type":"_doc","_id":"6138663588472832000","found":false}

It seems like an account document was somehow dropped.  I can't find anything obvious in the elasticsearch or nodeos logs to indicate that the insert was dropped, however.

I could not reproduce this on the second run, it got to block #6158000 before being oom-killed.  I will close this one but will have to reopen the OOM issue.


Ci against rails7.1

Why?
Rails7.1 was released on 2023/10/05.
Currently, Scimaenaga can only be used with Rails 7.0 or lower.
This Pull Request is to make Scimaenaga compatible with Rails 7.1.
What?

Relax rails dependency version.
Use the latest version of rspec-rails.

@mkmn
Thank you for this pull request.
As a result of running CI, Rubocop pointed out below, so could you please fix it?
Specify development dependencies in Gemfile. Raw Output: scimaenaga.gemspec:27:3: C: Gemspec/DevelopmentDependencies: Specify development dependencies in Gemfile.

@ katsuyasu-murata
It's late.
I was considering making corrections, but I would like to disable Gemspec/DevelopmentDependencies cop for the following reasons.

Having multiple gemfiles increases maintenance costs.
add_development_dependency can continue to be used.
reviewdog only checks added/modified lines.

Fixing it may result in new warnings.
I would like to address the warnings collectively in a separate pull request.

I'm okay with that :+1:


Changes where a numeric value is NULL do not get captured

Description:
I use the timescaledb-event-streamer to catch inserts and updates. There seems to be a bug that if you insert or update a row but the value of a numerical field (i.e. float8) is NULL, the change does not get captured due to the error:
[application] [WARN] [caller] [TaskDispatcher] Error while dispatching event: illegal value for data type conversion

Replicate:
CREATE TABLE event_table
(
    ts      timestamp with time zone NOT NULL,
    details float8 NULL
);

SELECT create_hypertable('event_table', 'ts');

-- works as expected and catches the change
-- {"payload":{"op":"c","after":{"ts":"2023-09-14T07:34:48.414565Z","details":1},"source": ... }
INSERT INTO event_table (ts, details) VALUES (now(), 1);

-- this change does not get captured an results in the following warning/error
INSERT INTO event_table (ts) VALUES (now());

The log for the failing insert:
[2023/09/14T09:34:57.980] [application] [DEBUG] [caller] [ReplicationHandler] EVENT: {messageType:Begin finalLSN:0/220A650 commitTime:2023-09-14 09:34:57.967287 +0200 CEST xid:902}
[2023/09/14T09:34:57.980] [application] [DEBUG] [caller] [ReplicationHandler] EVENT: {messageType:Relation relationId:19101 namespace:_timescaledb_internal relationName:_hyper_1_3_chunk replicaIdentity:100 columnNum:2 columns:[{name:ts flags:1 dataType:1184 typeModifier:-1} {name:details flags:0 dataType:701 typeModifier:-1}]}
[2023/09/14T09:34:57.980] [application] [DEBUG] [caller] [ReplicationHandler] EVENT: {messageType:Insert newValues:map[details:<nil> ts:2023-09-14 09:34:57.966826 +0200 CEST]}
[2023/09/14T09:34:57.980] [application] [DEBUG] [caller] [ReplicationHandler] EVENT: {messageType:Commit flags:0 commitLSN:0/220A650 transactionEndLSN:0/220A680 commitTime:2023-09-14 09:34:57.967287 +0200 CEST}
[2023/09/14T09:34:57.980] [application] [WARN] [caller] [TaskDispatcher] Error while dispatching event: illegal value for data type conversion
[2023/09/14T09:34:57.980] [application] [DEBUG] [caller] [EventEmitter] Transaction xid=902 (LSN: 0/220A680) marked as processed
[2023/09/14T09:34:57.980] [application] [DEBUG] [caller] [ReplicationContext] Acknowledge transaction end: 0/220A680

I guess the problem is that in go ints and floats are not nillable

Hey @flexwende! Thanks for the report. That may be a good catch. I think not all converters may be implemented to support NULL yet!
Let me look into it. There may be more. Guess I'll just extend the integration tests to automatically test every single data type with a NULL value.

Release build 0.9.2 includes the necessary fixes: https://github.com/noctarius/timescaledb-event-streamer/releases/tag/v0.9.2


Reqest entity too large

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
If the user flow involves some big HTTP requests and operations, the error logging fails with "Reqest entity too large" because Sentry tries to send full stack-trace and HTTP request logs to the backend server.
What is the expected behavior?
The SDK should truncate the body if it's too large.
Which versions of Raven.js, and which browser and OS are affected by this issue?
I'm using the version 3.6 with your SaaS offering.

because Sentry tries to send full stack-trace and HTTP request logs to the backend server.

Raven.js does not send HTTP request logs, unless you're doing something custom.

The SDK should truncate the body if it's too large.

It truncates frames. It doesn't send HTTP request bodies so it wouldn't know to do this.
Are you appending your own data manually or using a plugin like redux-raven-middleware? Please attach a copy of a sample outbound report and I can act on this.

I use Angular plugin with default configurations but you're right, the version is pretty out of date. I upgraded to the last version and let you know if it occurs again. Thanks.
Here is a sample:
curl 'https://app.getsentry.com/api/app_id/store/?sentry_version=7&sentry_client=raven-js%2F3.9.2&sentry_key=KEY' -H 'Pragma: no-cache' -H 'Origin: https://app.rakam.io' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36' -H 'Content-Type: text/plain;charset=UTF-8' -H 'Accept: */*' -H 'Cache-Control: no-cache' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary $'{"project":"62493","logger":"javascript","platform":"javascript","request":{"headers":{"User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"},"url":"https://app.rakam.io/report-playground?query=select%201,2,3&options=%7B%22chartType%22:%22geo-cluster%22,%22limit%22:1000,%22enableStatistics%22:true,%22tableOptions%22:%7B%22enabled%22:true%7D,%22chartOptions%22:%7B%22latitude%22:0,%22longitude%22:1,%22title%22:2%7D,%22variables%22:%5B%5D,%22segments%22:%5B%5D,%22measures%22:%5B0,1,2%5D,%22dimensions%22:%5B%5D%7D&queryOptions=%7B%7D"},"exception":{"values":[{"type":"TypeError","value":"Failed to execute \'appendChild\' on \'Node\': parameter 1 is not of type \'Node\'.","stacktrace":{"frames":[{"filename":"https://app.rakam.io/static/components/raven-js/dist/raven.js","lineno":372,"colno":29,"function":"HTMLImageElement.wrapped","in_app":false},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":3,"colno":20753,"function":"HTMLImageElement.o [as _leaflet_click27]","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":17248,"function":"e._onMouseClick","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":1,"colno":5028,"function":"e.fireEvent","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":23910,"function":"e.togglePopup","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":23724,"function":"e.openPopup","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":23431,"function":"e.openPopup","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":1,"colno":21249,"function":"e.addLayer","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":1,"colno":29748,"function":"e._layerAdd","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":19069,"function":"e.onAdd","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":20008,"function":"e.update","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":21514,"function":"e._updateContent","in_app":true}]}}]},"culprit":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","extra":{"cluster":"4c8e91d0-471e-4041-a11f-f818b2b757a1","name":"rakam-prod","type":"ui","session:duration":3993424},"breadcrumbs":{"values":[{"timestamp":1488365533.259,"category":"ui.click","message":"td > i.margin-horizontal-5.pointer.fa.fa-play-circle.green"},{"timestamp":1488365533.335,"type":"http","category":"xhr","data":{"method":"GET","url":"https://api.mixpanel.com/track/?data=%3D&ip=1&_=1488365533262","status_code":200}},{"timestamp":1488365533.497,"type":"http","category":"xhr","data":{"method":"POST","url":"https://webapp-1290649538.us-west-
.......... TONS OF HTTP REQUESTS

TONS OF HTTP REQUESTS

So ... those look like breadcrumbs, and we limit those to 50 per page. If your URLs are explosively large, they could present a problem. So you could set maxBreadcrumbs.

They're quite large because these Mixpanel calls have a big base64 encoded query parameter that includes the event body. I believe that the best option here is to discard some third party analytics service calls such as Mixpanel or truncate the url if it's too large but it seems that both of them requires some work do be done in this repo, right?

Ah, that's very helpful. Yes, we should truncate this client-side.

It looks like I also removed Mixpanel HTTP calls in the snippet since they may include some user sensitive data but that part is particularly large.

Note, in the meantime you can use breadcrumbCallback to truncate (or disable) HTTP breadcrumbs. See config.

We are seeing this today with many large breadcrumbs, and it crashes the browser.  I would argue that Sentry.io should NEVER crash the browser regardless of what the user does.  Could you please add some sort of automatic client-side checking and truncation of the packet size before throwing an error in the client, like Sentry's Python integration did.

I think this issue needs to be reopened.

What do you mean by "crash the browser"? Like... an actual crash? How did that happen? Can you provide some more details?

It threw an error in the browser, effectively crashing the JS application running there.

As for details on how to reproduce, log a lot of large breadcrumbs, then hit another error.  Raven throws its own exception and does not log the original error, either to the console or to the server.

Ok, looking closer at this on my end, I may have been a bit alarmist.  Sentry's thrown error is not crashing the application since it starts with our own JS error, but it IS true that the large number of breadcrumbs are preventing sentry from logging the error to the server.  I think truncating the breadcrumbs in the client to ensure that the stack trace gets logged to the server should be standard.  I would rather see the stack trace with a note about truncated breadcrumbs than nothing at all when my users are encountering real errors.

We'll rethink the approach to this issue in the next major release. Right now, however, it'd be too much overhead to do this operating for every single event and it may impact the performance of our SDK.
We do provide all the required tools so that developers can do that themselves if they are willing to - https://github.com/getsentry/raven-js/blob/master/src/utils.js#L457-L515
If you want to calculate the size of the breadcrumbs every single time and truncate them if they are too heavy, feel free to do that. Here's how you'd approach something like this:
const utils = require('../node_modules/raven-js/src/utils'); // or just extract it to your own file, its up to you

const MAX_SIZE = 50 * 1024;

function truncateBreadcrumbs (data) {
    // Handle edge-case to prevent an infinite loop
    if (utils.jsonSize(utils.stringify(data)) > MAX_SIZE) {
      data.breadcrumbs = data.breadcrumbs.split(0, -1);
      return truncateBreadcrumbs(data)
    } else {
      return data;
    }
}

Raven.config('__YOUR_DSN__', {
  dataCallback: function (data) {
    return truncateBreadcrumbs(data);
  }
}).install();

@kamilogorek - we're getting this error - the app is intrinsically data heavy.
Your code refers to functions that no longer exist.
Would I be right in assuming the only thing those non-existent functions do is allow you to encode JSON with circular references?
If so, if I were to choose another library and use their stringification function, in place of your old utilities, would your code sample work still?
Also, is there a better, built in solution to this? We could probably get away without sending any app state at all, just console log, the error in question and other minutiae.
Otherwise errors gobbled up by Vue don't make it up and aren't reported without the Raven/Vue plugin - unless I've setup the Sentry browser client incorrectly. As only an error allow to propagate uncaught appeared in Sentry.
Cheers

Your code refers to functions that no longer exist.

It does, but here (@sentry/utils package) - https://github.com/getsentry/sentry-javascript/blob/5d670a7781b6e2144809c002e9332e2cc9dd239c/packages/utils/src/object.ts#L109-L123

Would I be right in assuming the only thing those non-existent functions do is allow you to encode JSON with circular references?

Kinda, but also some more:
/**
 * normalize()
 *
 * - Creates a copy to prevent original input mutation
 * - Skip non-enumerablers
 * - Calls `toJSON` if implemented
 * - Removes circular references
 * - Translates non-serializeable values (undefined/NaN/Functions) to serializable format
 * - Translates known global objects/Classes to a string representations
 * - Takes care of Error objects serialization
 * - Optionally limit depth of final output
 */

Also, is there a better, built in solution to this? We could probably get away without sending any app state at all, just console log, the error in question and other minutiae.

You can investigate vue's context (and optional alter/remove it) with beforeSend callback.
Sentry.init({
  beforeSend(event) {
    // event.contexts.vue has all the metadata
    return event;
  }
})


MissingPluginException

On Android when I run this I get the following error:
E/flutter (21336): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: MissingPluginException(No implementation found for method identify on channel posthogflutter)
E/flutter (21336): #0      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:294:7)
E/flutter (21336): <asynchronous suspension>
E/flutter (21336): #1      PosthogMethodChannel.identify (package:posthog_flutter/src/posthog_method_channel.dart:14:7)
E/flutter (21336): <asynchronous suspension>

I've followed the the instructions here. I also tried to run flutter clean with no luck.

It looks like this occurs only with hot reload and works OK for initial load or hot restart.

@alexprice12 does this happen on Android, iOS or Web?

@alexprice12 any feedback?

Closing the issue as a part of large repository cleanup, due to it being inactive and/or outdated.
Please do not hesitate to ping me if it is still relevant, and I will happily reopen and work on it.
Cheers!

This just happened to me when running unit tests. I'm testing a method that calls Posthog().identify() and I'm getting this exception when the test runs. I can get around it by moving it out into a provider and then mocking that provider but that's not great. Happy to provide more info if that's helpful.

@justincbeck could it be that you did not call TestWidgetsFlutterBinding.ensureInitialized(); in your tests?
Please open a new issue if needed with more context, flutter version, SDK version, the test itself, the stack trace, thanks.


⚠ Mobitopup has degraded performance

In f3ddf8d, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 8596 ms

Resolved: Mobitopup performance has improved in 7f626a5.


It's very easy to create a shared instance array or object property without realising the implications.

Following on from the discussion here:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/polymer-dev/pa_Ak2ATRw8/F_507ptDGQAJ
I managed to get caught out because I had created an array type property with a default value of []. When creating multiple instances of my element this array was then shared across the instances. Although the polymer docs do mention this problem, and the solution, I didn't find the documentation to be particularly clear. I know its a general JavaScript issue, but having a property type that behaves very differently to the other available types leads to confusion.
Another option would be to modify the _configureProperties method to detect array/object property values and output a console message suggesting that the function wrapping workaround be used.

dup: https://github.com/Polymer/polymer/issues/2240

I like the idea of getting the linter to pick this up.

I will close this and we can migrate the discussion over to 2240


Release 2.2.3 [MOB-16270] - Fix android:export manifest error

Description
Release v2.2.3
Changes made to explicitly set element android:exported https://developer.android.com/guide/topics/manifest/activity-element#exported to BOOT_COMPLETED broadcast receiver.
For app's targeting Android 12 and if you are using Android Studio 2020.3.1 Canary 11 or later, you need to set android:exported on each activity, service, and receiver on your AndroidManifest.xml file. Else the build fails with error
Error: Apps targeting Android 12 and higher are required to specify an explicit value for android:exported when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details.

addresses #55


Form submitted when filed has error

Schema
export const profileDisplayNameSchema = v.object({
	displayName: v.string([
		v.minLength(8, "Too Short Display Name"),
		v.maxLength(32, "Invalid Display Name"),
	]),
});

function Test() {
	const [form, { Form, Field }] = createForm<ProfileDisplayNameSchemaForm>({
		validate: valiForm(profileDisplayNameSchema),
	});

	const submitHanlder: SubmitHandler<ProfileDisplayNameSchemaForm> = async (
		values,
		event,
	) => {
		event.preventDefault();
                event.stopPropagation();
	};
	return (
		<Form action={updateDisplayName} method="post" onSubmit={submitHanlder}>
			<Field name="displayName">
				{(field, props) => {
					return (

							<input
								{...props}
								value={field.value}
								data-invalid={field.error ? "" : undefined}
							/>
							{field.error &&
{field.error}
}

					);
				}}
			</Field>
		</Form>
	);
}

I found that if the form is valid, the event.preventDefault() works and the form is not submitted. However, if the filed is invalid, the form was submitted with the invalid value.
Is this a bug?
Edit
I looked at the source code. When the form is valid, onSubmit is called in Form 's onSubmit.
So is it ok to add stopPropagation when form is invalid?
					if (await validate(form, options)) {
						await onSubmit(getValues(form, options), event);
					} else {
						event.stopPropagation();
					}

Can you send me a minimal reproduction on StackBlitz? Do you use SolidStart?

Yes..I use solid start.
I just found a new problem. When use both action and onSubmit on Form component, useSubmission  in onSubmit(I use it to set error form error) will not get the result after submit since the action runs after onSubmit. So...no need to set the stopPropagation.
Instead I have to use useAction to submit the form value and useSubmission get the result.
We can close this issue.
BTW, The reason I want to use action on form is I want to use the native way to submit the form but I have not noticed that action runs after onSubmit.


Conversion to HDRP

Hi Sebastian, love your work.
I tried converting your project to HDRP and noticed a number of issues with it. Thought you might like to know. If you solve them, I'd be keen to hear about it.

The portals no longer appear. This seems to be because HDRP does not use the OnPreCull event in MainCamera. I tried using LateUpdate instead to fix it - I'm sure you know a better way.
The portal shader looks slightly too dark. I have no idea why this is occuring in HDRP.
There seems to be a graphical glitch when crossing the portal that was not there previously. This may be because of the change to LateUpdate.

I really enjoy watching your vids. Keep it up.

It would be helpful for you to publish your fork with the changes you've made so far.

So I got this working.
You need to:

Turn off the post-processing on both the portal cameras under the custom settings and then

In PortalTravelers change line 56 from foreach (var mat in renderer.materials) to foreach (var mat in renderer.sharedMaterials).

As far as OnPreCall you can put a call to the render pipeline to call back before the frame starts. I also recommend using Brackeys precut shader as that one works well with HDRP. Still polishing up as there is a slight flicker when going through the portals. Not 100% seamless but That so far fixes the dark look and gets us 99% there.

Regarding getting this working on URP all the points above are required except that you need to enable post-processing on both portal cameras not disable it.

I commented on the other issue but I think it applies to this one aswell:
https://github.com/SebLague/Portals/issues/8#issuecomment-695125411

@jmsether Do you have a link for the refered Brackeys precut shader, couldn't find that anywhere on the internet.
Might just being blind xD, Any help appreciated!

@joaoachando you can find it in the description of his video: https://www.youtube.com/watch?v=cuQao3hEKfs&t=89s

I beg for your help guys...
I'm on URP and basically as soon as the game starts, the portals get messed up; easier to show than it is to explain... -> https://drive.google.com/file/d/12XMdVidhfvOXO34zb5QXxUaYUfpEyrUx/view?usp=sharing

As far as OnPreCall you can put a call to the render pipeline to call back before the frame starts. I also recommend using Brackeys precut shader as that one works well with HDRP. Still polishing up as there is a slight flicker when going through the portals. Not 100% seamless but That so far fixes the dark look and gets us 99% there.

Could you elaborate on this part? I tried injecting the PrePortalRender, Render and PostPortalRender calls of the portals after the BeginFrameRendering or BeginCameraRendering pipeline callback, but I'm getting an error: "Recursive rendering is not supported in SRP (are you calling Camera.Render from within a render pipeline?)."

It's been a while but I will dig up my old project and get the scripts I used there. It's mostly down the disabling the hdr effects on the portals cameras

This work for me in URP:

Use RenderPipelineManager.beginCameraRendering as the alternative of OnPreCull as it doesn't support SRP.

void Awake()
    {
        portals = FindObjectsOfType<Portal>();
        mainCam = GetComponent<Camera>();
        RenderPipelineManager.beginCameraRendering += RenderPortal;
    }

    private void OnDestroy()
    {
        RenderPipelineManager.beginCameraRendering -= RenderPortal;
    }

    private void RenderPortal(ScriptableRenderContext context, Camera camera)
    {
        for (int i = 0; i < portals.Length; i++)
        {
            portals[i].PrePortalRender(context);
        }
        for (int i = 0; i < portals.Length; i++)
        {
            portals[i].Render(context);
        }
        for (int i = 0; i < portals.Length; i++)
        {
            portals[i].PostPortalRender(context);
        }
    }

Use UniversalRenderPipeline.RenderSingleCamera(context, portalCam) instead of portalCam.Render() in Portal.cs
Enable post-processing for all portal camera

Some Reference:
https://forum.unity.com/threads/onprecull-event-not-running.907634/
https://teodutra.com/unity/shaders/urp/graphics/2020/05/18/From-Built-in-to-URP/

I also needed to change the screen texture from _MainTex to _BaseMap.
static readonly int MainTexture = Shader.PropertyToID("_BaseMap");

void CreateViewTexture ()
{
    if (viewTexture == null || viewTexture.width != Screen.width || viewTexture.height != Screen.height)
    {
        if (viewTexture != null) viewTexture.Release();
        viewTexture = new RenderTexture(Screen.width, Screen.height, 0);
        portalCam.targetTexture = viewTexture;  // the view from the portal camera to the view texture
        linkedPortal.screen.material.SetTexture(MainTexture, viewTexture); // display it
    }
}

Somehow, my RenderTexture is still too dark: https://github.com/SebLague/Portals/issues/8#issuecomment-997226860

This work for me in URP:

Use RenderPipelineManager.beginCameraRendering as the alternative of OnPreCull as it doesn't support SRP.

void Awake()
    {
        portals = FindObjectsOfType<Portal>();
        mainCam = GetComponent<Camera>();
        RenderPipelineManager.beginCameraRendering += RenderPortal;
    }

    private void OnDestroy()
    {
        RenderPipelineManager.beginCameraRendering -= RenderPortal;
    }

    private void RenderPortal(ScriptableRenderContext context, Camera camera)
    {
        for (int i = 0; i < portals.Length; i++)
        {
            portals[i].PrePortalRender(context);
        }
        for (int i = 0; i < portals.Length; i++)
        {
            portals[i].Render(context);
        }
        for (int i = 0; i < portals.Length; i++)
        {
            portals[i].PostPortalRender(context);
        }
    }

Use UniversalRenderPipeline.RenderSingleCamera(context, portalCam) instead of portalCam.Render() in Portal.cs
Enable post-processing for all portal camera

Some Reference: https://forum.unity.com/threads/onprecull-event-not-running.907634/ https://teodutra.com/unity/shaders/urp/graphics/2020/05/18/From-Built-in-to-URP/

I originally had an issue where the Portal Camera's overwrote what the Main Camera rendered however I found the cause and fix. I found that using an additional Overlay Camera with Camera Stacking to render my Weapons interfered with the above code.
The solution is to add a Render Object to the URP and get it to render the Weapons that are on their own layer. Source

I know this is super old, but I've been trying to get this work with HDRP too and am having the same issue of calling Camera.Render() within the render pipeline via RenderPipelineManager.beginCameraRendering.
I tried replacing Camera.Render() with a call to RenderPipeline.SubmitRenderRequest(portalCamera, request) but am now getting an error saying recursive calls aren't allowed either with Camera.Render() OR RenderPipeline.SubmitRenderRequest().
Seems like Unity has a bunch of issues with this stuff. I'm currently on Unity 6000.0.23f1. Has anyone made any developments either on this version or another one?


When exception catched,make sure console application can show the exception

add   Console.ReadLine() for every Console Application Example

？any complain？
in the console application,have you tested when you catch an exception?

@chucklu Thanks for the contribution. I've relocated the ReadLine() method inside the catch{} block

actually,put the ReadLine() outside the catch is better
Curl.GlobalCleanup();
then exit the application ?
actually,I want to see what I have get which should be showd on Console

These are very simple console demos and are supposed to be run inside a command line environment. The purpose of these simple apps is to demonstrate the capabilities of the library. That's why I'd like to keep them lean. Also take note that these apps are almost line-by-line C# port of the official libcURL samples. You don't want to deviate too much from the source.

ok,I always run the console application in Debug mode rather than command line environment


Double entries for RFCs in group documents list on feat/rfc

Describe the issue
RFC documents are listed twice in the last section of the group documents page. They appear once for the RFC and once for the draft that became that RFC. These should be merged into a single entry.
@kivinen
Code of Conduct

[X] I agree to follow the IETF's Code of Conduct

This appears only to be in the "Related Internet-Drafts" section. Production shows RFCs in this section - it is mislabeled.

Closing as fixed on the feat/rfc branch


fix: create new context for bare-metal deployer

for bare-metal mode, deployment and operation are using the same ctx, causing the running binaries to shutdown if we pass the value to --timeout.
so the fix here is: separate the ctx of deployment and operation, do not let --timeout disturb the running binaries.

Codecov Report
Patch coverage: 33.33% and project coverage change: +0.15% :tada:

Comparison is base (6366341) 31.86% compared to head (99095fd) 32.02%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #148      +/-   ##
===========================================
+ Coverage    31.86%   32.02%   +0.15%
===========================================
  Files            9        9
  Lines         1293     1296       +3
===========================================
+ Hits           412      415       +3
  Misses         787      787
  Partials        94       94

Files Changed
Coverage Δ

pkg/cmd/gtctl/cluster/create/create.go
0.00% <0.00%> (ø)

pkg/deployer/baremetal/deployer.go
35.86% <37.50%> (+0.70%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Issue in funnel run query

Describe the Bug
Testing with MySQL, there's an error when I run a funnel report query.
No matter what params I put, weither a match is expected or not, the request doesn't succeed.
I didn't manage to see what's causing the issue for now.
Prisma error isn't very helpful...
and generated query doesn't fail when run through phpMyAdmin.
Database
MySQL
Relevant log output
Invalid `prisma.$queryRawUnsafe()` invocation:
Raw query failed. Code: `N/A`. Message: `N/A`

Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
Test VM

Confirmed bug. We are looking into this.

Fixed in v2.4.0.


M.M.H.C. Voordaan logo issue

Hi Joost,
Everything working except for the Voordaan logo. The logo attribute of Voordaan is empty.
Rgrds,
Timo

Hi Timo,
It gets the data from the api hockey.nl uses. If you check https://hockey.nl/match-center/#/search and search for Vordaan, you will see it does not have a logo. Other clubs such as GroenGeel also have this. You could try contacting the Knhb or the clubs themselves to get them to host the logo. I dont plan on hosting the logo's themselves or getting them from a different api when it doesnt find a logo. If you want you can contact them, or implement this yourself.

Thanks for the prompt reply, I will check with the ITC of the club to solve this. Rgrds Timo


WIP:Update hypershift deployment crd

Signed-off-by: zhujian<EMAIL_ADDRESS>issue reference: https://github.com/stolostron/backlog/issues/22689
This crd is synced from https://github.com/stolostron/hypershift-deployment-controller/blob/main/config/crd/cluster.open-cluster-management.io_hypershiftdeployments.yaml , we need replicas field to config the desired number of nodes the nodepool should maintain.

cc @stolostron/acm-hypershift @JakobGray

This says it's removing over 5000 lines. Is that correct? Is this PR still being updated?

@JakobGray I just copied it from hypershift-deployment-controller repo which is the one matches the code, so I think it is right.
And I applied this CRD in my environment, it works fine.
cc @jnpacker @nelsonjean

@zhujian7 can you add to this comment, but we took a look and this aligns us with what is expected from the Hypershift commit SHA that's being used.
The big change is that Kubevirt hanged their CRD references.  We applied the CRD and it is not a concern for us. @rokej ran the Kubevirt test about 2x weeks back, we will re-run those tests to verify, but the CRD change for the kubevirt change aligns to hypershift-operator image SHA.
Again, we feel the risk is minimal, as the CRD applies and the changes are 98% in a dev preview portion of the code.
It's so big a change because we incorporate the Hypershift HostedClusterSpec and NodePoolSpec in our CRDs.
Again the risk after review is LOW.

@zkayyali812 Are we getting this CRD from somewhere via automation or is it manually updated?

Are we getting this CRD from somewhere via automation or is it manually updated?

it is generated by make manifests for the hypershift-deployment-controller repo.
╰─# make manifests
/home/go/src/github.com/stolostron/hypershift-deployment-controller/bin/controller-gen rbac:roleName=hypershfit-deployment-controller crd paths="./..." output:crd:artifacts:config=config/crd


Connection timeout on websocket

For some reason that I cannot determine (I am using a ws proxy), the WebSocket connection sometimes takes a lot of time to fail (probably based on the browser's "connecting timeout").
When this happens (ua.transport.ws.readyState stuck in WebSocket.CONNECTING), I can manually call ua.transport.ws.close() which will trigger a disconnected (code: 1006) connection error and, in turn, the onError handler which will retry to connect.
It would be nice if I could set a wsServerConnectingTimeout setting that would trigger a new attempt at connecting the WS with a lower delay that the browser's default timeout.

My current workaround, in case someone else needs this:
ua.on('connecting', handleConnecting);
ua.on('connected', handleConnected);

function handleConnecting () {
  var ua = this;

  if (defaults.wsServerConnectingTimeout > 0) {
    clearTimeout(ua.transportConnectingTimer);
    ua.transportConnectingTimer = setTimeout(function () {
      if (ua.transport.ws.readyState !== WebSocket.OPEN) {
        ua.logger.warn('WebSocket connection timeout.');
        ua.transport.ws.close();
      }
    }, defaults.wsServerConnectingTimeout * 1000);
  }
}

function handleConnected () {
  clearTimeout(this.transportConnectingTimer);
}

These are probably good ideas.  I know that the WebSocket connection is a particularly messy area of the code right now, especially with regards to recovering from dropped connections.  It could probably use a giant overhaul.
I'll mark this as a feature request for the time being, and we'll loop back around to it when the time comes.

Yes, nice feature, I need it too.

This feature has been added in 0.11.0 as part of a major Transport refactor. There is a new connectionTimeout configuration option on the new Transport and can be set like so:
new SIP.UA({
  uri<EMAIL_ADDRESS>  transportOptions: {
       wsServers: [wss://my.favorite.server],
       connectionTimeout: 5
  }
});

If the implementation isn't quite how you imagined, we're always open to PRs- but I'm going to close this for now.


Remove all references to Mike Lieberman's personal github

All images should be part of this org. Where applicable all code that will be specifically part of the SSF should be under this repo or at least under this org.

IMAGE=ghcr.io/mlieberman85/rust:latest is the final line that references my personal github. Once that is removed we can close this issue.

IMAGE=ghcr.io/mlieberman85/rust:latest is the final line that references my personal github. Once that is removed we can close this issue.

I couldn't find any reference in the code to the build-docker.sh script. Could you please clarity in what situations this script is being used?

IMAGE=ghcr.io/mlieberman85/rust:latest It is this line in one of the demos.

IMAGE=ghcr.io/mlieberman85/rust:latest It is this line in one of the demos.

Yeah. I've noticed this line in the build-docker.sh script. Let me clarify my question.  Where can I find the instructions on how to execute the demo?


Update dashboard.php

User description

Closes #
📑 Description

✅ Checks

[ ] My pull request adheres to the code style of this project
[ ] My code requires changes to the documentation
[ ] I have updated the documentation as required
[ ] All the tests have passed

☢ Does this introduce a breaking change?

[ ] Yes
[ ] No

ℹ Additional Information

Description

Enhanced the dashboard by adding support for recentPullRequests.
Improved code clarity by renaming functions and parameters.
Updated the title for better representation.

Changes walkthrough 📝
Relevant filesEnhancement

dashboard.phpEnhance dashboard functionality and improve code clarity

Src/dashboard.php
Updated data structure to include recentPullRequests.  Changed title from "Activity Dashboard" to "Dashboard".  Renamed function showAjaxErrorAlert to showErrorAlert.  Modified populateIssues function to accept id instead of type.

+9/-9

💡 Penify usage:
Comment /help on the PR to get a list of all available Penify tools and their descriptions

@gstraccini codacy bypass


Cookie banner confirmation adjust

What
Adding padding
Why
Adjusting visual for cookie banner
Visual Changes

This has already been fixed by Max's PR https://github.com/alphagov/govuk_publishing_components/pull/1936 It may be that it just hasn't been merged and released yet.

This has already been fixed by Max's PR #1936 It may be that it just hasn't been merged and released yet.

Ah sweet, closing!


Update docs

Proposed change(s)
Update README and Add success message to prepare_sd.py
Useful links (GitHub issues, forum threads, etc.)
Types of change(s)

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Code refactor (non-breaking change which refactor the code)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] Documentation update only
[ ] Other (please describe)

Checklist

[x] I have read the CONTRIBUTING guidelines.
[x] I have checked to ensure there aren't other open Pull Requests for the same change.

Other comments

I am fine with the changes.
Of we drop Ubuntu 20.04 support I would promote the cloud-config installation method from experimental to standard.
We need an automatic procedure to create an image that can be downloaded and installed as alternate method for users that have slow network connections

We need an automatic procedure to create an image that can be downloaded and installed as alternate method for users that have slow network connections

:+1:


Handwritten Digit Recognition

Is your feature request related to a problem? Please describe.
The Handwritten Digit Recognition project aims to address the challenge of accurately identifying and classifying handwritten digits, which is crucial in various applications such as automated data entry and postal services. The complexity arises from the variability in handwriting styles, which can significantly impact recognition accuracy. By leveraging the MNIST dataset, which contains a vast collection of handwritten digits, this project seeks to develop a robust neural network model using TensorFlow and Keras. This model will learn to distinguish between the different digits through training, thereby enhancing its ability to generalize and make accurate predictions on unseen data. Implementing this feature not only showcases the effectiveness of deep learning in image classification but also provides a practical solution for automating processes that rely on handwritten digit recognition.
Describe the solution you'd like
The proposed solution for the Handwritten Digit Recognition project involves developing a convolutional neural network (CNN) model that leverages the capabilities of TensorFlow and Keras to accurately classify handwritten digits from the MNIST dataset. The solution will consist of several key components: data preprocessing to normalize and augment the dataset for improved model training; the architecture of the CNN, which will include convolutional layers for feature extraction, pooling layers for dimensionality reduction, and fully connected layers for final classification; and a training process that utilizes appropriate loss functions and optimization algorithms to enhance model performance. Additionally, the solution will include evaluation metrics to assess the model's accuracy and loss on both the training and validation datasets, ensuring that it generalizes well to new data. By employing this structured approach, the model aims to achieve high accuracy in recognizing handwritten digits, ultimately providing an efficient tool for applications that require digit recognition.
Describe alternatives you've considered
In developing the Handwritten Digit Recognition project, several alternative approaches and methodologies were considered to achieve the desired outcome. One alternative was using traditional machine learning algorithms, such as Support Vector Machines (SVM) or k-Nearest Neighbors (k-NN), for digit classification. While these methods can be effective for smaller datasets, they often struggle to generalize well with the complexity and variability inherent in handwritten digits compared to deep learning approaches.
Approach to be followed (optional)
A clear and concise description of the approach to be followed.
Additional context
Add any other context or screenshots about the feature request here.
add LABELS hacktoberfest and GSSOC EXT 24
ASSIGN ME THIS PROJECT

Proceed !


user-functions doesn't show in cmdline-completion since v8.2.3337

Describe the bug
When pressing <Tab> to complete function name, user-functions (starting with uppercase character) does not show.
To Reproduce
Detailed steps to reproduce the behavior:

start vim: vim -u NONE
define a user function: (set nocp to make <Tab> completion work)

function! AUserFunction()
endfu

In command line mode, press call A<Tab> to trigger completion.

Expected behavior
AUserFunction (as we defined in step 2) should be put in cmd line.
Screenshots
none.
Environment (please complete the following information):

Vim version since v8.2.3337
OS: tested in Alpine Linux 3.14, macOS 11.5.2
Terminal: tmux inside alacritty

Additional context
I use git bisect to figure out that v8.2.3337 (069f90852f1444cac50491c10dfbd339a3f9e0c8) introduces this bug.

Sorry the reproduce step seems not to trigger the behavior. (define user-function via command line mode)
The following step should work:
define user-function in vimrc.

This is strange, for me presence of the file /.vim/pack/plug/start/vim-swap/plugin/swap.vim:
" The vim plugin to reorder delimited items.
" Last Change: 07-Jun-2019.
" Maintainer : Masaaki Nakamura<EMAIL_ADDRESS>
" License    : NYSL
"              Japanese <http://www.kmonos.net/nysl/>
"              English (Unofficial) <http://www.kmonos.net/nysl/index.en.html>

if &compatible || exists("g:loaded_swap")
  finish
endif
let g:loaded_swap = 1

" keymappings
nnoremap <silent> <Plug>(swap-interactive) :<C-u>call swap#prerequisite('n')<CR>g@l
xnoremap <silent> <Plug>(swap-interactive) :<C-u>call swap#prerequisite('x')<CR>gvg@
nnoremap <silent> <Plug>(swap-prev) :<C-u>call swap#prerequisite('n', repeat([['#', '#-1']], v:count1))<CR>g@l
nnoremap <silent> <Plug>(swap-next) :<C-u>call swap#prerequisite('n', repeat([['#', '#+1']], v:count1))<CR>g@l
noremap <silent> <Plug>(swap-textobject-i) :<C-u>call swap#textobj#select('i')<CR>
noremap <silent> <Plug>(swap-textobject-a) :<C-u>call swap#textobj#select('a')<CR>

""" default keymappings
" If g:swap_no_default_key_mappings has been defined, then quit immediately.
if exists('g:swap_no_default_key_mappings') | finish | endif

nmap gs <Plug>(swap-interactive)
xmap gs <Plug>(swap-interactive)
nmap g< <Plug>(swap-prev)
nmap g> <Plug>(swap-next)

If I do :finish on the top of the file it still "influences" and I can't get function completed.

I find an even easier reproduce step:

run vim with VIMRUNTIME=/a-not-exist-dir vim -u test-vimrc.vim;
try to call :call D<Tab> to complete function.

content of test-vimrc.vim:
set nocp

function! Date()
endfunction

set rtp=
source empty.vim
source non-empty.vim

empy.vim is empty. (I guess file with any content should also behave the same)
content of non-empty.vim:
function! s:a_dict_func() dict
endfunction

@habamax can you post the output of :scr in vim?
With this minimal setup, the output is:
  1: ~/repos/vim/tmp.vimrc
  2: ~/repos/vim/empty.vim
  3: ~/repos/vim/non-empty.vim

I can't repro it with your steps, @lixiao-tap4fun .
But I have this issue with a single and only (all others are removed) vim-swap plugin (and even with a single file from that plugin). Weird.

For me it looks like this:

I will try to share the step to build, run vim and reproduce this inside docker tomorrow (I think a clean docker container should be easy to setup for everyone).

Additional context
I use git bisect to figure out that v8.2.3337 (069f908) introduces this bug.

I have reverted this patch and all functions could be completed.

I have verified that https://github.com/vim/vim/issues/8765#issuecomment-900168795 is reproducable on gVim for win32 (using 32-bit installer in Windows 8.1; installer is here: https://github.com/vim/vim-win32-installer/releases/tag/v8.2.3356)

I can reproduce the issue without sourcing empty.vim in https://github.com/vim/vim/issues/8765#issuecomment-900168795.

Run vim -Nu NONE.

:source this script:
function Date()
endfunction
let lines =<< END
  function s:a_dict_func() dict
  endfunction
END
call writefile(lines, '/tmp/s.vim')
source /tmp/s.vim
call feedkeys(":call D\<C-a>\<Home>\"\<CR>", 'tx')
echo @:

Actual:
"call D

Expect:
"call Date()

Vim 8.2.3358
macOS 10.15.7
iTerm2

When making the dictionary function global, the issue disappears:
function Date()
endfunction
let lines =<< END
  function ADictFunc() dict
  endfunction
END
call writefile(lines, '/tmp/s.vim')
source /tmp/s.vim
call feedkeys(":call D\<C-a>\<Home>\"\<CR>", 'tx')
echo @:

"call Date()

Even the writefile can be omitted:

Run vim -Nu NONE.
:source this script:

function Date()
endfunction

function s:a_dict_func() dict
endfunction

call feedkeys(":call D\<C-a>\<Home>\"\<CR>", 'tx')
echo @:

will get the same result.

@lixiao-tap4fun Oh, you are right. Presumably that is the simplest way to reproduce.

So the added check *name != NUL somehow conflicts with dict functions.

I've created a pull-request #8774 that will fix this issue. I'm not sure it is a proper way though...


Auto scroll should be disabled when scrolling left in line graph view or when scrolling up in table/console views

This should already be supported for the line graph. Are you having issues making it work?
It’s not currently supported for the table and console, but that seems like a reasonable addition.

So I was not sure about the graph but included it just in case it wasn't implemented already.

Thanks for the suggestion, this is now implemented for the table in addition to the line graph.


Should requirePermission be true?

Hi @klings. Found this setting in Web.config:
<sectionGroup name="nwebsec">
    <!-- For information on how to configure NWebsec please visit: https://docs.nwebsec.com/ -->
    <section name="httpHeaderSecurityModule"
        type="NWebsec.Modules.Configuration.HttpHeaderSecurityConfigurationSection, NWebsec, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=3613da5f958908a1"
        requirePermission="false" />
</sectionGroup>

When would we set the requirePermission flag to true?

Never. This was related to the good ol' trust levels for ASP.NET applications, which turned out to be dead by design. You can find a short description of it here: https://msdn.microsoft.com/en-us/library/system.configuration.sectioninformation.requirepermission.aspx


Addli prefix autofill

Adding an Insurance is a real pain so let me help you:

pressing enter on an incomplete addli will fill in all the prefixes for you
also redirects your caret to the empty field

Side stuff:
Code claiming
Debuged parsePremium to handle NumberFormatException (if you typed pr/hahaha you will get an undetected exception)
Docs update

Changes Unknown when pulling db2df156f56be711cbaa08be1e08258afb9295de on Juxarius:master into ** on CS2103AUG2017-T13-B1:master**.

Changes Unknown when pulling db2df156f56be711cbaa08be1e08258afb9295de on Juxarius:master into ** on CS2103AUG2017-T13-B1:master**.


[Issue 1713] Support optional booleans in annotation processor

Fixes #1713

@remkop can you retry the CI one more time? Maybe it had something to do with the latest CloudFlare issue.

Merged. Thank you for the contribution!

Hi @nineinchnick, I will probably move some of the items that are currently in scope for 4.7 to a later release.
That said, there are currently 4 tickets of type:bug (actually 5 but one is not reproducible), and one PR that I want to look at and resolve before releasing 4.7.
I currently don't have much time to work on picocli, so I cannot give any date estimates.
Hopefully this calendar year! 😅

@nineinchnick and the Trino CLI looks great! Nice work!


react-table sorting is not working for negative deciamal values in table

A better description and an example on Code Sandbox would be nice.
I had a similar issue, maybe it is the same problem:
https://github.com/tannerlinsley/react-table/issues/1967

Change the columns sortType to basic


first message text box in threads is not styled by the theme

Describe the bug
A clear and concise description of what the bug is.
The first message text box that shows when you create a new thread is not styled by this theme. However, after you send that first message, the normal thread text box is styled.
Screenshots
If applicable, add screenshots to help explain your problem.

Yeah i think that's a problem and i wanted to post that right now but i saw ur post rn lmao. Maybe it will be fixed


Investigate UX when Deferred and/or Sensitive wrapping of conjur::secret function is not done

We don't have a good idea what the UX looks like when the user forgets to wrap our conjur::secret function with Sensitive(Deferred()) block. This issue is to ensure we have acceptable UX for it.
AC:

[x] Investigate (and possibly open an issue) for UX when Sensitive() is forgotten around conjur::secret
[x] Investigate (and possibly open an issue) for UX when Deferred() is forgotten around conjur::secret
[x] Investigate (and possibly open an issue) for UX when Sensitive(Deferred()) are forgotten around conjur::secret

Original issue: #114

Results
Expected usage (Sensitive(Deferred(conjur::secret, ['...'])))
Notice: Writing secret Sensitive [value redacted] to /tmp/test.pem...
Notice: /Stage[main]/Main/Node[default]/Notify[Writing secret Sensitive [value redacted] to /tmp/test.pem...]/message: defined 'message' as 'Writing secret Sensitive [value redacted] to /tmp/test.pem...'
Notice: Writing secret to /tmp/test.pem...
Notice: /Stage[main]/Main/Node[default]/Notify[Writing secret to /tmp/test.pem...]/message: defined 'message' as 'Writing secret to /tmp/test.pem...'

Sensitive() is forgotten (Deferred(conjur::secret, ['...']))
Notice: /Stage[main]/Main/Node[default]/Notify[Grabbing 'inventory/db-password' secret...]/message: defined 'message' as 'Grabbing \'inventory/db-password\' secret...'
Notice: Writing secret Deferred({'name' => 'conjur::secret', 'arguments' => ['inventory/db-password']}) to /tmp/test.pem...
Notice: /Stage[main]/Main/Node[default]/Notify[Writing secret Deferred({'name' => 'conjur::secret', 'arguments' => ['inventory/db-password']}) to /tmp/test.pem...]/message: defined 'message' as 'Writing secret Deferred({\'name\' => \'conjur::secret\', \'arguments\' => [\'inven
tory/db-password\']}) to /tmp/test.pem...'
Notice: Writing secret to /tmp/test.pem...
Notice: /Stage[main]/Main/Node[default]/Notify[Writing secret to /tmp/test.pem...]/message: defined 'message' as 'Writing secret to /tmp/test.pem...'

Deferred() is forgotten (Sensitive(conjur::secret('...')))
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Internal Server Error: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- conjur/puppet_module/config
Error: Could not retrieve catalog; skipping run

Sensitive(Deferred()) are forgotten (conjur::secret('...'))
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Internal Server Error: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- conjur/puppet_module/config
Error: Could not retrieve catalog; skipping run

This probably warrants a troubleshooting section. CC @izgeri

@sgnn7 it looks like nothing really errors on missing Sensitive - is that right?
Is there anything we can add to make it fail more gracefully on missing Deferred?
We can add a Troubleshooting section to the README.

it looks like nothing really errors on missing Sensitive - is that right?

No errors but value of params is not redacted in logs

Is there anything we can add to make it fail more gracefully on missing Deferred?

Unknown - we may be able to figure out it's a server-side request through some hackery and print an error but I'm not sure it's trivial

We can add a Troubleshooting section to the README.

This is definitely the approach I think we should take for the first handling of these changes.

Stories are now split. The documentation needed for Sensitive is now in story https://github.com/cyberark/conjur-puppet/issues/198. Closing.


NIFI-13294 Deprecate Apache Knox SSO Authentication

Summary
NIFI-13294 Deprecates Apache Knox Single Sign-On authentication for subsequent removal in NiFi 2.
OpenID Connect and SAML 2 integration provide standards-based SSO without reference to particular identity providers.
This deprecation does not impact Apache Knox proxy gateway access to NiFi, which continues to be supported using X.509 client certificates and the X-ProxiedEntitiesChain HTTP header.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 21

UI Contributions

[ ] NiFi is modernizing its UI. Any contributions that update the current UI also need to be implemented in the new UI.

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

+1 merged to support/nifi-1.x