docs: 📚 add README.md

Description
Update README.md adding usage instructions and a link to the source
documentation.
Type of change

[x] Docs

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 76.92%. Comparing base (8f6e09a) to head (9ee98f6).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main       #4   +/-   ##
=======================================
  Coverage   76.92%   76.92%
=======================================
  Files          12       12
  Lines         520      520
  Branches      109      105    -4
=======================================
  Hits          400      400
+ Misses        115      114    -1
- Partials        5        6    +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Upgrade JDK?

We're still on JDK11.
Should we jump all the way to JDK 17 or JDK 21? These are the LTS choices:
https://www.oracle.com/ca-en/java/technologies/java-se-support-roadmap.html

My plan is to bump up to JDK 21. This will be my next big project.

See #2410

+1, we would also get some benefits on the Lucene side (e.g., vector comparison speedups via Panama Vector API).
perhaps this might require some version bump?

Done!


[Bug]: Disconnect happening randomly and exiting application, WebSocket in aborted state

Check The Docs

[X] I double checked the docs and couldn't find any useful information.

Verify Issue Source

[X] I verified the issue was caused by Discord.Net.

Check your intents

[X] I double checked that I have the required intents.

Description
My application seems to exit completely following a NULL logged as warning and a disconnection from Discord.NET. (see the stacktrace)
I'm not sure how it happens, since it happens very randomly (2 to 10 times per 24 hours with my production workload).
This makes me believe it has something to do with Discord and the users doing stuff with my bot.
I am running sharded discord client with 8 shards per process. Running 6 processes for a total of 48 shards at the moment.
However I had the same happening with 1 process per shard using the regular discord client.
They are running on a k8s cluster with .NET 6.
Some things I have done to try and solve this with no success:

I have made sure all events are using a Task.Run(...) so they do not block the gateway.
Used both Discord.NET 3.0 and labs 3.4.x / 3.5 in production.
Added try-catch blocks to just about everything Discord.NET related

I hope someone is able to help me trace the problem and work around it.
Version
Currently 3.0.0 and also had this problem in 3.5.0-labs
Working Version
Not exactly sure when it started
Logs
I have been logging first chance exceptions as well to hopefully find the issue, although these might be unrelated.
Perhaps the The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived' is where things go wrong, but I know too little of Discord.NET and the API to say anything about this.

Log Excerpt
[16:04:12 WRN] [null]
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()
[16:04:12 INF] Disconnecting
[16:04:12 INF] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
[16:04:12 INF] First chance exception occurred at application level: The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
System.Net.WebSockets.WebSocketException (0x80004005): The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
   at System.Net.WebSockets.WebSocketValidate.ThrowIfInvalidState(WebSocketState currentState, Boolean isDisposed, WebSocketState[] validStates)
[16:04:12 INF] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
System.Net.WebSockets.WebSocketException (0x80004005): The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
   at System.Net.WebSockets.WebSocketValidate.ThrowIfInvalidState(WebSocketState currentState, Boolean isDisposed, WebSocketState[] validStates)
   at System.Net.WebSockets.ManagedWebSocket.CloseOutputAsyncCore(WebSocketCloseStatus closeStatus, String statusDescription, CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
[16:04:12 INF] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)[16:04:12 INF] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TResult](Memory`1 payloadBuffer, CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:12 INF] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)   at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TResult](Memory`1 payloadBuffer, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[16:04:13 INF] Disconnected
[16:04:13 INF] First chance exception occurred at application level: Operation canceled
System.Net.Sockets.SocketException (125): Operation canceled
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Net.Sockets.SocketReceiveFromResult>.GetResult(Int16 token)
[16:04:13 INF] First chance exception occurred at application level: Operation canceled
System.Net.Sockets.SocketException (125): Operation canceled
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Net.Sockets.SocketReceiveFromResult>.GetResult(Int16 token)
   at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

Sample
I have not been able to reproduce the issue.

What docker base are you running the bot in?

cc @lhjt

They are running on a k8s cluster with .NET 6.

Are you running your services behind any sort of service mesh, such as Istio?
Have you experienced any other connectivity issues with deployments on your cluster?

I am running sharded discord client with 8 shards per process.

Is each process running in its own pod?

If so, do all of the processes encounter this issue at the same time?

When you experience this error, do all of the shards on the same process also encounter this error or is it just the one shard?

What docker base are you running the bot in?

mcr.microsoft.com/dotnet/aspnet:6.0.0-alpine3.14

Are you running your services behind any sort of service mesh, such as Istio?

Nope. I have no sidecars at all in the pod.

Have you experienced any other connectivity issues with deployments on your cluster?

None that I have witnessed.

Is each process running in its own pod?

Yes. Within that process is the sharded client with 8 shards at the moment.
However, I have had the same issue using non-sharded client with a single shard in a process per pod.

If so, do all of the processes encounter this issue at the same time?

They do not.
For example, the following image is from after 28h of running them.

The number of restarts is differs for each pod, but they all seem to have been killed with a similar disconnection log as the one I posted.

When you experience this error, do all of the shards on the same process also encounter this error or is it just the one shard?

I assume it's just one shard, since I just see a single Disconnecting log.
How can I find out from which shard a log message originates? I am currently just hooking in on the Log event and passing it through to the ILogger and its corresponding severity, but I see no shard or anything in the log message.

I think it may have something to do with the voice channels, since at the same time a FFmpeg process streaming to the AudioOutStream also seems to die with: Error writing trailer of pipe:: Broken pipe.

What are your process configurations for starting FFMPEG, in my experience with ffmpeg I get that error of a broken pipe but my bot doesn't close

This is quite a peculiar problem, but based on this information it doesn't seem like it would be related to your deployment environment.
Do you happen to have any other action logs produced by your bot just before the disconnect and program exit happens? If not, would you be able to change the logging level to debug and continue running your workload until you run into this issue again, and then attach the logs before the exception happens?

What are your process configurations for starting FFMPEG, in my experience with ffmpeg I get that error of a broken pipe but my bot doesn't close

I suppose the broken pipe could just be an after-affect of the socket being aborted.
My ffmpeg process comes down to this:
using var ffmpeg = Process.Start(new ProcessStartInfo
{
    FileName = "ffmpeg",
    Arguments = $"-hide_banner -i {inputFilePath} -ac 2 -f s16le -ar 48000 -",
    UseShellExecute = false,
    RedirectStandardInput = false,
    RedirectStandardOutput = true
});

await using var output = ffmpeg.StandardOutput.BaseStream;

try
{
    await output.CopyToAsync(audioStream, 3840, cancellationToken);
}
finally
{
    await audioStream.FlushAsync(cancellationToken);
}

If this was related I would expect this problem to happen much more often,
since this process runs many more times per day than the issue occurs.

This is quite a peculiar problem, but based on this information it doesn't seem like it would be related to your deployment environment.

Indeed, I'm really at a loss as to what it can be.

Do you happen to have any other action logs produced by your bot just before the disconnect and program exit happens? If not, would you be able to change the logging level to debug and continue running your workload until you run into this issue again, and then attach the logs before the exception happens? This may help us in producing an example where this issue can be reliably reproduced.

At the moment sadly I do not. I already added the first chance exception logging for more information, but it did not help much.
I will see if I can add even more logging and enable the debug logs and I will reply here again once I get some more data.
It may take a few days though.

I found this other similar issue: https://github.com/Discord-Net-Labs/Discord.Net-Labs/issues/203
Could it have something to do with this?

Can you test if dragging your bot to a different vc crashes the pod?

Can you test if dragging your bot to a different vc crashes the pod?

Yeah, I tried that, but it does not make it crash.
I figured maybe it could be something similar, but perhaps an unhandled exception at the Discord.NET end?
That would probably make the app exit.

@lhjt I was able to get some debug logs already.
This is from the last second before one of the pods crashed:
[09:50:13 DBG] POST channels/xxx/messages: 304.38 ms
[09:50:13 DBG] Received Dispatch (MESSAGE_REACTION_ADD)
[09:50:13 DBG] Received Dispatch (MESSAGE_CREATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_CREATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_REACTION_ADD)
[09:50:13 DBG] Received Dispatch (MESSAGE_UPDATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_CREATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_UPDATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_REACTION_ADD)
[09:50:13 DBG] Received Dispatch (MESSAGE_UPDATE)
[09:50:13 DBG] Received Dispatch (GUILD_MEMBER_UPDATE)
[09:50:13 DBG] Malformed Frame
[09:50:13 DBG] Received Dispatch (MESSAGE_CREATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_CREATE)
[09:50:13 DBG] Received Dispatch (MESSAGE_CREATE)
[09:50:13 DBG] Received Dispatch (CHANNEL_UPDATE)
[09:50:13 INF] First chance exception occurred at application level: The operation was canceled.

System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[09:50:13 INF] First chance exception occurred at application level: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[09:50:13 INF] First chance exception occurred at application level: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[09:50:13 DBG] Keepalive Stopped
[09:50:13 DBG] Heartbeat Stopped
[09:50:13 WRN] [null]
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()
[09:50:13 INF] Disconnecting
[09:50:13 DBG] Disconnecting ApiClient
[09:50:13 INF] First chance exception occurred at application level: The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
System.Net.WebSockets.WebSocketException (0x80004005): The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
   at System.Net.WebSockets.WebSocketValidate.ThrowIfInvalidState(WebSocketState currentState, Boolean isDisposed, WebSocketState[] validStates)
[09:50:13 INF] First chance exception occurred at application level: Operation canceled
System.Net.Sockets.SocketException (125): Operation canceled
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Net.Sockets.SocketReceiveFromResult>.GetResult(Int16 token)
[09:50:13 INF] First chance exception occurred at application level: The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
System.Net.WebSockets.WebSocketException (0x80004005): The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
   at System.Net.WebSockets.WebSocketValidate.ThrowIfInvalidState(WebSocketState currentState, Boolean isDisposed, WebSocketState[] validStates)
   at System.Net.WebSockets.ManagedWebSocket.CloseOutputAsyncCore(WebSocketCloseStatus closeStatus, String statusDescription, CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[09:50:13 INF] First chance exception occurred at application level: Operation canceled
System.Net.Sockets.SocketException (125): Operation canceled
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Net.Sockets.SocketReceiveFromResult>.GetResult(Int16 token)
   at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[09:50:13 DBG] Sent VoiceStateUpdate
av_interleaved_write_frame(): Broken pipe  64.5kbits/s speed=33.1x
Error writing trailer of pipe:: Broken pipe
size=    1533kB time=00:03:05.13 bitrate=  67.8kbits/s speed=  21x
video:0kB audio:1537kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

I did remove some unrelated logs, otherwise it would be too massive.
Hope this helps. Thanks so much!

The only Discord.Net trace I see here is the Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext(). The connection manager is used for audio sockets and the gateway socket so I'm unable to tell if this is an audio related issue or gateway issue.
Would you be able to directly add the project source to your bot? This should give you a full stack trace including the line numbers for dnet, this will help me figure out the root cause.

Would you be able to directly add the project source to your bot? This should give you a full stack trace including the line numbers for dnet, this will help me figure out the root cause.

Sure, here you go:

Log with discord-net 3.1.0 source added

[08:23:05 DBG] Received HeartbeatAck
[08:23:05 DBG] Latency = 200 ms
[08:23:05 DBG] Received Dispatch (MESSAGE_REACTION_ADD)
[08:23:05 DBG] Received Dispatch (GUILD_MEMBER_UPDATE)
[08:23:05 DBG] Malformed Frame
[08:23:05 DBG] Received Dispatch (MESSAGE_CREATE)
[08:23:05 DBG] Received Dispatch (MESSAGE_UPDATE)
[08:23:05 DBG] First chance exception occurred at application level: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[08:23:05 DBG] First chance exception occurred at application level: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[08:23:05 DBG] Heartbeat Stopped
[08:23:05 DBG] First chance exception occurred at application level: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[08:23:05 DBG] Keepalive Stopped
[08:23:05 WRN] [null]
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext() in /src/discord-net/Discord.Net.WebSocket/ConnectionManager.cs:line 79
[08:23:05 INF] Disconnecting
[08:23:05 DBG] Disconnecting ApiClient
[08:23:05 DBG] First chance exception occurred at application level: The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
System.Net.WebSockets.WebSocketException (0x80004005): The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
   at System.Net.WebSockets.WebSocketValidate.ThrowIfInvalidState(WebSocketState currentState, Boolean isDisposed, WebSocketState[] validStates)
[08:23:05 DBG] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
[08:23:05 DBG] First chance exception occurred at application level: The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
System.Net.WebSockets.WebSocketException (0x80004005): The WebSocket is in an invalid state ('Aborted') for this operation. Valid states are: 'Open, CloseReceived'
   at System.Net.WebSockets.WebSocketValidate.ThrowIfInvalidState(WebSocketState currentState, Boolean isDisposed, WebSocketState[] validStates)
   at System.Net.WebSockets.ManagedWebSocket.CloseOutputAsyncCore(WebSocketCloseStatus closeStatus, String statusDescription, CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[08:23:05 DBG] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[08:23:05 DBG] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
[08:23:05 DBG] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[08:23:05 DBG] First chance exception occurred at application level: Unable to read data from the transport connection: Operation canceled.
System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[08:23:05 DBG] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
[08:23:05 DBG] First chance exception occurred at application level: The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
[08:23:05 DBG] Sent VoiceStateUpdate
av_interleaved_write_frame(): Broken pipe1536.9kbits/s speed=1.11x
Error writing trailer of pipe:: Broken pipe
size=    6325kB time=00:00:33.73 bitrate=1536.0kbits/s speed=1.02x
video:0kB audio:6329kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

Does this error occur running the same application outside of docker? Are you able to run one process with 8 shards outside of docker for a day and see if you get the same logs? I've heard of people having issues with k8s/docker and websockets so it might be something there.
Looking at the one stack we get from the logs that exception is handled properly. One piece of evidence to back this up is the continuation of code executing by the other logs below it.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext() in /src/discord-net/Discord.Net.WebSocket/ConnectionManager.cs:line 79
[08:23:05 INF] Disconnecting // here
[08:23:05 DBG] Disconnecting ApiClient // here

Maybe it has to do with the [08:23:05 DBG] Sent VoiceStateUpdate update message. Maybe its trying to send the voice state even though the socket is closed? I'll have a look into this portion of the library.

Does this error occur running the same application outside of docker? Are you able to run one process with 8 shards outside of docker for a day and see if you get the same logs? I've heard of people having issues with k8s/docker and websockets so it might be something there.

I'm afraid it's not possible for me to run it elsewhere, since there is shared infrastructure within the cluster (such as RabbitMQ and Redis) and nothing is reachable from the outside.

I have been checking Discord.NET code a bit, and been trying to trace what is happening...
So it seems it is Disconnecting but before the OnDisconnected event fires it dies.
I have a log on the Disconnected event so I would expect it to show up in the log, but it does not.
private async Task DisconnectAsync(Exception ex, bool isReconnecting)
{
    if (State == ConnectionState.Disconnected) return;
    State = ConnectionState.Disconnecting;
    await _logger.InfoAsync("Disconnecting").ConfigureAwait(false);

    await _onDisconnecting(ex).ConfigureAwait(false);

    await _disconnectedEvent.InvokeAsync(ex, isReconnecting).ConfigureAwait(false);  // it never gets here ?
    State = ConnectionState.Disconnected;
    await _logger.InfoAsync("Disconnected").ConfigureAwait(false);
}

So I am assuming an exception is thrown within the DiscordSocketClient.OnDisconnectingAsync(...) somewhere.
I will throw some try blocks in the Discord.NET code that I added to my project to see if I get some more output.

Feel free to experiment but I think it might be related to this pr. I'll make a change to how it works and get you to try it out.

I think I'm already seeing some results. Caught this exception, which was otherwise uncaught:
System.ObjectDisposedException: The semaphore has been disposed.
   at System.Threading.SemaphoreSlim.Release(Int32 releaseCount)
   at Discord.Audio.DiscordVoiceAPIClient.DisconnectAsync() in /src/discord-net/Discord.Net.WebSocket/DiscordVoiceApiClient.cs:line 208
   at Discord.Audio.AudioClient.OnDisconnectingAsync(Exception ex) in /src/discord-net/Discord.Net.WebSocket/Audio/AudioClient.cs:line 127
   at Discord.ConnectionManager.DisconnectAsync(Exception ex, Boolean isReconnecting) in /src/discord-net/Discord.Net.WebSocket/ConnectionManager.cs:line 189
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext() in /src/discord-net/Discord.Net.WebSocket/ConnectionManager.cs:line 94

Does this give you any ideas?

Does catching that exception stop the pod from restarting?

Does catching that exception stop the pod from restarting?

Well, I have only had it running for 37m so far, but no restarts yet at all so it seems like it.
I will let it run for a whole day at least to be 100% sure.

Does using .NET 5 change anything?

Does using .NET 5 change anything?

That's not something I can downgrade to quickly and easily, so it will be a last resort test I can do.
However if you're talking mainly about the BackgroundServiceExceptionBehavior that has changed in .NET 6,
I have that set to Ignore, so it should be the same as .NET 5 in my app.
For now I will continue to find places with unhandled exceptions in discord-net code and add try-catches and logging where I can. The code is rather hard to follow, so I hope I can keep my sanity.

So far I have traced it to a Segmentation fault (exit code 139).
I am assuming it is caused by either ffmpeg, libopus, or maybe libsodium?
More likely libopus/libsodium since discord-net is using it directly, it would make sense that the app would crash.
Since ffmpeg is run as an external process it should not crash the .NET process as far as I know.
Things I have done that did not help:

Upgraded my k8s cluster to the latest version
Used a different docker base image mcr.microsoft.com/dotnet/aspnet:6.0.1-bullseye-slim
Made sure libopus is latest version 1.3.1
Made sure libsodium is latest version 1.0.18

FFmpeg version seems recent enough too:
ffmpeg version 4.3.3-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10 (Debian 10.2.1-6)
configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 51.100 / 56. 51.100
libavcodec     58. 91.100 / 58. 91.100
libavformat    58. 45.100 / 58. 45.100
libavdevice    58. 10.100 / 58. 10.100
libavfilter     7. 85.100 /  7. 85.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  7.100 /  5.  7.100
libswresample   3.  7.100 /  3.  7.100
libpostproc    55.  7.100 / 55.  7.100

I don't know anything about how this all works in discord-net,
so it would be great if someone could give their insights in what the problem could be.

I have chosen to circumvent the issue by using Lavalink instead.
My bot has now been running for many hours without any restarts, so it is safe to say the issue is gone.
It might be good to warn people that the Discord.NET audio client can kill your app with segmentation faults.
Perhaps it's only when using it in Linux containers, but it's hard to say for sure.


Download of pccommon.sh should use raw URL

Download of pccommon.sh should use raw URL.
(cherry picked from commit db701371f7d974a7cea8146cc3220fcff3060854)

:+1:

:+1:


Wrong docs of behavior test for setting up .env

Docs of behavior test (and also docs of fuzzer) gives an example to set up .env with OPENDAL_FS_ROOT=/tmp, but tests would fail if .env was set up according to it with error as follows:
thread 'main' panicked at src/raw/tests/utils.rs:70:45:
must succeed: Unexpected (permanent) at Builder::build => create root dir failed

Context:
   root: /tmp162b96c0-ac99-44ac-89cf-839db11dce66/

Source:
   Permission denied (os error 13)

This is due to that the given root and random uuid were glued together without a slash.
Maybe we should change the docs to /tmp/ or to utilize Path::join to generate random roots.

Maybe we should change the docs to /tmp/

Let's fix the docs. Thanks for the catch!


feat(Algebra/Module/CharacterModule): definition of character module

For an R-module M, its character module is defined as the group homorphisms from M to the unit rational circle where the R-module structure is given by rf(m) = f(rm). Let us denote the character module by M*
Then each linear map f : A -> B gives a linear map f* from B* to A* by composition.
When f is injective, f* is surjective.
There is a natural isomorphism between Hom(A, B*) and (A ⊗ B)*

[x] depends on: #8905
[x] depends on: #10766 (for defining the module structure)

Thanks!
bors r+


Error (Xcode): : The method 'withValues' isn't defined for the class 'Color'.

Steps to reproduce
upgrade to version 3.27.0
Card(
 elevation: 8,
 shadowColor: AppColors.blackBackGroundColor.withValues(alpha:0.4),
 color: AppColors.whiteBackGroundColor,
 shape: RoundedRectangleBorder(
   borderRadius: BorderRadius.circular(Dimens.padding8),
 ),)

while running flutter build ios --target lib/flavor/main_prod.dart , getting above error.
Expected results
it should run without error as this change was showing in waring.
Actual results
Error (Xcode): lib/app/ui/find_class/tablet_find_class_page.dart:223:62: Error: The method 'withValues' isn't defined for the class 'Color'.
Encountered error while building for device
Code sample
Code sample
Card(
  elevation: 8,
  shadowColor: AppColors.blackBackGroundColor.withValues(alpha:0.4),
  color: AppColors.whiteBackGroundColor,
  shape: RoundedRectangleBorder(
    borderRadius: BorderRadius.circular(Dimens.padding8),
  ),)

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
Error (Xcode): lib/app/ui/find_class/tablet_find_class_page.dart:223:62: Error: The method 'withValues' isn't defined for the class 'Color'.

Encountered error while building for device```

</details>

### Flutter Doctor output

<details open><summary>Doctor output</summary>

```console
[✓] Flutter (Channel stable, 3.27.0, on macOS 15.1 24B83 darwin-arm64, locale en-IN)
[✓] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2024.2)

Hi @richanshah, I tried to reproduce the issue on my end using the below code sample, I am unable to reproduce the issue. The app builds fine on both ios and android.
Can you try running

flutter clean
flutter pub get
and share the output of flutter run -v

code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      home: Scaffold(
        backgroundColor: Colors.white,
        body: Center(
            child: Card(
          elevation: 8,
          shadowColor: Colors.red.withValues(alpha: 0.4),
          color: Colors.white.withValues(alpha: 0.8),
          shape: RoundedRectangleBorder(
            borderRadius: BorderRadius.circular(10),
          ),
          child: const Text(
            'Hello World',
            style: TextStyle(fontSize: 30),
          ),
        )),
      ),
    );
  }
}

flutter doctor -v
[✓] Flutter (Channel stable, 3.27.0, on macOS 15.1.1 24B2091 darwin-arm64, locale en-US)
    • Flutter version 3.27.0 on channel stable at /Users/mahesh/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 8495dee1fd (2 days ago), 2024-12-10 14:23:39 -0800
    • Engine revision 83bacfc525
    • Dart version 3.6.0
    • DevTools version 2.40.2


Invalid utf-8 strings are invalidly parsed as valid json

See attached file. It should not be parsed correctly.
invalid_json.json.gz

Yes, the parser does not perform utf8 validation. I'm not sure if it should. The place to do it is here:
https://github.com/vinniefalco/json/blob/develop/include/boost/json/impl/basic_parser.ipp#L497

The JSON Data Interchange Syntax says (emphasis mine):

A string is a sequence of Unicode code points wrapped with quotation marks (U+0022).

So I think it should be validated since invalid UTF-8 is just a sequence of UTF-8 code units, but not also a sequence of Unicode code points.

To do this we need to

modify detail::count_unescaped to stop when it encounters a character with the high-bit set.

check if (c & 0x80) != 0 here: https://github.com/CPPAlliance/json/blob/fcdf8856512b51109ec4fe6792a40302bb30768c/include/boost/json/basic_parser.hpp#L768

then, if we have at least 3 more characters, implement "fast" utf-8 validation (all bytes present), otherwise do a goto into a new state where we handle one character at a time and check for the end of the stream.

This looks doable, but it will add an additional branch. We could put it after all the other checks for escapes, and then it would be zero cost. However, users will then complain that they want the parser to accept invalid UTF-8....

However, users will then complain that they want the parser to accept invalid UTF-8....

Similar to accepting comments in JSON.

Similar to accepting comments in JSON.

Yes and then if we parse and ignore comments, users will 1. want a way to turn it off, and 2. ask to have the comments preserved when serializing.

Probably it would be better to add a precondition that input string should be valid utf-8 (and same for json::string).

Well, we can expect all we want but if the JSON comes from an untrusted network source that won't do much good. As for json::string, yes the caller is responsible for putting only valid UTF-8 in the string.

Yep, in this case parse should accept something like valid_utf8_string_view instead :smile: (or just an overload). And well, that overload can be added in future when there will be a standard way to pre-validate utf-8 strings.

Well I think in this case we can afford to always do the validation, and provide a run-time switch to turn it off, without affecting performance.

Implemented in d98b565887834bca73fde2cc8f0216de6c7992f8. Validation is enabled by default, but can be disabled by passing parse_options to parser::parser.


Cameratrap DP

It will be interesting to compare  Cameratrap DP with AC and see what attributes are not covered and do we need tool at adding some of those.

Pinging @peterdesmet I would love to see coordination between Audubon Core and the Machine Observations Interest Group on this topic.

@baskaufs definitely up for that. Maybe good to schedule a call?

I'm interested in this as well!

I've tagged this as "help wanted". We should change the tag to "Meeting" once one is actually scheduled.

I'm closing this since we are having the meeting on this topic in a few minutes. We can open more specific issues around this topic as they arise.


feat: Add Storybook info to Architecture doc and add initial support

This commit adds information about Storybook to the Architecture.md
doc and also adds initial project support for Storybook.

Contributes to: #14
Signed-off-by: Andrew Borley<EMAIL_ADDRESS>

We have a fair ammount of customisation in our build. I know Storybook has a built in/uses its own webpack config for building the storybook - but I wonder if we need to provide one so our customisations (like aliases) are available. We should be able to have a webpack.storybook.config (akin to our dev and prod build config) for this. What do you think?

The latest commit updates the .storybook/main.js file to pick up the dev webpack config and merge it into the storybook config 👍

I like the set of add ons mentioned. Storybook 6 brought in a bunch of changes from 5 and older, like args: https://medium.com/storybookjs/introducing-storybook-args-2dadcdb777cc , which reduces the ammount of boilerplate involved. Is there anything there we could take advantage of?

Yep, looks good. When we have components and stories we should use it.


Issue #211: added flag FLAG_INCLUDE_STOPPED_PACKAGES

This adds a new directory and all the gcm.jar files. Added the flag in src/android/com/google/android/gcm/GCMBroadcastReceiver.java

@vikasing I can't merge this. The plugin doesn't even use the classes in gcm.jar that this PR over-rides.


Large XLSX-Files Wrap-up

There are numerous issues on the problem of loading large XLSX's, with partly contradicting details.
My view of the problem:

Some large XLSX files cannot be read.
It's not the number of rows for me adding a column lead to the error
In my case it's the call to XLSX.readFile(filename, options); that just does not return then
Memory limitation is not the reason for the problem: increasing the max_heap_size with --max-old-space-size=10096 does not help

Open questions:

Could it help to switch versions? I am using node 17.4 (and electron 13), and sheetjs 0.17.0
How can I evaluate if the pro packages offered by @SheetJSDev can handle my files? I cannot upload these file to some website, for privacy reasons.
The xlsx cli tool is able to open my problematic sheet, what is the difference between the tool shipped and my program using the same lib???

Most of the open issues involve the hardcoded V8 string length limit (a limit that is unaffected by max-old-space-size and other tuning parameters) and will also hit limits in NodeJS and the CLI tool.
The CLI tool sets the option dense: true in the XLSX.readFile call:
var dense_wb = XLSX.readFile("test.xlsx", { dense: true });

It is an (currently) undocumented option and will change the worksheet type.  As long as you stick to the API functions and avoid directly addressing cells like ws["A3"], then you should be able to set the same option.  The actual structure of dense workbooks will change at some point, so it's best to stick to functions like sheet_add_aoa to write data into the worksheet and sheet_to_json to extract data from the worksheets.
.
More details below:
The current worksheet structure is a large object where each key is an address.  Various V8 changes over the years degraded performance of this access pattern (performance has actually regressed).  We've reported bugs to NodeJS and to Chromium, including https://bugs.chromium.org/p/v8/issues/detail?id=6696
dense changes the structure to an array of arrays of cell objects.
.
Array of Arrays should have been the obvious choice.  There were two early decisions that led to the current situation: "sparse worksheets" and "web workers".
"sparse worksheets": Imagine a worksheet that only sets A1 and XFD1048576.  In the sparse structure, this creates an object with a "A1" key and a "XFD1048576" key.  In the dense structure, to ensure the bottom cell could be stored, the worksheet is an array of length 1048576 and the last element is an array of length 16384.  This just crashed IE in our testing :(
"web workers": Back in 2012 (It's been nearly 10 years since the first commit!) Web Workers and File API were relatively new.  We had explored the idea of passing the binary data (be it a binary string or a Uint8Array) to the worker and passing back the worksheet object.  We encountered issues trying to pass back the arrays in IE10 and ultimately concluded that it was safer to send simple objects in Worker messages.
.
IE is long gone, so why not flip the default?  There are a few reasons:
"relevance": Photoshop and InDesign CC 2022 suffer the same problem.
"fallbacks": We looked into a migration using Proxy (underlying dense worksheet with a proxy that lets you access cells by address) but proxies are still incredibly inefficient in 2022.
"change": Changing defaults is hard, especially given the size of the developer base.  We've recently revisited all of the demos (https://docs.sheetjs.com/docs/demos/) and quite a few of them didn't work in recent testing because some project decided to abandon CJS or changed APIs in ways that required rewriting the demo from scratch.

Thanks a lot for this verbose response. I can confirm that the dense parameter did solve my problem. I have not tested what my new limit is, but for the time being I am fine.
Actually my list of contracts is indeed dense, so this representation is ideal. The mentioned sheet with only two far distant non-empty cells is an edge case. One might think about setting dense to true by default.
Thanks!


[Feature Request]: Allow users to change the generated resource name for workbenches and other components

Feature description
When creating any of the components in a DS Projects, users don't have the option to change the automatically generated resource name. While, it is availble when creating a DS Project.
In order to keep consistency, we could add the same option to the rest of the components (where technically feasible). This was raised in an internal slack thread.
Describe alternatives you've considered
remove the option from DS Project creation form (maybe not ideal)
Anything else?

UX to be onboarded
originally raised internally by @xianli123 :)

@bdattoma Looks good. I will come up with a design for this feature.

Hi @bdattoma I created a design for this issue. Here is the design link for your reference. To sum up the design:

Added a new field named Resource name in the workbench creation page.
Added 2 help texts to make users better understand the naming rules.

Please take a look and feel free to let me know if there are any concerns. Thanks!

Hi @bdattoma I found that you added the other components in the title. Could you please elaborate on what components will show the resource name during the creation process? I can provide some designs for these parts if needed.

@xianli123 We should look at a way to be able to hide the "resource name" section universally as well. Some ideas to make it less prominent: use an expandable section, either controlled by the expandable caret, or a button next to or near the Name field

Hi @kywalker-rh An expandable section might be better than an icon next to the Name field. I used a clickable expandable link to show the resource name as shown in the screenshot below.

In addition, I put a question mark next to the link. The tooltip goes that: Resource names are what your resources are labeled in Openshift and not editable after creation. It reuses Name by default and automatically converts Name to a valid resource name.  What do you think?

I think this looks good to me!
@andrewballantyne you had some thoughts around not wanting to use the caret, but after seeing this I'm not sure I understood your reasoning for it, can you elaborate here?
@xianli123 Two minor points, first, I think the "Resource name" label and field should be indented to make it clear there is a hierarchy, and second, we should get @kaedward to look at the text for the tooltip.

My point was I saw no value in wasting the vertical space on something only "power users" (for lack of a better term) will want to use. Having it as an inline "Display Name Action" allowed it to be more "extra" and less in your face.
This has to be singularly applied to all modals, pages, and flows that support Display Name changes with a backing K8s name. On the Workbench page, you have the space, on a Create Data Connection or Create Cluster Storage modal, it really didn't need to take up valuable vertical real estate... but that is my two cents -- PF may disagree with me too, so don't take me on my opinion. I just didn't want it to be always "there" taking up space.

@kywalker-rh @xianli123 Looks like we already have a tooltip for this purpose:
"Resource names are what your resources are labeled in OpenShift.
Resource names are not editable after creation."
Although I like the idea of combining these two statements into one sentence - how about this?
"The resource name is used to identify your resource in OpenShift, and is not editable after creation."

Thanks! @kywalker-rh @kaedward.

Looks like we already have a tooltip for this purpose
Yes. There is a tooltip in the current UI. Combining them makes more sense. I will update the design based on your suggestions!

@andrewballantyne @kywalker-rh If we don't want it to always take up space, we could get rid of the expandable link. I checked the PF and found a component named Form group with additional label info. We might change the additional label to a checkbox toggle as shown below.

After marking the checkbox, the Resource name will appear. But I am not sure if PF can support this kind of change. I conveyed this idea to the PF team and am waiting for their reply. What do you think of this idea?

@xianli123 (cc @kywalker-rh) The top right hand corner is not supported by PF as far as I can see in their docs, but to the right of the input field is valid space we can put something.
This is what I had in my head (obv without polish):
Start at:

Clicking on the link does this:

@andrewballantyne I have noticed this component that you proposed. However, I initially thought that the link action should only have an impact on the front field. So I gave it up. If we all agree with using this component, we can use it. What do you think? @kywalker-rh

@xianli123 PF is often times a composable set of items, so usually whatever you see in the docs we can do. Your linked example you used is the design guidelines, not the component itself; but I imagine they'd never recommend a non PF component in their designs.

Yes @andrewballantyne. I understood.
One more thing I want to share is that the PF team thinks we can change the additional label to the checkbox toggle. So the approach I first shared should be possible. A PF engineer helped to build this sandbox to demo it. The video below is a demo of that.
https://github.com/opendatahub-io/odh-dashboard/assets/53931519/ef32e49d-f876-447e-9cf6-df7f01db5008
Using a link button next to the Name field is a good approach to trigger the Resource name. In addition, using a checkbox toggle to show/hide the Resource name is also a feasible way. Which one do you prefer? What are your thoughts about these 2 approaches? @kywalker-rh

@xianli123 In this use-case I prefer the toggle. The one thing I want to be sure is that this can work for other use-cases where there might be more complicated "Advanced" settings). I think it could, but just want to explore what that would look like too, to make sure we're designing something that allows us to be more consistent.

@kywalker-rh If there is a hidden Advanced part, we can use the Field Groups component. We can put the advanced settings into a field group. When the checkbox are marked, the Advanced settings part will appear as a separate section. Does it make sense?

I created a template about how to show/hide the resource name or other components. Please take a look at the design. This template will work for most situations where users show/hide resource names or advanced settings. If there is any special case, I can modify the design to suit that.

This looks great @xianli123! I think this works perfectly for the name and for the advanced settings in the future. I'm completely on board, let's implement this everywhere 😄

@kywalker-rh Thanks for reviewing it. Let's go ahead.

@uidoyen and I talked about implementation of this functionality in with his current PR. I've gone ahead and moved it forward to in progress as he works on his PR.

@andrewballantyne Sounds good. If there are any questions during the implementation process, please feel free to let me know.

Migrated to:  https://issues.redhat.com/browse/RHOAIENG-1161


Updates key-exchange/diffie-hellman-key-exchange.md

Fix typos.
Signed-off-by: Evgeny Bachinin<EMAIL_ADDRESS>

Issue https://github.com/nakov/Practical-Cryptography-for-Developers-Book/issues/38
and pending pull request https://github.com/nakov/Practical-Cryptography-for-Developers-Book/pull/40
are in progress.
Close mine.


no containers and services create in k8s template

Rancher versions:
rancher/server: v1.6.0
rancher/agent: v1.2.2
Infrastructure Stack versions:
healthcheck:v0.3.1
ipsec:  rancher/net:holder
network-services:  network-manager:v0.7.0|metadata:v0.9.1
scheduler:
kubernetes (if applicable):
v1.5.4-rancher1-4
Docker version: (docker version,docker info preferred)
Client:
Version:      1.12.6
API version:  1.24
Go version:   go1.6.4
Git commit:   78d1802
Built:        Tue Jan 10 20:26:30 2017
OS/Arch:      linux/amd64
Server:
Version:      1.12.6
API version:  1.24
Go version:   go1.6.4
Git commit:   78d1802
Built:        Tue Jan 10 20:26:30 2017
OS/Arch:      linux/amd64
Operating system and kernel: (cat /etc/os-release, uname -r preferred)
Distributor ID:	Ubuntu
Description:	Ubuntu 14.04.4 LTS
Release:	14.04
Codename:	trusty
3.13.0-86-generic
Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
ECS
Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
single node rancher
internal DB
Environment Template: (Cattle/Kubernetes/Swarm/Mesos)
Kubernetes
Steps to Reproduce:

add k8s template
add host
add wordpress  catalog from k8s template
launch

Results:
there are no containers and services create

I dont believe there are catalog items for kubernetes within rancher anymore. You should be using helm/tiller to deploy to kubernetes, unless you have your own services defined.

These k8s templates were no longer valid as of v1.3.99 but we didn't appropriately put in maximum rancher version on the templates. They are now hidden in Rancher.


Address field does not pre-fill when passing data into submission object

Describe the bug
Passing an address to the submission object does not pre-fill a address component field.
I'm trying to pre populate an address field by passing an address into the submission object. I've tried passing it as a string and also tried passing the individual fields of a address component as follows:
// as a string
submission={{
  data: {
    address: '123 Address, Chicago, IL 60007'
  },
}}

// also tried the following
submission={{
  data: {
        address: {
          address1: 'Test',
          address2: '',
          city: 'Chicago',
          state: 'IL',
          zip: '60007',
        },
  },
}}

Expected behavior
I would expect to be able to pass this into the submission object and the value be set on the input.
Screenshots
This is my address field that is remaining blank after passing data into the submission object:

If there is anything else I can provide to clarify the issue please let me know!

What provider are you using?
Here's what I have done to reproduce your scenario...

Add Address Component with provider: OpenStreetMap Nominatum
Post submission via API using postman with the following json object

{
    "data": {
        "address": "123 Address, Chicago, IL 60007"
    }
}

Result
Submission view shows up empty, but the data is still available in the submission object itself:

Here's what I did to get it to successfully display:

Add Address Component with provider: OpenStreetMap Nominatum
Create a dummy submission via UI to capture the submission object expected by the provider.
Post submission via API using postman with the following json object

    "data": {
        "address": {
            "place_id": 25871661,
            "licence": "Data © OpenStreetMap contributors, ODbL 1.0. http://osm.org/copyright",
            "osm_type": "way",
            "osm_id": 210815525,
            "lat": "41.72382345",
            "lon": "-87.60908342557869",
            "class": "building",
            "type": "yes",
            "place_rank": 30,
            "importance": 0.00000999999999995449,
            "addresstype": "building",
            "name": "",
            "display_name": "9361, South Saint Lawrence Avenue, Roseland, Chicago, Hyde Park Township, Cook County, Illinois, 60619, United States",
            "address": {
            "house_number": "9361",
            "road": "South Saint Lawrence Avenue",
            "quarter": "Roseland",
            "city": "Chicago",
            "municipality": "Hyde Park Township",
            "county": "Cook County",
            "state": "Illinois",
            "ISO3166-2-lvl4": "US-IL",
            "postcode": "60619",
            "country": "United States",
            "country_code": "us"
            },
            "boundingbox": [
            "41.7237891",
            "41.7238579",
            "-87.6091811",
            "-87.6089858"
            ]
        }
    }
}```

Posting that worked.

Then after some further tinkering I found this worked as well:
```{
    "data": {
        "address": {
            "display_name": "9361, South Saint Lawrence Avenue, Roseland, Chicago, Hyde Park Township, Cook County, Illinois, 60619, United States"
        }
    }
}```

![image](https://github.com/formio/react/assets/110409244/c6f9c3a7-026a-4e98-a62b-2ee5fc135c81)

@lane-formio Appreciate the detailed response! I wasn't aware we could pass in only the display_name, but that worked. Is there documentation somewhere that says what each of these form components (such as Adddress Component, etc) expect as the submission object, it wasn't clear the entire provider object OR only the display_name was required? Our team is new to the formio ecosystem, so apologies if I missed it.
I'm going to go ahead and close this out, thank you!

Our commercial offering does have inline documentation available via the "API Explorer" but the address component is certainly a little unique in that regard as the submission object varies between providers.


Refactor _create_session

Breakout logic to find highest protocol version
@knifewine you closed #486, but I still think this refactoring needed done, and will be useful. Can you review? Thanks

great idea, +1


Error occurred when finalizing GeneratorDataset iterator

Hello,
I cannot train my own model when running the train_val.py script on macOS in PyCharm (conda env).
System information:
tensorflow 2.5.0
Python 3.9
Keras 2.4.3
45/45 [==============================] - ETA: 0s - loss: 2.0532 - accuracy: 0.5686 - truncated_acc: 0.4852 - truncated_loss: 2.3367Traceback (most recent call last):
  File "/Users/ljudmilapetkovic/PycharmProjects/deep-spell-checkr/deep-spell-checkr/train_val.py", line 109, in <module>
    model.fit_generator(train_loader,
  File "/Users/ljudmilapetkovic/.conda/envs/deep-spell-checkr/lib/python3.9/site-packages/keras/engine/training.py", line 1918, in fit_generator
    return self.fit(
  File "/Users/ljudmilapetkovic/.conda/envs/deep-spell-checkr/lib/python3.9/site-packages/keras/engine/training.py", line 1176, in fit
    self._eval_data_handler = data_adapter.get_data_handler(
  File "/Users/ljudmilapetkovic/.conda/envs/deep-spell-checkr/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 1348, in get_data_handler
    return DataHandler(*args, **kwargs)
  File "/Users/ljudmilapetkovic/.conda/envs/deep-spell-checkr/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 1138, in __init__
    self._adapter = adapter_cls(
  File "/Users/ljudmilapetkovic/.conda/envs/deep-spell-checkr/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 793, in __init__
    peek, x = self._peek_and_restore(x)
  File "/Users/ljudmilapetkovic/.conda/envs/deep-spell-checkr/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 850, in _peek_and_restore
    peek = next(x)
  File "/Users/ljudmilapetkovic/PycharmProjects/deep-spell-checkr/deep-spell-checkr/utils.py", line 176, in datagen
    encoder_input, decoder_input = next(inputs)
  File "/Users/ljudmilapetkovic/PycharmProjects/deep-spell-checkr/deep-spell-checkr/utils.py", line 168, in batch
    data_batch[i] = ctable.encode(token, maxlen)
  File "/Users/ljudmilapetkovic/PycharmProjects/deep-spell-checkr/deep-spell-checkr/utils.py", line 44, in encode
    x[i, self.char2index[c]] = 1.0
KeyError: 'k'
2021-06-15 11:46:16.159552: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
	 [[{{node PyFunc}}]]

Process finished with exit code 1

The only thing I modified in the script was this block:
train_books = ['train_Q354_dAlemBERT.txt']
val_books = ['val_PERRROT75.txt']

I have tried different solutions suggested here, but without success.
Do you have any idea why this is happening?

P.S. I also have the Load settings error:
Cannot load settings from file '/Users/ljudmilapetkovic/Library/Application Support/JetBrains/PyCharmCE2021.1/options/actionSummary.xml': content truncated File content will be recreated

Is it related to the original problem above?

for me, this error happened in the middle of training when I had no space left on my system.

I installed anaconda on my 5 TB external hard drive and tried to run the programme from there, but I still receive the same Error occurred when finalizing GeneratorDataset iterator error with the  Key Error: '\\':
57/57 [==============================] - ETA: 0s - loss: 1.9530 - accuracy: 0.5800 - truncated_acc: 0.4953 - truncated_loss: 2.23532021-06-20 16:37:19.874595: W tensorflow/core/framework/op_kernel.cc:1755] Unknown: KeyError: '\\'
Traceback (most recent call last):

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/ops/script_ops.py", line 249, in __call__
    ret = func(*args)

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 645, in wrapper
    return func(*args, **kwargs)

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 961, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 821, in wrapped_generator
    for data in generator_fn():

  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/utils.py", line 176, in datagen
    encoder_input, decoder_input = next(inputs)

  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/utils.py", line 168, in batch
    data_batch[i] = ctable.encode(token, maxlen)

  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/utils.py", line 44, in encode
    x[i, self.char2index[c]] = 1.0

KeyError: '\\'

Traceback (most recent call last):
  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/train_val.py", line 110, in <module>
    model.fit_generator(train_loader,
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/keras/engine/training.py", line 1918, in fit_generator
    return self.fit(
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/keras/engine/training.py", line 1189, in fit
    val_logs = self.evaluate(
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/keras/engine/training.py", line 1464, in evaluate
    tmp_logs = self.test_function(iterator)
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 924, in _call
    results = self._stateful_fn(*args, **kwds)
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3023, in __call__
    return graph_function._call_flat(
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 1960, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 591, in call
    outputs = execute.execute(
  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnknownError:  KeyError: '\\'
Traceback (most recent call last):

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/ops/script_ops.py", line 249, in __call__
    ret = func(*args)

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 645, in wrapper
    return func(*args, **kwargs)

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 961, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "/Volumes/LaCie/.conda/envs/spell/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 821, in wrapped_generator
    for data in generator_fn():

  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/utils.py", line 176, in datagen
    encoder_input, decoder_input = next(inputs)

  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/utils.py", line 168, in batch
    data_batch[i] = ctable.encode(token, maxlen)

  File "/Volumes/LaCie/PycharmProjects/spell/deep-spell-checkr/utils.py", line 44, in encode
    x[i, self.char2index[c]] = 1.0

KeyError: '\\'

	 [[{{node PyFunc}}]]
	 [[IteratorGetNext]] [Op:__inference_test_function_10400]

Function call stack:
test_function

2021-06-20 16:37:21.520890: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
	 [[{{node PyFunc}}]]

Process finished with exit code 1


[Feature][UI] Support to view and manage all timing settings of a project.

Purpose of the pull request
This PR will close #14139 .

Brief change log
Verify this pull request
This change added tests and can be verified as follows:

Manually verified the change by testing locally.

The test result is as follows:

Codecov Report

Merging #14178 (0dfc49c) into dev (9979ea3) will decrease coverage by 0.04%.
The diff coverage is 0.00%.

:exclamation: Current head 0dfc49c differs from pull request most recent head d1c5c49. Consider uploading reports for the commit d1c5c49 to get more accurate results

@@             Coverage Diff              @@
##                dev   #14178      +/-   ##
============================================
- Coverage     38.22%   38.19%   -0.04%
+ Complexity     4461     4452       -9
============================================
  Files          1227     1227
  Lines         42793    42747      -46
  Branches       4751     4720      -31
============================================
- Hits          16358    16326      -32
+ Misses        24630    24622       -8
+ Partials       1805     1799       -6

Impacted Files
Coverage Δ

...nscheduler/api/controller/SchedulerController.java
82.60% <ø> (ø)

...heduler/api/service/impl/SchedulerServiceImpl.java
30.21% <0.00%> (ø)

... and 39 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Setting the executor in a @ct.lattice decorator does not use the right configuration parameters

Environment

Covalent version: 0.226.0-rc.0 or v0.224.0-rc.0
Covalent-Slurm plugin version: 0.12.1 or main
Python version: 3.10.12
Operating system: Ubuntu (WSL)

What is happening?
Passing in a ct.executor.SlurmExecutor definition into a @ct.lattice decorator's executor kwarg does not work properly. I am getting the dreaded "username is a required parameter in the Slurm plugin" error message even though the username is clearly shown in the UI. Unlike in #70, this is not a sublattice but rather a very simple workflow. It's clear that the covalent.conf file is being read whenever the executor is passed to the Lattice object. Passing it to the Electron object works as expected.
How can we reproduce the issue?
Let's take the example from the docs:
executor = ct.executor.SlurmExecutor(
    username="myname",
    address="test",
    ssh_key_file="/path/to/my/file",
    remote_workdir="/scratch/user/experiment1",
    options={
        "qos": "regular",
        "time": "01:30:00",
        "nodes": 1,
        "constraint": "gpu",
    },
    prerun_commands=[
        "module load package/1.2.3",
        "srun --ntasks-per-node 1 dcgmi profile --pause",
    ],
    srun_options={"n": 4, "c": 8, "cpu-bind": "cores", "G": 4, "gpu-bind": "single:1"},
    srun_append="nsys profile --stats=true -t cuda --gpu-metrics-device=all",
    postrun_commands=[
        "srun --ntasks-per-node 1 dcgmi profile --resume",
    ],
)

@ct.electron # (executor=executor) works fine here!
def my_custom_task(x, y):
    return x + y

@ct.lattice(executor=executor)
def workflow(x, y):
    return my_custom_task(x, y)

dispatch_id = ct.dispatch(workflow)(1, 2)

The UI will show "username is a required parameter in the Slurm plugin" even though the username is provided. In this case, I am starting from a default Covalent configuration file, which has no username by default.
Note: Using @ct.electron(executor=executor) instead of @ct.lattice(executor=executor) works fine.
What should happen?
The username should be detected.
Any suggestions?

@wjcunningham7: I'm still getting this username error despite using a simple toy job and nuking my conda environments to start fresh. Not quite sure what I've changed, but basically it seems like the covalent.conf is getting pulled as higher priority rather than the user-specified lattice decorator kwarg.
If I edit my covalent.conf to have my username and credentials, I am able to submit a job.

Closed in https://github.com/AgnostiqHQ/covalent/pull/1736


Remove support for Goerli chain ID

Feature Request
Goerli networks are expected to shut down by Aprill 11, after that we should remove hardcoded Goerli chian ID.
If possible, we should also remove all mentions of Goerli in demos, examples, tests etc.

Blocked for now, let's unblock this in April

https://github.com/software-mansion/starknet-jvm/issues/384


Symmetry checks for matrix atoms

Hi, I added symmetry checks for the arguments of some matrix atoms which require symmetric input (like @rileyjmurray suggested in issue #932).

Thanks for taking the initiative on this @tuelwer!
There is small catch here though. CVXPY's attribute propagation system isn't good at tracking symmetry. For example,
>>> X = cp.Variable(shape=(4,4), symmetric=True)
>>> A = np.diag([1,2,3,4])
>>> Y = cp.multiply(A,X)  # Elementwise
>>> Y.is_symmetric()  # False

If we move forward with this PR as-is, it would raise errors in cases like the above.
Now, CVXPY doesn't blindly assume that users provide symmetric inputs to these atoms.
Before passing a problem to a solver, CVXPY looks for matrix atoms that require symmetry, and if an argument arg to such an atom has arg.is_symmetric() == False, then we essentially add arg == arg.T to the constraints (strictly speaking we don't enforce symmetry in that way since it introduces redundant constraints, but you get the idea).
The issue in #932 happened because the user called .value on the atom when the numerical value of the argument was not symmetric. By my explanation above, this can only happen when arg.value is something other than the result of solving a CVXPY problem. In the user's specific case, arg was a constant numpy array. We do need CVXPY to account for such inputs, but the best way forward will be to only require symmetry constraints in calls to .value, where we can check for numerical (rather than symbolic) symmetry.

Thanks for the clarification! I will continue to work on this and implement the checks in the .value() function.

Sorry for the delay; this looks good! Thanks for contributing.


Solution for loading Raven.js in multiple Polymer components

Via the sentry.io support channel:

Currently I load the main raven js at index.html. Then inside each dom element, I have to include this line at every ready()

ready() {
    Raven.config("my_sentry_dsn").install();
},

This works but I have to include this install call on each of my component. Do you have a better approach?

@benvinegar FYI I figured out this issue, I loaded the install at index.html and all sub-components got the lib loaded now. Thanks!

Awesome!


Ensure an ObjectPoolProvider is registered

e.g. take advantage of aspnet/HttpAbstractions#561 fix wherever cookies are used

/cc @davidfowl @Tratcher

:up::date:

:shipit:

b06a844


Selenium shows full path on local fs

https://github.com/smileytechguy/issueTesting/issues/112
Needs to be fixed to protect against information stealing

fixed


Fixed spelling mistake

Fixed spelling mistake for changing the font of latex text. It used to be "tricker" but has been fixed to "trickier"

Motivation

The documentation now has less mistakes.

Reviewer Checklist

[ ] The PR title is descriptive enough
[ ] The PR is labeled correctly
[ ] The changelog entry is completed if necessary
[ ] Newly added functions/classes either have a docstring or are private
[ ] Newly added functions/classes have tests added and (optional) examples in the docs
[ ] Newly added documentation builds, looks correctly formatted, and adds no additional build warnings

Added that too, weird I did not notice that...

No worries 🎉


Disabled components no longer focus lock pointers.

Overview
To check if a pointer is focus locked, we see if there are any IMixedRealityPointerHandler when the OnPointerDown event is raised.
This PR fixes an edge case where an component implements that interface, but is itself disabled.
Changes

Fixes: #9921

/azp run


redirecting to dank memes

Courtesy of #6

I'll let you have a blank line too. Many thanks @piticent123 - and Happy Hacktoberfest!

The others were like that, so I figured I'd be consistent ;) Happy Hacktoberfest to you too! I love the concept of this repo!


feat(report): pretty print final JSON result

Merging this in as it's mainly to do with the report, PR is just here to let everyone know about the changes


Change media mixin to media-query

Possible typo in documentation. Couldn't follow example until I changed to @include media-query(...)
@mixin media-query($media-query) { ... } in src/styles/tools/mixins.scss
What are you trying to accomplish with this PR?
Please provide a link to the associated GitHub issue.
Cannot find an existing open issue for this.
Checklist
For contributors:

[ ] I have updated the docs to reflect these changes, if applicable.

For maintainers:

[x] I have :tophat:'d these changes.
[ ] I have bumped the package.json version in a separate PR, if applicable.

Good catch! Thanks @declank.


Zero fill left shift operator "<<" is getting unexpected value

I'm using an algorithm to decode a polyline that has a line with the folowing code:
(3 & 0x1f) << 30
running this line in dart results in:<PHONE_NUMBER>
but if run in javascript results in: -1073741824
The resulting javascript code value is as expected and correct. I would like to know if the use of this operator in dart is correct?
I apologize if this is not the proper repository for this issue

The negative result comes from Javascript running on browser console or nodejs.
I found this bug because I'm "translating" a Javascript library to Dart.

The Dart result is actually correct because Dart on JavaScript defined its shift operations that way.
Dart integers used to be arbitrary precision integers (what is currently retained as the separate type BigInt), and it does not make sense to have both >> and >>> on arbitrary precision integers.
Compiling Dart to JavaScript, without big integers, meant making some compromises. All JavaScript bitwise operations (|, ^, &, ~, >>, <<) would first truncate to a signed 32-bit number, then perform the operation, and return a signed number. The >>> operation was different in that it returned an unsigned 32-bit number.
Since Dart did not have both >> and >>>, we had to pick one behavior for >>. One of the factors were the speed of cryptographic functions, and to help those, it was chosen that all Dart bitwise operations compiled to JavaScript should return an unsigned number.
That is what you see now, Dart returning that unsigned interpretation of the 32 bits of the result.
C# gives you a negative result because the C# type here is a signed 32-bit integer. If you change the code to:
long x = 3;
x = x << 30;
Console.WriteLine(x);  // writes positive number.

The Swift result is what it is because Swift uses 32- or 64-bit integers depending on platform, and you are likely on a 64-bit platform.

Thanks for your explanations.
The algorithm I am implementing needs to be signed 32-bit int, so I found a workaround using the fixnum package (https://pub.dev/packages/fixnum)


[ConstraintSystem] Reinstate favoring more specialized of two generic…

… overloads.
This was originally commited in
3d32e89e335285e5274100ce47bb755c1f2837c6, and then backed out in
c40fd3966c28c119bd6e5ebd878a26cb5e8e5c4d due to concern over
behavioral changes.
This version avoids the optimization when there are any optional types
involved in the signatures of the functions.
For these cases, compareDeclarations can return the wrong order at the
moment.
I have another PR in the works that attempts to begin unraveling some
of the issues around fixing the overload comparisons.
(cherry picked from commit 1a2d7f5e991861c4d685cc64df0dd51cb45dcb4d)

@swift-ci Please test


question

if i wanted to use this instead of commando, like changing my existing code to work with it, would it work similarly to commando, or is a lot of it different

Commando is a framework focused in commands and is used with discordjs. And the Discord Akairo its a framework not only for commands but also the entire bot. An extension for discordjs.
So if you are going to use akairo it can be redoing your bot or in a new project.
I hope I have helped, I am learning English so there may be some mistakes even with the assistance of google translator.

if i wanted to use this instead of commando, like changing my existing code to work with it, would it work similarly to commando, or is a lot of it different

You would need to redo your bot. Most of the login for your code will be the same, just as the commands themself. But setting up the bot will be different. Akairo uses a more class based system out of the box than commando, so Events, commands, and other things are handled a bit differently.
I would just create a sample prod and test it out for yourself.

Alright, I guess I'd better read about how it's all handled, thanks!