Incomplete response when using HTTP2+CONNECT tunneling

Description:
envoy version: 1.25.6
We are using HTTP2+CONNECT to tunnel TCP connections over an HTTP2 connection. The setup looks like:
client --HTTP--> envoy1 --TCP over HTTP2 tunnel---> envoy2 --HTTP--> server
client makes HTTP 1.1 requests.
envoy1 runs a tcp_proxy filter and multiplex the tcp connection on an HTTP2 stream.
envoy2 runs 2 HCMs. The first HCM demultiplexes the HTTP2 stream and feeds to an internal listener with a second HCM. The second HCM routes the original HTTP request to upstream server.
The server serves large binary content using chunk encoding.
When client makes a request without "Connection: close" header, everything works fine. However, if it makes a request with "Connection: close" header, sometimes the response is incomplete - the tail of the response is not received.
With curl an error message shows: transfer closed with outstanding read data remaining
I did some investigation here. There are 5 connections involved:
C1: stands for client -> envoy1
C2: stands for envoy1 -> envoy2 external listener C2.log
C3: stands for envoy2 first HCM -> upstream (internal listener) C3.log
C4: stands for downstream (envoy2 first HCM) -> internal listener C4.log
C5: stands for envoy2 second HCM -> server C5.log
C3 and C4 are physically same connection but logically 2 connections from envoy2 perspective
From logs, I can tell that C5 received whole response from server and C4 wrote whole response. Also, C3 got the whole response. The problem is though C2 successfully encoded the whole response but C2 failed to write all the data before resetting the stream.
I also did a packet capture at envoy1 side and it shows envoy2 reset the stream thus following DATA frames are not being sent.

key logs from C2:
[2023-07-11 12:33:45.331][17][trace][http] [source/common/http/conn_manager_impl.cc:1606] [C2][S4803202076355283748] encoding data via codec (size=16384 end_stream=false)
[2023-07-11 12:33:45.332][17][debug][router] [source/common/router/upstream_request.cc:500] [C2][S4803202076355283748] resetting pool request
[2023-07-11 12:33:45.332][17][trace][http] [source/common/http/conn_manager_impl.cc:1606] [C2][S4803202076355283748] encoding data via codec (size=8597 end_stream=true)
[2023-07-11 12:33:45.332][17][debug][http] [source/common/http/conn_manager_impl.cc:212] [C2][S4803202076355283748] doEndStream() resetting stream
[2023-07-11 12:33:45.332][17][debug][http] [source/common/http/conn_manager_impl.cc:1649] [C2][S4803202076355283748] stream reset
[2023-07-11 12:33:45.332][17][trace][http2] [source/common/http/http2/codec_impl.cc:809] [C2] deferred reset stream
[2023-07-11 12:33:45.332][17][trace][http2] [source/common/http/http2/codec_impl.cc:1316] [C2] about to send frame type=3, flags=0
[2023-07-11 12:33:45.332][17][trace][http2] [source/common/http/http2/codec_impl.cc:1338] [C2] send data: bytes=13
[2023-07-11 12:33:45.332][17][trace][connection] [source/common/network/connection_impl.cc:483] [C2] writing 13 bytes, end_stream false
[2023-07-11 12:33:45.332][17][trace][http2] [source/common/http/http2/codec_impl.cc:1218] [C2] sent frame type=3, stream_id=1, length=4
[2023-07-11 12:33:45.332][17][debug][http2] [source/common/http/http2/codec_impl.cc:1247] [C2] sent reset code=0
[2023-07-11 12:33:45.332][17][debug][http2] [source/common/http/http2/codec_impl.cc:1363] [C2] stream 1 closed: 0
Though log says 'deferred reset stream' but actually there's no delay, it sends the RST immediately, causing nghttp2 to drop pending DATA frames.
Some questions:

I see here suggested using nghttp2_on_frame_send_callback to ensure DATA being sent before RST. But it seems envoy is not using it. Is there any special reason here?https://github.com/envoyproxy/envoy/blob/bd836c22030b20624142c78fc16773d4bb2e582e/source/common/http/http2/codec_impl.cc#L1231-L1243
I do not quite understand the logic of deferred reset stream. It seems to me that if a stream is set to be deferred reset, any send operation on any stream (even not same one) can cause the stream to be reset? Because sendPendingFrames() always iterates over all the deferred reset streams and resets them. Does the 'defer' take effect in fact?
(Correct me if this does not make sense) Today when the upstream closes the connection, the stream is reset because stream.filter_manager_.remoteDecodeComplete() is false. It makes sense for normal HTTP2 requests to reset the stream when request is not completed. However,for CONNECT, it's always false unless client ends the tunnel. Also, from CONNECT request perspective, the request is already completed so should we handle it in the same way as normal HTTP2 request completed?

my guess is it's related to this
https://github.com/envoyproxy/envoy/blob/main/source/common/http/conn_manager_impl.cc#L258
because that's the main code difference between the connection close case and not.  That said I have no idea how the internal listener handles fast connection close cc @kyessenov for how that might cause problems.

@alyssawilk It's not related to the internal listener. I also tested it without internal listener, the issue still persists. In that test the envoy2 setup looks like:
--HTTP2--> HCM --HTTP1.1--> (normal listener) --> HCM ------->

@alyssawilk @kyessenov comments are appreciated.

@alyssawilk @kyessenov would appreciate commenting on this or resetting the stale bot.


Ply not showing in scene

Hi Keijiro,
While I have no issues importing my Ply file, when I drag it into the scene it simply does not show up. All of the components are there and editable, but I cannot see anything. The imported Ply file still has a thumbnail that is visible and shows my pointcloud
I downloaded the richmond-azaelias.ply file, and that works perfectly fine. However, both files are imported and visualised with no issues in Cloud Compare. I've included a screenshot of how it looks like after I drag in my Ply file (richmond-azaelias.ply in there for reference as working).

As such it looks like it must be something to do with my Ply file; would you have any suggestions of what or how I may investigate further?
Any help would be very much appreciated.
(I am very very new to Unity, so my apologies if this question is very simple).

Could you provide the file that causes the issue? I'd like to test it myself.

I received test data from @Glorfendal and found that the point cloud lies too far from the origin. It's recommended to move them around the origin before exporting to avoid floating point precision error issues.
I'm closing this issue now. Please feel free to reopen for further questions.


/money pay typo

You currently have syntax showing /money pay
but the right way to pay is actually /money pay
This confuses players since its opposite
Latest Version

dupe


Added overview to Basic QA in README.md

Done

@kishore-narendran Thank you for migrating the other data from the spreasheet.
@sigit-prayoga Alright.
Maybe I will edit some minor editing style. Because in Markdown, there is a code syntax like branch not just -branch. I actually have updated them in the newer version of our spreadsheet, but not the public copy yet.

Almost forgot. Should we omit the Duration?

Okay, we should omit them. I actually planned to edit them tonight, but I should have a rest right now.


How to use deserializeBody function?

I want to download media from the message and it seems that I have to deserializeBody first to get the access hash. Well, the function gets two arguments: InputStream and TLContext. I have no idea what to send as argument! Can you help me with this?
for example for photos the function is:
@Override
    public void deserializeBody(InputStream stream, TLContext context)
            throws IOException {
        this.photo = ((TLAbsPhoto) StreamingUtils.readTLObject(stream, context));
        this.caption = StreamingUtils.readTLString(stream);
    }

but no example to learn what the arguments should be.
thanks

That method is not supposed to be used externally, only by the api itself


Deprecate most types in favour of Ecto 2.1 ones

@lau could this be merged? The type definitions still conflict

@archSeer Version 0.17.0 is out.

Thank you! 🎊


Upgrade to kafka-clients 3.4.0

This is an enabler for integration with the kroxylicious proxy framework because it depends on kafka-clients too, causing problems at runtime if we try to call the EncryptionModule built against an older kafka-clients.
Note: topic-encryption with the vertx-proxy will still require the proxied Kafka cluster or client to be less that version 3.1.0 because it does not yet work with topic ids in the FetchResponse (see #17 ).

@chris-giblin could you take a look?

@robobario Were you able to test these changes with older clients to verify topic encryption with names still works?

@robobario Were you able to test these changes with older clients to verify topic encryption with names still works?

I tested it with the vertx-proxy in front of a 3.4.0 broker, and observed that decryption did not work for a 3.4.0 consumer and did work with a 3.0.2 consumer. Happy to check other combinations (old broker versions) on Monday.

Thanks, testing with 3.0.2 suffices for me to verify the behavior for older versions is unchanged. It is nice that the kafka level has been brought up to a recent level with this PR. Thanks!
This change however does necessitate implementing the version interception referred to by Tom, above.  This is a good time for me to address https://github.com/strimzi/topic-encryption/issues/17.

@chris-giblin check out https://github.com/kroxylicious/kroxylicious/blob/00a42e7e7b06fb1ff9b82dd63267427fd06a484c/kroxylicious/src/main/java/io/kroxylicious/proxy/internal/filter/ApiVersionsFilter.java#L21 which has the logic you need.

@tombentley @robobario is there anything stopping this being merged?  I know the client is out of date already, but it still makes sense to take this.


Legacy string field's fielddata attribute parsing issue

NEST/Elasticsearch.Net version: 5.x
Elasticsearch version: 5.x
Description of the problem including expected versus actual behavior:
Upon 2.x to 5.x upgrade, Elasticsearch will also upgrade the existent indices mappings. One change is to add "fielddata": false to any not_analyzed string fields. And these new mappings seem to break the mappings serialization:
Unexpected initial token 'Boolean' when populating object. Expected JSON object or array. Path 'fielddata', line 1, position 368.

Thanks for submitting @astefan; I've opened PR #2728 with a fix

Merged in PR; this'll be in the next release on Nuget 👍


Add requestid to context

This PR is a proof of concept. So, it is very open for discussions.

Resolves https://github.com/TheWisePigeon/visio/issues/22
Add requestid to context, so we will have unique identifier for each request.
As starter, we are implementing it in KeyHandler.Create

Also one thing that I meant by

This middleware should only log requests either coming from the Frontend or authenticated requests going to the API.

is that, when I originally hosted the first version of Visio on a linode instance, I used Chi's logger middleware but my logs were bloated because of bot requests coming to the server. So I think passing an id down to the middlewares part can be handled by Chi, but we need another middleware that could tell if a request is coming either from our frontend or from a user, a very simple way I think we can solve this is by having a special value that would be passed in a special header in all requests coming from the frontend and as for the requests going to the API itself we could just check for the existence of the x-api-key header and reject the request if it is not set. Do you think it's a good approach?

Also one thing that I meant by

This middleware should only log requests either coming from the Frontend or authenticated requests going to the API.

is that, when I originally hosted the first version of Visio on a linode instance, I used Chi's logger middleware but my logs were bloated because of bot requests coming to the server. So I think passing an id down to the middlewares part can be handled by Chi, but we need another middleware that could tell if a request is coming either from our frontend or from a user, a very simple way I think we can solve this is by having a special value that would be passed in a special header in all requests coming from the frontend and as for the requests going to the API itself we could just check for the existence of the x-api-key header and reject the request if it is not set. Do you think it's a good approach?

i think that's a good idea
in addition, if the bots exhaust your server (i.e. our server processes bots' requests and returns HTTP 200 OK) we can also do rate limitting

yep, rate limiting is definitely something I want to implement :)


(WIP) pdk validate draft

@sanfrancrisko I think you can close this PR but keep your branch around for when we come back to validators


dev: Kakarot RPC Adapter Documentation initialization

To start implementing the RPC adapter, I created a similar documentation and workflow as on the kakarot repo.
Go check my pull request and suggest edits: https://github.com/sayajin-labs/kakarot-rpc-adapter/pull/3

Thanks for your PR!


Update Spark dependency to 2.4.1, Parquet to 1.10.1 or later

Spark 2.4.2 passed voting, will be released soon.


Handle multiple events with the same name in one contract

By indexing the events by signature instead of name. We should somehow do the same for functions since currently they are also indexed by name, but functions don't have a signature() method available. Another possible approach would be to use a map from name to a set of functions. Or to not use a map at all and use linear search on a vector.
This is a breaking change and will require a major release.

[clabot:check]

CC @debris could say what you think about it?

@tomusdrw @debris I've updated this to be backwards compatible with passing just a name, and to index the name to a vector of events, as per your suggestions. I'd like to do the same for functions but I'm not sure how that would work for the cli.

@tomusdrw @debris Any update on this one?

@tomusdrw Thanks for taking the time to take another look at this, I've addressed your review.

Thanks a lot!


Add support for switching between non user/channel and user/channel mode

Hey 👋 ,
I build conan center index recipes in-house, so that we get all our required build for our compilers. This works great, but one thing always occur.
The build script looks like
import os
import shutil
from cpt.packager import ConanMultiPackager
from conans import tools

def create_clean_build_folder(folder):
    if os.path.exists(folder):
        shutil.rmtree(folder)
    os.makedirs(folder)

if __name__ == "__main__":
    url = "https://github.com/conan-io/conan-center-index/archive/master.zip"
    recipe_folder = "conan-center-index-master/recipes/zlib/1.2.8"
    build_folder = "_build"
    reference="zlib/1.2.8@"

    tools.rmdir(build_folder)
    os.makedirs(build_folder)

    # download the conan index as zip and extract to a sub directory
    tools.get(
        url,
        filename="conan-index.zip",
        destination=build_folder,
        pattern=os.path.join(recipe_folder, "*"), # extract just zlib/1.2.8
        retry=2,
        retry_wait=5
    )
    conanfile = os.path.join(build_folder, recipe_folder, "conanfile.py")

    # normal cpt build, but with the downloaded conanfile
    builder = ConanMultiPackager(conanfile=conanfile, reference=reference)
    builder.add_common_builds()
    builder.run()

When we do this in the past we used channels to switch between channels. A half year ago we want to have the same reference as used in conan center index mypackage/0.1.2@ without channel and username and only upload on stable. This approach works but is not optimal for testing, because we need from time to time that the user can try out the new testing branch.
We think about using following approach in the near future:
stable branch: mypackage/0.1.2@
testing branch: mypackage/0.1.2@company/testing
We can do this in every build script and it would work but we have to do this over and over again. It would be nice when cpt would support that as well.
I've check the source code
and see that the actual impl. would not allow that because when it sees the @ in the reference it don't ask for the username/channel.
if self.partial_reference:
            if "@" in self.partial_reference:
                self.reference = ConanFileReference.loads(self.partial_reference)
            else:
                name, version = self.partial_reference.split("/")
                self.reference = ConanFileReference(name, version, self.username, self.channel)
        else:
            if not os.path.exists(os.path.join(self.cwd, self.conanfile)):
                raise Exception("Conanfile not found, specify a 'reference' "
                                "parameter with name and version")

I think with a minimal modification this could be support.
What are you thinking about that?
Greetings
Michael

It's not clear for me your problem... What do you propose?

Hey @uilianries ,
I want to be able to use cpt to switch between
stable branch: mypackage/0.1.2@
testing branch: mypackage/0.1.2@company/testing
by just using cpt environment variables
variables:
  CONAN_REFERENCE: "mypackage/0.1.0"
  CONAN_USERNAME: "company"
  CONAN_CHANNEL: "testing"
  CONAN_STABLE_BRANCH_PATTERN: "main"
  CONAN_STABLE_CHANNEL: "" # <= this seems not to work right now

@tonka3000 Now it's much clear. Thank you!


Inline elements should have correct Span (not zero-filled)

Consider the following markdown fragment:
a `b` **c**

The parser returns ContainerInline node with the following children:
[0] = {LiteralInline} a
[1] = {CodeInline} `b`
[2] = {LiteralInline}
[3] = {EmphasisInline} 42 '*' Count: 2

But all Span properties of these nodes are zero-filled (0-0) though they should not.
How can I get the correct position of inline nodes?

It works fine if I set PreciseSourceLocation in the following way:
var builder = new MarkdownPipelineBuilder
{
    PreciseSourceLocation = true
};
MarkdownDocument document = Markdown.Parse(content, builder.Build());
return new MarkdownContainerBlockNode(document, ParseHtmlMarkdown(document));

Sorry for the noise.


How to draw the visual point cloud simulation like in Project?

Hi, @yzqin @dexpc-rl
Thanks for the wonderful work.
How to draw the visual point cloud simulation like in Project?
Thanks for any help:)

Hi @Charlie0257 ,
The point cloud video showcased on the project page is created using the Mitsuba Renderer to display the point cloud.
I'm confident that a comparable result is achievable if you utilize a ray tracing method for visualization. This would require taking into account both the shadow and rendering material, and treating each point as a sphere instead of using a rasterizer such as Open3D.

Thanks for your reply! @yzqin
Could you please open-source the code to showcase the video with Mitsuba Renderer?


Managed Identity and Service Principal Support

Overview

Added MSAL Managed Identity and Service Principal Token Providers to Microsoft.Artifacts.Authentication Library.
Created new endpoint ARTIFACTS_CREDENTIALPROVIDER_FEED_ENDPOINTS environment variable with new json schema for MI/SP required fields.
Updated VstsBuildTaskServiceEndpointCredentialProvider to call  Microsoft.Artifacts.Authentication for MI/SP token providers.
Reverted #485 Changes to use system.text.json for de/serialization everywhere except for the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable.

Design Decisions

Intentionally not supporting SP secrets authentication to promote security best practices.
The new environment variable name and json schema were created instead of reusing or extending the existing VSS_NUGET_EXTERNAL_FEED_ENDPOINTS to reduce password usage and clarify the environment variable will be available to our other credproviders such as the artifacs-keyring not just NuGet.

Environment Variable
ARTIFACTS_CREDENTIALPROVIDER_FEED_ENDPOINTS
 {"endpointCredentials": [{"endpoint":"http://example.index.json", "clientId":"required", "clientCertificateSubjectName":"optional", "clientCertificateFilePath":"optional"}]}

endpoint: required. Feed url to authenticate against.
clientId: required for both MI/SP. For user assigned managed identities enter the Entra client id. For system assigned variables set the value to system.
clientCertificateSubjectName: Subject Name of the certificate located in the My/ CurrentUser or LocalMachine certificate store. Optional field. Only used by SP authentication.
clientCertificateFilePath: File path location of the certificate on the machine. Optional field. Only used by SP authentication.

Will throw error if both clientCertificateSubjectName or clientCertificateFilePath are specified.

using System;

add license header

Refers to: CredentialProvider.Microsoft/CredentialProviders/VstsBuildTaskServiceEndpoint/VstsBuildTaskMsalTokenProvidersFactory.cs:1 in 0dceaff. [](commit_id = 0dceaff4b5011ec085c15f3ae33bd0c967a108f0, deletion_comment = False)

using System;

same for all new (non test?) files

In reply to:<PHONE_NUMBER>

Refers to: CredentialProvider.Microsoft/CredentialProviders/VstsBuildTaskServiceEndpoint/VstsBuildTaskMsalTokenProvidersFactory.cs:1 in 0dceaff. [](commit_id = 0dceaff4b5011ec085c15f3ae33bd0c967a108f0, deletion_comment = False)

please address the build warnings as well, there are a few about package conflicts. surprised these are not failing the build :(


Bump SLE BCI image to 15.4

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Problem:
Harvester components base image needs to update.
Solution:

Upgrade base image from SLE BCI 15.3 to 15.4
Related Issue:
#3149

Need to backport to v1.1 branch as well. Thanks!


[Automated] Format markdown

Produced via:
prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github | grep -v docs/cmd/)

Cron. -knative-prow-robot
/cc knative-sandbox/client-writers
/assign knative-sandbox/client-writers

@maximilien not sure why this bot is still active (I deactivated it), but please don't merge the markdown formatting as it breaks the "one sentence per line" format (which is much easier to use for diffs and that we prefer). Also the auto generated markdown from cobra doesn't fit in this scheme, too.
No worries for now, but for the future.


Hook in testing data

It'd be great to see tests/per-million, cases per test, deaths per test etc.

Fixed in https://github.com/zbraniecki/covid19-timeseries/commit/1271880421844309de9c669ad4184a6fef8f848f


update min version of httparty to 0.15

HTTParty changed its decompression logic in 0.15 (https://github.com/jnunemaker/httparty/pull/513/commits/6f6bf6b726484eaf50e190769bbe14c9841a2c64#diff-580cbdf9b30c3492eaffdfa72a97aa58R189)
We had to remove the explicit gzip handling in this PR: https://github.com/Shopify/salesforce_chunker/pull/61
To ensure that the compression continues to work properly in the next version of the gem, we should set a minimum dependency of HTTParty 0.15

closing, doing this at the same time as a gem bump: https://github.com/Shopify/salesforce_chunker/pull/66


🌵 Support Responsive Of iPhone SE

🎫 Tickets

#75

👨🏼💻 What did you do?

I supported responsive for iPhone SE
I fixed the layout for SE to go to SignIn Page smoothly

👀 Images

🌋 TODOs

[x] nothing


chore: reenable devimint gateway liquidity channel opening

I audited the code for open_channel() to ensure that open_channel_between_gateways() matches it 1-to-1. I then re-enabled open_channel_between_gateways() for devimint (which was disabled here due to being slow and flaky). I measured the startup time for just mprocs 10+ times on my local machine before and after the change and with the changes in this PR it never failed and the time difference was within margin of error.

Will revert if it is causing issues. :shrug:  :rofl:

MQ found some LN test problems, worth looking into before kicking it back into the MQ imo.

Hmmm most of the CI errors are some form of the following out-of-disk-space:
fedimint-workspace-lcov-ci> 2024-05-22T22:56:47.489529Z ERROR connect_fed{payload=ConnectFedPayload { invite_code: "fed11qgqpw9thwvaz7te3xgmjuvpwxqhrzw33xymnsdf0qqqjqmlee0af0aap6fyvmslxhn94k9tsms6wtel7sq32z747qdhra0uuej7vxh" }}: ln_gateway::rpc::rpc_server: error=Failed to open the database: Error opening rocksdb: IO error: No space left on device: While appending to file: /build/test-14034218261564233250GBHOqh/gateway-16131950401517601773/6ff9cbfa97f7a1d248cdc3e6bccb5b1570dc34e5e7fe8022a17abe036e3ebf9c.db/000000.dbtmp: No space left on device
Whatever's causing this flakiness appears to be outside of the control of the code being changed in this PR.


Vite: Warnings on console from renderMarkdown sanitize-html dependency

Is it an issue related to Adaptive Cards?
No
Is this an accessibility issue?
No
What version of Web Chat are you using?
Latest production
Which distribution are you using Web Chat from?
NPM
Which hosting environment does this issue primarily affect?
Others or unrelated
Which browsers and platforms do the issue happened?
Others or unrelated
Which area does this issue affect?
Others or unrelated
What is the public URL for the website?
No response
Please describe the bug
Our project is a React SPA app and we use Vite (v4.4.4) for building. We use the default markdown which is exported as renderMarkdown botframework-webchat.
Running the application results to following warning on console:
Module <X> has been externalized for browser compatibility. Cannot access <X.Y> in client code. See http://vitejs.dev/guide/troubleshooting.html#module-externalized-for-browser-compatibility for more details.
Affected modules are:

path
fs
url
source-map-js
All of these are originated from renderMarkdown.ts dependency of sanitize-html.

Do you see any errors in console log?
Only warnings

How to reproduce the issue?

Create new project with Vite
Install botframework-webchat from NPM
Include in project and implement some chat using the default renderMarkdown function
Run using npm run vite
Our setup is this, but I would expect the warnings to appear with any usage of the function:
import { createDirectLine, Components, renderMarkdown, } from 'botframework-webchat'
<Components.Composer        directLine={createDirectline({token:token})}        renderMarkdown={renderMarkdown}      >
      <Components.BasicWebChat/>
</Components.Composer>

What do you expect?
No warnings
What actually happened?
Warnings as stated above
Do you have any screenshots or recordings to repro the issue?

Adaptive Card JSON
No response
Additional context
No response

AFAIK, there are dependencies outside of Web Chat that imported external and browserify them.
If they are from sanitize-html, could you file the issue to them? We will uptake their latest version when they have resolved the issue.


[Feature] Use a custom data loader for conversions

Instead we commit warcrimes against the JVM

We're waiting on #68 before taking a look at this.


Rancher Prime documentation for Azure

@guangyee, NV Azure password is exposed in the Cluster configuration. To address, we included Important Notes.. Do you see similar issue in Rancher ??
The current NeuVector deployment exposes the original password in the Cluster configuration settings in Azure Portal. Until this is resolved, it is suggested to change the Admin password after initial login.  Edit profile in the NeuVector console to change password.

Thanks @guangyee for updating Password Notes. Assume, update changes will be done in a different PR once update process is finalized. I will approve this PR


discussion: ReadConsoleInput can read 0 characters and return success and we're just not sure if that's by-design

More discussion notes in https://github.com/dotnet/runtime/issues/88697. I'm filing this here so we have a local reference to the discussion, even if we just close this out.
Here are some miscellaneous noted comments from that thread:

The Console.ReadKey function can throw an InvalidOperationException when "application does not have a console or when console input has been redirected". That makes sense. However, there is another case where it throws an InvalidOperationException, which happens when there are multiple processes attached to the same console, both waiting for input, and one dies or is killed.

BTW it's interesting that the official docs say that it's impossible to read 0 characters on success:

The function does not return until at least one input record has been read.

Is it a doc bug, or a code bug?

Bear with me as I collect some notes
ApiDispatchers::ServerGetConsoleInput looks like the guy that handles ReadConsoleInput<EMAIL_ADDRESS>That calls into ApiRoutines::GetConsoleInputImpl<EMAIL_ADDRESS>That always passes WaitForData as true, in the read from the input buffer. Doc comment on that method:

// - WaitForData - if true, wait until an event is input (if there aren't enough to fill client buffer). if false, return immediately

oh but then I think we start getting into the realm of the console read waits. Those are nasty to try and track down. My gut says it's not impossible that there's a case where a client disconnecting might try to complete the other waits that were pending. Tracking that down would be trickier. I also have no idea which is right, certainly not off the top of my head.

oh but then I think we start getting into the realm of the console read waits. Those are nasty to try and track down.

No actually, it's gonna be super easy, barely an inconvenience. ^1
GetConsoleInputImpl constructs a DirectReadData which is the callback that runs when input data is finally (supposedly) available sometime later. It passes false as WaitForData and so it returns STATUS_SUCCESS when the buffer is empty. It's been like that ever since the first commit on GitHub:
https://github.com/microsoft/terminal/blob/d4d59fa3395012ca37ba12665da4ec11c7dcf9cb/src/host/readDataDirect.cpp#L138
But if we compare that with the conhost v1 code where the callback was a function called DirectReadWaitRoutine in directio.cpp then that one passes:
!(a->Flags & CONSOLE_READ_NOWAIT)

Which I think is the expected behavior. In other words, I believe that this is a conhost v1 -> v2 regression.


Support for clipboard with X11 PRIMARY selection?

I have installed xrdp with xorgxrdp on CentOS 7.4.
The clipboard is working, but only for the X11 CLIPBOARD selection, that is text explicitly copied or pasted.
It would be really practical if it worked with the X11 PRIMARY selection as well, that is highlighted text and middle-mouse button paste.
In Xvnc (TigerVNC) it works for both when used from a VNC client (no xrdp involved).
Versions used:
xrdp-0.9.3-2.el7.x86_64
xorgxrdp-0.2.4-3.el7.x86_64
Windows 10 Remote Desktop client 10.0.15038

AFAIK xrdp doesn't support X11 PRIMARY clipboard so far.  IMO, maybe it is possible to implement it but there's little benefit for effort because there's some workaround using third-party tools.
Syncing PRIMARY and CLIPBOARD using parcellite will take the same effect.

Indeed, I also saw (after reporting this) that gpaste could also be an alternative, although I have not tested either.

@diego-santacruz Do you still want it as xrdp's feature? If no, parcellite or gpaste is enough for you, I'm willing to close this.

I have successfully tested gpaste and enabling the synchronization of clipboard and primary selection solves the issue.

Okay, if you want the feature as xrdp's feature again, please contribute with code. We'll consider merging it but not implement it by ourselves. Closing.


Repo sync for protected CLA branch

The pull request is created from master637613632499453330 to master to fix git push error for protected CLA branch

@Taojunshen : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Remove deprecated warnings in tests by updating to react-dom etc

Any reason I'm not aware of why I should put react-dom as a peerDependency instead?
There's one more warning I haven't removed, resulting from usage of setProps here.
I'm not familiar with using React in this way so I'm not sure what the appropriate refactoring would be. The message says call React.render again at the top level - not sure how to do this in this case! Give me a hint and I'll add it to the commit :wink:
Other than that, the deprecation messages were very helpful, which is great.

Ahhh, this is what I get for not testing properly. We should just push react-art 0.14 that's not compatible with react 0.13.
Can you change it so the node returned from render has ref={c => this.domNode = c} and then use this.domNode directly? You shouldn't need to depend on react-dom at all for this.
For the setProps thing, you can do:
var container = document.createElement('div');
var instance = ReactDOM.render(<TestComponent flipped={false} />, container);
// ...
ReactDOM.render(<TestComponent flipped={true} />, container);

Nice! react-dom is now only a devDependency.

@spicyj This should be ready for merging now.

Can you change the react and react-dom versions both to ^0.14.0-a, and also in examples/vector-widget/package.json? Thanks @ThomWright!

Done! Would it be worth raising an issue to replace react-tools with babel-jest? That should remove the last of the deprecations, AFAIK.

Thanks! Yes, I'd be happy to take a PR for that.


Storage layer needs proper error handling #749

Fixes #749, and partly fixes #725
There's no reason why XmlSerializableAddressBook must implement ReadOnlyAddressBook.  It can just have a toModelType() method like the other classes, and then we have the freedom of throwing checked exceptions to signal invalid data as well.

But hang on, it seems like we don't need this anymore because XmlSerializableAddressBookTest.java doesn't use this right?

@Zhiyuan-Amos  Yep, only XmlUtilTest uses this, the test data files for each test class are in their respective folders

@Zhiyuan-Amos Yep, only XmlUtilTest uses this, the test data files for each test class are in their respective folders

Ok then I guess we can revert the changes :)

@yamidark deleted some of our comments cos they were clogging up the reviewing screen. Also, since there's no major changes required, I'll tag prof for review :)


Disable interpolation of XObject when Interpolate entry is not true (issue9713)

According pdf reference section 4.8, image should not be interpolated unless Interpolate entry is True.

When the resolution of a source image is significantly lower than that of the out-put device, each source sample covers many device pixels. This can cause imagestoappear  “jaggy”  or  “blocky.”  These  visual  artifacts  can  be  reduced  by  applyingan image interpolationalgorithm during rendering. Instead of painting all pixelscovered by a source sample with the same color, image interpolation attempts toproduce a smooth transition between adjacent sample values. Because it may in-crease the time required to render the image, image interpolation is disabled bydefault; setting the Interpolate entry in the image dictionary to true enables it.

This looks, for all intents and purposes, like a duplicate of PR #8338; please note that that one was closed because it caused way too many outright regressions in existing test-cases.
Did this patch actually pass all existing tests locally, please refer to https://github.com/mozilla/pdf.js/wiki/Contributing#4-run-lint-and-testing additional information?

Yes, it is duplicated, and it didn't pass the test-cases.


Adding a README and link to repo

I'm using this module and it took me some time to find the code from npmjs.org so I just want to be helpful and contribute to the project by adding a README file and a link to the repo that should get visible no npmjs.org.
Thanks for the module and have a nice day.

Ta. I've had the same frustration in the past, but I've also got a habit of running npm init -f first then creating my repo, so it never lands in the package file!


StatSig react native not working

Hello! StatSig is not working with React Native for me.
The app builds find but adding StatsigProvider causes the app to crash
I'm using react native version 0.67.3 and react version 17.0.2
This has been happening for several months and has been a hard blocker from our company using your product. I'd appreciate any help getting this to work! Thanks :)

Copying this code snippet which was shared in slack.  It looks like you are trying to do your own custom initialization path and use the provider, which wont work


RbacCollector is currently broken

After a change in ZendDeveloperTools, the RbacCollector is broken.  See here:
https://github.com/zendframework/ZendDeveloperTools/commit/d3432681aa32177a741ad23604a40af9ad454acb#commitcomment-11090423

Same issue for me, tried a PR here https://github.com/ZF-Commons/zfc-rbac/pull/296.

Fixed in #296


Gusinacio/tap agent integration

This PR aims to lower the requisites to match what we use on tap-agent and indexer-service.
Some key points:

Remove Serde from unneeded places like ReceivedReceipts. (also remove typetag, because we don't need to store state of checks)
Remove stateful checking, now all the checks are done on verify
Split ReceiptStore into: ReceiptStore and ReceiptDelete
Remove query_id
Add a unique_hash() for signed messages
Remove update_receipt_by_id method because it's not needed anymore
Split Check trait into: Check and CheckBatch
Create a Checks type that is way more memory efficient to move things around.
Allow anyT: Display error type on verify_signer closure

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

65 of 85 (76.47%)  changed or added relevant lines in 7 files are covered.
11 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.8%) to 79.137%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

tap_core/src/tap_receipt/receipt_auditor.rs
5
6
83.33%

tap_core/src/tap_receipt/received_receipt.rs
19
23
82.61%

tap_core/src/checks/mod.rs
14
29
48.28%

Files with Coverage Reduction
New Missed Lines
%

tap_core/src/tap_manager/manager.rs
1
87.28%

tap_core/src/checks/mod.rs
10
60.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.8%

Covered Lines:
990

Relevant Lines:
1251

💛  - Coveralls


reformat module registry printing and add utest

Reformats printing of module registry to be ordered and grouped by module.
Add utest to check printing and warning when registering.

@nathanhhughes Turns out the behavior was already as intended, but now I'm sure and the printing looks clearer (hopefully) 🙂

@nathanhhughes This should now be final 🙂


Port to 1.21.3

Bumped versions and minor code adjustments so it works on 1.21.3.

Hello! I appreciate the effort but I typically try to get full version coverage for my mods. In this case I actually updated to 1.21.2 first (and did a little house cleaning while I was at it) and that turned out to be compatible up through 1.21.4!
Sorry for taking so long to getting around to this! I hope you were at least able to build it locally for your own use case in the meantime!

Makes sense, I do too. I did end up getting use out of it, so all's well that ends well.


Failed validation: #/definitions/QuoteSummaryResult

Problem getting quotes for ETFs and mutual funds

0P000071W8.TO
QQQ

const data = await yahooFinance.quoteSummary("0P000071W8.TO");
const data = await yahooFinance.quoteSummary("QQQ");

Mutual Fund Error:  The following result did not validate with schema: #/definitions/QuoteSummaryResult
[
  {
    keyword: 'required',
    dataPath: '/price',
    schemaPath: '#/definitions/Price/required',
    params: { missingProperty: 'regularMarketDayHigh' },
    message: "should have required property 'regularMarketDayHigh'"
  }
]

ETF Error:  The following result did not validate with schema: #/definitions/QuoteSummaryResult
[
  {
    keyword: 'additionalProperties',
    dataPath: '/summaryDetail',
    schemaPath: '#/definitions/SummaryDetail/additionalProperties',
    params: { additionalProperty: 'yield' },
    message: 'should NOT have additional properties'
  }
]

A simple fix would be https://github.com/gadicc/node-yahoo-finance2/issues/16

Hey @LesterLyu, thanks for reporting this.
I published a new version which let's you use all the options to skip validation and be less verbose with errors, as documented at https://github.com/gadicc/node-yahoo-finance2/blob/devel/docs/validation.md.  (This URL is given during the error too).
This was a bit of a rushed release pre-weekend but only in terms of the above, everything else is better tested.  Let me know if you run into trouble.
For the rest of it, will fix the schema hopefully over the weekend.

(talking about 1.5.0)

Thanks @LesterLyu, fixed in 1.5.1 with tests for all quoteSummary submodules for these two symbols.  Please let me know if you have any further issues.  I know these validation issues are annoying in the beta but we'll reap the rewards if we get it right now.  Of course you always still have the option to turn off validation.


Attached image is not properly rendered

Description
On root page (under commit heatmap) issues comments with attached images are not rendered properly.
If image is a markdown ![image] - it is okay
Screenshots

Gitea Version
1.22.1
Can you reproduce the bug on the Gitea demo site?
No
Operating System
No response
Browser Version
120

How is this comment displayed in the issue? The attached images should not be rendered on feed.

In the issue it is rendered correctly

But other images are rendered in feed if they are ![image]

Inside issue it is also correct
** updated to 1.22.2
Oh, may be I misused word "attached", not just attached, copy-pasted inside text


Bug when adding link to truncation attributed string

Hi, i have the same issue that:
https://github.com/TTTAttributedLabel/TTTAttributedLabel/issues/519#
I think it was fixed on this link
https://github.com/TTTAttributedLabel/TTTAttributedLabel/commit/f243cc5191131418e4fd27da456578834315b391
But the last version doesnt have that fix. I tried to download that commit, but when text end with \n the link of truncation attributed string doesnt run.
Any idea? Thanks!

I'm also having issues with truncation. The link seems to be added to the characters immediate before the truncation text at times - so tapping on the truncation text doesn't do anything. This appears to only affect the tap area of the link and not the actual visual styling of the text.

looks like this should be included in 2.0.0

@jessesquires Hi is this bug get resolve???I am downloading the latest sample..but still gets problem.

Hello is there a fix for this?  I amrunning into the same problem.

The issue is still present in 2.0.0.
It's reproducible if you press and hold slightly to the left (assuming LTR text) of the the truncation token.

Pls fix this error

Solved this issue: https://github.com/TTTAttributedLabel/TTTAttributedLabel/issues/519#issuecomment-752873366


Roadmap

For visibility cc'ing: @sayakpaul
Tech stack

Gemma 2B/7B models

Hugging Face(HF) transformers for modeling
HF peft for parameter efficient fine-tuning
HF text-generation-inference for serving
HF Hub for versioning fine-tuned models
HF trl for streamlining SFT fine-tuning process

Pipeline

(manual) Manually launching the LLM fine-tuning

(automate) Evaluating the fine-tuned LLM

(conditional) based on the results from the step 2.

(automate) if the performance is any better, replace the existing model on the target HF Model Repo
(manual) if the performance is bad, run a script to generate synthetic data that look similar to the existing ones

Go back to the step 1

(automate) from step 3, if the performance is determined to be good enough, the currently serving model by TGI will be replaced accordingly.

we assume TGI is running on somewhere in the cloud like dstack or HF Space repo with Hugging Chat.

Action items
0. dataset preparation

the target dataset is the filtered no-robots dataset with only Coding category.
the no-robots dataset comes with train and test splits, so we are going to use the train split for fine-tuning and test split for evaluating the fine-tuned model.

1. fine-tuning script

this script should look similar to the standard transformers + peft + trl usage (guide link1, link2).
additionally, we could add an option to run fine-tuning process on Vertex AI (guide link).

2. evaluation script

this script leverages service LLM (Gemini, ChatGPT, ...) to evaluate fine-tuned model on the test split.
each item in the test split contains messages which is basically a pair of user and assistant's contents.

this script let the fine-tuned model generate output of each item.
then, ask to service LLM how the generated output is similar to the recorded output/answer in 1-10 scale score.
aggregate scores on all items, then calculate the average score.

this script decides if the average score is any better

than a given threshold (baseline)
than the score recorded from the previous model

if yes, this script drops the following outputs (optionally, direct pushing to HF Model repo)

model checkpoint
evaluation results (average score, score on individual items)
(TBD) how to push the model to a HF Model repo should be discussed in detail

3. Synthetic data generation script

this script leverages service LLM (Gemini, ChatGPT, ...) to generate synthetic data and add it to the existing train split. This script shouldn't look into the test split at all, but generate similar data by referring to the train split.

4. Deployment script

this script serves the fine-tuned model on {local, cloud} environment with TGI framework.

Our primary target cloud environment is HF Space Repo with Hugging Chat template and dstack.

if there is already being served TGI instance, this script will terminate it and replace with the new one (new model)

Will start with Step 1.

@sayakpaul not step 0 but step 1, right?

I believe you should be able to indendepently work on Step 2? @deep-diver

@sayakpaul that is correct!

@sayakpaul not step 0 but step 1, right?

Step 0 will be coupled with Step 1.

Cool! Let's do this

Dataset created: https://huggingface.co/datasets/sayakpaul/no_robots_only_coding
from datasets import load_dataset

dataset = load_dataset("HuggingFaceH4/no_robots")
dataset["train_sft"] = dataset["train_sft"].filter(lambda example: example["category"] == "Coding")
dataset["test_sft"] = dataset["test_sft"].filter(lambda example: example["category"] == "Coding")

dataset.push_to_hub("no_robots_only_coding")

Going to spin up an experiment now.

Used the following config to start the training run with https://github.com/huggingface/alignment-handbook:
# Model arguments
model_name_or_path: google/gemma-7b
model_revision: main
tokenizer_name_or_path: philschmid/gemma-tokenizer-chatml # Custom tokenizer with <|im_start|> and <|im_end|> tokens
torch_dtype: bfloat16
use_flash_attention_2: true

# LoRA arguments
load_in_4bit: true
use_peft: true
lora_r: 16
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj

# Data training arguments
dataset_mixer:
  sayakpaul/no_robots_only_coding: 1.0
dataset_splits:
- train_sft
- test_sft
preprocessing_num_workers: 12

# SFT trainer config
bf16: true
dataset_kwargs:
  add_special_tokens: false  # We already wrap <bos> and <eos> in the chat template
  append_concat_token: false # No need to add <eos> across samples
do_eval: true
evaluation_strategy: epoch
gradient_accumulation_steps: 2
gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false
hub_model_id: gemma-2b-sft-qlora-no-robots
hub_strategy: every_save
learning_rate: 2.0e-04
log_level: info
logging_steps: 5
logging_strategy: steps
lr_scheduler_type: cosine
max_seq_length: 2048
max_steps: -1
num_train_epochs: 1
output_dir: data/gemma-2b-sft-qlora-no-robots
overwrite_output_dir: true
per_device_eval_batch_size: 8
per_device_train_batch_size: 4
push_to_hub: true
report_to:
- tensorboard
save_strategy: "steps"
save_steps: 100
save_total_limit: 1
seed: 42
warmup_ratio: 0.1

And then ran (from the root of https://github.com/huggingface/alignment-handbook):
ACCELERATE_LOG_LEVEL=info accelerate launch --config_file recipes/accelerate_configs/multi_gpu.yaml scripts/run_sft.py recipes/zephyr-7b-gemma/sft/config_qlora.yaml

Model: https://huggingface.co/sayakpaul/gemma-2b-sft-qlora-no-robots/
It's incorrectly named. Should have been 7b.

cool! gonna experiment with that model.
in the meantime, I have created a prompt to assess the outputs.
Given an instruction and two responses—one generated by a human and the other by a language model—I'm seeking to evaluate how closely the language model's response mirrors the human-generated one. Additionally, I want to assess the accuracy and relevance of the language model's response to the original instruction.

Instruction:
I have 2 lists below. Can you show me how to make a bar chart from them using Python?\n\nlist1 = [\"USA\", \"China\", \"Russia\", \"Brazil\", \"Japan\"]\nlist2 = [22, 53, 32, 45, 56]

Human Response:
To install it with PIP, open your command line and enter: pip install matplotlib
To install it with Conda, enter: conda install -c conda-forge matplotlib
Then you can import the module into your Python code. Here is the full code:
from matplotlib import pyplot as plt

list1 = ["USA", "China", "Russia", "Brazil", "Japan"]
list2 = [22, 53, 32, 45, 56]

plt.bar(list1, list2)
plt.show()
This will display a bar chart with list1 on the x-axis and list2 on the y-axis.

Language Model Response:
import matplotlib.pyplot as plt

# Create the bar chart
plt.bar(list1, list2)

# Set labels and title
plt.xlabel("Country")
plt.ylabel("Number of people")
plt.title("Number of people in each country")

# Show the chart
plt.show()

Please analyze the similarity between the Human Response and the Language Model Response on a scale of 1 to 100, where 1 indicates no similarity and 100 indicates identical responses. Then, evaluate the Language Model Response for its precision and relevance in answering the given Instruction on the same 1 to 100 scale.

Provide the assessment in the following JSON format:

{
  “similarity_assessment”: {
    “score”: [Insert similarity score here],
    “reason”: [Insert how the similarity score is determined],
  },
  “precision_assessment”: {
    "score": [Insert precision score here],
    “reason”: [Insert how the precision score is determined],
  }
}

then I get the following output:
{
   “similarity_assessment”: {
      “score”: 80,
      “reason”: The responses include the essential elements of creating a bar chart using
matplotlib in Python, such as importing the necessary library, specifying the data for the x and y axes, and displaying the chart. The main difference is that the human response provides more detailed installation instructions, while the language model response focuses on the code for generating the bar chart.
   },
   “precision_assessment”: {
      "score": 95,
      “reason”: The language model's response accurately and concisely provides the steps needed to create a bar chart using Python. It includes the relevant code, sets labels and a title for the chart, and displays it. The response demonstrates a good understanding of matplotlib and adheres to the original instructions effectively.
   }
}

If there is something you want to add your thoughts on, please let me know!

Thank you! Will set up a GitHub Action for this. Could you set the Gemini API Key in the secrets?

sure @sayakpaul
but, first let me write this in a script.
and, the model you shared "https://huggingface.co/sayakpaul/gemma-2b-sft-qlora-no-robots/" is already fine-tuned version?

Yes it is. But the number of samples is incredibly low so the performance might be bad. But I think that is not a blocker.

very nice :)
I will drop the work on my side tomorrow!

@sayakpaul
created a notebook
: https://colab.research.google.com/drive/1xvfWWjPNRLubsRcOVvumJXzDcvNrVZiW?usp=sharing
If you confirm it is OK, I will make a PR for this notebook first. Then, we can work on making things as a standalone codebase.

NOTE: the evaluation prompt currently works super bad. The model's output simply repeats the same word Survivor many times, but Gemini assessed it with about 80-90 score

def get_model(model_name):
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForCausalLM.from_pretrained(
        model_name, torch_dtype=torch.bfloat16, load_in_4bit=True, device_map="auto"
    )

    return tokenizer, model

I used a different tokenizer. Refer to the comment. We should also follow how chat template is exactly applied to the inputs.
https://github.com/huggingface/alignment-handbook/blob/ba7e0e4fcaa60b8ffa35373de230d5c5cd2eccde/src/alignment/model_utils.py#L65
I will let you figure this out first and then we can take a closer look. WDYT?

Sounds cool!

@sayakpaul I don't fully understand the second part. Are you saying that get_tokenizer() is preferred way to get tokenizer?

Yes, because that ensures that the same tokenizer is used during training and validation.


Support golden file image diffing in testWidgets

After https://github.com/flutter/flutter/pull/16758 lands, we'll ostensibly have the ability to capture a screenshot in a widget test running via flutter_tester and compare it to a golden file, as follows:
testWidgets('test', (WidgetTester tester) async {
  await tester.pumpWidget(new MyApp());

  // Get the bytes of the screenshot
  RenderRepaintBoundary repaintRenderBoundary = tester
      .allRenderObjects
      .firstWhere((RenderObject object) => object is RenderRepaintBoundary);
  ui.Image image = await repaintRenderBoundary.toImage();
  ByteData bytes = await image.toByteData(format: ui.ImageByteFormat.png);

  // Get the bytes of the golden file
  File golden = new ui.File('/path/to/golden.png');
  List<int> goldenBytes = await golden.readAsBytes();

  // Fail the test if the screenshot isn't within tolerance of golden
  ...
});

But doing so causes the following exception due to flutter_test's use of FakeAsync:
  TimeoutException after 0:00:05.000000: Test timed out after 5 seconds.
  dart:async  _startMicrotaskLoop

A hacked up attempt to call _fakeAsync.flushTimers() ends up yielding the same exception when we await that call.

we should just have a "tester.compareToGolden" or something that jumps through the appropriate hoops.

Within Google, there's an existing system set up to do golden file comparisons (it saves results to a location where they can be visualized, etc.).  It has roughly the following API:
typedef Future<List<int>> ScreenshotTaker();

class GoldenFileComparisonService {
  /// User-defined function for taking a screenshot.
  ScreenshotTaker takeScreenshot;

  /// Makes a diff request against a golden image and publishes the result.
  Future<Null> diffScreenshot(String key);
}

Then test code is expected to hook into the API like so:
void main() {
  GoldenFileComparisonService service;

  setUp(() {
    service = new GoldenFileComparisonService();
  });

  testWidgets('matches golden file', (WidgetTester tester) async {
    service.takeScreenshot = () async {
      // Use `tester` to get image bytes
    };

    await service.diffScreenshot('imageKey.png');
  });
}

So at the heart of it, the service expects two things:

We have a method that can return Future<List<int>> for the screenshot bytes
We can safely await arbitrary futures in our test code

We potentially have some amount of influence over that API, but we do want to delegate to the service to do the comparison, because it exports its findings in such a way that they can be visualized in test results (providing a suite of UI tools for visualizing the differences and approving the test files as the new goldens).

SGTM.
Rather than --local-goldens how about just putting the checkout in bin/cache? That way everyone has it in a known location.
We'll have to make sure the flutter_test_config.dart logic is protected by a lock to prevent multiple simultaneous checkouts or some such.

Currently blocked on https://github.com/dart-lang/sdk/issues/32982

As of https://github.com/flutter/flutter/pull/17141 (out for review), the only item left is to build a Flutter-specific test configuration in packages/flutter/test/flutter_test_config.dart that sets up a custom comparator that loads its golden files from a newly created flutter/goldens repo.

Ok, marking this as done.  There are still several improvements we should make, being tracked in:

https://github.com/flutter/flutter/issues/17258 -- Implement better logic in golden file comparators
https://github.com/flutter/flutter/issues/17259 -- Allow centralized place where tests can output files
https://github.com/flutter/flutter/issues/17260 -- Visualize golden image file failures in test output
https://github.com/flutter/flutter/issues/17262 -- Test golden files without shadows

This this in production already?

@FrankApiyo Just found this:
https://medium.com/flutter-community/flutter-golden-tests-compare-widgets-with-snapshots-27f83f266cea


problems with posix/GNU API to strerror_r

I got following error during compiling on alpine linux v3.5 (for use in docker)
hazelcast-cpp-client/external/include/asio/asio/include/asio/impl/error_code.ipp:110:22: error: could not convert 'strerror_r(value, ((char*)(& buf)), sizeof (buf))' from 'int' to 'std::__cxx11::string {aka std::__cxx11::basic_string}'
return strerror_r(value, buf, sizeof(buf));
The issue is corrected in official boost
https://github.com/chriskohlhoff/asio/commit/443bc17d13eb5e37de780ea6e23157493cf7b3b9
after some discussions
https://github.com/chriskohlhoff/asio/issues/94
A similar problem was in
hazelcast-cpp-client/hazelcast/src/hazelcast/util/Util.cpp:137:44: error: invalid conversion from 'int' to 'char*' [-fpermissive]
char *errStr = ::strerror_r(errnum, strerrbuf + numChars, buflen - numChars);
I applied similar solution with static functions - see attachment.
util_cpp.txt
Afterwards the client compiles well, but I have not run the tests.

Can you share the docker image if not private so that I can test it as well?

Here is the Dockerfile and docker-compose-test.yml files with a short description, how to run the compiling and tests.
testhz.zip

@wsw2016
I tried your docker image. I got the following error: make[2]: *** [CMakeFiles/HazelcastClient3.9.1-SNAPSHOT_64.dir/build.make:855: CMakeFiles/HazelcastClient3.9.1-SNAPSHOT_64.dir/hazelcast/src/hazelcast/client/aws/impl/DescribeInstances.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/normaluser/hazelcast-cpp-client/hazelcast/include/hazelcast/client/internal/socket/TcpSocket.h:46:0,
from /home/normaluser/hazelcast-cpp-client/hazelcast/src/hazelcast/client/connection/Connection.cpp:33:
/usr/include/sys/errno.h:1:2: error: #warning redirecting incorrect #include <sys/errno.h> to <errno.h> [-Werror=cpp]
#warning redirecting incorrect #include <sys/errno.h> to <errno.h>
^~~~~~~
cc1plus: all warnings being treated as errors
It found this: https://github.com/armon/bloomd/issues/40
It seems as if we need special porting for alpine linux (e.g. not including sys/errno.h), is that correct?
Furthermore, our library is compiled with c++98 to be compatible with older compiler versions. Hence, we do not normally test with -std=c++11 flag. Hence, it is better not use this flag.

We switched to C++11, and use official boost library as dependency, hence closing this issue.


Feature/base typography styles

Hey I finished the styling of the headings, code, paragraphs, etc.
I left all my changes in the old lib folder so you can take a look at the structure, and tell me if I need to change anything, etc.
Point the gulp task to that old folder so you can see the changes live. So please do that so you can see it live.

That sounds good! I'll  make the changes and resubmit the PR.
PS: What do you think about adding an .editorcofigto the project. Something simple like this:
root = true

[*]
charset = utf-8
end_of_line = lf
indent_size = 2
indent_style = tab
insert_final_newline = true
trim_trailing_whitespace = true

[*.md]
trim_trailing_whitespace = false

Hey I updated and the architecture of the project. Take a look and let me know what you think


Align coding style

The last instantiation of call line #42 had mismatched showing filter's return value with the two previous instantiations lines #19, #30 from the same code block.

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

Agreed. I must mention, the reasoning to elect lookup ease from retaining the explicit return value was this being canonical documentation.


Read FIDs into index of data frame

Return ndarray of FIDs from read() in addition to fields and geometry.  Use this for the index in read_dataframe.

May want to put this behind a keyword because this is unnecessary overhead for some drivers (e.g., Shapefile).


Cleanup toyota tuning

I like this, it should add more friction to create new tunes. After merging this, we should do the same for at least Honda and Hyundai.


Error creating new order :: Cannot issue for "": Domain name is empty

Relevant software and versions

chef-client: 13.9.4
acme cookbook: ~> 4.1.1

When I ran my cookbooks, it failed with the following error:
================================================================================     Error executing action create` on resource 'acme_certificate[*******.com.ar]'
================================================================================
Acme::Client::Error::RejectedIdentifier
---------------------------------------
Error creating new order :: Cannot issue for "": Domain name is empty

Cookbook Trace:
---------------
/var/chef/cache/cookbooks/acme/libraries/acme.rb:47:in `acme_order_certs_for'
/var/chef/cache/cookbooks/acme/resources/certificate.rb:76:in `block in class_from_file'

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/AppLoadBalancer-acmev2/recipes/lb_reverse_proxy.rb

157:   acme_certificate host['server_name'] do
158:     crt     "#{node['AppLoadBalancer']['cert_path']}/#{host['server_name']}.crt"
159:     #Se deshabilita por la nueva version de la receta de ACME - AP 20200514
160:     #chain   "#{node['AppLoadBalancer']['cert_path']}/#{host['server_name']}-chain.crt"
161:     key     "#{node['AppLoadBalancer']['cert_path']}/#{host['server_name']}.key"
162:     wwwroot '/var/lib/haproxy'
163:     alt_names ["#{host['server_alias']}"]
164:     owner node['AppLoadBalancer']['haproxy']['user']
165:     group node['AppLoadBalancer']['haproxy']['group']
166:     only_if { host['balancer']['https'] }
167:   end
168:

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/AppLoadBalancer-acmev2/recipes/lb_reverse_proxy.rb:157:in `block in from_file'

acme_certificate("*******.com.ar") do
  action [:create]
  default_guard_interpreter :default
  declared_type :acme_certificate
  cookbook_name "AppLoadBalancer-acmev2"
  recipe_name "lb_reverse_proxy"
  crt "/etc/ssl/lojack/*******.com.ar.crt"
  key "/etc/ssl/lojack/*******.com.ar.key"
  wwwroot "/var/lib/haproxy"
  alt_names [""]
  owner "haproxy"
  group "haproxy"
  cn "*******.com.ar"
  only_if { #code block }
end`

Site name was replaced with ******* for confidentiality purpose.

Line 163 was misspelled... my bad


Got HTML content but no text found.

Hi guys,
Is there a way to get notified on this change ? Basically if, there isn't any text found, Changedetection doesn't send any notification, even though the CSS selector is valid as a filter...Am I missing something here ?
Basically you put the CSS selector as a CSS/JSON/XPATH Filter, and you should get a notification when the CSS selector you typed is not found on the page, and  another notification when the CSS selector is found on the page regardless if any text is available on the webpage...when changedetection finds my CSS selector I get this error "Got HTML content but no text found." ??  but no notification via Telegram...
Shouldn't changedetection send the notification even though it returns this status : "Got HTML content but no text found." ??? That is a change on the webpage, even though no text is found...shouldn't I get a notification ?

Also even with the box from the screenshot bellow checked, you do not get a notification when your filter isn't found on the webpage, so that is definitely a bug...

Please follow the issue template...


Check current notebook cell when refreshing intellisense cache

This PR fixes #21252

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.002%) to 42.16%

Files with Coverage Reduction
New Missed Lines
%

extensions/notebook/src/dialog/configurePython/configurePathPage.ts
1
57.02%

Totals

Change from base Build<PHONE_NUMBER>:
-0.002%

Covered Lines:
28567

Relevant Lines:
62933

💛  - Coveralls


Where did plotFuncs and plotFuncsDer go?

Lots of things import these from HARK.utilities, but they seem to be missing from utilities.py.

the functions are renamed to plot_funcs and plot_funcs_der according to the PEP8 convention.


fix: request type annotation

Orders expires_at allows a datetime or a time interval e.g. (P)2d as an input but not an integer, which Days was an alias for.
Days is also used in the market data API for the to params and manually converted to p{x}d. We could also adjust it there

Didn't see the api handler automatically converts the integer into a interval


AWS Seoul region support

[add] ap-northeast-2 region

Wow..... that was quick.... You need to update the Region.h and Region.cpp
file in core and then update the endpoint computation in the generator.
Most of those files you manually updated are generated, so the fix needs to
happen there.
Thanks for your work on this.
On Jan 6, 2016 6:40 PM, "Seungmo Koo"<EMAIL_ADDRESS>wrote:

[add] ap-northeast-2 region
You can view, comment on, or merge this pull request online at:
https://github.com/awslabs/aws-sdk-cpp/pull/68
Commit Summary

*added: new region support - AWS Seoul region

File Changes

M aws-cpp-sdk-autoscaling/source/AutoScalingEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-0 (2)
M aws-cpp-sdk-cloudformation/source/CloudFormationEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-1 (2)
M aws-cpp-sdk-codedeploy/source/CodeDeployEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-2 (2)
M aws-cpp-sdk-cognito-identity/source/CognitoIdentityEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-3 (2)
M aws-cpp-sdk-cognito-sync/source/CognitoSyncEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-4 (2)
M aws-cpp-sdk-core-tests/aws/RegionMapperTest.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-5 (1)
M aws-cpp-sdk-core/include/aws/core/Region.h
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-6 (1)
M aws-cpp-sdk-core/source/Region.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-7 (2)
M aws-cpp-sdk-datapipeline/source/DataPipelineEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-8 (2)
M aws-cpp-sdk-dynamodb/source/DynamoDBEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-9 (2)
M aws-cpp-sdk-ec2/source/EC2Endpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-10 (2)
M aws-cpp-sdk-ecs/source/ECSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-11 (2)
M aws-cpp-sdk-elasticache/source/ElastiCacheEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-12 (2)
M aws-cpp-sdk-elasticbeanstalk/source/ElasticBeanstalkEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-13 (2)
M aws-cpp-sdk-elasticfilesystem/source/EFSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-14 (2)
M
aws-cpp-sdk-elasticloadbalancing/source/ElasticLoadBalancingEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-15 (2)
M aws-cpp-sdk-elasticmapreduce/source/EMREndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-16 (2)
M
aws-cpp-sdk-elastictranscoder/source/ElasticTranscoderEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-17 (2)
M aws-cpp-sdk-email/source/SESEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-18 (2)
M aws-cpp-sdk-glacier/source/GlacierEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-19 (2)
M aws-cpp-sdk-kinesis/source/KinesisEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-20 (2)
M aws-cpp-sdk-kms/source/KMSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-21 (2)
M aws-cpp-sdk-lambda/source/LambdaEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-22 (2)
M aws-cpp-sdk-logs/source/CloudWatchLogsEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-23 (2)
M aws-cpp-sdk-mobileanalytics/source/MobileAnalyticsEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-24 (2)
M aws-cpp-sdk-monitoring/source/CloudWatchEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-25 (2)
M aws-cpp-sdk-opsworks/source/OpsWorksEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-26 (2)
M aws-cpp-sdk-rds/source/RDSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-27 (2)
M aws-cpp-sdk-redshift/source/RedshiftEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-28 (2)
M aws-cpp-sdk-s3/include/aws/s3/model/BucketLocationConstraint.h
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-29 (1)
M aws-cpp-sdk-s3/source/S3Endpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-30 (2)
M aws-cpp-sdk-s3/source/model/BucketLocationConstraint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-31 (7)
M aws-cpp-sdk-sdb/source/SimpleDBEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-32 (2)
M aws-cpp-sdk-sns/source/SNSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-33 (2)
M aws-cpp-sdk-sqs/source/SQSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-34 (2)
M aws-cpp-sdk-sts/source/STSEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-35 (2)
M aws-cpp-sdk-swf/source/SWFEndpoint.cpp
https://github.com/awslabs/aws-sdk-cpp/pull/68/files#diff-36 (2)

Patch Links:

https://github.com/awslabs/aws-sdk-cpp/pull/68.patch
https://github.com/awslabs/aws-sdk-cpp/pull/68.diff

—
Reply to this email directly or view it on GitHub
https://github.com/awslabs/aws-sdk-cpp/pull/68.

Yep! just fixed RegionEndpointMapper.java
Thanks for adding AWS Seoud region :+1:

can you shoot me a pull request without the updates to the clients? I just need the client generator update and the region.h and region.cpp updates.

b.t.w. you can use endpoint override and authenticationRegion in the client config anytime you need a region we haven't explicitly called out.

OK, I fixed this PR in the contribution guidelines.
3 files (region.h, region,cpp, and regionendpointmapper.java) are changed eventually.

One last thing. Could you rebase this into one commit?

done.
(but your last commit (contribution guidelines) has been included while squashing my commits)
anyway, thanks for your kindness.


How to plot two combined DateTime range into one chart?

Hi, I love this crate for plotting. But after reading all codes under examples directory, I wonder how I can draw two sepreted and not overlaped time range into one chart without some gap between my data.
For example, I have two sets of data:
((time1..time2), y) and ((time3..time4), y)
I want to draw them in one chart.
I can achieve this by split_horizontally() the root drawing area, and draw two chart, one with y-axis and label x-axis start from time1 to time2, the other does not have y-axis and label and with x-axis start from time3 to time4. This "looks like" my two data sets are drawed together. But I think this is not a good way.

Yep, that's the work around currently. But this is a nice feature to support


mkgshmm does not install

The Makefile.in/Makefile for mkgshmm does not include instructions to build mkgshmm from mkgshmm.in. When "make install" is issued, the process fails on the
@INSTALL_PROGRAM@ $(TARGET) @bindir@
step, as the $(TARGET) ("mkgshmm") does not exist.

Please ignore; it appears that I did something wrong.


[Linux] Enable frame pointers when building Swift libraries.

Turn on frame pointers for the Swift runtime libraries.  This makes backtraces that go through the runtimes more reliable without having to parse DWARF data.
rdar://116112040

@swift-ci Please test

Explanation: Set -fno-omit-frame-pointer when building the Swift runtime libraries.
Original PR: #68986
Reviewed by: TBD
Resolves: rdar://116112040
Tests: Adds a test that checks that we get a decent backtrace when someone calls fatalError().

@swift-ci Please test


Update tsaoptions.json (#4464)

Update tsaoptions.json to update area path for TSA bug filing.
CP of PR #4464 to 1.4
--
A microsoft employee must use /azp run to validate using the pipelines below.
WARNING:
Comments made by azure-pipelines bot maybe inaccurate.
Please see pipeline link to verify that the build is being ran.
For status checks on the main branch, please use TransportPackage-Foundation-PR
(https://microsoft.visualstudio.com/ProjectReunion/_build?definitionId=81063&_a=summary)
and run the build against your PR branch with the default parameters.

/azp run


Api-integration-with-lambda Project added

Adding a project i build in python

/assign
Give this a hacktober tag

Go ahead, @ayushlohmod


genesis: make genesis a transaction type

Today Genesis is just a bunch of objects that get dumped into a node's database at startup. Instead we'd like to make genesis more of a first-class citizen in sui by making it be a specific transaction type.
Also include work from #2769, which is folded into this:

As the reconfiguration process becomes activated, a lot of our reliance on the genesis blob can be refactored away, especially in tests. See https://github.com/MystenLabs/sui/pull/2746#issuecomment-1168231545.

Closed out #2769, combining that into this change here.


Update mouse enter/exit on TouchEvent

It was only updated on Drag/Motion this PR updates the state on TouchEvents closes https://github.com/godotengine/godot/issues/1212
Tested on Android.
The red area should turn green on touch. It stays red without this change.
New Game Project55.zip

This is good and necessary to avoid hacks you'll otherwise need for things to work across both desktop and mobile with on_mouse_enter/exit, this still alive?

There shouldn't be touch-specific code for GUI. As long as touch-to-mouse translation is enabled (the default), mouse events are sent. Therefore GUI code only has to deal with them.
Probably the right fix would be to send a mouse move event when the touch gesture starts.
Or is it the other way round? I mean, updating GUI hovering when the mouse button is pressed (for the mouse, that's usually not needed because surely the move event will have timely reported the pointer position before the click).

Moving to 4.0 milestone as it's not ready to merge yet as per https://github.com/godotengine/godot/pull/21566#issuecomment-502222040.

@AlexHolly Is this still desired? If so, it needs to be rebased on the latest master branch.
If not, abandoned pull requests will be closed in the future as announced here.

This PR has not received any new commits for almost a 2 years and is abandoned, closing.


bug: Cuda not used

Cortex version
<IP_ADDRESS>
Describe the Bug
Cortex for Windows downloaded from the link provided here: https://cortex.so/. REST API works as decribed. Chat/Completion works also, but does not use CUDA. CUDA 12.2 is installed and worked well with Nitro.ex e and also Jan.
Steps to Reproduce
start cortex.exe
Init llamma.cpp engine
(pull mode) llamma 3.1
Start model llamma 3.1
Chat (summarize a text with about 200 words)
Works but takes 11 minutes, which looks like a CPU attempt for me. Jan makes the same in some seconds.
if i start nvidia-smi is does not show GPU activity, during completion. if you repeat the chat request it takes only 2min40 sec, but this is still much too slow.
Screenshots / Logs
No response
What is your OS?

[ ] MacOS
[X] Windows
[ ] Linux

What engine are you running?

[X] cortex.llamacpp (default)
[ ] cortex.tensorrt-llm (Nvidia GPUs)
[ ] cortex.onnx (NPUs, DirectML)

Hey @DAVIDSystems apologies for the confusion, we (un)released Cortex, and are currently rewriting the entire thing in c++.
The new stable launch is ETA end October or Nov.
There is a nightly build right now that has breaking changes quite often. If you wanted to help us test it. Can I ask you to try the latest build from https://discord.gg/nGp6PMrUqS instead?


Enhance Container Component with additional styling properties

Description:
We need to enhance the Container component by adding the following properties to improve its customization and styling options.
Properties to Add:

Responsiveness:

Min Width
Width
Max Width

Border:

Border With
Border Type includes (solid, dotted, and dashed)
Border Color
Border Radius

Overflow Inclide  (auto, scroll, hidden)
StylingBox (margin, padding)

Considerations:

Overlay (opacity, blur):
Affix property for Container

Overlay property has been moved to ticket #1892.


Enable custom WebDriver

When I run splinter with Google Chrome Canary installed, I got this error
selenium.common.exceptions.WebDriverException: Message: u'unknown error: cannot find Chrome binary\n  (Driver info: chromedriver=2.10.267517,platform=Mac OS X 10.9.2 x86_64)'

It is possible to configure ChromeWebDriver to use Canary, though splinter currently does not expose this functionality. Expose a Driver interface to customize WebDrivers.

Custom browser paths for chrome can be set via the Chrome Options object:
from selenium import webdriver

chrome_options = webdriver.chrome.options.Options()
chrome_options.binary_location = "/path/to/canary"
browser = Browser('chrome', options=chrome_options)


No support for yolov5v6

Hi,
I got the following error when I try to convert the yolov5v6.onnx
[CVU-Info] Backend: Tensorrt-<IP_ADDRESS>-cuda
[CVU-Info] Building and Optimizing TRT-Engine for input_shape=(384, 640). This might take a few minutes for first time.
[TensorRT] INFO: [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 0, GPU 353 (MiB)
[CVU-Info] Platform has fp16 support. Setting fp16 to True
[TensorRT] WARNING: onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
In node 279 (importRange): UNSUPPORTED_NODE: Assertion failed: inputs.at(0).isInt32() && "For range operator with dynamic inputs, this version of TensorRT only supports INT32!"
[TensorRT] ERROR: 4: [network.cpp::validate::2411] Error Code 4: Internal Error (Network must have at least one output)
Traceback (most recent call last):
  File "detect.py", line 126, in <module>
    not opt.no_auto_install)
  File "detect.py", line 74, in detect_image
    preds = model(image)
  File "/usr/local/lib/python3.7/dist-packages/cvu/detector/yolov5/core.py", line 102, in __call__
    outputs = self._model(processed_inputs)
  File "/usr/local/lib/python3.7/dist-packages/cvu/detector/yolov5/backends/yolov5_tensorrt.py", line 307, in __call__
    self._load_model(self._weight)
  File "/usr/local/lib/python3.7/dist-packages/cvu/detector/yolov5/backends/yolov5_tensorrt.py", line 142, in _load_model
    self._input_shape)
  File "/usr/local/lib/python3.7/dist-packages/cvu/detector/yolov5/backends/yolov5_tensorrt.py", line 262, in _build_engine
    trt_engine_file.write(engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'

Any instructions how to convert?

Hello @NaveedAI, we have identified the issue and are working on a solution.

@NaveedAI I think your onnx model is with dynamic shapes?
You need to export onnx model without the --dynamic flag from ultralytics/yolov5 and this would work just fine. The problem you mentioned is a known issue with latest tensorrt and onnx versions.

So, export your onnx model with yolov5 python export.py --imgsz "height" "width" (without the dynamic flag).
And use the generated onnx model to create the tensorrt engine.

These are my logs for reference.:
static/yolov5s6.onnx
(384, 640, 3)
(384, 640, 3)
[CVU-Info] Building and Optimizing TRT-Engine for input_shape=(384, 640). This might take a few minutes for first time.
[06/11/2022-10:41:34] [TRT] [I] [MemUsageChange] Init CUDA: CPU +313, GPU +0, now: CPU 362, GPU 244 (MiB)
[06/11/2022-10:41:34] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 362 MiB, GPU 244 MiB
[06/11/2022-10:41:34] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 497 MiB, GPU 278 MiB
[CVU-Info] Platform has fp16 support. Setting fp16 to True
[06/11/2022-10:41:34] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[06/11/2022-10:41:35] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +518, GPU +224, now: CPU 1071, GPU 502 (MiB)
[06/11/2022-10:41:35] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +115, GPU +54, now: CPU 1186, GPU 556 (MiB)
[06/11/2022-10:41:35] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[06/11/2022-10:42:40] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[06/11/2022-10:50:18] [TRT] [I] Detected 1 inputs and 5 output network tensors.
[06/11/2022-10:50:18] [TRT] [I] Total Host Persistent Memory: 169040
[06/11/2022-10:50:18] [TRT] [I] Total Device Persistent Memory: 23151104
[06/11/2022-10:50:18] [TRT] [I] Total Scratch Memory: 0
[06/11/2022-10:50:18] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 31 MiB, GPU 225 MiB
[06/11/2022-10:50:18] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 33.7113ms to assign 11 blocks to 139 nodes requiring 18726403 bytes.
[06/11/2022-10:50:18] [TRT] [I] Total Activation Memory: 18726403
[06/11/2022-10:50:18] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 1761, GPU 816 (MiB)
[06/11/2022-10:50:18] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +1, GPU +8, now: CPU 1762, GPU 824 (MiB)
[06/11/2022-10:50:18] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +24, GPU +32, now: CPU 24, GPU 32 (MiB)
[CVU-Info] Engine serialized and saved !
[06/11/2022-10:50:18] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +10, now: CPU 1749, GPU 800 (MiB)
[06/11/2022-10:50:18] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 1749, GPU 808 (MiB)
[06/11/2022-10:50:18] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +39, now: CPU 24, GPU 71 (MiB)

Total Time Taken:  525.28 seconds.
Engine File Saved at:  static/yolov5s6.engine

And these are the outputs after inference:
root@a20af9be6c00:/home# python examples/example_video_simple.py -v static/zidane.jpg
[CVU-Info] Building and Optimizing TRT-Engine for input_shape=(384, 640).  Feel free to re-open if the suggestion does not resolve your issue


Another approach to PyObject handling

seems custom class implementation works well. Should we try to use same approach for python native types?
idea behind class implementation:

developer does not have access to class instance, only to reference, reference is tight to gil lifetime
to store reference to instance to rust struct, instance ref needs to be converted to Py<CLS> instance.

same approach could be used for native objects:

access to PyObject can be restricted to &'p PyObject, 'p is gil livetime
to store ref, it would need to be converted to Py<PyObject>. alternative: instead of Py<PyObject>, PyObject itself can be smart pointer, (it can be universal py pointer),
but specific classes like PyList or PyTuple can be converted to Py<PyTuple> for storage purpose
or to PyObject pointer.

pros: for native object references:

PyClone can be replace with normal Clone
no need to acquire Gil for reference counting
no need for py parameter for any function call.

@messense

That's interesting. I would prefer PyObject pointer, it seems less confusing. The overall design should focus on safe and ergonomic.
I'm ok with passing py parameter around, it actually makes it clear that you are dealing with Python objects, but not opposed to remove it to make function signatures simpler.

I don't like py, it is to invasive. Because of it we have to create a lot of traits. With standard trait learning curve would be simpler.

That's right.

I played with this approach. in genera it is useful. but it become not very good when
native type returns other native types, it has to return some kind of Py<PyObject> instance, which makes everything not very good.

some explanation. I want to disallow native object instances in free form. i.e. you can not
specify function that accept PyString or PyTuple

fn my_func(&self, py: Python, args: PyTuple) -> PyResult<()>;

in my previous attempt I tried to add lifetime, so it would look like:
fn my_func<'p>(&'p self, py: Python<'p>, args: PyTuple<'p>) -> PyResult<()>;

but it ended up being very non-ergonomic. first of all any type had to include lifetime. second mirror type required, each native type with lifetime would require type without that just holds reference to ffi::PyObject.
there was another attempt that introduced Py<'p, T> pointer, it was even less ergonomic.
so what I am trying to now. If I disallow instances at all and allow only references then it seems works much better. i.e
fn my_func(&self, py: Python, args: &PyTuple, kwargs: &PyDict) -> PyResult<()>;

but we still need ability to store py objects in rust structs, so we need to introduce special pointer
Py<T> that needs to be Sync and Send and we should be able to extract object references:
impl<T> for Py<T> {
    fn as_ref(&self, py: Python) -> &T;
}

that is actually easy, but if py class api can return other py instance, then it becomes a problem.
here is PyList api:

impl PyList {

    fn new(py: Python, items: &[PyObject]) -> Py<PyList>;

    fn get_item(&self) -> PyObject;
}

new method, because free instances are disallowed we have to return Py<PyList>, but if you need to manipulate this list, you would immediately would do something like this:

let ob = PyList::new(py, &[...]);
let list = ob.as_ref(py);
list.set_item(...);
...
return ob;

I closed this ticket because of this annoyance. but then I realized, instead of Py<T> can we use reference fn new(py: Python, items: &[PyObject]) -> &PyList. in rust we need owner, we can not have free references. so I added simple api that can hold PyObject instances and then release all of them when GIL object get dropped.
https://github.com/PyO3/PyO3/blob/py-ptr/src/pythonrun.rs#L103
Thoughts? Ideas?

That's a great explanation.
I was under the impression that mutable global variables are bad? And how is the performance impact?

I think performance should be the same. performance could be affected if you would use a lot of &PyObject refs otherwise performance should actually increase. in current implementation, PyObject's Drop impl acquire Gil, even if it recursive lock, it should have negative impact

so PyObject now is just smart pointer, all other native types are available only as references. py parameter is removed from most apis. I had to add PyInstance (I don't like name) which is just ref PyObject.
I don't really like PyInstance name, any ideas?
@messense

I don't like PyInstance either.
PyObjectRef, PyValue(doesn't make much sense)

I like PyObjectRef more, I'll change PyInstance to PyObjectRef


Flamerobin corrupt loaded sql file when saved again (empty)

Flamerobin corrupt loaded sql file when saved again.
I do not know if this is becaouse of encodding?
But if you load file and change anything and press save, it will empty the file on the HDD :-(
Try attached file (tested with newest filamerobin)
Przepięcie danych sprawozdania na nowe sprawozdanie2.txt

Hi, sorry but I couldn't replicate your problem. Can you reproduce and detail it to me?
I tried with 32 and 64 bits file

Just to reference the code, I think it could have an else statement if not saved, but this is not the cause of your problem
https://github.com/mariuz/flamerobin/blob/de8f182d405bba002056d7756be27fbe22e0d23d/src/gui/ExecuteSqlFrame.cpp#L1347-L1368

Looking at your attached code, it looks like error only possible inside
styled_text_ctrl_sql->SaveFile(filename) - if you have source code of that control, then maybe possible to catch this.
But first you must replicate problem on your side :/

maybe, have you tried in another partition or folder? (assuming some crazy permission error)
Also, if you "save as" overwriting or not the file does it work?
I cant try now, but as soon as I can I'll give you feedback

styled_text_ctrl_sql->SaveFile(filename) is from wxWidget

maybe, have you tried in another partition or folder?
no, if i without load file and write something it is saved without problems

Also, if you "save as" overwriting or not the file does it work?

tested now option "save as" new file is created but it is empty :/
So something in this file made this control(wxWidget) to save nothing :/
If it will not erease my file it will be "less problematic" but good that i have script backup from previous day :)

hello, again, can't reproduce here
Just out of curiosity, does your SQL history work?
Also, have you tried to save a simple 1-2 rows SQL, also in a new folder/partition?

Hi, yes sql history is working ok.
I have tried this file on third computer and the same problem.
Any other scripts are working ok, but this one not. I am really interested in fixing the problem, as i like to still trust flamerobin as i have trusted it for years.

FlameRobin 0.9.3 (git hash e601534) Unicode (x64)

Yes, tried with exactly this one release, saved the file
Downloaded the file Przepiecie.danych.sprawozdania.na.nowe.sprawozdanie2.txt, at "Desktop", "D:\custonFolder", "user\Documents".
I opened in Fr with database charset NONE, UTF8 and ISO8859_1 (shouldn't matter but anyway I tried) inserted a space at line 20, saved, reopened and nothing
I'm looking for how to show error message from wxWidgets, as soon as I found (I'm not a wx expert) I'll publish a snapshot for you to try
Thanks

I have tried this file on third computer and the same problem.

Just to be sure, have you tried using the file from this link you passed me? (downloaded and tried from this source, not "directly from your disk file"
https://github.com/mariuz/flamerobin/files/8876909/Przepiecie.danych.sprawozdania.na.nowe.sprawozdanie2.txt

Please, try this setup, I add a simple error dialog
https://github.com/mariuz/flamerobin/commit/165463b66c645984c1f7a37891b1a9d651a05fa7
Download and test please:
https://github.com/mariuz/flamerobin/releases/tag/<IP_ADDRESS>%23259

Tested, but the dialog not show up.
But one more thing happened, for a second i have seen in the same location file with .tmp extension
with exact same size as my loaded file. Then it dissapear and the dialog was show to reload:

And main file go to empty size.

@livius2 can you try this snapshot?
https://github.com/mariuz/flamerobin/releases/download/<IP_ADDRESS>%23259/flamerobin-0.9.3.c3c51c4f-setup-x64.exe
Check this new option and try saving the file.
2 more tests to you try to save:
-Same content, different file name (simple and short ANSI file name)
-Same file name, different content (simple and short ANSI SQL text)

"Use alternate save SQL file mothod" is working ok :)
No problem with saving this file and any modifications to it.

I have looked at your code and it looks safe to me. It can be used instead of saving from widget itself.
I suppose removing then option from config will be also good.
PS. i am really curious how save method look in widget itself that it can fail.


Update active_record_basics.md

правка перевода

I've prepared a stage to preview changes. Open stage or view logs.

:+1:


add scalike-jdbc to community build

this would have caught https://issues.scala-lang.org/browse/SI-9918

Good to hear. If there is anything I can help, please let me know.

https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/598/console has
[scalikejdbc] Running "test" in: syntax-support-macro
[scalikejdbc] [info] Formatting 4 Scala sources {file:/home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalikejdbc-f4576e833930d18d9e6284f9b85e13c432540cce/}syntax-support-macro(test) ...
[scalikejdbc] [info] Formatting 19 Scala sources {file:/home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalikejdbc-f4576e833930d18d9e6284f9b85e13c432540cce/}core(compile) ...
[scalikejdbc] [info] Reformatted 19 Scala sources {file:/home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalikejdbc-f4576e833930d18d9e6284f9b85e13c432540cce/}core(compile).
[scalikejdbc] [info] Compiling 4 Scala sources to /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalikejdbc-f4576e833930d18d9e6284f9b85e13c432540cce/scalikejdbc-syntax-support-macro/target/scala-2.12.0-4fa4b83-nightly/test-classes...
[scalikejdbc] scala.reflect.internal.Types$TypeError: type mismatch;
[scalikejdbc]  found   : scalikejdbc.package.SQLSyntaxSupport.type
[scalikejdbc]  required: SQLSyntaxSupportFeature.this.SQLSyntaxSupport.type
[scalikejdbc] 	at scala.tools.nsc.typechecker.Contexts$ThrowingReporter.handleError(Contexts.scala:1374)

@seratch have you published for 2.12.0-RC1?

@SethTisue We've not published for RC1 because we couldn't compile due to SI-9918.

let's try again with scala/scala@be43eb5: https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/599/consoleFull

FYI compile and test successful with Scala 2.12.0-RC1-be43eb5
https://github.com/xuwei-k/scalikejdbc/commit/8a48c8fc732d4518635211572ac8f4cae1de72bf

dammit, there is no scala-be43eb5-nightly, because https://github.com/scala/scala-jenkins-infra/issues/181 . I'll have to come back to this

oh, but hey wait, there is a 2.12.0-RC1-be43eb5 because Jason wanted to use it as STARR. so, https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/600/consoleFull

@seratch @xuwei-k any idea what this is about?
[scalikejdbc] [info] - should be faster when using TrieMap instead *** FAILED ***
[scalikejdbc] [info]   false did not equal true (SQLInterpolationSpec.scala:954)

from https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/600/consoleFull

@SethTisue The test is a kind of naive micro benchmark code just to verify the benefits of simple cache built with TrieMap. The tests rarely fail on my local mache or Travis CI builds. But I found the test looks too naive. I just omitted the assertions in the test suite. Would you please try the latest again?

new test run: https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/607/consoleFull (404 temporarily while Jenkins catches up)

oops, needs specs2 v3. try again: https://scala-ci.typesafe.com/view/scala-2.12.x/job/scala-2.12.x-integrate-community-build/609/consoleFull

@seratch how about this one?
[scalikejdbc] [info] - should convert timeZone *** FAILED ***
[scalikejdbc] [info]   "2016-01-09 [19]:43:42" did not equal "2016-01-09 [07]:43:42" (DBSessionSpec.scala:1090)

fixed by #293, but running tests is disabled, unless/until we can sort out the DBSessionSpec/timeZone thing

@SethTisue Thanks! After fixing the test failure issue, I'll make a pull request to enable tests for th project.

I'll make a pull request to enable tests for th project.

no need, I took care of it in https://github.com/scala/community-builds/pull/303


bot is not working again

Please tick all applicable boxes.

[x] I am using Python 3.5.3 or higher (run python --version on the command line)
[x] I have followed the official guides to install the bot for my system
[x] I have updated my dependencies to the latest version using the appropriate update script

Which version are you using?

[x] The latest master version (release-260419)
[x] The latest review version

What type of issue are you creating?

[x] Bug
[ ] Feature request
[ ] Question

Description of issue
Steps to reproduce

Log file
Please attach your MusicBot log file (located at logs/musicbot.log) to this issue. You can do so by dragging and dropping the file here. If you do not include your log file, you WILL be asked to provide one.

Try the changes mentioned in #2103

Caused by outdated discord.py run python -m pip install -U discord.py[voice]

Caused by outdated discord.py run python -m pip install -U discord.py[voice]


export getCreateTx

Can getCreateTx get exported from client b/c i need a way to make transactions dynamically without the typed api since i am not generating descriptors.
Context: https://github.com/paritytech/substrate-connect/pull/2074/files#diff-b8a6045c9dc87c6479d7a15973b74671ef75892a404e9a681da35f95d469a31bR94

Most definitely not, because it's an implementation detail, it's not part of the public API and it will change quite a bit over time, given the fact that the current implementation (and API) was the fastest way that I found to re-use the code that I used to implement the tx-helper.
That being said, in the future we want to provide a proper library that facilitates the creation of transactions. In the meanwhile, there is nothing stopping you from either:
a) make your own implementation, while improving on the current one (we may benefit from that b/c as I mentioned before the current implementation is not ideal)
b) copying the parts that you need (it's not a lot of code that you will have to copy, I think)
c) using the old libraries


Implement Peg-in & Peg-out Maturity

Since peg-out coins share some of the same properties as coinbases, it seems wise to similarly lock those coins for a number of blocks before they can be spent. This minimizes the risk of loss of funds during unexpected reorgs. This was mentioned briefly in LIP-0002: https://github.com/litecoin-project/lips/blob/master/lip-0002.mediawiki#peg-out-transactions
It is unclear whether peg-in coins should similarly be locked, but I expect we'll probably choose to do this out of an abundance of caution. It has not yet been decided how many confirmations should be necessary before pegged-in or pegged-out coins should be spendable.

As part of this, we'll have to actually commit to "output features" in the bulletproof. Otherwise, the field could be malleable.

Maturity was implemented, but is now being removed as it seems unnecessary, and results in excessive complications when supporting the auto-pegin/pegout logic we've implemented.