De-couple connection setup and resource ACL messages

This is required for #6335 and also fixes the race condition we currently face where the WireGuard tunnel is established before allow_access is received on the corresponding Gateway, causing packet loss to occur for the first few packets right after a new connection.
sequenceDiagram
    Client->>Portal: prepare_connection + allow_access
    Portal->>Gateway: allow_access
    Portal->>Client: connection_details
    Client->>Portal: request_connection (only ICE creds)
    Portal->>Gateway: request_connection (only ICE creds)
    Gateway->>Portal: reply: request_connection
    Portal->>Client: connection_accepted(only ICE creds)
    Gateway-->>Client: candidates
    Client-->>Gateway: candidates

Here is my suggestion for how to implement this:

We merge #6403 ASAP.
We introduce a successor message to prepare_connection that in addition to resource_id and connected_gateways also includes a nullable payload that gets forwarded to the selected gateway. I'll refer to this as connection_intent for now but it doesn't matter what it is called.
connection_intent will only be sent by clients >= 1.3.0
clients >= 1.3.0 should only be paired with gateways >= 1.3.0
Upon receiving a connection_intent, the portal will perform the same logic as for prepare_connection but in addition send an allow_access request to the gateway where it forwards the contained payload.
Because gateways >= 1.3.0 will have #6403, the gateway will be able to handle the allow_access message just fine. In theory, we could also omit this and even send allow_access to existing gateways but it will result in warnings being printed on older gateways.
We introduce a successor message to request_connection that does not contain a resource_id. The ICE credentials & embedded "allow-access" message are part of client_payload so portal doesn't care about them. Clients >= 1.3.0 will not embed the "allow-access" message though so (4) is important to ensure compatibility with gateways.
The gateway also needs a successor message to request_connection that doesn't contain any of the resource stuff. In fact, all we need is the client-ID and the ICE credentials.

The following is optional but may be worth doing given that we are already making changes:
As far as I understand, the portal is currently keeping some state in the form of the reference field in these messages to ensure it can route the messages back and forth between the client. I think we can get rid of this if we make the ICE credentials more explicit and also implement #6404.
Essentially, I think we can implement the following:
sequenceDiagram
    Client->>Portal: `connection_intent`
    Portal->>Gateway: `allow_access`
    Portal->>Client: `connection_details`
    Client->>Portal: `setup_connection` (gateway, secret-key, ICE creds)
    Portal->>Portal: generate gateway ICE creds
    Portal->>Client: reply `setup_connection` (ICE creds generated by portal)
    Client-->>Gateway: candidates
    Portal->>Gateway: `setup_connection` (client ICE creds + generated ICE creds)
    Gateway-->>Client: candidates

That way, the portal doesn't need to maintain any state between messages sent by either side but can always answer them immediately.
One thing that we should also do is: When a client sends setup_connection, the portal should check whether the client has access to any resource on that gateway and reject the connection setup if not. This is a defense-in-depth mechanism to ensure we don't establish unnecessary connections or connections that could potentially be misused.

@thomaseizinger a few questions:

Is there any reason why client can't generate ICE creds and send them over in connection intent and eliminate an async process of distributing them from the portal? If client still sees them and generates own credentials I don't see why it can't make one for gateway too and now wait for async reply?
Gateway doesn't need to know anymore difference between allowing access and setting up a new connection? (allow_access vs request_connection)
At what moment a new network flow is added to the audit log? Now it's on connection intent? (I assume yes because that is what adds access authorization to the Gateway).
Why we can't merge connection_intent and setup_connection?

Is there any reason why client can't generate ICE creds and send them over in connection intent and eliminate an async process of distributing them from the portal? If client still sees them and generates own credentials I don't see why it can't make one for gateway too and not wait for a separate reply for it?

That is a good idea. We can generate them on the client and send them over to the gateway, yes.

Gateway doesn't need to know anymore difference between allowing access and setting up a new connection? (allow_access vs request_connection)

They are two completely different things:

Allowing access ensures we will forward packets from this client's IP to the resource. Without allow-access, we drop them.
request_connection sets up the actual tunnel and does not need to be concerned with which resources will be contacted. For the packet-buffering change, this is important because I may now need to set up a new connection and allow-access to several resources at once (for all the packets the I buffered on the client)

If it doesn't, why we can't merge connection_intent and setup_connection?

If we already have a connection to the gateway that gets selected by the portal as a result of connection_intent, then we don't need to do anything and can just start sending packets. That is why they need to be separate. Setting up a connection is conditional but allowing-access (consequence of connection_intent in the above design) needs to happen for each resource once.

At what moment a new network flow is added to the audit log? Now it's on connection intent? (I assume yes because that is what adds access authorization to the Gateway).

Yes.

Do we need to wait for setup_connection to be delivered to the Gateway before we send it to the Client? Is it possible to have a race where one party received it and started sending candidates before another one is expecting that?

No, but we have also identified this race condition. We can resolve that by buffering candidates for unknown connections. It should be very easy.

I see we don't wait for a single reply on the gateway, how we prevent races when allow_access (or other) message was delivered but not still processed (eg due to CPU load) and client sends the next one?

Receiving allow_access out of order doesn't matter. We do assume reliable delivery though but that is already the case today. Packets from the client will be dropped until the allow-access is processed. By sending it already as part of connection_intent, we should avoid that case though.

the portal should check whether the client has access to any resource on that gateway and reject the connection setup if not

allow_access should be not for any resource on the gateway (checking that would be both expensive and pointless) but for a specific resource the access is granted (because we need to create a network flow for it).

Yes. We only emit a connection_intent once for:

each CIDR resource
each non-wildcard DNS resource
each domain accessed within a wildcard DNS resource

The check I was talking about above is only for the connection setup (i.e. ICE credential exchange). We should not allow that if the client does not have access to any resource on this gateway. Currently we get this "for free" because connection-setup implies an allow-access but that is what we'd like to remove.

@thomaseizinger what if, to eliminate 2nd roundtrip, the portal will figure out if tunnel can be reused? You already send connected_gateway_ids so if I select one of the gateways that is already connected, I can send a different kind of messages back to client and gateway. The only downside for this would be that there will be extra payload included in each connection_intent message to the portal.

@thomaseizinger what if, to eliminate 2nd roundtrip, the portal will figure out if tunnel can be reused? You already send connected_gateway_ids so if I select one of the gateways that is already connected, I can send a different kind of messages back to client and gateway. The only downside for this would be that there will be extra payload included in each connection_intent message to the portal.

Unfortunately, that opens us up to a race condition where the connection we have locally just fails (due to being idle for example) inbetween sending connection_intent and receiving connection_details.
Even though it is another roundtrip, I think we will end up with a more robust system if we can let the client chose, when to make a new connection.
In addition, this also gives us the flexibility in the protocol to make connections to gateway whenever we want which could be used to e.g. prime connections to certain gateways, failover to a different one etc.

@thomaseizinger but this race is not solved by having a separate message, you still can send me connection_details and something can fail on the client or gateway while it's being handled by the parties - you will have to do some error handling to prevent that races in the connlib anyways?

In addition, this also gives us the flexibility in the protocol to make connections to gateway whenever we want which could be used to e.g. prime connections to certain gateways, failover to a different one etc.

Client can not connect to gateways - it would mean that clients know about gateways before it picks resources and exposing this information to clients would allow easier network enumeration for the attackers, plus it's really hard to authorize such connection. We can only authorize access to resources, and gateway ID along any it's information client gets in that process should be opaque to it.

Here are two ideas. First if we don't care about gateway_payload and response from gateway for reused connections in general (but there will be race if client tries to send ice candidates to gateway which did not process allow_access yet):
sequenceDiagram
    Client->>Portal: `connect` (resource_id, connected_gateway_ids, preshared-key, payload[eg. ICE creds])
    Portal->>Portal: Authorize and Log Access
    alt Reuse Connection
        Portal->>Gateway: connect (flow, resource, client, client_payload)
        Gateway-->>Portal: pending_connection (gateway_payload)
        Portal-->>Client: setup_connection (gateway_id, gateway_pubkey, gateway_public_ip, gateway_payload)
    else New Connection
        Portal->>Gateway: allow_access (flow, resource, client)
        Portal-->>Client: reuse_connection (gateway_id)
    end

    par Candidate Exchange
        Client->>Gateway: candidates
        Gateway->>Client: candidates
    end

or the one with reply:
sequenceDiagram
    Client->>Portal: `connect` (resource_id, connected_gateway_ids, preshared-key, payload[eg. ICE creds])
    Portal->>Portal: Authorize and Log Access
    alt Reuse Connection
        Portal->>Gateway: connect (flow, resource, client, client_payload)
        Gateway-->>Portal: pending_connection (gateway_payload)
        Portal-->>Client: setup_connection (gateway_id, gateway_pubkey, gateway_public_ip, gateway_payload)
    else New Connection
        Portal->>Gateway: allow_access (flow, resource, client)
        Gateway-->>Portal: acknowledge (gateway_payload)
        Portal-->>Client: reuse_connection (gateway_id, gateway_payload)
    end

    par Candidate Exchange
        Client->>Gateway: candidates
        Gateway->>Client: candidates
    end

BTW keep in mind the messages between client and gateway processes in the portal can also have a costly roundtrip/network splits/other failure scenarios once we scale out our API - they can be hosted on cluster nodes in different regions.

Thanks for drafting up more ideas. How does the above solution support sending multiple concurrent connects for different resource IDs where some of them may get routed via the same gateway? The gateway can only ever use a single pair of ICE credentials for a client. How do you as the portal decide, which ones of the concurrently arriving connect calls you send to the gateway? How does the client know which one it should use?
The main problem we need to solve for #6335 is:

We need to authorize N resources concurrently.
We may need to set up M connections to actually talk to those resources.

Keeping these concerns separate makes it much easier to manage the state for these operations.

@thomaseizinger but this race is not solved by having a separate message, you still can send me connection_details and something can fail on the client or gateway while it's being handled by the parties - you will have to do some error handling to prevent that races in the connlib anyways?

Failing to set up a connection is not a race condition. We have timeouts that reset the state there. The race condition is:

We have a connection to a gateway.
We send connection_intent to the portal, signalling that we a connection.
The connection goes idle and we close it (or fails for some other reason).
The portal responds to us, telling us to "reuse" a connection that we no longer have.
The client is left in a failed state.

The only way to fix this is to restart the entire process, i.e. send another connection_intent to the portal, this time not including the now failed connection. The portal may then either pick the same or a different gateway and respond again. If we are really unlucky, it again picked a gateway that we were just connected to and are now disconnected, so this may happen multiple times in a row.
This seems like unnecessary trial-and-error that can be avoided by just deferring the condition of "reuse or create connection" to the point where the client learns about which gateway to use. This is what we do today and works pretty well.
The thing that changes with #6335 is that we now have to authorize access to N different resources as part of the connection setup but today's message only allows us to embed one.
This can be fixed by implying the allow-access directly after connection_intent so the only alternative flow after is either setting up a new connection or reusing an existing one.

In addition, this also gives us the flexibility in the protocol to make connections to gateway whenever we want which could be used to e.g. prime connections to certain gateways, failover to a different one etc.

Client can not connect to gateways - it would mean that clients know about gateways before it picks resources and exposing this information to clients would allow easier network enumeration for the attackers, plus it's really hard to authorize such connection.

I don't see the security problem you are describing. As long as a client is allowed to access at least one resource on a gateway, it should just be able to make a connection to it. We still need to know the gateway ID to connect to it and we only learn that as a result of sending a connection_intent. You can't do any network enumeration that you couldn't also do today unless you start guessing gateway IDs which is pretty unlikely?

Here is the design we settled on:
sequenceDiagram
    Client->>Portal: connect (resource_id, connected_gateway_ids)
    Portal->>Portal: Authorize and Log Access
    Portal->>Portal: Generate preshared_key (random 32 bits)
    Portal->>Portal: Generate Ice Credentials (hash of PKs + Client/Gateway IDs)
    Portal->>Gateway: setup_connection (flow, resource, client, preshared_key, ice_credentials)
    Gateway-->>Portal: ack
    Portal-->>Client: setup_connection (gateway_id, gateway_pubkey, gateway_public_ip, preshared_key, ice_credentials)

    par IPs Echange (will be removed in favor of direct C-G communication later)
        Client->>Portal: DNS IPs
        Portal->>Gateway: DNS IPs
    end

    par Candidate Exchange
        Client->>Gateway: candidates
        Gateway->>Client: candidates
    end

I wonder if we can find a different name from setup_connection? We could treat it as a "connection to a resource" but it is still kind of an overloaded term, especially because a "connection to a resource" doesn't exist in the clients.
Should we call it setup_flow maybe?

Should we call it setup_flow maybe?

Or something that implies the upsert behaviour? Maybe upsert_flow?
Also, on the client, it will just be the "response" to connect, right? And not a separate message?

@thomaseizinger renamed, lmk if you want to change more - I'm don't have any strong opinions about naming there

@thomaseizinger renamed, lmk if you want to change more - I'm don't have any strong opinions about naming there

LGTM, thank you

@AndrewDryga While implementing this, I noticed a subtlety that I wanted to spell out explicitly:
When the ICE credentials are generated, it is important that they are unique to this pair of client and gateway and they should only ever change together, i.e. if a client A restarts, the ICE credentials for gateway "foo" and client A need to change but the ICE credentials for gateway "foo" and client B should remain the same.
In other words: We should use a single hash function state for both credentials that is seeded with both IDs and keys (as opposed to perhaps seeding a separate hash function with just the client's / gateway's ID + key).
The goal is that each party can detect based on their own credentials, whether their local state is still valid.

@conectado @AndrewDryga I discovered something:
Currently, when we receive an allow_access request, we always check that the specific domain is actually a sub-domain of the resource domain. With the above protocol, the message for sending proxy IPs is separate and thus the allow_connection message doesn't actually contain the domain, meaning we can't make this check.
I think we need to embed the domain and proxy IPs in the initial upsert_flow message because this check is part of the authorisation and we shouldn't authorize the flow if the client wants to access a domain outside of the resource domain.
In other words, I think we still need a portal-opaque payload field that we can use to send across the domain + IPs.

@thomaseizinger I'll add payload field

@thomaseizinger

I think we need to embed the domain and proxy IPs in the initial upsert_flow message because this check is part of the authorisation and we shouldn't authorize the flow if the client wants to access a domain outside of the resource domain.

Good catch. We should also do that check when new domains are added to connection (for wildcards).

@thomaseizinger

I think we need to embed the domain and proxy IPs in the initial upsert_flow message because this check is part of the authorisation and we shouldn't authorize the flow if the client wants to access a domain outside of the resource domain.

Good catch. We should also do that check when new domains are added to connection (for wildcards).

Yeah! "New domains being added" is equivalent to use having to send the proxy IPs to the gateway.

@thomaseizinger I'll add payload field. Does the gateway need to send any payload back to the client?

Thinking about it, I am not sure we should make it an opaque payload field. Don't we want to log an audit entry for each domain? The domain information is only in the payload so it wouldn't be opaque actually.

Summary from discussion with @AndrewDryga:
We only ever want to authorise each resource once, even if it is just a wildcard resource, otherwise we'll see quite a lot of spam in the audit log and fairly heavy DB operations with each access.
The consequence of that is that the gateway no longer knows, which specific domain is getting accessed at the time of a resource authorisation. Without the specific domain (and the assigned proxy IPs), we cannot yet perform any name resolution and NATing.
This has three consequences:

We need to break compatibility with < 1.1.0 clients because we don't have a response we can send back (see #6708).
We need some more refactoring on the gateway to be able to handle connection authorisations without domain names and corresponding proxy IPs. Instead, the name resolution needs to happen as part of handling a different message.
The upcoming packet-buffering might be tricky because domain names and IPs need to be sent separately but that can only happen once we have a new connection. Until then, we need to buffer packets. I am thinking of designing "Send DNS IPs" as a batch-message and simply sending over all resource IPs that we have resolved so far and not one-by-one. That will ensure that packets won't get dropped on the gateway because the IPs have already been shared ahead-of-time.

Summary from discussion with @AndrewDryga:
We only ever want to authorise each resource once, even if it is just a wildcard resource, otherwise we'll see quite a lot of spam in the audit log and fairly heavy DB operations with each access.
The consequence of that is that the gateway no longer knows, which specific domain is getting accessed at the time of a resource authorisation. Without the specific domain (and the assigned proxy IPs), we cannot yet perform any name resolution and NATing.
This has three consequences:

We need to break compatibility with < 1.1.0 clients because we don't have a response we can send back (see chore(gateway): remove domain response (breaks 1.1.0 clients) #6708).
We need some more refactoring on the gateway to be able to handle connection authorisations without domain names and corresponding proxy IPs. Instead, the name resolution needs to happen as part of handling a different message.
The upcoming packet-buffering might be tricky because domain names and IPs need to be sent separately but that can only happen once we have a new connection. Until then, we need to buffer packets. I am thinking of designing "Send DNS IPs" as a batch-message and simply sending over all resource IPs that we have resolved so far and not one-by-one. That will ensure that packets won't get dropped on the gateway because the IPs have already been shared ahead-of-time.

@thomaseizinger  so the idea is to have a response for DNS IPs?

I'd like to avoid that because its only purpose would be to know explicitly, for how long we can buffer packets. It would add a delay of 2 RTTs though and make the code more complicated.
Instead, I am thinking of doing it optimistically:

If we don't have a connection to the gateway yet, we can send all proxy IPs as soon as we learn which gateway to use. We still need to perform ICE and the WG handshake at that point, meaning it is likely that the DNS IP message made it to the gateway by the time we actually flush the packets.
Every time we resolve new IPs for a DNS resource, we can optimistically those IPs to every gateway that we connect to. Depending on how quickly the application makes use of the resolved IPs, some packets might get dropped there.

Do you think that is good enough?

Thinking more about it, it might be better to buffer explicitly. That way, the experience is guaranteed to always be the same, regardless of the latency.
We can still optimistically pre-send IPs for resolved resource names to reduce the impact of the added roundtrip for the response.

Thinking more about it, it might be better to buffer explicitly and wait for a response. That way, the experience is guaranteed to always be the same, regardless of the latency between client <> gateway and client <> portal.
We can still optimistically pre-send IPs for resolved resource names to reduce the impact of the added roundtrip for the response.

I think this is the way to go, we will need to mark ips as sent or pending and buffer ips for those packets until they are sent.
I wonder if it'd be too much load on portal/gateways if we send all ips to all connected gateways as soon as we resolve them.

I wonder if it'd be too much load on portal/gateways if we send all ips to all connected gateways as soon as we resolve them.

I think it should be fine. It would be very unusual for a client to make a DNS query and not do anything with the result so chances are, we'll send them eventually?

I've thought about what the messaging between client and gateway could be here and I think it is possible but it isn't very clean and kind of brittle.
The gateway needs to send two kinds of responses to setup_dns_resource_nat (That is what I am naming the message about the proxy IPs for now):

An OK response that the setup was successful
An ERR response that the setup wasn't successful

We need the error to avoid running into a timeout on the client.
In terms of the messaging protocol, the client doesn't know that the message is forwarded to the gateway. Instead it is using the normal "request-reply" mechanism of phoenix-channel. In the design of phoenix-channels, replies don't have any type information. Instead, they look like this:
{
    "ref":null,
    "topic":"client",
    "event": "phx_reply",
    "payload": {
        "response": {
			// some kind of response
        },
        "status":"ok"
    }
}

Without any type-information, all our deserialisation framework can do is try all possible types one-by-one to see which one fits. That by itself isn't pretty (produces bad error messages when things are slightly off for example) but it works.
The problem is that we are now mangling together two kinds of things:

Responses generated by the portal, which have a particular shape.
Responses forwarded by the portal.

In order to be forwards-compatible with any changes to this protocol between client and gateway, the gateway needs to now embed some kind of type information into this forwarded payload, otherwise the client doesn't know, which response type it needs to deserialise things into.
This is getting very convoluted and error prone.
How difficult would it be to write our own transport layer for the websocket on the portal side that lets us design our own wire format?

I'll let @AndrewDryga chime in here more thoroughly, but I know that we're currently leveraging quite heavily PhoenixChannel in the control plane. It's tied not only into the Client - Gateway coms, but also to any browser sessions that may be active (e.g. online/offline status).
To replace that, we'd need to have a control protocol for the coms here in this issue, and then write glue code to tie the messages to the PhoenixChannel/LiveView code for the admin portal.
I know it costs a few extra bytes on the wire, but could we keep this PhoenixChannel protocol intact, and instead build the desired control protocol on top of its standard reply messages? We would need to define a wire format to embed inside the response field, but this is sounding like more of a serialization/deserialization contract than an issue with the transport (PhoenixChannel) per se, no?

To replace that, we'd need to have a control protocol for the coms here in this issue, and then write glue code to tie the messages to the PhoenixChannel/LiveView code for the admin portal.

Yeah, if I understand things correctly, what we'd need to replace is use Phoenix.Socket.

I know it costs a few extra bytes on the wire, but could we keep this PhoenixChannel protocol intact, and instead build the desired control protocol on top of its standard reply messages? We would need to define a wire format to embed inside the response field, but this is sounding like more of a serialization/deserialization contract than an issue with the transport (PhoenixChannel) per se, no?

I am not worried about the bytes on the wire, I am more worried about forwards/backwards-compatibility bugs because the messages in phoenix channel aren't typed (a reply does not indicate, what its content is). In general, the JSON format of phoenix channel is quite complex but we aren't really benefiting from that complexity, at least not on the clients.
In an ideal world, I'd like to have a protocol with the portal where each message follows something like:
{
	"type": "...",
	"payload": ...
}

And we get to decide, what the different types are. We don't need a concept of "responses", as those can be expressed by convention in the message type. We also don't need sequence numbers. For messages where it matters, we can put that in the payload itself.

@conectado and I chatted about this just now and I'll try to implement this using a p2p control protocol through the tunnel instead of using the portal. That should fix this particular issue.

@thomaseizinger
I'm afraid that if we replace battle tested in 1000s of productions WebSocket transport layer we will:

Have to maintain it for eternity without getting any critical fixes from upstream (and sometimes fixing breaking changes from HTTP server lib or be locked from updating them);
Will introduce bugs we don't know about and make everything less stable overall. The chances of this is way higher than hitting deserialization issues in our own codebase.

I believe it won't worth it in the end.

a client cannot differentiate whether a reply is from the portal itself or whether its payload is a forwarded payload from the gateway

In a Phoenix channel any party can send a message at any time, messages sent from the client can be replied by the portal (event will be phx_reply and ref will be equal to the ref you sent to the portal). We leverage it for forwarded messages but don't have to - I can just send those as regular messages like we do with event: init, event: config_change, etc.
Typically for replies you can know what that would be using the ref knowing what you have sent and what types of responses you expect there.
However, then you will loose the ability to do synchronous actions - you won't be able to send a message and await for it's reply. Do you care about it? Also, would it be better if you add type into the payload itself? It's opaque for the portal and will make deserialization eaiser.

As we grow the team anyone seeing self-written transport protocol will be WTFing all the time :).

At the moment, I am WTFing at our control protocol is complex in its implementation when in principle, it doesn't need to be :)

2. The chances of this is way higher than hitting deserialization issues in our own codebase.

Having a simpler control protocol extends to more benefits than that. We've had several issues in the past with the complexity of phoenix channels (disconnected rooms for example).

In a Phoenix channel any party can send a message at any time, messages sent from the client can be replied by the portal (event will be phx_reply and ref will be equal to the ref you sent to the portal). We leverage it for forwarded messages but don't have to - I can just send those as regular messages like we do with event: init, event: config_change, etc.

Things would be much easier if we can entirely ditch phx_reply and treat every message as an incoming message. How difficult would it be to do that?

using the ref knowing what you have sent and what types of responses you expect there.

The ref is a runtime value so it is impossible to know at compile-time, what the format of the payload is. phx_reply is untyped so the only way of deserialising is trial-and-error of all possible reply types that we have.

Also, would it be better if you add type into the payload itself? (BTW you can nest anything in there). It's opaque for the portal and will make deserialization eaiser.

Yeah, it is a possibility. Most designs are possible but that doesn't mean we should implement them :D
Considering the entire message, having multiple layers of "types" isn't very nice.
I'll try to implement a p2p control protocol now, then we don't need to deal with these shenanigans.

How difficult would it be to do that?

In case p2p control protocol would be too much work we can do it for sure. We use replies just thinking that it would be better for clients/gateways, but we don't have to do it. We can also add type to all payload's too.

@AndrewDryga If you don't mind, we'd also like to rename upsert_flow to create_flow. We noticed that upsert isn't correct because we don't ever call this if we have a flow already for this resource.

How difficult would it be to do that?

In case p2p control protocol would be too much work we can do it for sure. We use replies just thinking that it would be better for clients/gateways, but we don't have to do it. We can also add type to all payload's too.

It could be useful even outside the p2p-control protocol discussion. For example, when you reply to upsert_flow with an error, we don't have any context in the error, which one failed. This means we need to keep state around on what actually failed.
it would be easier if we had a message create_flow_failed that just contained the resource ID.

@thomaseizinger we can change that, just one question:

we don't have any context in the error

Why can't we use ref to identify for which action the reply happened?
I think we just think about the programing models differently here. Typically messages in a channel are sort of RPC's, where you call a function (send a message) and get a reply to it (with a matching ref), aka call. Or if you don't care about reply then you just send a message, aka cast. In Elixir we would typically block the lightweight process until reply is received. For example, if I would need to buffer something until gateway replies I would send a message, save the ref and keep buffering in a separate process until there is a message back with that ref, which then will flush the buffer. Is our approach on the Rust side is different? Just trying to understand better our event handling model :).

Updated naming, WDYT?
Here is the design we settled on:
sequenceDiagram
    Client->>Portal: create_flow (resource_id, connected_gateway_ids)
    Portal->>Portal: Authorize and Log Access
    Portal->>Portal: Generate preshared_key (random 32 bits, unique per flow)
    Portal->>Portal: Generate Ice Credentials (hash of PKs + Client/Gateway IDs)
    Portal->>Gateway: authorize_flow (flow, resource, client, preshared_key, ice_credentials)
    Gateway-->>Portal: flow_authorized
    Portal-->>Client: reply :ok (gateway_id, gateway_pubkey, gateway_public_ip, preshared_key, ice_credentials)

    par IPs Echange (will be removed in favor of direct C-G communication later)
        Client->>Portal: DNS IPs
        Portal->>Gateway: DNS IPs
    end

    par Candidate Exchange
        Client->>Gateway: candidates
        Gateway->>Client: candidates
    end

@thomaseizinger we can change that, just one question:

we don't have any context in the error

Why can't we use ref to identify for which action the reply happened?
I think we just think about the programing models differently here. Typically messages in a channel are sort of RPC's, where you call a function (send a message) and get a reply to it (with a matching ref), aka call. Or if you don't care about reply then you just send a message, aka cast. In Elixir we would typically block the lightweight process until reply is received. For example, if I would need to buffer something until gateway replies I would send a message, save the ref and keep buffering in a separate process until there is a message back with that ref, which then will flush the buffer. Or even without the ref just make that process a state machine that knows current state and valid transitions. Is our approach on the Rust side is different? Just trying to understand better our event handling model :).

The way we implemented the phoenix channel client in Rust is a state machine that is configured it with 2 type parameters: An enumeration of all incoming types and an enumeration of all reply types. The reply itself is untyped, so the possible reply types is also an enumeration. Whenever we receive a reply, we attempt to deseriailise it into one the possible types and emit it together with the ref of the original request.
In order to achieve a 1-to-1 mapping of the expected response type for a request, I'd have to create some kind of task / process and all tasks that currently have a reply outstanding know, which ref they are waiting for and some orchestrating task dispatches the response to the correct one. By modelling the req-res flow in a separate type, I could express that for a given request, only a certain set of responses is allowed.
We can build that but I'd prefer not to. A stateless protocol where each message can be interpreted on its own is easier to handle on the client than the multiplexing protocol at hand here.

Updated naming, WDYT?

LGTM!

@thomaseizinger ah, I see. I think the issue is that we did not follow the idiomatic use of the Phoenix channels (that kinda expect to use ref's for that). We can totally switch to messages instead of replies. Would that open a door for race conditions though? Without ref for two messages you won't ever be able to tell a difference in case the client sends two identical messages for which one the reply is.
Eg. you send a create_flow, then restart (for whatever reason) and send a new one for the same resource, the gateway still will broadcast an old flow_created back to you with a resource_id before second message is processed, and then broadcast reply for the 2nd message too. ref tracking will make 100% sure you always get what you expect and that you can have a per-task/process timeout waiting for reply, so that you can repeat certain actions.

@thomaseizinger ah, I see. I think the issue is that we did not follow the idiomatic use of the Phoenix channels (that kinda expect to use ref's for that).

Yeah, writing a full-fledged phoenix channel seemed overkill for the small subset that we need :)

Eg. you send a create_flow, then restart (for whatever reason) and send a new one for the same resource, the gateway still will broadcast an old flow_created back to you with a resource_id before second message is processed, and then broadcast reply for the 2nd message too. ref tracking will make 100% sure you always get what you expect and that you can have a per-task/process timeout waiting for reply, so that you can repeat certain actions.

Yeah, there can be race conditions here but I don't think they are very bad:

If we only disconnected from the websocket and still have the same public key, the connection will succeed anyway and it doesn't matter that we send two create_flows. In fact, I am planning to resend create_flow if we don't get a reply within 2s and there are still packets for these resources so multiple concurrent create_flows can happen, even for the same resource, depending on the latency for example.
If we have a new public key, the connection will fail anyway.

I think it is much more important that we designed this new protocol to be idempotent and thus don't have to worry about reorderings instead of trying carefully seqeuence things.

@AndrewDryga Here is a suggestion that uses messages instead of replies on the client:
sequenceDiagram
    Client->>Portal: create_flow (resource_id, connected_gateway_ids)
    Portal->>Portal: Authorize and Log Access
    Portal->>Portal: Generate preshared_key (random 32 bits, unique per flow)
    Portal->>Portal: Generate Ice Credentials (hash of PKs + Client/Gateway IDs)
    Portal->>Gateway: authorize_flow (flow, resource, client, preshared_key, ice_credentials)
    Gateway-->>Portal: flow_authorized

    alt Success
        Portal-->>Client: create_flow_ok (resource_id, gateway_id, gateway_pubkey, gateway_public_ip, preshared_key, ice_credentials)
    else Offline
        Portal-->>Client: create_flow_err (resource_id, reason: "offline")
    end

    par Candidate Exchange
        Client->>Gateway: candidates
        Gateway->>Client: candidates
    end

By convention, the "responses" to create_flow are named as:

create_flow_ok
create_flow_err

reason in create_flow_err is a string (other than offline, what else do we have?)
Note that I also added resource_id to create_flow_ok

Does that make sense from your perspective? Am I missing something?
Perhaps to be consistent, we should name flow_authorized from gateway -> portal to authorize_flow_ok? There is no authorize_flow_err at the moment but I think it would make sense if we just follow the convention of always naming the reply message after the request.

Here is a full example of create_flow_ok as I am currently working with locally:
{
  "event": "create_flow_ok",
  "ref": null,
  "topic": "client",
  "payload": {
    "resource_id": "f16ecfa0-a94f-4bfd-a2ef-1cc1f2ef3da3",
    "gateway_id": "b3d34a15-55ab-40df-994b-a838e75d65d7",
    "site_id": "eb94482a-94f4-47cb-8127-14fb3afa5516",
    "gateway_public_key": "OR2dYCLwMEtwqtjOxSm4SU7BbHJDfM8ZCqK7HKXXxDw=",
    "preshared_key": "sMeTuiJ3mezfpVdan948CmisIWbwBZ1z7jBNnbVtfVg=",
    "client_ice_credentials": {
      "username": "PvCPFevCOgkvVCtH",
      "password": "xEwoXEzHuSyrcgOCSRnwOXQVnbnbeGeF"
    },
    "gateway_ice_credentials": {
      "username": "PvCPFevCOgkvVCtH",
      "password": "xEwoXEzHuSyrcgOCSRnwOXQVnbnbeGeF"
    }
  }
}

And create_flow_err:
{
  "event": "create_flow_err",
  "ref": null,
  "topic": "client",
  "payload": {
    "resource_id": "f16ecfa0-a94f-4bfd-a2ef-1cc1f2ef3da3",
    "reason": "offline"
  }
}

@jamilbk @AndrewDryga @conectado I've updated the description of this issue with all related PRs / tasks. The open ones are in the order in which they need to be completed.
The gateway PR is ready for review and merge. My recommendation would be to not yet bump the gateway version. We need to have the gateway deployed to staging in order to properly test the protocol using clients built off the (unmerged) client PR. Once we are happy with the testing, I'd suggest to bump the gateway version (PR is ready and linked above) and release it.
After that, we need to communicate to all admins that they should upgrade their gateways.
After the grace period (2 weeks?) we can merge the client implementation and eventually bump their versions as well and release them which finally completes this issue! :)

Just as final confirmation:

clients 1.3 are compatible with gateway 1.3 and 1.4
clients 1.4 are compatible with only gateway 1.4

correct?

All merged, we just need to release.


hwmv2: ite_ec: port to HWMv2

Port ite_ec soc and boards to hwmv2.

Compliance is still not having it though, is there some known issues with Kconfig symbols?

@nordicjm thanks, all done, compliance seems to be happy now even though I don't quite understand why, guess the behavior of the v2 Kconfig check

@nordicjm thanks, all done, compliance seems to be happy now even though I don't quite understand why, guess the behavior of the v2 Kconfig check

Kconfig.soc is used by things like documentation whereas Kconfig is only used by the actual image build itself


ERROR: BoundsError: attempt to access 15-codeunit SubString{String} at index [0:15]

Sorry, it happened again (GMT)
julia> checkdocs(GMT)
ERROR: BoundsError: attempt to access 15-codeunit SubString{String} at index [0:15]
Stacktrace:
  [1] checkbounds
    @ .\strings\basic.jl:216 [inlined]
  [2] SubString
    @ .\strings\substring.jl:46 [inlined]
  [3] SubString
    @ .\strings\substring.jl:43 [inlined]
  [4] getindex
    @ .\strings\substring.jl:292 [inlined]
  [5] check(::CheckDoc.Checks.ParagraphTermination, doc::CheckDoc.DocContext{Markdown.MD})
    @ CheckDoc.Checks C:\Users\j\.julia\packages\CheckDoc\KNlbc\src\checks.jl:343

Hah! It's a good test package it seems :laughing:, should be fixed now :crossed_fingers:

Thanks for trying this out and letting me know :slightly_smiling_face:

Thanks. It's working now (and found tons of things)


add a sample math problem

Provide the example in the top left space of the 3-by-3 problem matrix (on the main page). This will be presented in a pop-up image modal, similar to the editor.

The title for this "problem" should be: "

Using this Tool"

done


[SMALLFIX] Use static imports for standard test utilities in ModeTest…

….java

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/16949/
Test FAILed.


Middle click to paste selection in terminal does not work when middle clicking the selection itself

Steps to Reproduce:

Open terminal on Linux
Select text
Middle click on the selected text, nothing happens

This has been fixed since adding the text area back in #175


Question: Will WinUI 3.0 be buildable with Clang/LLVM (and CMake/Ninja)?

[Assuming that WinUI 3.0 will be compilable from source, rather than relying on Nuget packages...]
Given CMake is a first class citizen in Visual Studio now, will it be a supported build system for WinUI 3.0? Also, for cross platform projects that use clang on other platforms (e.g. macOS) it is useful to use the same compiler on Windows to minimise (or compare) performance and compilation differences. Will WinUI 3.0 be compilable with clang & lld-link?

When WinUI 3 goes open source, it will be compilable from source. It uses MSBuild and the msvc compiler to build.
With regards to your questions about CMake and Clang. Are you asking if you will be able to build a WinUI 3 app using these tools? Or are you asking if you will be able to build the WinUI runtime itself from source using these tools?
When it comes to compiling a WinUI app in C++ with Clang; I expect it would be possible. C++/WinRT does not depend on any msvc specific features and can be compiled using any standards compliant C++ compiler. With regards to using CMake as the build environment; all of the Visual Studio templates that we ship are MSBuild based. We do not currently plan to ship any CMake-based templates. But there should be nothing that would fundamentally prevent someone from creating a CMake-based build process that produces a WinUI app. There would be a certain amount of work involved in this, for example the build process would need to invoke some of the sdk tools, such as the Xaml Compiler or midlrt to produce the required output. So it is not something that would be supported "out of the box", but it certainly seems feasible. We would welcome any community contributions in this area if this was seen as a useful scenario.
When it comes to compiling the WinUI platform itself: As mentioned, the platform is currently built with msvc. It is not currently possible to compile the WinUI platform using Clang, as not all of the code is fully standard compliant C++ code. There are a couple of places where we use msvc-specific features. However, we certainly would value getting the code into a more compliant state and this is another area that we would welcome any community contributions.
With regards to building the WinUI platform itself with CMake: we currently use MSBuild to define our build process. Though it is theoretically possible that CMake could be used instead, given that WinUI build process is a quite large and complex with many components involved, it would not be feasible without a very significant effort. We do not have any plans to move away from MSBuild.
Let me know if you have any further questions, or if you have any specific scenarios in mind.

Hi @kmahone , I'm talking about the compilation of the actual WinUI library, not an app that uses WinUI. My plan is to compile WinUI from source, within our solution. Currently we have a .sln and a CMakeLists.txt which we maintain side-by-side. The app is developed and built using Visual Studio (via .sln file), but we also compile for performance tests and validation using CMake | clang | Ninja.
As you've mentioned in your reply, there are two parts to this question:

Can we compile the WinUI library with Clang - if not, can we submit PRs to enable this?
Can we write a CMakeLists.txt for WinUI - this doesn't have to be in the upstream repo (i.e. your repo) but I'd be interested to know about any steps that Visual Studio or VSIX plugins performs for the developer behind the scenes.

Can we compile the WinUI library with Clang - if not, can we submit PRs to enable this?

With the source code as it is today (and as it will be when it is make is OSS), it would not be possible to compile it with clang. The code is close to C++ standards conformance, but not entirely. For example there are cases where msvc will allow an implicit cast that is not standard. There are also a few places where we use msvc specific features such as __super. These are just some issues that I know of off the top of my head. So these and any other issues would need to be resolved before it could be compilable in Clang. We have never tried to compile with Clang, but we have tried to compile using msvc with /permissive- set, which switches msvc to a more standards compliant mode. We don't yet compile successfully in permissive- mode. But we would accept any PRs that would help get the code base into a more standards compliant state.

Can we write a CMakeLists.txt for WinUI - this doesn't have to be in the upstream repo (i.e. your repo) but I'd be interested to know about any steps that Visual Studio or VSIX plugins performs for the developer behind the scenes.

I expect it would be possible. However, it would not be a trivial task, as the WinUI build is quite complex. CMake would need to be able to invoke many of the sdk build tools (such as midlrt, the XamlCompiler, etc.). When WinUI 3 goes OSS, you will be able to examine the msbuild files and see what it is doing. Porting to CMake would be quite an involved task, but I can't think of a reason why it would not be possible. As for whether we would accept a PR into this repo that added the CMakeLists.txt files to build WinUI: there would have to be a discussion about that, but my guess is that we would likely not accept such a PR due to cost of keeping two build systems working and in sync.
I have a broader question for you: why do you want to compile WinUI from scratch using CMake instead of consuming the built binaries from the nuget package? Building WinUI from source as part of your app would dramatically increase the build time of your app and would introduce a significant amount of complexity to your build. Is there a reason you would want to do that over just picking up the dll and winmd from the nupkg?

A couple of reasons why I'd like to build WinUI from source and preferably with CMake (and Ninja):

We're a cross-platform application. CMake is the cross-platform build system.
All third party libraries are built from source as part of our application build (we have approx. 35 third party libraries built this way).
Building from source avoids potential pitfalls such as C++ ABI breaks, missing dependencies, etc
If WinUI and C++/WinRT are "just" C++ with no extensions (i.e. not C++/CLI) then it should be compilable by any compliant compiler.
We can't use the LLVM toolset within Visual Studio, as Clang doesn't obey the /MP flag (parallelism should be provided by the build system, not the compiler). Release builds take 2 hours 45 minutes with 28 cores, so we can't entertain the idea of dropping down to a single core build.

Slight side question, but can WinUI be included in vcpkg  with this approach?

Thanks for the info. I should also point out that WinUI itself has some dependencies on nuget packages. So even if you were compiling WinUI from source, there would still be a requirement to pull down some packages.
We do not currently have any plans to publish WinUI as a vcpkg since nuget will be our primary distribution method. But this might be something that could be driven by the community.

@kmahone  +1 for making MIDL, MDMERGE, CPPWINRT, and the "Xaml Compiler" usable from CMake.  This is something I have been struggling with for some time: there is very little documentation for using any of those tools directly in a command-line, batch file, or CMake script.  I think I've finally got a handle on most of them, except for the "Xaml Compiler" -- which I have not been able even to locate, let alone to use from CMake.  Do I have to resort to having a "standard" Visual Studio solution setup to generate the *.xaml.g.{cpp,h} files?

@RealTommyKlein created a CLI version of XamlCompiler that is included in the Nuget package. We ourselves don't use it yet, though.


Add Cloud Run Button

This would improve the usability of this project greatly!

@jthegedus You should be able to use this from my fork: https://github.com/jamesward/badger/tree/cloud-run-button


ChatHuggingFace using local model with HuggingFacePipeline wrongly checks for available inference endpoints

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

        # Case token is explicitly provided
        if isinstance(token, str):
            return token

        # Case token is explicitly forbidden
        if token is False:
            return None

        # Token is not provided: we get it from local cache
        cached_token = get_token()

        # Case token is explicitly required
        if token is True:
            if cached_token is None:
>               raise LocalTokenNotFoundError(
                    "Token is required (`token=True`), but no token found. You"
                    " need to provide a token or be logged in to Hugging Face with"
                    " `huggingface-cli login` or `huggingface_hub.login`. See"
                    " https://huggingface.co/settings/tokens."
                )
E               huggingface_hub.errors.LocalTokenNotFoundError: Token is required (`token=True`), but no token found. You need to provide a token or be logged in to Hugging Face with `huggingface-cli login` or `huggingface_hub.login`. See https://huggingface.co/settings/tokens.

/opt/poetry-cache/virtualenvs/sagacify-example-llm-8EXZSVYp-py3.10/lib/python3.10/site-packages/huggingface_hub/utils/_headers.py:158: LocalTokenNotFoundError

Description
I am trying to use the langchain-huggingface library to instantiate a ChatHuggingFace object with a HuggingFacePipeline llm parameter which targets a locally downloaded model (here, Meta-Llama-3-8B).
I expect the instantiation to work fine even though I don't have a HuggingFace token setup in my environment as I use a local model.
Instead, the instantiation fails because it tries to read a token in order to list the available endpoints under my HuggingFace account.
After investigation, I think this line of code should be at line 456 instead of line 443 in file langchain/libs/partners/huggingface/langchain_huggingface/chat_models/huggingface.py
def _resolve_model_id(self) -> None:
  """Resolve the model_id from the LLM's inference_server_url"""

  from huggingface_hub import list_inference_endpoints  # type: ignore[import]

  available_endpoints = list_inference_endpoints("*")  # Line 443: This line is not at the right place
  if _is_huggingface_hub(self.llm) or (
      hasattr(self.llm, "repo_id") and self.llm.repo_id
  ):
      self.model_id = self.llm.repo_id
      return
  elif _is_huggingface_textgen_inference(self.llm):
      endpoint_url: Optional[str] = self.llm.inference_server_url
  elif _is_huggingface_pipeline(self.llm):
      self.model_id = self.llm.model_id
      return                      # My code lies in this case where it does not use available endpoints
  else:
      endpoint_url = self.llm.endpoint_url
                                  # Line 456: The line should be here instead
  for endpoint in available_endpoints:
      if endpoint.url == endpoint_url:
          self.model_id = endpoint.repository

  if not self.model_id:
      raise ValueError(
          "Failed to resolve model_id:"
          f"Could not find model id for inference server: {endpoint_url}"
          "Make sure that your Hugging Face token has access to the endpoint."
      )

System Info
huggingface-hub           0.23.2          Client library to download and publish models, datasets and other repos on the huggingface.co hub
langchain                 0.2.1           Building applications with LLMs through composability
langchain-core            0.2.2           Building applications with LLMs through composability
langchain-huggingface     0.0.3           An integration package connecting Hugging Face and LangChain
langchain-text-splitters  0.2.0           LangChain text splitting utilities
sentence-transformers     3.0.0           Multilingual text embeddings
tokenizers                0.19.1
transformers              4.41.2          State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow

platform: linux
python: Python 3.10.12

@c-pletinckx Fixed!

@keenborder786 Cool ! Is it possible to merge the pull request and release a new version of langchain-huggingface with the fix ?

Same issue. I am still getting this error

Same issue. I am still getting this error

tokenizer = AutoTokenizer.from_pretrained(name,use_auth_token=auth_token)

bnb_config = BitsAndBytesConfig(
    load_in_8bit=True,

)

model_config = AutoConfig.from_pretrained(
    name,
    use_auth_token=auth_token,
    tempreature=0.2,
    top_p=0.95,
    repetition_penalty=1.15,
    max_new_tokens=1200,
)

model = AutoModelForCausalLM.from_pretrained(
    name,
    trust_remote_code=True,
    config=model_config,
    quantization_config=bnb_config,
    device_map='auto',
    use_auth_token=auth_token,
)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, max_new_tokens=4096, device_map="auto", streamer = streamer)
llm = HuggingFacePipeline(pipeline=pipe)

chat_model = ChatHuggingFace(llm=llm)

i think line 320 should not be there if i am using local llm
def __init__(self, **kwargs: Any):
        super().__init__(**kwargs)

        from transformers import AutoTokenizer  # type: ignore[import]

        self._resolve_model_id()

        self.tokenizer = (
            AutoTokenizer.from_pretrained(self.model_id)
            if self.tokenizer is None
            else self.tokenizer
        )

320 self._resolve_model_id()

please modify  def _resolve_model_id(self) in langchain_huggingface/chat_models/huggingface.py
like this

we can load and use local model from huggingface

same issue here, running llama3-8b-instruct locally yet the huggingface hub looking for a token. In the thread above one claimed it's fixed, but I installed the package yesterday and it's not fixed.

@c-pletinckx Fixed!

I dont think it's fixed man


Change video functionality to use GStreamer backend (Trac #108)

Migrated from http://microfluidics.utoronto.ca/microdrop/ticket/108
{
    "status": "closed",
    "changetime": "2014-04-17T19:39:01",
    "description": "",
    "reporter": "cfobel",
    "cc": "",
    "resolution": "fixed",
    "_ts": "1397763541728826",
    "component": "microdrop",
    "summary": "Change video functionality to use GStreamer backend",
    "priority": "major",
    "keywords": "",
    "version": "0.1",
    "time": "2012-06-20T21:59:02",
    "milestone": "Microdrop 1.0",
    "owner": "",
    "type": "enhancement"
}

Trac update at 20120623T17:22:48<EMAIL_ADDRESS>commented:

commit c17eab023b1746994001e9735c93106d318a3f0b
Author: Christian Fobel<EMAIL_ADDRESS>Date:   Thu Jun 21 10:24:20 2012 -0400
Add GStreamer libs and SDK to install dependencies

See #112.
Note that the GStreamer libraries must be added to the system PATH
variable and the GStreamer Python bindings must be added to PYTHONPATH.
Also, if using a version of Python other than 2.6, the correct libraries
and Python bindings must be copied into the main GStreamer lib
directory.
Need to perform check in install_dependencies.py to verify these steps
have been done.

Trac update at 20120623T17:22:52<EMAIL_ADDRESS>commented:

commit 2b27d5d80b334a1335c2227d12c7d3cef7009f5c
Author: Christian Fobel<EMAIL_ADDRESS>Date:   Wed Jun 20 18:00:22 2012 -0400
Change video code to not use camerabin

Change video code to not use camerabin, since camerabin does not work in
Windows.
Disable video recording, since it is broken in Windows.
Close down video pipeline when program is closed with ALT-F4.
See #112.

Trac update at 20120623T19:55:35:

<EMAIL_ADDRESS>commented:

commit 634b53f012b38c48169bc2e5ccf28903f1e2c00f
Author: Christian Fobel<EMAIL_ADDRESS>Date:   Sat Jun 23 15:51:42 2012 -0400
Add post-install configuration for GStreamer

Copy correct version of Python binding to system site-packages path.
Instruct user to add %GST_PLUGIN_PATH% to the system PATH.
Fixes #112.

<EMAIL_ADDRESS>changed resolution from "" to "fixed"
<EMAIL_ADDRESS>changed status from "new" to "closed"

Trac update at 20140313T20:02:55: ryan changed milestone from "1.1" to "1.0"

Trac update at 20140417T19:39:01: ryan changed component from "" to "microdrop"


✨ Adding VN Agent PodIP Provider

What this PR does / why we need it:
This adds the ability to use the vn-agent as a DaemonSet but uses vn-agent pod IPs as the routable address to configure with the vnode UWS'ed object. Because this interface is used for all syncing any updates to the node status object trigger a reconciliation. We handle vn-agent pod failures with status updates which triggers a resync.

⚠  For anyone implementing this you should have a node level check for example node-exporter or an initContainer on the vn-agent which updates a condition on the Node or other part of the node object. By doing this the syncer will update all your VirtualClusters VNodes anytime the vn-agent is rescheduled, this can be done without rebinding.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
No issue filed.
Signed-off-by: Chris Hein<EMAIL_ADDRESS>

/cc @Fei-Guo

Blocked on #270

/rebase

/retest

One last nit, please change the copyright year for all touching files.

@Fei-Guo ✅ done, just waiting for tests to pass again.

/lgtm
/approve


FC001 should not fire node.run_state[:foo]

node.run_state[:foo] is valid. We should not alert on these.

Fixed in Foodcritic 11.3


Setup CI

[ ] Add integration tests with git ls-files (current one is with recursive list)
[ ] Build and push to Github releases jar+GraalVM binaries with https://github.com/actions/upload-release-asset#example-workflow---upload-a-release-asset

GraalVM Gradle (they're not using new releases by default):
🍏 ➜  gw cli:nativeImage

> Task :cli:nativeImage
Build on Server(pid: 5699, port: 63970)
[kowners:5699]    classlist:     536.98 ms
[kowners:5699]        (cap):   3,553.83 ms
[kowners:5699]        setup:   3,957.01 ms
[kowners:5699]   (typeflow):   3,555.78 ms
[kowners:5699]    (objects):   3,218.45 ms
[kowners:5699]   (features):     151.14 ms
[kowners:5699]     analysis:   7,028.08 ms
[kowners:5699]     (clinit):      98.73 ms
[kowners:5699]     universe:     210.73 ms
Warning: Reflection method java.lang.Class.newInstance invoked at kotlin.internal.PlatformImplementationsKt.<clinit>(PlatformImplementations.kt:50)
Warning: Reflection method java.lang.Class.newInstance invoked at kotlin.internal.PlatformImplementationsKt.<clinit>(PlatformImplementations.kt:41)
Warning: Reflection method java.lang.Class.getMethods invoked at kotlin.internal.PlatformImplementations$ReflectAddSuppressedMethod.<clinit>(PlatformImplementations.kt:18)
Warning: Aborting stand-alone image build due to reflection use without configuration.
Warning: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Build on Server(pid: 5699, port: 63970)
[kowners:5699]    classlist:      89.58 ms
[kowners:5699]        (cap):   2,611.17 ms
[kowners:5699]        setup:   2,837.82 ms
[kowners:5699]   (typeflow):   1,339.39 ms
[kowners:5699]    (objects):   1,486.96 ms
[kowners:5699]   (features):     102.55 ms
[kowners:5699]     analysis:   2,982.65 ms
[kowners:5699]     (clinit):      65.71 ms
[kowners:5699]     universe:     156.91 ms
[kowners:5699]      (parse):     213.96 ms
[kowners:5699]     (inline):     647.39 ms
[kowners:5699]    (compile):   1,647.42 ms
[kowners:5699]      compile:   2,696.35 ms
[kowners:5699]        image:     220.43 ms
[kowners:5699]        write:     171.18 ms
[kowners:5699]      [total]:   9,195.55 ms
Warning: Image 'kowners' is a fallback image that requires a JDK for execution (use --no-fallback to suppress fallback image generation).
native image available at build/graal/kowners (3 MB)

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 24s
7 actionable tasks: 4 executed, 3 up-to-date

Seems not good regarding this post.


initial look at configuring source list

Hello, as a preface, this is my first open source pr. I'm also more or less making a suggestion, I wouldn't use the code I've pushed as it's fairly crude (and it's my first time writing rust), its just to show what I'm suggesting can work. Maybe this should be a feature request rather than a pr?
I'm a big fan of this project. I was interested in being able to configure the source lists instead of having to do so manually in ~/.local/share/flavours/base16/sources.yaml after retrieving the default schemes and templates with flavours update all. I've only poked around the code briefly and slapped together a simple way of achieving this.
The Config struct now has four new fields:

schemes: A string of the link to a repo of a list of schemes of this format
schemes: A string of the link to a repo of a list of templates of this format
extra_schemes: A list of strings, each a link to a base16 colorscheme repo of this format. This is not implemented in this pr. Let's say I wanted to use the default scheme list with additional schemes, I'd add them to this list.
extra_templates: A list of strings, each a link to a base16 template repo of this format. This is not implemented in this pr. Let's say I wanted to use the default template list with additional templates, I'd add them to this list.

I'm not familiar with the base16 scheme and template repos or how frequently prs are accepted. For the purpose of adding new schemes/templates maybe this solution is overkill if new themes/templates are easily added there. But I would like to be able to specify my own scheme/template lists that are used when I run flavours update all.
Again I'm not the most familiar with your design choice and source code. Is this a reasonable suggestion, does it conflict with any of your design choices? Is there a way of achieving this as is? Would you be interested in implementing this feature at all (I of course would be willing to help with this)?

Regardless of if this idea is implemented, I'm going to use this fork once I finish the functionality I mentioned. I'll make a new when I feel it is ready.


Allow passing a custom client instance.

Resolve #429
Hi there. Apparently this came up with another user too: https://github.com/FormidableLabs/urql/issues/429
I had the same need so instead of continuing to roll a custom useQuery hook I thought I'd write a quick PR.

I also stumbled upon this problem and ended up having to wrap my component into another urql.Provider to support this usecase. This is not ideal as it means the children hierarchy of the wrapped component has a different provider, and this might be unexpected for the children (in my case it's fine, the component is very small and its children do not use useQuery).
It would be much simpler if useQuery and consorts could accept a client parameter.
@kitten Can you expand on why this would not be natural for this hook and why it would not be inline with React ?


Use E5-multilingual as a default

Instruct-xl indeed is on rank 2 atm, while E5-multilingual is on place 5, however multilingualism will greatly enhance the overall usability of the document search imo, it performs really well when used.
I guess just make it optional by default, just as the LLM, but I suggest using that as a default embedding model. It's also only 2.2 Gb not ~5GB large...
Great Repo!

Thanks for the suggestion, I will add a support for it.

Added support for e5 models through sentence-transformers. Great model indeed - subjectively pulling more relevant results compared to the instructor-large. Thanks again!


Device Login

Allow users to sign into the API via device login instead of using client credentials to login. This allows then to use the CLI tool from any directory instead of ensuring they have an up-to-date .env config in their cwd.
Acceptance Criteria

[ ] Printout the URL to sign in the device, as well as the one-time authenticator code
[ ] Poll for 1-2 minutes
[ ] Get the security schemes from the portal's openapi.yaml
[ ] Use the well-known configurations to extract the OIDC endpoints, so we don't have to emded them in the CLI

Feature branch details:

Endpoint: https://api-services-portal-feature-gwa-cli-v2.apps.silver.devops.gov.bc.ca/ds/api
The endpoint will have a "link" header in a standard format like: </ds/api/v2/openapi.yaml>; rel="service-desc".  Take the link of the service-desc and get the OpenAPI spec
OpenAPI spec: https://api-services-portal-feature-gwa-cli-v2.apps.silver.devops.gov.bc.ca/ds/api/v2/openapi.yaml
Under components/securitySchemes/openid/openIdConnectUrl this will be the URL for the .well-known/openid-configuration

Following RFC6749 (https://www.rfc-editor.org/rfc/rfc6749#section-4.4.3), the client credential login will not return a refresh token, so when the token expires, the user will either have to run gwa login again with the client id or secret, or the credentials could be stored in a local file under user's home directory.


Executing get_tree().quit() on the HTML 5 export crash the site, other browser tabs still works.

Godot version:
3.2.3
OS/device including version:
GTX 1650
Issue description:
Executing get_tree().quit() on the HTML 5 export crash the site, other browser tabs still works.

This should be implemented in 3.2.3 thanks to https://github.com/godotengine/godot/issues/23010, but it's seems it's not working on your end?
Which browser are you using? Also, please upload a minimal reproduction project to make this easier to troubleshoot.

Hello Calinou as always :D
I am currently using Firefox developer edition v82.0b6
It really is quite simple, a button with get_tree().quit() attached to the Esc keyboard key, export to HTML 5.
Press it and watch it hang ;p

It really is quite simple, a button with get_tree().quit() attached to the Esc keyboard key, export to HTML 5.
Press it and watch it hang ;p

So you actually get what you want. The next step is to close the web-browser tab :wink:

I'm unable to reproduce this in latest beta.
Can you provide a screenshot of the "crash". I don't really understand what "crash the site" means.

Closing since there is no reply.
Feel free to open a new issue if you have more details, or comment below (and I'll reopen).


Removed an unwanted file and fixed an example

I accidentally pushed a binary to the repo so I have to delete it
I also forgot to unload the images in textures_image_generation.c
Maybe I tried to go a little too fast 😅

Hey! No worries! Thanks again for all your effort!


Please describe HelpKeyword in more detail

The current doc's say:

HelpKeyword : Optional String parameter. The Help keyword to associate with the error.

Which kind of is not very helpful.
Please document what can and can't be done using HelpKeyword.
(e.g. I can "see" that whatever I input there it seems to always redirect to docs.microsoft.com. Is there any way to get this to link to somewhere else?)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 92240a7d-b3d0-997d-4608-0cbf9b0772b8
Version Independent ID: 730a0427-8083-e401-0f3b-6b75b9c6f94a
Content: Error Task - Visual Studio
Content Source: docs/msbuild/error-task.md
Product: visual-studio-windows
Technology: msbuild
GitHub Login: @ghogen
Microsoft Alias: ghogen

@nils-a I think this is used by Visual Studio to generate an F1 query, which are exclusively processed by docs.microsoft.com, so if that's the case, there's no way to direct this traffic anywhere else, or any way to add user keywords.

I think [...]

This illustrates my point, doesn't it. 😄
For me, adding HelpKeyword will always result in a link to https://msdn.microsoft.com/query/dev16.query?appId=Dev16IDEF1&l=EN-US&k=k(MY-KEYWORD-HERE)&rd=true.
If that is really the case, and there is nothing to do about it then the docs shoutd read "Internal use by Microsoft. Nothing can be gained by using this property." (or something like it...)
(The thing is.. When developing a roslyn analyser, there is a HelpLink, which will create a nice linkt to whatever url is given - so I'm guessing this is not only Visual Studio doing things here...)

@ghogen in https://github.com/dotnet/msbuild/issues/4029 @davkean writes:

HelpKeyword has a different meaning; it means the keyword that we use to look up a help topic in help.

So it seems, that the documentation really should read something along the lines of "Keyword to a help-topic. Visualization of this keyword is build-system dependent. For Visual Studio this points to a Keyword-Search on msdn.microsoft.com."

Action is still required.


Can't boot on 1.21

Type of Error
Crash
Description
The game crashed whilst initializing game
Error: java.lang.RuntimeException: Could not execute entrypoint stage 'client' due to errors, provided by 'vmc-mc'!
Mod Version
0.4.1
Minecraft Version
1.21
Mod Loader
Fabric
Steps to Reproduce

Crash Log
The game crashed whilst initializing game
Error: java.lang.RuntimeException: Could not execute entrypoint stage 'client' due to errors, provided by 'vmc-mc'!

Can you provide the crash log?

Issue solved as of version 0.4.2


size of the terms of use and privacy policy should be the same as CanBeWell 1.0

the privacy policy and terms of use in landing page in transgender is in different size as CanBeWell 1.0

tested successfully


Multiple Variants of Search on Trie

Description of Change
An implementation of 3 variants of search function on trie. A simple search determining if the word exists in the trie. A Suggestions Search which suggests all possible words in trie which share the longest prefix with the search key. And a frequency based Suggestions Search, which suggests top3 most frequently searched words in trie which shares the longest prefix with search key.
Checklist

[x] Added description of change
[x] Added file name matches File name guidelines
[x] Added tests and example, test must pass
[x] Added documentation so that the program is self-explanatory and educational - Doxygen guidelines
[x] Relevant documentation/comments is changed or added
[x] PR title follows semantic commit guidelines
[x] Search previous suggestions before making a new one, as yours may be a duplicate.
[x] I acknowledge that all my contributions will be made under the project's license.

Notes: 3 variants of search explicitly defined, to be performed on trie. Explicitly defined so people can refer/use them directly.

Not sure if we still accept algorithms in the operations_on_datastructures folder? Should we add a note? @kvedala @ayaankhan98.

Not sure if we still accept algorithms in the operations_on_datastructures folder? Should we add a note? @kvedala @ayaankhan98.

This problem is more about organising data in data structure in such a way that we can have frequent fast  access to data. It's not about any purely mathematical algortihm but a task of general aptitude thinking. we can accept this under operations_on_datastructure. Moreover it's one of the basic system design interview question which is often asked as a design of auto complete feature.

Please fix clang-tidy warnings.
All Checks Cleared.

👍 Good code.

Thank you. Have changed the documentation as suggested.

Have implemented the suggested changes. Also removed #include as  includes std::vector.

Amazing work, I like how you've improved & made the code so well. 🎉
Code and documentation look good and very refined; LGTM. 😄 👍

Thanks! And really thank you for guiding me through this David. You helped a lot.

Great job @g-s-k-zoro

Thanks! Also thanks for your suggestions.


Refactor/deletions

Several deletions for code unused on onwanted

Please add a message on the commit saying that some deprecated libraries were removed

Done


Define case-sensitivity for models

As far as I can tell, your options for enabling case-sensitivity are
a) define it on the adapter, which is shared by all of your database tables
b) define it on a per-query basis
Given we have already created a ton of indexes to hit the lowercased query, changing the adapter to be case sensitive is a nonstarter. Per-query is also annoying because you need to define it for each query a table uses.
Any way to set a caseSensitive attribute on a model object? that way queries using that table would be case sensitive; it seems like the right unit of on/off-ness for case sensitivity.

actually now that I double check it looks like the syntax proposed here: https://github.com/balderdashy/waterline/issues/239 does not work, the generated query is
SELECT ... WHERE LOWER("table"."driverId") undefined AND "table"."driverId" undefined


Improve Sync Diagnostics

reduce the log levels in the block verifiers, because some peers on mainnet
provide blocks from low-height side chains
log the height when the block verifiers reject blocks
make sync diagnostics and metrics more consistent
tweak some sync log levels

See #819 and #835 for analysis and alternative fixes.


Box Plot

Can React-Vis be used for a Boxplot? I need to plot several boxplots on the same scale so they can be compared.

It sure can! It's not baked in as part of the main library, but can be built super easily. Checkout this example (https://github.com/uber/react-vis/tree/master/examples/candlestick) for one technique on how to do it.

Thanks!

Marking as resolved, feel free to re-open if you have additional questions


Updated content SDK to release version

Verified that @brianytang has signed the CLA. Thanks for the pull request!


Allow for style property

This is just a small extension of the ImageLoader component to allow specifying a style property to be applied as a style attribute to the rendered DOM node.

Thanks for this PR! I'd like to just pull this in, but I'm afraid it would have to change in fairly short order, as we're currently looking at a general solution for delivering props to the underlying <img> in #18. So, a merger between the solution we're discussing there and your solution might look something like:

// `imgProps` are passed as props to the underlying `<img>` DOM node
<ImageLoader imgProps={{style: {border: 'none'}}} />

Would that work for you?

Hi, thanks for the feedback.
I like the proposal from #18, but I don't think that an added imgProps property would interfere with my proposal for an style property, because imgProps will be applied to the contained img element (when — this is, if — successfully loaded and rendered), whereas the style would be applied to the container component (the wrapper), regardless of the success or even progress of the contained image.
In the end, I personally would like to see both of them eventually. What do you think?

Aha, yes, you're right, i misunderstood your intention! I"m still inclined to pull this in, but i'd like to pull in just the change, and reserve the version bump and changelog updates for when we are ready to cut the next release (which will be soon). If you've got the time, would you mind updating the PR to include just the change and the documentation (a simple test would be icing on the cake, but again, if you have the time)? If not, no worries, i can work with you've already got here.
Longer-term, we're considering simply forwarding any props given to ImageLoader (that aren't specific to it) to the wrapper element, and then adding something like the imgProps feature discussed in #18 to allow customization of the underlying <img> itself.

I modified the PR accordingly. Let me know if the test is sufficient.

Looks great, thanks a lot for the test!


fixed subtitle errors in ep.33 (Enligsh Version)

timestamp: 00:00:26,640 --> 00:00:28,620
modification: notification --> authentication
timestamp: 00:04:21,113 --> 00:04:22,923
modification: of --> or

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Automatic man-page generator

Provide some hooks to generate a man page file for the configured arguments and docstrings.

Thank you for reporting this issue and appreciate your patience. We've  notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.


Functions to implement for the package listwiz

Sorting Algorithms

~Bubble Sort: https://en.wikipedia.org/wiki/Bubble_sort~
Insertion Sort: https://en.wikipedia.org/wiki/Insertion_sort
~Selection Sort:~
~Merge Sort for us.~

Lists manipulation/analysis

the longest increasing stick
the longest decreasing
the location and value of the smallest value
the location and value of the largest value
the element that occurs the most often

I have created an individual issue for all of these now, I think, so closing.


docs(v2): Elaboration of raw-loader in markdown react component

Motivation

Make it clear that one needs to install raw-loader before using it
Removed the ; after CodeBlock component, otherwise it will render the unintended semicolon after code block
Added that one call pass the title prop to CodeBlock component as well.

Have you read the Contributing Guidelines on pull requests?
Yes

Removed the ; after CodeBlock component, otherwise it will render the unintended semicolon after code block

I'm afraid this is automatically added by prettier.  I see why the lint test failed after earlier pull req. Thank you for catching it.

Thanks
About the ; yes it's prettier because we use a jsx code block. Unfortunately there's no support for mdx in prism so using jsx is required to get highlighting working.

@etho201 I guess you have to wrap the title prop in quotes, i.e., ```bash title="script.sh".

Thank you both for the comments. I'm really not sure what happened... I tried with and without quotes earlier and it still didn't work. Once I reverted back (using git reset --hard origin/main) the problem went away. Afterwards I was hesitant about trying raw-loader, but I just tried it again and this time everything is working just fine. 🤔

Could be a caching problem, should've run yarn clear...

Thanks for the tip. I'll definitely keep that one in mind.
On Fri, Aug 13, 2021, 1:24 AM Joshua Chen @.***> wrote:

Could be a caching problem, should've run yarn clear...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/facebook/docusaurus/pull/5210#issuecomment-898199937,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADS7EBZPKYRRE43W2TW73T3T4SUBNANCNFSM5A3MLQTQ
.


MOTECH-2695: Add element id naming conventions to email module

@nickdotreid Did you have a chance to take a look at the PR after fixes?

@sebbrudzinski thanks for bringing this back up... it looks good and I merged it.... I will follow up with Piotr in the related ticket to see if this fits his needs....


Legion: non informative error in case of impossible compact sparse instance covering

If the call to compute_covering (https://gitlab.com/StanfordLegion/legion/-/blob/control_replication/runtime/legion/region_tree.inl#L372) fails during the creation of a compact sparse instance, the runtime ends up computing the potential memory footprint of the instance using a NULL layout. As a result, the default mapper issues errors like
[0 - 7f525eff6840]  169.168499 {5}{default_mapper}: Default mapper failed allocation of size<PHONE_NUMBER>49674 bytes for region  requirement 1 of task UpdateUsingDiffusionFluxZ (UID 7548) in memory 1e00000000000000 (SYSTEM_MEM) for processor 1d00000000000003 (OMP_PROC). This means the working set of your application is too big for the allotted capacity of the given memory under the default mapper's mapping scheme. You have three choices: ask Realm to allocate more memory, write a custom mapper to better manage working sets, or find a bigger machine.

whereby a non-sensical memory requirement is reported.
Having a more informative error would help the debugging of the layout constraints.

I think I will need a reproducer because Legion I can't see how Legion can return something other than 0 or the right size of the instance. We initialize the instance layout size here:
https://gitlab.com/StanfordLegion/legion/-/blob/control_replication/runtime/legion/region_tree.inl#L350
And that should propagate all the way back out to the footprint that the mapper sees if the covering computation fails. This probably also needs to be fixed, but that means we still don't understand how you get a crazy large size for the instance, and we need to understand that before fixing anything.

The issue has been fixed in 90085a37.
Thank you very much!


Fix tld catching when subdomains and TLD options declared

Currently when both subdomains and TLDs are set some of the TLDs are missed.
Re-visiting the conditional filter for the logic on handling both conditions.

Closing, the other pages were outside the domain. Working as intended.

Okay, I'll take a look, thanks!


test sources not being compiled

I was expecting the plugin to compile schemas in src/test/avro as well, alas it did not.

just add the following lines to SbtAvro.scala
// test schema location
sourceDirectory <<= (sourceDirectory in Test) { _ / "avro" },
// generated files
javaSource <<= (sourceDirectory in Test) { _ / "java" },


Notation: from beta to w

Codecov Report

Merging #32 into master will not change coverage.
The diff coverage is 66.66%.

@@           Coverage Diff           @@
##           master      #32   +/-   ##
=======================================
  Coverage   95.06%   95.06%
=======================================
  Files           4        4
  Lines         162      162
  Branches       14       14
=======================================
  Hits          154      154
  Misses          4        4
  Partials        4        4

Impacted Files
Coverage Δ

celer/dropin_sklearn.py
100% <ø> (ø)
:arrow_up:

celer/homotopy.py
100% <100%> (ø)
:arrow_up:

celer/wrapper.py
76% <33.33%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c4ddc2...8ad321a. Read the comment docs.


Provide ability to publish and consume binary data directly

CASE:
OutboundMessage#msg is a String,
In this way, it forces us to use marshalling (XML, JSON, YML) insted of serialization (CBOR, protobuf)
In the case of converting a binary array to we at least lose time. In addition, the string representation of the binary array will weigh twice as much since char = 16 bit vs byte = 8 bit
For instance you can compare implementtion of methods dump and dumps in kotlinx/serialization/SerialImplicits.kt kotlinx-serialization-runtime-0.11.1

Another bad thing I discovered is that your API is non-persistent in terms of data types.
As I already said OutboundMessage#msg is a String, Not so bad on its own.
But in com.viartemev.thewhiterabbit.consumer.ConfirmConsumer all methods use the following lambda (com.rabbitmq.client.Delivery) -> kotlin.Unit
And com.rabbitmq.client.Delivery#_body is a byte[]
So we publish the String and consume byte[]
CASE1: if we publish JSON (for i.e.) we consume byte representation of JSON (looks simple)
CASE2: if we publish CBOR first of all we encode it as string and we consume byte representation of what we encode.
Now it our object that was bynary encoded then represenrted as string and this string represented as byte array  (looks hard)

The PR with fix
https://github.com/viartemev/the-white-rabbit/pull/268

The PR with fix is https://github.com/viartemev/the-white-rabbit/pull/268
it was based on v.0.0.5 as the master branch is not in production-ready state
For master com.viartemev.thewhiterabbit.channel.TxChannel.Transaction line 37 fix needed.

@laserg thank you for contribution.
I totally agree with you, that OutboundMessage should have ByteArray instead of String. As a part of work on RPC implementation, I added a new RabbitMqMessage type which has only basic properties and body.
In the future, all methods of the library should operate with this type. Feel free to contribute.

Give me a couple of minutes. I will fix PR.

@laserg Moving to new RabbitMqMessage type required adding Envelop field to this type also

@laserg I agree with you about inconsistency. But this problem should be solved in another issue. Before that, all required parameters should be collected and after we can move to one general type.

@viartemev Yep. Now https://github.com/viartemev/the-white-rabbit/pull/268
Contains backward compatible API fix,

This feature available in new version

@viartemev, please do not forget to close this issue.
Thank you for fast release.


publish v1.2.0 of Elemental

publish v1.2.0 of Elemental

I haven't dived into this much. Has the chart / output been validated in anyway?

Yes @richard-cox . The extension has been validated extensively by QA, which has given me the green light (along with the Elemental backend team) to publish it.


Find a way for proto item migrations to merge rule element sources

Currently we just leave the sources on the item as they are. This is not always intended.

Maybe we could split them up or mark them as system or user provided.


Added precompiled header support

Automatically includes the precompiled header file and you can choose the name of this header file from the Configuration.cpp file

You benefit exactly zero seconds from using pch with this project, everything is done in the dllmain.cpp file.

Not necessary and requires people to manually modify their config file themselves if they want to use the latest update, not backwards compatible friendly. Any good compiler has a force include pch header option in settings anyway, don't need to manually do #include in your files if you have that setup.


Thumbnail Template Management

Rather than store thumbnail templates as files that have to manually copied to the edit nodes, store the thumbnails and associated metadata in the database (#358). Additionally, the crop and location thumbnail parameters can be overridden (#377) and a one-off template can be combined with a video frame to produce a custom thumbnail (#366).
This is a work in progress and has not been tested (although it does build).

I've made the minimal change to thrimbletrimmer to keep it working using the new thumbnail apis, and am going to merge this so that I can get the database changes into master.
Just to check my understanding, after this PR is merged, we have the following left to do:

[ ] Auto-attribution needs to actually happen (added to description in thrimshim)
[ ] UI for template management
[ ] UI for one-off templating
[ ] Better way of doing default templates

Did I miss anything?


mingw compiler support

Hi
in the main page , there is ( Supported compilers : GCC 4.6 and above ), is this for linux only ?

No, MinGW is supported...
See full list:
https://github.com/bkaradzic/bx/blob/master/scripts/toolchain.lua#L11


Projectile Speed Not Relative To Grid It's Shot From

Description

When you shoot a gun into space, on a shuttle that is currently moving, the projectile flies back into the shuttle, as it's speed is not relative to the shuttle.
Reproduction

Make hole in shuttle.
Move shuttle in some direction.
Shoot through hole in direction of movement
???

Demonstration is made better by shooting something explosive like an RPG, as you can clearly see where the projectile landed.
Additional context

Shiptest literally impossible because of this.

https://user-images.githubusercontent.com/44201461/205448180-fd1e8e88-e230-42c3-8314-cb9554b31f22.mp4

(the bug is demonstrated in the end of the video)


2020-11 Redeploy CyHy Environment

Procedure
Add the following to the Redeployments tracker
- [ ] [202X-XX](https://github.com/cisagov/cyhy_amis/issues/###)

Follow the build and deploy instructions.
Check off each service as it is redeployed to track the overall status. Once all services have been redeployed and functionality confirmed, close this issue and check off the month on the Redeployments tracker.
Status

[x] BOD

[x] Bastion
[x] Docker

[ ] CyHy

[x] Bastion
[x] Dashboard
[ ] Database
[x] Reporter
[ ] Scanners

[ ] Portscanners (nmap)
[ ] Vulnscanners (Nessus)

The scanner and database redeploys are on-hold until the merger of the following PRs:

https://github.com/cisagov/cyhy_amis/pull/292
https://github.com/cisagov/cyhy-runner/pull/2
https://github.com/jsf9k/cyhy-commander/pull/17


Stuck on post where neither "deleting" nor "hiding" was an option

Maybe related to #17. The program seems to have been stuck on a post from 2013 where neither "deleting" nor "hiding" was an option. I manually "untagged" me from the post/photo and restarted the program, which worked fine, but this might want to be considered fixing as a future enhancement.

I'm going to close this again, since I am not sure anymore this is what actually happened and I might have jumped the gun here.

I think this is a real thing, but it's hard to reproduce. I think Facebook has removed certain features over the years so things are a bit inconsistent.

Reopening since I think it might be a real issue but it's hard to reproduce.

I'm consistently stuck on this step, and the situation is this:
A friend posted a picture on another friends' wall in which I was tagged. The option presented in the context menu is to "Remove Tag".

Found another post with it: the parameter for selenium is:
data-feed-option-name="UNTAG"

Thanks, I think I can add that, even if I can't test it
On Mon, May 27, 2019, 2:05 PM Joseph Bennett<EMAIL_ADDRESS>wrote:

Found another post with it: the parameter for selenium is:
data-feed-option-name="UNTAG"
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/weskerfoot/DeleteFB/issues/22?email_source=notifications&email_token=AAC4L3YQXYTLXFEIXUM6OV3PXQPIFA5CNFSM4HOWWKP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWKJRKY#issuecomment-496277675,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAC4L37Q3DKCENG6RHCF7W3PXQPIFANCNFSM4HOWWKPQ
.

It should just need an additional case here to handle it https://github.com/weskerfoot/DeleteFB/blob/565e01f6d0499f2b405ef50c006d772068e253fe/deletefb/tools/wall.py#L27

https://github.com/weskerfoot/DeleteFB/pull/44 may solve this
Could someone test? @jsmbennett @Neugierdsnase

Should be fixed now


Implement Application Rollback into the PIpeline

[ ] Implement Application Rollback into the PIpeline

Version of the application that will be deployed to an environment will depend on deployment.properties.
Implemented reading version and build number from the properties file and deploy it to the development environment.

Tested the rollback process in the development environment.
Used TAG "rollback-dev.v0.0_rc4" and  pipeline was triggered and extracted application version details from deployment.properties.
Based on the information that was extracted, application package was downloaded S3 and deployed in the development environment.


Fix CORS and mixed content issue

In this PR I used  Nuxtjs serverMiddleware to create a proxy server and pass the requests through the proxy and get over CORS and mixed-content issue's
resolving https://github.com/liyasthomas/postwoman/issues/175 and https://github.com/liyasthomas/postwoman/issues/2

Hey @hosseinnedaee,
Your changes look good to me!
View build log
TravisBuddy Request Identifier: 48432ae0-f2b3-11e9-8575-bb3eea981d29

Oh- just one other issue, we need to add Access-Control-Allow-Headers: * to the proxy response

See #200 which includes these commits with some finishing touches and bug fixes.


Merge main into develop

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.39%. Comparing base (d61da57) to head (724edee).
Report is 6 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #543      +/-   ##
===========================================
- Coverage    90.47%   90.39%   -0.09%
===========================================
  Files          148      148
  Lines        10838    10838
  Branches       511      511
===========================================
- Hits          9806     9797       -9
- Misses         868      877       +9
  Partials       164      164

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests


[VS Code] Sign-in failed.

Error:
Sign-in failed.
Steps to Reproduce:

Connect to NetMotion (VPN)
Open Visual Studio or Visual Studio Code.
Attempt to start a new sharing session (Visual Studio) or sign into Live Share (VS Code)
Visual Studio will immediately fail with the error:

[2018-03-09 22:02:37.396 VSIX I] Acquired an access token for the VSO account '<14:eb047f03>'
[2018-03-09 22:02:37.420 VSIX E] Failed to create a collaboration session. An internal error occurred. Please contact Microsoft Product Support Services.

VS Code will get to 'Ready to Collaborate' page on browser, but will display 'Sign-in Failed' on actual instance of VS Code.
Note: If I disable VPNing, which probably also disables use of my corporate proxy, everything works as expected.
Note: My system currently works (on the VPN + Proxy) with Spotify, MS Teams, NuGet in Visual Studio, and Docker, which either natively support IE proxy settings or have their own setup for proxy details.
Note: When I use bash shell on windows and CURL https://download.microsoft.com, it fails with (56) Proxy CONNECT aborted.
Note: I have the HTTP_PROXY and HTTPS_PROXY environment variables set up with my corporate proxy url (url only, no credentials).

Version Data

extensionName
VSLS

extensionVersion
0.2.206

protocolVersion
2.1

applicationName
VSCode

applicationVersion
1.21.0

platformName
Windows

platformVersion
10.0.14393

@killnine Thanks for reporting! The fact this only happens when you are on VPN makes me suspect this may be #86.  @jasongin Is that your thinking as well?
To know for certain, it would be good to have the logs from your machine when you hit this.  Could you repro and run the "Live Share: Export Logs" command to get a zip and then either attach here or email<EMAIL_ADDRESS>Assuming that is the case we can merge your issue with the thread on the other issue.

Yes, it's likely our bug with authenticating proxies, as reported in #86. To confirm, look for the "407 Proxy Authentication Required" message in the log.

Thanks Jason.  @killnine Are you okay with us closing this bug and moving the details to #86 to consolidate the conversation for others as well?

Yeah that’s okay with me

Great - Closing as dupe and moved your details over.