[WiP][Windows] Support original_dst/sidecar

Signed-off-by: Sotiris Nanopoulos<EMAIL_ADDRESS>Commit Message:
Fixes #13955
Additional Description:
The architecture on Windows is the following:

When the original_dst filter is used on Windows, we use SO_ORIGINAL_DST to query for the original address.
If the traffic direction is Outbound and address has been restored. Then query for the WFP redirect records (think about this as the firewall rules) and apply the redirect records to the outbound connection.

Pending work

[ ] Validate case where traffic is ipv6
[ ] Make a cohesive story of the #ifdefs to ensure a smooth experience between older and newer platforms
[ ] Add unit/integration tests
[ ] Document design choices regarding traffic_direction
[ ] Document platform version support

Risk Level: Low (Windows only, it should be a null-op for Unix)
Testing: Pending
Docs Changes: Document the usage of inbound traffic property
Release Notes: N/A
Platform Specific Features: Windows Parity

@ggreenway some points that I am looking for some feedback on:

is stream_info appropriate info to store the redirect records
do I break any existing connection handling scenario?
Should the logic for getting the traffic direction be moved  to the filter level?

@ggreenway some points that I am looking for some feedback on:

is stream_info appropriate info to store the redirect records
do I break any existing connection handling scenario?
Should the logic for getting the traffic direction be moved  to the filter level?

I have pending work to test this ( I have been testing this manually so far) but I am waiting for some feedback on the high level structure direction of the changes and then I can move into that

I have pending work to test this ( I have been testing this manually so far) but I am waiting for some feedback on the high level structure direction of the changes and then I can move into that

Given that this data isn't present most of the time (only when using original-dst, and only on windows), it might come out cleaner to set something in the filterState() on the stream info instead of adding it at the top-level on StreamInfo.

Given that this data isn't present most of the time (only when using original-dst, and only on windows), it might come out cleaner to set something in the filterState() on the stream info instead of adding it at the top-level on StreamInfo.

Does the original dst stuff only work for direction == outbound on linux? A quick skim of the source makes it appear that direction is only used for draining-related behavior. So could you just omit the check for direction?

Does the original dst stuff only work for direction == outbound on linux? A quick skim of the source makes it appear that direction is only used for draining-related behavior. So could you just omit the check for direction?

Given that this data isn't present most of the time (only when using original-dst, and only on windows), it might come out cleaner to set something in the filterState() on the stream info instead of adding it at the top-level on StreamInfo.

Seems reasonable. I will look into it.

Does the original dst stuff only work for direction == outbound on linux?

Why do you think so? My understanding is that Istio configures a virtual inbound listener with the original_dst property to capture incoming traffic.

A quick skim of the source makes it appear that direction is only used for draining-related behavior.

Quite likely. AFAIK, I am the only one that uses the traffic_direction property in that way. The issue is that on Windows we need to distinguish between inbound and outbound traffic to apply the redirect records. This was the best way I could find to distinguish them. From inspections in Istio codebase,  it always applies that property.

Given that this data isn't present most of the time (only when using original-dst, and only on windows), it might come out cleaner to set something in the filterState() on the stream info instead of adding it at the top-level on StreamInfo.

Seems reasonable. I will look into it.

Does the original dst stuff only work for direction == outbound on linux?

Why do you think so? My understanding is that Istio configures a virtual inbound listener with the original_dst property to capture incoming traffic.

A quick skim of the source makes it appear that direction is only used for draining-related behavior.

Quite likely. AFAIK, I am the only one that uses the traffic_direction property in that way. The issue is that on Windows we need to distinguish between inbound and outbound traffic to apply the redirect records. This was the best way I could find to distinguish them. From inspections in Istio codebase,  it always applies that property.

Does the original dst stuff only work for direction == outbound on linux?

Why do you think so? My understanding is that Istio configures a virtual inbound listener with the original_dst property to capture incoming traffic.

Maybe I'm misunderstanding (or I misread your change). It looked to me like the change you're making only applies to listeners with direction == OUTBOUND. Can you explain the intention of that?

A quick skim of the source makes it appear that direction is only used for draining-related behavior.

Quite likely. AFAIK, I am the only one that uses the traffic_direction property in that way. The issue is that on Windows we need to distinguish between inbound and outbound traffic to apply the redirect records. This was the best way I could find to distinguish them. From inspections in Istio codebase, it always applies that property.

Can you elaborate on that? I just want to understand the flow of how this works on windows, and the constraints.

Does the original dst stuff only work for direction == outbound on linux?

Why do you think so? My understanding is that Istio configures a virtual inbound listener with the original_dst property to capture incoming traffic.

Maybe I'm misunderstanding (or I misread your change). It looked to me like the change you're making only applies to listeners with direction == OUTBOUND. Can you explain the intention of that?

A quick skim of the source makes it appear that direction is only used for draining-related behavior.

Quite likely. AFAIK, I am the only one that uses the traffic_direction property in that way. The issue is that on Windows we need to distinguish between inbound and outbound traffic to apply the redirect records. This was the best way I could find to distinguish them. From inspections in Istio codebase, it always applies that property.

Can you elaborate on that? I just want to understand the flow of how this works on windows, and the constraints.

It looked to me like the change you're making only applies to listeners with direction == OUTBOUND. Can you explain the intention of that?

Sorry, I realized something must have not been clear just by reading the code. With the newer Windows SDK the definition SO_ORIGINAL_DST is now defined on Windows. So the existing code for inbound traffic traffic works as it is. This is why there are no changes for the inbound direction.
We have two cases for traffic redirection:

Inbound traffic which is handled by a Virtual Inbound Listener. In this case the Envoy code work as is, assuming a version of Windows that supports SO_ORIGINAL_DST socket option.
Outbound traffic which is handled by a Virtual Outbound Listener. In this case Windows and UNIX divert (hence the changes). For outbound traffic, on Windows, we need to respect the existing firewall rules of the container.

Imagine the following scenario, which is only applicable for the outbound traffic direction:
Players: container, Sidecar Container, google.com (upstream)
When container makes a request to google.com the traffic (outbound) gets intercepted by the sidecar container. This means that we have (i) a socket between container(downstream) and sidecar and (ii) another socket between sidecar and google.com (upstream).
In this scenario the platform admin might have set firewall rules blocking traffic from container 1 to google.com. The sidecar should respect these rules when proxying the connection. This means that we should query for the redirect records of the downstream socket and re apply these records to the upstream socket.
The bulk code changes in this PR are to support the logic described above.

Can you elaborate on that? I just want to understand the flow of how this works on windows, and the constraints.

The above lead us to a subtle problem. Since Envoy needs to do "extra" work on outbound connections, how can it distinguish between the two cases?
Here I considered 3 approaches:

Rely on the IP of the address and the destination. I think this is a fragile approach with many edge cases like handling traffic within the same POD.
Try to apply the redirect records in both inbound and outbound traffic and accept that this will fail 50% of the time. This is a bit awkward in terms of the user experience. Because if something goes wrong and the outbound direction does not work then we would silently fail thinking the connection is inbound.
Use the listener traffic_direction property and rely on service mesh frameworks setting this property to the listeners. I have verified that Istio already does that.

IMHO approach (3) is the best but I am always open to suggestions.

It looked to me like the change you're making only applies to listeners with direction == OUTBOUND. Can you explain the intention of that?

Sorry, I realized something must have not been clear just by reading the code. With the newer Windows SDK the definition SO_ORIGINAL_DST is now defined on Windows. So the existing code for inbound traffic traffic works as it is. This is why there are no changes for the inbound direction.
We have two cases for traffic redirection:

Inbound traffic which is handled by a Virtual Inbound Listener. In this case the Envoy code work as is, assuming a version of Windows that supports SO_ORIGINAL_DST socket option.
Outbound traffic which is handled by a Virtual Outbound Listener. In this case Windows and UNIX divert (hence the changes). For outbound traffic, on Windows, we need to respect the existing firewall rules of the container.

Imagine the following scenario, which is only applicable for the outbound traffic direction:
Players: container, Sidecar Container, google.com (upstream)
When container makes a request to google.com the traffic (outbound) gets intercepted by the sidecar container. This means that we have (i) a socket between container(downstream) and sidecar and (ii) another socket between sidecar and google.com (upstream).
In this scenario the platform admin might have set firewall rules blocking traffic from container 1 to google.com. The sidecar should respect these rules when proxying the connection. This means that we should query for the redirect records of the downstream socket and re apply these records to the upstream socket.
The bulk code changes in this PR are to support the logic described above.

Can you elaborate on that? I just want to understand the flow of how this works on windows, and the constraints.

The above lead us to a subtle problem. Since Envoy needs to do "extra" work on outbound connections, how can it distinguish between the two cases?
Here I considered 3 approaches:

Rely on the IP of the address and the destination. I think this is a fragile approach with many edge cases like handling traffic within the same POD.
Try to apply the redirect records in both inbound and outbound traffic and accept that this will fail 50% of the time. This is a bit awkward in terms of the user experience. Because if something goes wrong and the outbound direction does not work then we would silently fail thinking the connection is inbound.
Use the listener traffic_direction property and rely on service mesh frameworks setting this property to the listeners. I have verified that Istio already does that.

IMHO approach (3) is the best but I am always open to suggestions.

Thanks, that makes more sense now.
You'll need to document somewhere that this flow won't work on windows unless the traffic direction is configured properly. I don't think the traffic direction configuration affects this functionality on linux. But I agree that this is a reasonable use of that configuration, in order to get what you need to make it work on windows.

Thanks, that makes more sense now.
You'll need to document somewhere that this flow won't work on windows unless the traffic direction is configured properly. I don't think the traffic direction configuration affects this functionality on linux. But I agree that this is a reasonable use of that configuration, in order to get what you need to make it work on windows.

@envoyproxy/api-shepherds care to chime in? This wouldn't change any .protos, but it is new use of existing config.

@envoyproxy/api-shepherds care to chime in? This wouldn't change any .protos, but it is new use of existing config.

You'll need to document somewhere that this flow won't work on windows unless the traffic direction is configured properly.

+1, for sure

You'll need to document somewhere that this flow won't work on windows unless the traffic direction is configured properly.

+1, for sure

I incorporated the first comment and added the redirect_records into a filter_state instead of stream_info

I incorporated the first comment and added the redirect_records into a filter_state instead of stream_info

@ggreenway @davinci26 can you provide a TL;DR from a user/API perspective? I.e. if I configure original_dst in a management server and I speak to a Linux and Windows Envoy, what differences would I expect for traffic? What assumptions about the environment?

@ggreenway @davinci26 can you provide a TL;DR from a user/API perspective? I.e. if I configure original_dst in a management server and I speak to a Linux and Windows Envoy, what differences would I expect for traffic? What assumptions about the environment?

if I configure original_dst in a management server and I speak to a Linux and Windows Envoy, what differences would I expect for traffic?

The experience should the similar. In the sense that the traffic get's intercepted and redirected from the application container to the sidecar container. Note that this works well for container traffic redirection.
Also we the platform only supports redirection for tcp and not udp.

What assumptions about the environment?

A version of Windows that supports traffic redirection (support SOL_IP and SO_ORIGINAL_DST).
Traffic redirection has been setup for the environment. In Windows we don't have iptables so the way to set this up is by modifying the HNS policies.
Set the traffic_direction property into the listener.

if I configure original_dst in a management server and I speak to a Linux and Windows Envoy, what differences would I expect for traffic?

The experience should the similar. In the sense that the traffic get's intercepted and redirected from the application container to the sidecar container. Note that this works well for container traffic redirection.
Also we the platform only supports redirection for tcp and not udp.

What assumptions about the environment?

A version of Windows that supports traffic redirection (support SOL_IP and SO_ORIGINAL_DST).
Traffic redirection has been setup for the environment. In Windows we don't have iptables so the way to set this up is by modifying the HNS policies.
Set the traffic_direction property into the listener.

This seems like a reasonable implementation without API semantic divergence, @ggreenway did you have some concern around this from the API side?

This seems like a reasonable implementation without API semantic divergence, @ggreenway did you have some concern around this from the API side?

@htuch My only concern is that this setup would work correctly on linux without setting the traffic direction, but the direction is required on windows. But given that windows requires this additional information, I think using the existing direction is better than creating a new field for this. I just wanted another opinion on it from an API PoV.

@htuch My only concern is that this setup would work correctly on linux without setting the traffic direction, but the direction is required on windows. But given that windows requires this additional information, I think using the existing direction is better than creating a new field for this. I just wanted another opinion on it from an API PoV.

@ggreenway yeah, I agree, I think as long as we can sensibly configure both Linux/Windows with the traffic direction field set for consistent behavior, then we're all good. The idea we're aiming for is that a management server can be client neutral, i.e. not condition its behavior on the OS. In this case, there's a reasonable way to do this I think.

@ggreenway yeah, I agree, I think as long as we can sensibly configure both Linux/Windows with the traffic direction field set for consistent behavior, then we're all good. The idea we're aiming for is that a management server can be client neutral, i.e. not condition its behavior on the OS. In this case, there's a reasonable way to do this I think.

@davinci26 One other thing to try/consider: could the code for getting the redirect record be put in the original_dst listener filter, as dynamicMetadata (which can be set on the listener filter callbacks)?
I think the code in router/tcp_proxy and cluster makes sense, for handing off that information at the various points.
@lizan Do you have any thoughts on the approach to this?

@davinci26 One other thing to try/consider: could the code for getting the redirect record be put in the original_dst listener filter, as dynamicMetadata (which can be set on the listener filter callbacks)?
I think the code in router/tcp_proxy and cluster makes sense, for handing off that information at the various points.
@lizan Do you have any thoughts on the approach to this?

One other thing to try/consider: could the code for getting the redirect record be put in the original_dst listener filter, as dynamicMetadata (which can be set on the listener filter callbacks)?

I think that we could do that but we would need to know traffic_direction property of the filter listener on the filter. I could not find any precedent of filters being aware of the configuration of the listener.
Do we have any examples were we pass additional info like the above to the filters? If we can do it at the filter level the info could be store as part of the restored local_address of the socket and then move to filter_state by the connection handler.

One other thing to try/consider: could the code for getting the redirect record be put in the original_dst listener filter, as dynamicMetadata (which can be set on the listener filter callbacks)?

I think that we could do that but we would need to know traffic_direction property of the filter listener on the filter. I could not find any precedent of filters being aware of the configuration of the listener.
Do we have any examples were we pass additional info like the above to the filters? If we can do it at the filter level the info could be store as part of the restored local_address of the socket and then move to filter_state by the connection handler.

@ggreenway when you get some time, this should be ready for another review.
@rmiller14 this should be a safe commit for internal testing. Feel free to hop in for a review as well.

@ggreenway when you get some time, this should be ready for another review.
@rmiller14 this should be a safe commit for internal testing. Feel free to hop in for a review as well.

/retest

/retest

@sunjayBhatia yeah you got this all correct.
Will add a link to https://docs.microsoft.com/en-us/windows-hardware/drivers/network/using-bind-or-connect-redirection, this is good public documentation

/retest

@ggreenway I expect the CI to go green now, PTAL.
I have modified the listener .proto to add some more comments because they are used in documentation to communicate the new use for the traffic_direction property

@ggreenway I addressed all your comments, thanks a lot for all the reviews.
Should I drop the coverage for the win32Ioctl function which is not exercised in the linux code path and drops the LCOV?

Should I drop the coverage for the win32Ioctl function which is not exercised in the linux code path and drops the LCOV?

Yes, that's fine

Is it possible to add an integration test for this, or would the tests never be run in the proper setup?

For now the CI image does not support this feature so it is not possible. Even after that I am not sure if it would be possible to test it since everything runs inside a docker container.

Would two requests for the same original_dst address have different values for the redirect-records?

yes it is possible this is why I added the hash key in the option.
in the following scenario:

request to google.com
add firewall rules to block google.com
the new request to google.com will have different redirect records

I will test that scenario explicitly. I was hoping that this covered by existing tests

@ggreenway do we want more testing for this PR? or any other changes?

I thought it was only in PR description but I guess it is also needed in docs/root/version_history/current.rst

@ggreenway is away this week, @htuch do you want to do the last review to get this PR merged?

I'm happy to take a look this week some time.

Thanks Matt! This comment https://github.com/envoyproxy/envoy/pull/14753#issuecomment-763240675 gives a lot of additional context and should help a lot with the review.

Do we need another review? Since Greg is senior maintainer, I think we can just go ahead and merge. But if @mattklein123 wants to take another pass that is good as well :)

@davinci26 can you fix the merge conflict?

/retest

/retest


Import additional plugins

I just recently updated my dependencies but couldn't find any way to continue using my compression plugins together with customize-cra.
I got the following three plugins:

But I don't know how to include them with customize-cra.
I tried the ...addBabelPlugins but it doesn't work.
Is there any way to do this with customize-cra yet? Else I gotta stay on the older versions of cra and react-app-rewired unfortunately.

I tried it like this:

Like this:

Like this:

and even like this:

But it doesn't work.

That won’t work - you can’t add actual webpack plugin instances. You’re modifying the webpack config object, here.
Check out the code (or customize-cra) to see how that works.

Is there any method or doc you could point me to, to find out how I am able to integrate those plugins?

The way it was done before when compose and injectBabelPlugin were still supported is not possible anymore and I couldn't find any documentation showing how to do it now.

Look at how other plugins are implemented.
https://github.com/arackaf/customize-cra/blob/master/index.js


Update circe-core, circe-jawn, ... to 0.14.5

Updates

io.circe:circe-core
io.circe:circe-jawn
io.circe:circe-testing

from 0.14.4 to 0.14.5.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.circe" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "io.circe" }
}]

labels: library-update, early-semver-minor, semver-spec-patch, version-scheme:early-semver, commit-count:1

Superseded by #391.


WIP: Hook up to new Buy Now Echo config, make MP query

This PR does a few things:

It updates Echo.
It uses a newly added flag in Echo to enable Buy Now for our users remotely.
It adds an admin flag to allow Artsy staff to enable the Buy Now flow immediately.
It adds a metaphysics mutation to create and order and parse its order ID.
It routes the user to a new route defined in Echo, with that order ID.

Regarding that last point, it is apparently not possible to tell JLRoutes "I want to go to this route but use this in place of :id", so I did a string replacement. Let me know if you have better ideas.
This PR is still a work-in-progress because:

[ ] We need to define how error-handling should work.
[ ] I need to write some tests.

This PR fixes PURCHASE-388 and PURCHASE-389

          1 Warning

:warning:
Needs testing on a Phone if change is non-trivial

  Generated by :no_entry_sign: Danger

Okay, testing this locally is blocked by Force/Reaction having an old schema. I've opened a PR here to update them, but it's still a WIP itself: https://github.com/artsy/reaction/pull/1193

Okay so here's what this looks like in the admin menu:

This setting circumvents the Echo config, so we can test without enabling the feature for everyone.
Right now, this routes to /order2/:id, but that's configurable through Echo too. However, the infrastructure for redirecting /order2/:id to /order2/:id/shipping hasn't been set up yet, so it'll 500 if you try it right now. Work to do that is ticketed in PURCHASE-418.
I'll wrap this PR up with some unit tests for the switchboard changes.


Add text&title widget autoheight support

Added "Autoheight" setting to text and title widgets.

i18n-check

The following components have not been translated
dash.dashkit-plugin-common.view (ru)dash.dashkit-plugin-common.view (en)

Wait for the reviewers to check your changes in Weblate and try running github action again.

i18n-check

The following components have not been translated
dash.dashkit-plugin-common.view (ru)dash.dashkit-plugin-common.view (en)

Wait for the reviewers to check your changes in Weblate and try running github action again.

i18n-check

The following components have not been translated
dash.dashkit-plugin-common.view (ru)dash.dashkit-plugin-common.view (en)

Wait for the reviewers to check your changes in Weblate and try running github action again.


chore: bump indexer beta-4

Description

PR simply bumps the indexer versions for the beta-4 toolchain

Does it make sense to bump it for both release candidate versions?

@JoshuaBatty @kayagokalp Changes made (I think)

Do you want to change beta-4 toolchain or beta-4-rc-2 toolchain? @ra0x3

@kayagokalp I changed the file prefixed with "-2" due to this comment. Did I read it incorrectly (should I change the other one)?

One more attempt @kayagokalp  :)

I think you wanted to change this one https://github.com/FuelLabs/fuelup/blob/gh-pages/channel-fuel-beta-4.toml instead of
https://github.com/FuelLabs/fuelup/blob/gh-pages/channel-fuel-beta-4-rc.toml cc @ra0x3


Multi root: decide where to store workspace information

We should revisit where we store multi root metadata. Things to consider:

free form setup of roots (no more master details)
having a stable identifier to use for: UI state, API storagePath, API workspaceState
how to identify windows on the main side that now requires a workspacePath?
should we realpath the root configuration to be in sync with what we do today in the workbench startup?
should we install watchers that notify when a root disconnects/connects and introduce a new event for this?
how to share this?

Related: https://github.com/Microsoft/vscode/issues/29456

Closing, the main work has been done and is available in insiders.


DE-7110: Show Example of Test DataStream with more than 1 record

Nothing fancy, just adds a second message to the stream in the test for demonstration purposes

Can we assume the data will be outputted in the same order they came in? I am not sure how the tests are set up if it's on a single machine I think we should be ok.

My understanding is we run all this in a testcontainer and add messages in the same thread, so yeah, I did assume that ordering will be preserved. Maybe @gunnarmorling can confirm.


The future of Huginn

Next Steps
I want to explore the next steps for Huginn and to talk about something much more ambitious than the current implementation.
Huginn, the Protocol
Goals:

Huginn Agents shouldn't have to be implemented in Ruby inside of a Rails app.  Agents should be implementable in any language.
You should be able to interact with both local and remote Agents in the same way, with the same API.
The system should be fast enough to tackle a wide variety of problems.

Proposal:

Agents can be written in any language on any platform that can open persistent connections.
Agents create and consume JSON Event objects.
To consume events from an Agent you connect to that Agent's IP address and port and subscribe to a standing query for matching events.  The default query is something like subscribe to *, returning all generated events, but specific Agents can allow for more complex predicates, such as subscribe where zipcode = 12345 (syntax TBD).

This could easily be wrapped in WebSockets.
This predicate language should be standardized and, importantly, very simple to implement.
It's possible that Agents should create events in a rolling event log and allow for short-term event replay from a specific timestamp so that a re-connecting Agent can recover any lost events.

Agent configuration is not part of the spec.  Some Agents may be configured via the command line, others via a web GUI, still others can be paid remote services.

The reference implementation will make it easy to customize and deploy Agents on your own system with configuration via a web GUI similar to the current system.
The reference implementation may allow Agents to be sandboxed (a la docker) so that you can install semi-trusted 3rd-party code.

Agents should be runnable locally or remotely.  The goal is to allow you to maintain control of your data by running everything locally, or to subscribe to remote Agents and participate in a larger data ecosystem.  Some questions:

How should local Agents be packaged and distributed?
What would a marketplace look like where you could easily subscribe to, or download and run, Agents for any purpose?

What do you think?
Get Involved!
Want to help create the future of Huginn?  Get involved!  You can comment and contribute here, send pull requests, follow @tectonic on Twitter, or contact me directly on my personal site to discuss further involvement.

If you do not mind I would like to revive this thread to talk a little about the possibilities of integrating other programming languages in Huginn.
I think the best bet is to create an interface that somehow allows those languages to communicate with Huginn.
It can be approached in different ways, for example one option would be to create a small GUI that allows creating files that will later become agents. And these files once saved will be executed by the interpreter of the selected language.
For example, the language that I'm used to PHP only requires to have one of the versions installed on the server.
Once this file is created let's suppose that the list of agents that appears when creating a new agent is able to find this file and in some way Huginn is able to execute the interpreter and execute the first method that would be for the creation of the GUI of the agent. Suppose that it returns an XML/HTML that Huginn is able to reinterpret and create the fields.
Once viewing the GUI of the specified agent we would see fields and configurations defined.
I've created a simple example class in PHP that creates a simple Telegram Agent. As you can see Huginn should interface with this class in order to pass $options variable and the $event.
<?php
namespace App\Agents;

use App\AgentInterface;
use \React\EventLoop\Factory;
use \unreal4u\TelegramAPI\HttpClientRequestHandler;
use \unreal4u\TelegramAPI\TgLog;
use \unreal4u\TelegramAPI\Telegram\Methods\SendMessage;

class Telegram implements AgentInterface
{
    public $name = 'Telegram agent';
    public $description = 'The Telegram Agent receives and collects events and sends them via Telegram.';

    public function process($options, $event)
    {
        $loop = Factory::create();
        $tgLog = new TgLog($options->token, new HttpClientRequestHandler($loop));
        $sendMessage = new SendMessage();
        $sendMessage->chat_id = $options->chat_id;
        $sendMessage->text = $event->text;
        $tgLog->performApiRequest($sendMessage);
        $loop->run();
    }

    public function gui()
    {
        return '
        <input name="token" type="text">
        <input name="chat_id" type="text">
        ';
    }
}

IF the direct interfacing possibility is not possible then we can define a communications protocol to ensure this works via API or something like that.
What do you think?

Not sure if I’ve mentioned it here before, but these days a lot of my thought process has tended more towards AWS lambda and similar ‘serverless’ technologies in place of what huginn would have solved for me in the past.
One ‘run it yourself’ serverless tech that would marry well with this concept of ‘agents in any language’ is OpenFaaS ( https://github.com/openfaas/faas )
While i’m sure you could approximate an integration with the web agent, I think a more directly supported bridge/maybe even some howto could be a really powerful addition to opening up huginn to a much larger pool of potential agents.
I feel like huginn has the ‘workflow’ and UI angle down pretty solidly, where OpenFaaS/Lambda/etc could provide nicer agent plugin potentials.

Could be that a solution for someone with a simple huginn server on a 3€/month VPS like me?
Don't know if we are using huginn the same way but at least for me the most condensed solutions on one location the best for me. Spreading all my flux through other platforms are a possibility but also adds another mesh of costs and complexity. My server with Huginn is always dancing free of load and could be doing this work.
While I agree with you I feel that I need this on huginn to be flexible enough to become a finished product with full potential.
In fact you gave me an Idea of making a simple platform that collects events from huggin over web or shell and process them using just functions that are defined by the user but as I said I prefer this to be integrated in huginn and not as a different application.

What about extending the Scheduler and Agent registry inside Huginn (I don't know what else to call the subsystem where Huginn keeps a list of the Agent types it supports) so that a user could upload custom agents that they wrote in other languages (PHP, Python, shell scripts, etc), tell Huginn that they exist (so that, when setting up a new Agent they're available as options), and run them with all the others.
An interface would need to be defined to pass events to those user-defined agents in the same way that they are to the internal ones, and an interface would need to be defined to pass events from those user-defined agents to everything else.
Heck, I'd kick some money in on Bountysource for this kind of functionality now that I think about it a little.  Can we crowdfund features on Bountysource?

This is definitely the most desirable option. Don't know if possible without a major refactoring, shame on me that I'm not a Ruby developer.
But meanwhile and slowly unless a solution for this need happens I'm creating a port of Huginn in Laravel for PHP, intending to have all this requirements in mind.

IF the direct interfacing possibility is not possible then we can define a communications protocol to ensure this works via API or something like that.

We have the ShellCommandAgent which can execute any command that is installed on the Huginn server (granted that only really works when Huginn is not installed using docker). The fact that the needed interpreter would need to be present limits the potential. I don't see us adding every interpreter on the planet to the docker image ontop of that dependency management for the other languages could be annoying.

I feel like huginn has the ‘workflow’ and UI angle down pretty solidly, where OpenFaaS/Lambda/etc could provide nicer agent plugin potentials.

I think I agree, OpenFasS and co are nice because they already have packaging and deployment figured out.
I believe the universal way to interact with other languages would be some sort of HTTP interface. It might be possible to have an Agent in Huginn that once the endpoint is configured shows the Agent description, validates the options and runs the Agent by making HTTP calls to the "agent server" that is written in a different language and is deployed somewhere (docker, locally, openfass, labda).

I think that there should be at least two ways of communicating.

I believe the universal way to interact with other languages would be some sort of HTTP interface. It might be possible to have an Agent in Huginn that once the endpoint is configured shows the Agent description, validates the options and runs the Agent by making HTTP calls to the "agent server" that is written in a different language and is deployed somewhere (docker, locally, openfass, labda).

One of them you exposed perfectly right here. But there should be another way of doing this in the same machine. Take a look to https://capnproto.org/
If we develop some kind of RPC protocol there's no need to aggregate every interpreter in the world to the builds.
The user should if working inside the same machine use this protocol after installing everything needed for their reference language. If outside the machine just use HTTP protocol.

So lets imagine this HTTP API:
Somewhere in Huginn you define a new HTTP Agent and sets the URL.
After setting the URL a GET somewhere.com/my-agent/get-info will be done to that URL, and will return:

A definition of the agent (Name, description ...)
A definition of GUI for that agent in XML that will be drawn for users in Huginn.

After saving the agent. The event flow will:
Send a request to the agent POST somewhere.com/my-agent/run and the response be parsed by huginn in order to get the new event.
If visualizations are needed in the show option of the agent then you could add another URL that get this kind of visualizations.
What do you think?

My /2c is that in trying to ‘invent’/imagine this new protocol, you’re basically duplicating a lot of the work that has already been done in IMO much more active (and this more maintained) projects. I think the issue of not wanting to have to setup/interact with/learn another system are pretty minor compared to the dev costs of building that same system again (similar thoughts for translating an existing system from one language to another.. but that’s just my personal /2c)
Have a look at the API OpenFaaS uses, and perhaps the interfaces there. A lot of thought and work has gone into designing around basically exactly this problem (how can I make a small piece of language independent code run my thing). Huginn calls it an agent. OpenFaaS calls it a function. But they are practically identical in function.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Marcos Sigueros Fernández<EMAIL_ADDRESS>Sent: Friday, June 14, 2019 10:08 pm
To: huginn/huginn
Cc: 0xdevalias; Manual
Subject: Re: [huginn/huginn] The future of Huginn (#34)
So lets imagine this HTTP API:
Somewhere in Huginn you define a new HTTP Agent and sets the URL.
After setting the URL a GET somewhere.com/my-agent/get-info will be done to that URL, and will return:

A definition of the agent (Name, description ...)
A definition of GUI for that agent in XML that will be drawn for users in Huginn.

After saving the agent. The event flow will:
Send a request to the agent POST somewhere.com/my-agent/run and the response be parsed by huginn in order to get the new event.
If visualizations are needed in the show option of the agent then you could add another URL that get this kind of visualizations.
What do you think?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/huginn/huginn/issues/34?email_source=notifications&email_token=AAFYBY66UZUWHRKMQSIKZGDP2OC4XA5CNFSM4AEKVFIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXWTEVA#issuecomment-502084180, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAFYBYYPMEREBJS44FJMZTDP2OC4XANCNFSM4AEKVFIA.

Ok ok I'm going to try OpenFaaS if you all guys say that is what I need I trust you. I don't want to reinvent the wheel obviously. If the wheel that you guys suggest work easily over a cheap vps I'm ok with that. Let me learn and try to work with this over the next few days to have a real opinion based in something real.

I've been testing Open FaaS from this morning. Wow, it is really easy to setup on my shabby VPS server. It works flawlessly with PHP so I'm really happy with it.
With solutions like this then this protocol interface is clearly not needed.
The only complain that I have with Open FaaS is that the interface is awful as hell and faulty of options... but I will survive with it anyways.
Thanks guys.

I don't really have the capacity to explore/implement it right now.. (maybe one day..) but my feeling is that it wouldn't be all that hard to create an OpenFasS Huginn Agent, to allow it to easily be plugged into Huginn workflows, and then everyone could easily get the best of both worlds!

Maybe by writing the .yml and create a function inside somewhat called "functions" directory trough the API and the file system. And redirecting the events through HTTP. I mean with Huginn and a UI interface that allows the user to create new functions including the source and the ability to build and deploy.
The OpenFaaS API does not allow to create a function with code so you need to create the code first on the server. Or at least that's what I've found until now.
I think that I would be more than happy with something that fixes that.
It really seems not so hard for me to implement that in PHP instead of continuing porting huginn. Which really is itself a project that I've started just for fun sometime ago. And this should be simpler and useful than a huginn port to PHP.

@alrik11es If you follow the link I pasted above through to http://editor.swagger.io/ and import the swagger URL provided (on my link above), you can see  that a new function can be deployed with POST /system/functions.
OpenFaaS is built around docker containers, so you're basically telling it where to get the pre-made function docker container from.

Though when I said "create an OpenFasS Huginn Agent" above, I meant implementing the OpenFaaS API on the Huginn side. Initially I wouldn't even care so much about deploying new functions, the tools in OpenFaaS (WebUI, CLI, etc) are already perfectly good at that. The thing I would see value in implementing is the ability for Huginn to invoke OpenFaaS functions, and retrieve the result.
The swagger docs linked above show that this would be done with POST /function/{functionName} or POST /async-function/{functionName} (though for the latter, you would need to have a 'webhook' style receiver in Huginn to get the result I believe). So basically you could already connect these really easily using the Huginn Web Agent.
Where I would see more value in a 'deeper' integration is to automate some of the setup/selection of functions. For example, you could use GET /system/functions and GET /system/function/{functionName} to list the available OpenFaaS functions/details about them, and then build some UI showing them within Huginn as 'external agents'. You could connect Huginn's concept of secrets with OpenFaaS's concept of secrets, managed by the /system/secrets API endpoint.
Those are the sorts of things I envisage when I think about a Huginn/OpenFaaS integration.

I'm a little embarrassed of not expressing myself perfectly. Sometimes expressing myself correctly in English is really difficult for me. It happens sometimes to me in Spanish which is my native language so...

The swagger docs linked above show that this would be done with POST /function/{functionName} or POST /async-function/{functionName} (though for the latter, you would need to have a webhook agent in Huginn to get the result I believe). So basically you could already connect these really easily using the Huginn Web Agent.

Yes it works like that, I've tested it.
{
  "service": "nodeinfo",
  "network": "func_functions",
  "image": "functions/nodeinfo:latest",
  "envProcess": "node main.js",
  "envVars": {
    "additionalProp1": "string",
    "additionalProp2": "string",
    "additionalProp3": "string"
  },
  "constraints": [
    "node.platform.os == linux"
  ],
  "labels": {
    "foo": "bar"
  },
  "annotations": {
    "topics": "awesome-kafka-topic",
    "foo": "bar"
  },
  "secrets": [
    "secret-name-1"
  ],
  "registryAuth": "dXNlcjpwYXNzd29yZA==",
  "limits": {
    "memory": "128M",
    "cpu": "0.01"
  },
  "requests": {
    "memory": "128M",
    "cpu": "0.01"
  },
  "readOnlyRootFilesystem": true
}

POST /system/functions until what I've seen this request do not allow creating new code but it does push existent from a docker registry.
When I talked like in my other comment I though like if I were implementing this kind of technologies for my company's agnostic users. They are no ops, not even specialist developers. So this people are in the need of something that makes their lives easier not over complicate it. Of course you can connect through CLI and create a new function and so on. But I don't want my users have the need to. It's a matter of efficiency. What is efficient for them is efficient for me as well.
So your latest answer is a good opinion and I respect it and I agree mostly with it. But in my opinion the value is having everything that allows someone to create new agents/functions or whatever condensed in the same place or at least in the same application.
It's a matter of different way of thinking. Just my opinion. Not trying to deny other opinions or solutions.
Anyways I have to really thank you for pointing me towards new architectures that really makes my life easier. I'm really up into implementing this for lots of minor tasks on my company when I arrive on Monday. Thank you very much.

I've been playing around a little with OpenFaaS, and I think a couple of POST Agents would be ideal for triggering functions (though probably not writing functions).  POST Agent would also be ideal for triggering the OpenFaaS API rail for updating its local directory of ready-to-run function images.

How active is this project? It seems like a lot of Pull Requests and Issues have amassed so I was wondering if Huginn is still being maintained or even developed further. What's the plan?

Huginn is OSS, everyone is welcome to contribute. As far as I know all pull requests that are still open are either stale because requested changes were not made or never got to a merge able state because of disagreements.
I don't think we ever had a collaborative plan for Huginn. Contributors added the features and Agents (for that we now have huginn-agent) they wanted/needed.

That is absolutely fine I and I didn't want to put any pressure/blame on regular contributors. The way it currently looks to me when I see the issue and PR count as well as created/updated dates on issues/milestones is that Huginn does not get much attention anymore.
If so many pull requests and issues are stale it would benefit the project (imho) to close these if they are old. I've seen there are already quite a few labels setup as well and even very well made milestones but they seem to have been unused for the last 2-3 years.
I think a good cleanup of all the old/stale issues & PRs (& milestones) as well as triaging what remains with labels & milestones would make it a lot clearer what is being worked on and what is going on with the project. Maybe even creating isse & PR templates (and a stale bot) could help organize this for the future.

It would be nice if we can have some translation service, so anyone can contibute with translations.

Is this issue currently being worked on? Should we close it?

I am not pursuing this at the moment. It was more of a vision statement.

Thanks. Just running some triage and wanting to close up old issues. 🙂

On the subject of the future of Huginn; it appears that the current version of Huginn doesn't work on the latest version of Ubuntu (v22.04). There's no reason why it should; the Huginn installation notes only specify Ubuntus v14 and v18 as pre-requisites.
Even so, I have managed to get Huginn running on Ubuntus v20 and v21 without too much problem.
One reason appears why Ubuntu v22 doesn't play well with Huginn is that the elderly versions of Ruby that Huginn relies on mandatorily require OpenSSL v1, and Ubuntu v22.04 installs OpenSSL v3. Downgrading OpenSSL, or reconfiguring Ruby v2.6.9 to support OpenSSL v3 (or possibly some version of LibreSSL) is a fiddly task with potential security issues that complicates future upgrade paths; it's not a goer for most users, and I would be disinclined to try it if I wanted a reliable instance of Huginn. And then there may be other problems as Huginn and its pre-requisites become increasingly aged...
Ubuntu v21 was an interim release that's been EOL'ed; there are no further updates to it, and you can't install a new instance of Ubuntu v21 because the installer can't find said updates, and you wouldn't be able to install Huginn OS pre-requisites on it anyway.
This leaves Ubuntu v20.04 LTS as the longest-lived option known to work with Huginn which will receive maintenance updates until 2025, and security updates until 2030. Ubuntu v18.04, the latest version specified as supported by Huginn will have maintenance updates until 2023 and security updates until 2028.

@ohingardail That seems like something that would be worth a new issue for for visibility/tracking, as I suspect it will get lost/buried away tacked onto the end of this now closed issue.

Acknowledged. I wasn't sure how important this Ubuntu EOL issue was, since it's some way off and there may be technical details and mitigations the community knows about that I don't. However, a new issue report may be the best way to find out!

On the subject of the future of Huginn; it appears that the current version of Huginn doesn't work on the latest version of Ubuntu (v22.04). There's no reason why it should; the Huginn installation notes only specify Ubuntus v14 and v18 as supported.

I'm running Huginn on Ubuntu Server 22.04 LTS and have been since a month after it came out.  One thing I'm doing is using Ruby installed with rvm, not the Ubuntu Ruby packages.


Split docs project into website and docs

This divides the site into two projects, docs and website.  Like now, website will be published off master and docs will be published off the release branches.  A key difference is that each project gets its own static assets, so we can change the website project without needing to republish all versions of docs.  It also becomes much easier to preview the site from either project.
This starts all the way back on release-0.15.x, as it's the earliest branch published.  This is a one-time merge train so each site gets its own copy of the static assets.
The upgrade of sbt-ghpages should pave the way toward an sbt-1.x upgrade.
I envision the work done in the http4s.org repo being moved here, but only to master.  Trying to keep the changes minimal on the older branches.

Codecov Report

Merging #1381 into release-0.15.x will decrease coverage by 0.15%.
The diff coverage is n/a.

@@                Coverage Diff                 @@
##           release-0.15.x    #1381      +/-   ##
==================================================
- Coverage           69.24%   69.08%   -0.16%
==================================================
  Files                 211      211
  Lines                4458     4458
  Branches              294      294
==================================================
- Hits                 3087     3080       -7
- Misses               1371     1378       +7

Impacted Files
Coverage Δ

.../scala/org/http4s/testing/ArbitraryInstances.scala
58.82% <0%> (-35.3%)
:arrow_down:

...server/middleware/authentication/NonceKeeper.scala
92.59% <0%> (-7.41%)
:arrow_down:

core/src/main/scala/org/http4s/HttpVersion.scala
87.5% <0%> (-6.25%)
:arrow_down:

...cala/org/http4s/client/blaze/Http1Connection.scala
76.85% <0%> (+0.92%)
:arrow_up:

...src/main/scala/org/http4s/client/PoolManager.scala
48.48% <0%> (+1.51%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 923f9a4...cf860ad. Read the comment docs.


New version: Spotify.Spotify version <IP_ADDRESS>2.g949d5fd0

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.4 schema?

Microsoft Reviewers: Open in CodeFlow

@louis993546

@stephengillie It looks like the bug is not fixed.

Yes, Spotify requires install as non-admin, but I believe the validation pipeline only installs as admin.


fw/config: ensure params are obj_dict's

As part of finalizing a job spec, ensure all *_parameters are
obj_dict's.

Typo fixed.


COVID-19 documentation tweaks

Comments, minor fixes: roll up in your next PR:
https://github.com/castorini/anserini/blob/master/docs/experiments-covid.md
1.
${DATA_DIR} -> add the date of the collection in the path; I assume we're going to get frequent updates, so let's not get them confused.
2.
ls ${DATA_DIR}/*.tar.gz | xargs --replace tar -zxvf {} -C ${DATA_DIR}

I don't think --replace works on Mac.

will do
@lintool can you confirm if -I instead works on mac

nope:
$ ls ${DATA_DIR}/*.tar.gz | xargs -I tar -zxvf {} -C ${DATA_DIR}
xargs: illegal option -- z
usage: xargs [-0opt] [-E eofstr] [-I replstr [-R replacements]] [-J replstr]
             [-L number] [-n number [-x]] [-P maxprocs] [-s size]
             [utility [argument ...]]

It's only four files.... maybe it's easier just to uncompress each explicitly, rather than futz with cross-platform compatibility? Like, what about Windows?

Also, next iteration, add link from main README.md.

Resolved by #1049


Fix Misindexed File Name (groomerServicesRouter.js)

Description
File names were previously changed to adhere to naming convention, but one got stuck indexed with incorrect file name and prevented import of module.  Had to have another team member pull down branch to get the filename to index correctly.
Type

[x] This is a bugfix.
[ ] This is a new feature.
[ ] This is part of a feature.

Organization

[x] Have you linked this pull request to a Trello card?
[x] Have you named the branch in the format of feature/describe_feature_or_change OR bug/describe_bug_this_is_fixing

Review

[ ] Is this a work in progress?
[x] Is this ready to be reviewed?
[x] Have you added two reviewers to this pull request?

 Fix bug caused by a filename that which will not push to github properly renamed, thus preventing a proper import.


Kafka monitor failing due to missing topic with topic creation enabled

I'm just getting started with kafka-monitor, and my first run is failing with the following exception:
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at com.linkedin.kmf.core.KafkaMonitor.<init>(KafkaMonitor.java:46)
        at com.linkedin.kmf.core.KafkaMonitor.main(KafkaMonitor.java:132)
Caused by: java.lang.RuntimeException: Can not find valid partition number for topic kafka-monitor-topic. Please verify that the topic has been created. Ideally the partition number should be a multiple of number of brokers in the cluster.
        at com.linkedin.kmf.services.ProduceService.<init>(ProduceService.java:75)
        at com.linkedin.kmf.tests.BasicEndToEndTest.<init>(BasicEndToEndTest.java:54)
        ... 6 more

This error is technically correct as I have not set up a topic and the target Kafka is a blank dev instance.  However, I'd expect kafka-monitor to start as I have produce.topic.topicCreationEnabled set to true.  For the config I'm running the example config in the repo with the zookeeper.connect and bootstrap.servers pointed at my dev Kafka setup.  Rather than die with an exception I'd expect kafka-monitor to set up it's test topic.

Looking into it.

I created the topic and I'm still seeing this error:
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
[2016-11-08 16:05:14,465] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
        at com.linkedin.kmf.core.KafkaMonitor.<init>(KafkaMonitor.java:46)
        at com.linkedin.kmf.core.KafkaMonitor.main(KafkaMonitor.java:132)
Caused by: java.lang.RuntimeException: Can not find valid partition number for topic kafka-monitor-topic. Please verify that the topic has been created. Ideally the partition number should be a multiple of number of brokers in the cluster.
        at com.linkedin.kmf.services.ProduceService.<init>(ProduceService.java:75)
        at com.linkedin.kmf.tests.BasicEndToEndTest.<init>(BasicEndToEndTest.java:54)
        ... 6 more

kafkat partitions
Topic           Partition       Leader          Replicas                                                ISRs
kafka-monitor-topic     0               0               [0]                                                     [0]

So perhaps this is something else?  I'm still gathering data myself.

I confirmed it is not creating the topic initially, however it's throwing the same error with the topic created, which is a little confusing.
Also, I'm running Confluent Kafka 0.9.0_2.0.0 so I'm running kafka-monitor off the <IP_ADDRESS> branch, 893c580a3f3267ee9fa753a466bf8207eee093a8

AAH this is a PEBCAK issue.  I was passing the wrong ZK path.

Glad you where able to figure it out.


[PAN-OS Upgrade Assurance][dependabot/github_actions/peter-evans/find-comment-3] documentation PREVIEW - do NOT MERGE

Description
DO NOT MERGE - preview PR made for changes on branch: dependabot/github_actions/peter-evans/find-comment-3.
Types of changes
New feature (non-breaking change which adds functionality)

Hi @horiagunica, any updates on these open PRs?

Hi @horiagunica, is this PR still needed? Just doing a bit of housekeeping for stale PR's. Thanks!


Incorrect types for columns are chosen during the export to new table from another table

System information:

Operating system (distribution) and version win10
DBeaver version 21.0.5

Connection specification:

Database name and version CockroachDB 20.1.5

Describe the problem you're observing:
Export SQLite - Cockroach
Integer type is mapped to varbit
BLOB - to bytea
Datetime - to varchar
Numeric - to varbit

The BLOB ⇒ bytea conversion looks fine to me

Fixed for NUMERIC, NUMERIC(10,2) (ore another modifiers), INTEGER, INT.
Search data types in PG/PG forks bases generally improved.
Search for types with modifiers improved.
Datetime is string type in SQLite -  https://www.sqlite.org/datatype3.html

verified


add javascript in docs to eslint

Lint stuff in docs/ that's not generated or vendored.

Coverage remained the same at 93.228% when pulling 0c7014da7e96dee92b2edc1c38954484b8334363 on boneskull/lint-docs into 1a4646dd2f7b35010c2488edcb2d5f4cfb2f14a8 on master.


mat-dialog-content

hello
mat-dialog-content typography hierarchy wont render correctly in Dialog With Scrollable Content example

Hello, do you have an example to reproduce the error or a screenshot?

hello yes here is a screenshot from the scrollable dialog component example where the p tag and h3 heading tag have the  some font size and the  typography hierarchy from angular material didnt applied correctley

Thanks, for a quick fix, I added prose CSS class instead of mat-typography. And for mat-typography I will add CSS styles soon. I already updated the demo, check it out https://demo.elementarui.com/pages/components/dialog.

ok thanks man keep good work,i ll check it


Revert "Set OverwriteOutput in case kmz already exists"

Reverts Esri/distance-direction-addin-dotnet#600 This PR is causing ArcMap to crash:
start up arcmap
create graphic from dd
export graphic as a kmz
open in earth
close earth
close armap
reopen arcmap
result is arcmap normal mxt is curropted and does not open back up.
Symbology is not correct in ArcGIS Earth as well

It appears that this commit was reverted in error and was not related to the problem - so I was going to put back.
FYI @topowright @ACueva @dfoll


Static class properties and eslint

Static class properties are caught by the linter. E.g.
class Foo extends React.Component {
  static propTypes = { bar: PropTypes.string }; <--- Eslint parsing error: Unexpected token =
}

I can't for the life of me figure this out.
Something goes wrong in the linter configuration. It seems to be missing the transform-class-properties plugin, but it should be part of the babel preset I'm using
This is a minor annoyance, but any help is greatly appreciated.

Almost a year later, encountered this bug again somewhere else.
This fixes the issue

:+1: just hit the same issue. Adding the babel-eslint package and setting as the parser in the eslintrc worked.


Double check epsilon implementation and implement constants and weights for epsilon and nu in the import function rather than in each script

Discussed the test of epsilon with @eimrek

Hi, just to have a reply on this issue:
I tested epsilon and nu for "Fr2O5" directly in https://github.com/aiidateam/acwf-verification-scripts/blob/main/acwf_paper_plots/plots/supplementary/periodic_tables_all/generate_all_periodic_table.py
and they matched with my javascript implementation (using gaussian quadrature), given here: https://github.com/materialscloud-org/acwf-verification/blob/main/src/components/ComparisonMeasures.js
(numbers also available at https://acwf-verification.materialscloud.org)
Should there be any more thorough testing?


Accessibility / ARIA implementation : multiple issues

Hello
Some issues regarding your ARIA implementation on qtip2 :
1/ role should not be "alert", but "tooltip", as specified in the ARIA specification,
here : http://www.w3.org/WAI/PF/aria/roles#tooltip
and here : http://www.w3.org/WAI/PF/aria-practices/ ( chapter <IP_ADDRESS> )
2/ In the same time, the aria-live attribute should be removed, as it is too obstrusive, even in "polite" mode. For example, in actual implementation, tooltips activating on focus make the screen reader read twice the tooltip content : once because of aria-live, and second time because the focused element is aria-describedby the tooltip content. (also, aria-live on tooltips is not even suggested by the specification)
The aria-describedby on tooltip's target is the only thing that matter here.
3/ aria-describedby can support MULTIPLE ids ( aka aria-describedby="id1 id2 id3" ).
see http://www.w3.org/WAI/PF/aria/states_and_properties#aria-describedby
In your current implementation, adding a tooltip to an element already having an aria-describedby attribute erase the current id references instead of adding the tooltip's id to it (same behaviour when the tooltip is destroyed : the original aria-describedby value is lost)

I second this.  While #3 is not so much an issue for me, my poor users are getting bombarded w/definitions reading 2-3 times.


Compress HTML output via gzip

https://andycroll.com/ruby/compress-your-rails-html-responses-on-heroku/
PageSpeed (mobile) before deploying

https://railsdevs.com - 95, 97, 97
https://railsdevs.com/developers - 85, 45, 78
https://railsdevs.com/developers/1 - 69, 69, 63

PageSpeed (mobile) after deploying

https://railsdevs.com - 97, 99, 99
https://railsdevs.com/developers - 89, 93, 96
https://railsdevs.com/developers/1 - 57, 68, 73


Extern simulator core

https://github.com/FRUK-Simulator/SimulatorCore/pull/2 will allow this to work correctly.

Haha it was a typo! Nice


OSX/M1 window resizing crashes the engine when rendering thread model is set to multi-threaded

Godot version
3.5.2
System information
MacBook Air M1 2020 8GB Sonoma 14.0
Issue description
On some M1 Mac systems, setting the rendering thread model to multi-threaded causes the engine to crash when the window size is changed.
This seems to be a race condition and gets progressively worse when the game is actually doing stuff - to the point where a game like ΔV: Rings of Saturn will crash on any attempt to resize the window or switch between full-screen and windowed modes. I have verified that adding a command line of --render-thread safe fixes the problem, so I'm confident that it's caused by the multi-threaded rendering model.
You can replicate this with an empty project without code - it just requires more vigorous mouse movements to trigger.

Crash recording: https://imgur.com/a/ERIE01f
OS crash report: crash-report.txt

Steps to reproduce
This replicates only on M1 Macs.

Create a new project
Set the rendering thread model to multi-threaded
Create an empty Node2D
Run it
Resize the window rapidly

MRP attached.
Minimal reproduction project
mrp.zip

Also happening in 3.6 beta 3.

Fixed by #81442.

TL;DR: Huge stability improvement, but I still get reports of crashes in production.
Unfortunately, it seems that this was not the root cause. While the #81442 does greatly reduce the rate of crashes (from "in a second" from before the patch to "in few minutes if you play with setting" after the patch), there seems to be another underlying problem that causes the instability, and disabling the live resize just makes it not immediately apparent. It seems that window resizing still has a small chance of crashing the engine, but after the patch we are just not resizing 60 times per second.
I am unable to replicate this anymore with a minimal project, but it does still happen in production. If you grab the free demo build from itch.io, enter the settings and either keep changing the fullscreen mode, or set the mode to windowed and keep resizing the window (and releasing the mouse button), it will cause a crash in a dozen of tries or so on an M1 system.

The .pck file in there should be compatible with any 3.5.3 build.

The crash seems to be caused by particles processing (deep inside the GPU driver after the RasterizerStorageGLES3::_particles_process call). So it is might be related to https://github.com/godotengine/godot/issues/72469, https://github.com/godotengine/godot/issues/72960 and https://github.com/godotengine/godot/issues/54052, If it is, I'm not sure if we can fix it on engine side, and since OpenGL is deprecated on macOS, driver won't be fixed either.

I would probably disable GPUParticles rendering on macOS if not using ANGLEMetal.

If this is caused by particles, and disabling such particles fixes the issue, an approach that seems reasonable is to blanket-disable all GPUParticles when the resize procedure starts and then re-enable them when it finishes.
I could probably implement that on the game side, but this is a nasty gotcha for anybody developing cross-platform, so it should probably be either prominently documented or handled on the engine side.


code check with eslint

replace jshint with eslint which is more modern for js codebase.

Hi. Thanks, I'm sure these are good fixes. However I'm not really fond of a single PR containing bugfixes, formatting changes and a switching out the CI system. In fact, I'm not sure why we need to change CI at all?

alright, i gonna separate them.

the travis failure here:
SyntaxError: Use of const in strict mode.
is due to using nodejs 0.10. I am not objected to increasing the minimum required node version to 4.x (which apparently is the minimum version having enabled es6 also in strict mode) og 6.9.x even. However others may not have that luxury...
http://stackoverflow.com/questions/22603078/syntaxerror-use-of-const-in-strict-mode

thats why i switch to circle-ci, only the module eslint need ES6 syntax which is a dev- dependency , so we can develop and run lint in node 4.x+ , and run only npm test on node 0.10.
just like
machine:
  node:
    version: 4

test:
  override:
    - npm run lint && npm run test
    - nvm use 0.10; npm test

that what my previous commit from here

It will not increase the minimum node requirement since we can skip lint on lower version.
The problem is that wether we should to a hint/lint on CI check.
If yes, that is the solution.
If not, it does not matter what hinter / linter to choose.

Node 0.10 is simply what was the thing when I created that file, feel free to update as needed. Today I'd probably prefer circle before Travis as well, but either way works for me. Staying with Travis probably creates slightly less work for me, which is preferable from that standpoint.

Lets increase the node version then...


⚠ Jenkins Plugins Artifactory has degraded performance

In 605f3d4, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1570 ms

Resolved: Jenkins Plugins Artifactory performance has improved in b3cfc47 after 11 minutes.


Misbehavior of parameters & form-data

I encountered a couple of an odd behavior when dealing with special parameters and form-data. Here is the source code:
var superagent = require("superagent");
var fs = require('fs');

var req = superagent.get("http://example.com/test?abc=def");
req.request().assignSocket(fs.createWriteStream("./rwparams.txt"));
req.end();

var req = superagent.post("http://example.com/test?abc=def");
req.type('form-data')
req.data({"a":"1","b":2})
req.request().assignSocket(fs.createWriteStream("./rwparams2.txt"));
req.end();

Output of rwparams.txt:
GET /test HTTP/1.1
Host: example.com
Connection: close

The parameter "abc=def" is just gone.
Output of rwparams2.txt:
POST /test HTTP/1.1
Host: example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 5
Connection: close

a=1&b

the variable "b" no longer has a value. At least it could throw an error that it has invalid input.

assignSockets overwrites the output from the "socket" to the file stream, so when node start writing the request, instead of opening a new socket, it will notice it already has an output and use the file stream.
It is more of a debugging method that I found while poking around with the library.
What you see at both the file outputs is the request HTTP headers.

ah :) I use netcat's server feature nc -l 3000

I'll try and get some tests for superagent soon

Awesome! Thanks.


Simple prompt utility

Lesson learned from lmify, adding prompter to core is not a good idea, however making CLIs can benefit from such utility if exposed directly with minimum dependencies and smart fallbacks for CI and Test environments.
A simple async prompt utility and docs how to integrate it with other commands would be useful.

consola v3 supports prompts to integrate


Building rocPRIM issue

Hi @jszuppe I need some help with build rocPRIM. I am running into this:
root@29f5659f5f83:~/rocPRIM/build# cmake -DBUILD_BENCHMARK=ON ../.
CMake Error at cmake/Dependencies.cmake:138 (find_package):
  Could not find a package configuration file provided by "ROCM" with any of
  the following names:

    ROCMConfig.cmake
    rocm-config.cmake

  Add the installation prefix of "ROCM" to CMAKE_PREFIX_PATH or set
  "ROCM_DIR" to a directory containing one of the above files.  If "ROCM"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:63 (include)

-- Configuring incomplete, errors occurred!
See also "/root/rocPRIM/build/CMakeFiles/CMakeOutput.log".
See also "/root/rocPRIM/build/CMakeFiles/CMakeError.log".

I have rocm-dkms package installed from https://github.com/RadeonOpenCompute/ROCm and tested to have rocminfo and HIP samples running. Not sure what is needed extra for rocPRIM.

I got this to work with SiuChi's suggestion to install rocm-cmake. Could you update your README to include something like: Required: ROCm cmake modules can be installed from here https://github.com/RadeonOpenCompute/rocm-cmake

Actually, cmake script should handle this and download required rocm-cmake. I really don't know why it didn't.


Binding RepeatBehavior

Hi,
I am trying to bind the RepeatBehavior parameter to a view model property like this:
    <Image Stretch="None" gif:ImageBehavior.RepeatBehavior="Forever">
        <Image.Style>
            <Style TargetType="Image">
                <Style.Triggers>
                    <DataTrigger Binding="{Binding Path=HasInvalid, Mode=OneWay, UpdateSourceTrigger=PropertyChanged}" Value="False">
                        <Setter Property="gif:ImageBehavior.AnimatedSource" Value="pack://application:,,,/Resources/a.png" />
                    </DataTrigger>
                    <DataTrigger Binding="{Binding Path=HasInvalid, Mode=OneWay, UpdateSourceTrigger=PropertyChanged}" Value="True">
                        <Setter Property="gif:ImageBehavior.AnimatedSource" Value="pack://application:,,,/Resources/b.gif" />
                    </DataTrigger>
                    <DataTrigger Binding="{Binding Path=AnimateInvalid, Mode=OneWay, UpdateSourceTrigger=PropertyChanged}" Value="False">
                        <Setter Property="gif:ImageBehavior.RepeatBehavior" Value="1x" />
                    </DataTrigger>
                    <DataTrigger Binding="{Binding Path=AnimateInvalid, Mode=OneWay, UpdateSourceTrigger=PropertyChanged}" Value="True">
                        <Setter Property="gif:ImageBehavior.RepeatBehavior" Value="Forever" />
                    </DataTrigger>
                </Style.Triggers>
            </Style>
        </Image.Style>
    </Image>

I see the property changing but the animation doesn't stop.
What am I missing?

Not sure why this doesn't work... does the problem also affect the AnimatedSource property, or just RepeatBehavior? (i.e. does the image change when the HasInvalid property changes?)

yes, the image change works fine.
is it possible that the gif has something inside that overwrites the repeat?

is it possible that the gif has something inside that overwrites the repeat?

I don't think so... A change to either AnimatedSource or RepeatBehavior should cause the animation to restart with the new settings.
But what are you trying to do exactly anyway? You say that the animation doesn't stop; is it what you're trying to do? You could try setting AutoStart to false instead of changing the RepeatBehavior

ah, that was stupid
I had on the image:  gif:ImageBehavior.RepeatBehavior="Forever"
because I thought this would be a default value that I can override with the binding
now I have removed it and it works
sorry for wasting your time

Ah, indeed! I think it would have worked in a DataTemplate trigger, but not in a Style trigger


Improve Javadoc

Remove duplicated line.
Description
I think this duplicated line was just missed in previous reviews.

Woops, thank you for your attention to detail @lukas-vlcek!  And sorry for the crazy long time to respond.  I'll merge this.


Remove config-transformer from packages/ as moving

Ref https://github.com/symplify/config-transformer/pull/6

@TomasVotruba the pending workflows can be removed:
 Split Tests of config-transformer Expected — Waiting for status to be reported
 rector (config-transformer) Expected — Waiting for status to be reported

CI should be All green after that 👍


[Issue] Unable to upload files

Description
I'm unable to upload my artefacts to the remote device. No problem to connect BUT there is no commands results, no folder created and nothing to push.
Severity (1-5)
4=Low (annoyance), 5=High (crashes visual studio)
Steps To Reproduce
Steps to reproduce the behavior:

Click on Build and Deply
Error message :

12:08:44.468: Build was successful
12:08:45.039: Connected to tiflex@<IP_ADDRESS>:22 via SSH and SFTP
12:08:45.058: BASH> uname -m
12:08:45.081: BASH> which curl ; echo $?
12:08:45.102: BASH> [ -d /.vs-debugger/vs2022 ] || curl -ksSL https://aka.ms/getvsdbgsh | bash /dev/stdin -v latest -l /.vs-debugger/vs2022
12:08:45.177: Returned:
12:08:45.177: BASH> mkdir -p
12:08:45.196: BASH> sudo chown -R tiflex:tiflex
12:08:45.250: BASH> rm -rf Ockham125/{*,.*}
12:08:45.268: BASH> mkdir -p Ockham125
12:08:45.289: Destination Tar.GZ: 'Ockham125/vsldBuildContents.tar.gz'
12:08:45.289: Getting bin files for transfer...
12:08:45.301: Compressing files for transfer...
12:08:53.588: Update file count: 181; File Size: [137163064 bytes] of Total Files: 181 [137163064 bytes] need to be updated
12:08:53.588: Uploading...
12:08:53.593: Error while uploading file. No such file
   à Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)
   à Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)
   à VsLinuxDebugger.Core.SshTool.<>c__DisplayClass22_0.<PayloadCompressAndUploadAsync>b__1()
   à System.Threading.Tasks.Task.Execute()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   à System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   à VsLinuxDebugger.Core.SshTool.<PayloadCompressAndUploadAsync>d__22.MoveNext()
12:08:53.593: BASH> set -e;cd "Ockham125";tar -zxf "vsldBuildContents.tar.gz"
12:08:53.610: Payload Decompress results: '' (blank=OK)
12:08:53.610: Upload completed with failure.

Dev environment:
Win 10.0.19044
VS2022:latest
LinuxDebugger:  latest
Linux Distribution:

OS: Raspbian 64b / Debian
Version: 11 Bullseye

Ok my bad the system memory was full ... ;)
Thanks for your work it's damn great !

@dedounet no worries, thank you for the heads up. I'll keep error handling around this in mind for a future release
Appreciate your support using this. Please feel free to give the project a thumbs up on the Visual Studio Extension store

I'm facing the same problem and I have no idea what's happening.
@dedounet What did you mean with the system memory was full. Did you run out of disk space or RAM and does this apply to the remote machine or your local machine.


RKE2 Registries - When multiple registry authentication hosts exist and one is removed, an incorrect secret name is rendered

Setup

Rancher version: 2.9-head

Describe the bug

When you have multiple registry authentication hosts if you remove one of them the other one doesn't show its original secret name anymore. This is just a rendering issue since after save the payload still has the original secret name.
Screenshots

https://github.com/rancher/dashboard/assets/135728925/2eee1e79-98ff-45ce-90b4-c92ca87735e7

possible automation-candidate if the cluster can be faked to go to EDIT view, something based of https://github.com/rancher/dashboard/blob/master/cypress/e2e/blueprints/nav/fake-cluster.ts as an example

https://github.com/rancher/dashboard/pull/10898#issuecomment-2101104810

I think I've missed some other PRs where the tests were updated. Moving to done.


Monero Wallet RPC: createSubaddress returns same value (on linux)

Hello there,
i am not sure if it's a problem of this library.
First of all, i've tried monero version v<IP_ADDRESS> and v<IP_ADDRESS>
Magically, when i call the WalletRPC on a windows machine, i'll retrieve a new value each time i call createSubaddress(accIdx).
Using the same code on a linux machine (ubuntu/debian with openjdk) it returns each time the first address which have been generated through code.
I'm starting the wallet-rpc on both achines as follows
Windows:
 monero-wallet-rpc.exe --wallet-dir D:\00_disk_e\coins\monero-wallet\testwallet --rpc-bind-port 4711 --prompt-for-password --rpc-login admin:password
Linux:
./monero-wallet-rpc --wallet-dir /media/shopsrv/42ECF9B9ECF9A777/monero-wallet/testwallet2 --rpc-bind-port 4711 --prompt-for-password --rpc-login admin:password
Of course, the rpc-login is just for testing at the moment.
My call code calls are as follows:
final MoneroWalletRpc walletRpc = new MoneroWalletRpc(url, walletConfig.getWalletConfig().getUsername(), walletConfig.getWalletConfig().getPassword());
walletRpc.openWallet(walletConfig.getWalletConfig().getWUsername(),
walletConfig.getWalletConfig().getWPassword());
final String primaryAddress = walletRpc.getPrimaryAddress();
final MoneroSubaddress moneroSubaddress = walletRpc.getAddressIndex(primaryAddress);
newMoneroAdress = walletRpc.createSubaddress(moneroSubaddress.getAccountIndex());

all you see at the getters are simple data-objects containing the information and such.
Do you have an idea or hint where there's a difference on two Operating Systems? I strongly suspect the monero implemenations, but i am not sure about this. maybe you have an idea?

okay, seems either a call issue or really library issue.
sometimes it works, sometimes not. I've got it managed, that on my windows machine the wallet addresses are always the same.
however, when i startet the monero-gui, the newly created adress was not within the wallet.
when i pressed "create new address" in the gui, a new address was generated, but just like the address from createSubaddress() method.
after another hit on "create new address" within the gui created simply a new address.
Here's the full-code call, any ideas?
		try {
			walletRpc.openWallet(walletConfig.getWalletConfig().getWUsername(), walletConfig.getWalletConfig().getWPassword());
			final String primaryAddress = walletRpc.getPrimaryAddress();
			final MoneroSubaddress moneroSubaddress = walletRpc.getAddressIndex(primaryAddress);
			newMoneroAdress = walletRpc.createSubaddress(moneroSubaddress.getAccountIndex());
			return newMoneroAdress == null ? null : newMoneroAdress.getAddress();
		} finally {
			if (!walletRpc.isClosed()) {
				walletRpc.close();
			}
		}
	}

This is sounding like a discrepancy with monero-wallet-rpc as the library should be making RPC calls the same way on both platforms.  To confirm, you could log and compare the RPC requests and responses to ensure they're identical.  This can be done by uncommenting the System.out.println() statements in MoneroRpcConnection.java (needs proper logging config :/).
Note that unused created subaddresses will not carry over when restoring a wallet from mnemonic, but they should be saved to wallet files.

somehow i managed to provoke this on windows, too. which is "nice".
i'll try the sysouts or just inject the log4j logger in there and replace the calls.
let me try this out, i guess there's something i am not doing right.

soooo, after a while i figured out that i need to call walletRpc.save()
then the address is really persisted on the wallet.


Message batching

Problem
The current implementation is limited in the number of send messages it can serve, it behaves by submitting a transaction for each send mutation. The issue in doing so resides in the number of transaction signed by the same account a block can contain, it is limited to one transaction which in this case leads to a possibility of only one send mutation per block.
Proposed solution
The simplest way to solve this is by batching the messages to submit maximum a transaction per block. This will certainly break the API as the send mutation will be asynchronous, some informations can thus not be returned anymore (e.g. the transaction hash).
To allow a client to still be able to retrieve such transaction specific data, we should consider implementing the send operation through GraphQL subscriptions also.

:tada: This issue has been resolved in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Getting a 500 internal server error when inviting a parent user to a suborganization

Describe the issue:

Getting a 500 internal server error when inviting a parent user to a suborganization
https://github.com/wso2/product-is/assets/40081059/c4ba2c2f-0189-4a20-a0e0-225316487ea6
How to reproduce:

1.Login to the console
2.Create a user
3.Create a sub org and switch into it
4.Go to users section
5.Go to add user
6.Select a single user  from drop down
7.Click on  invite parent user
Expected behavior:

Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: [e.g., IS 5.10.0, IS 5.9.0]
OS: [e.g., Windows, Linux, Mac]
Database: [e.g., MySQL, H2]
Userstore: [e.g., LDAP, JDBC]

Optional Fields
Related issues:

Suggested labels:

This occurs when trying to invite a user who is not existing in the parent organization

The issue is not yet resolved.
Exception occurs at https://github.com/wso2-extensions/identity-organization-management/blob/d86a85c41dc69eb5fa67abda0b1b2e16e19df6aa/components/org.wso2.carbon.identity.organization.user.invitation.management/src/main/java/org/wso2/carbon/identity/organization/user/invitation/management/InvitationCoreServiceImpl.java#L610-L611 due to invitedUserId being null


RFC: eslint-config-standard-with-typescript to depart from standard

https://github.com/standard/standard/issues/1957

Please discuss there.


[IOTDB-1572]Should not change object's state in toString()

https://issues.apache.org/jira/browse/IOTDB-1572

Try to run mvn spotless:apply?


Add Code of conduct material in the jinja template.

There was code of conduct of this template repository but the jinja template was blank...!

There is a non-jinja file already: https://github.com/scipp/copier_template/blob/main/CODE_OF_CONDUCT.md
I don't know if it means we don't need a template file, or if the blank template served a purpose?

I thought copier only copies jinja files, but requirements don't have any jinja files... hmm...
Maybe we can just delete this file since it is not using any of those variables set by copier....??

Copier also copies non-jinja files. Since we COC for copier is the same as for other projects this should not be necessary. See https://scipp.github.io/sciline/developer/index.html#code-of-conduct where this works.

Sorry, closed by accident: Yes, the empty file should be removed, I suppose?

I just removed the jinja file...! Thanks.


feat: add support for remaining args

I don't know how to make this change testable because it happens at the creation of the Pipeline so I added a demo script to showcase what it does and as an easy way to check that all is good for you.
One idea for making it testable was to change:
https://github.com/slaveOftime/Fun.Build/blob/3053b5eb9fa259ae5d6a454e6b2aa375fbb2544f/Fun.Build/PipelineContextExtensions.fs#L13-L15
to
PipelineContext with

     static member Create(name: string, ?forceArgs : string array) =

but I don't think this is good idea to change an API just to make it testable. Because I don't think we ever want to override the Environment.GetCommandLineArgs() in a normal usage.

@MangelMaxime for the unit test, I think you can write code like:
let mutable actualAllCmdArgs = []
let mutable actualRemainingArgs = []

pipeline "demo" {
    cmdArgs ["-p"; "demo"; "test1"; "v1"; "--"; "test2"; "v2"]
    stage "shows arguments" {
        run (fun ctx ->
            actualAllCmdArgs  <- ctx.GetAllCmdArgs()
            actualRemainingArgs  <- ctx.GetRemainingsCmdArgs()
        )
    }
    runImmediate
}

// Assert accordingly

Regarding the unit test, originally I wrote something like you suggested but in the middle of writing this PR I changed how I implemented RemainingArgs and removed the tests because I think cmdArgs take effect after PipelineContext.Create has been called.
I will give it a try and check what happens.

I checked and with my current implementation I can't test it the way you proposed.
However, looking at the comment in the code, I think I should also add the Args/Remaining split logic at this place too no?
https://github.com/slaveOftime/Fun.Build/blob/bedb12fedab68204d9b3985c9fe34dcea94272d9/Fun.Build/PipelineBuilder.fs#L174-L181

Yes, I think we should do that.

@albertwoo This is ready for review.
By using making cmdArgs support Remaining args too and using an utility type to transform a list of args into a list of args + remaining args, I am able to indirectly test both places where this transformation takes place.

The failing test doesn't seem related to my changes but caused by a slow CI runner.

The failing test is related to a paralle tests, existing before, but did not figure out why 😂.
I will merge your PR, thanks for your contribution.


color code drop-down list of Tags

Would be great to see all Tag lists color coded to know which Category on:

As far as styling , here are some good tips worth experimenting with:

selectInput--CSS customization - shiny - RStudio Community

Yay, color-coded dropdowns 🥳

Yay, finito!
https://github.com/marineenergy/apps/pull/48