Add support for CosmosDB Emulator and Storage Emulator (Azurite)

Is your feature request related to a problem? Please describe.
Not related to a problem
Describe the solution you'd like
I would like to see setups for CosmosDB Emulator and Azurite to allow usage for testing storage or cosmosDB functions.
Describe alternatives you've considered
I have one implemented locally before I found this repo. I can contribute and add these changes but need to make sure I find everything I need to make sure it works. I see there is ExecuteScripts functions and I don't think they come with CLIs in the images so I would have to check.
Additional context

Thanks for your feature request. Supporting Azure Cosmos DB would be great.
Right now, .NET Testcontainers does not ship any 3rd party database or message broker libraries etc. This makes it easier to maintain the library, but it has a disadvantage. All vendor specific calls, features, etc. go straight to the container. We do not use any client libraries at all.
I'd like to move modules in the future (databases, message brokers, etc.) to their own library. This allows much better support incl. features for databases, message brokers, etc.
What do you think about a specific project, like DotNet.Testcontainers.CosmosDB?

Thanks for your feature request. Supporting Azure Cosmos DB would be great.
Right now, .NET Testcontainers does not ship any 3rd party database or message broker libraries etc. This makes it easier to maintain the library, but it has a disadvantage. All vendor specific calls, features, etc. go straight to the container. We do not use any client libraries at all.
I'd like to move modules in the future (databases, message brokers, etc.) to their own library. This allows much better support incl. features for databases, message brokers, etc.
What do you think about a specific project, like DotNet.Testcontainers.CosmosDB?

That would be great! Really anything to take advantage of all the awesome work you have done here. Maybe DotNet.Testcontainers.CosmosDB and DotNet.Testcontainers.Azurite ?

Yep, sounds good.

Built in integration would be a god send!
I'm having trouble trying to spin up an instance of CosmosDb - I'm not able to access the cosmos explorer, do you know if the cosmos image should be compatible with this library?
Running this command works fine:
docker run --name=Cosmos_DB -d -p 8081:8081 -p 10251:10251 -p 10252:10252 -p 10253:10253 -p 10254:10254 -e AZURE_COSMOS_EMULATOR_PARTITION_COUNT=29 -e AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=<IP_ADDRESS> -e AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
However the following creates the container but I can't connect to the explorer:
    var cosmosDbContainerBuilder = new TestcontainersBuilder<TestcontainersContainer>()
        .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
        .WithName("Cosmos_DB")
        .WithPortBinding(8081, 8081)
        .WithPortBinding(10251, 10251)
        .WithPortBinding(10252, 10252)
        .WithPortBinding(10253, 10253)
        .WithPortBinding(10254, 10254)
        .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "3")
        .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
        .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
        .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(8081));

    await using (var cosmosDbContainer = cosmosDbContainerBuilder.Build())
    {
        await cosmosDbContainer.StartAsync();

        //...
    }
    ```

Built in integration would be a god send!
I'm having trouble trying to spin up an instance of CosmosDb - I'm not able to access the cosmos explorer, do you know if the cosmos image should be compatible with this library?
Running this command works fine:
docker run --name=Cosmos_DB -d -p 8081:8081 -p 10251:10251 -p 10252:10252 -p 10253:10253 -p 10254:10254 -e AZURE_COSMOS_EMULATOR_PARTITION_COUNT=29 -e AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=<IP_ADDRESS> -e AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
However the following creates the container but I can't connect to the explorer:
    var cosmosDbContainerBuilder = new TestcontainersBuilder<TestcontainersContainer>()
        .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
        .WithName("Cosmos_DB")
        .WithPortBinding(8081, 8081)
        .WithPortBinding(10251, 10251)
        .WithPortBinding(10252, 10252)
        .WithPortBinding(10253, 10253)
        .WithPortBinding(10254, 10254)
        .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "3")
        .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
        .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
        .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(8081));

    await using (var cosmosDbContainer = cosmosDbContainerBuilder.Build())
    {
        await cosmosDbContainer.StartAsync();

        //...
    }
    ```

You need to expose the port 8081 too. It's not exposed by the Dockerfile.
Then I recommend assigning random ports.
In addition to that, it's not enough to just wait for port 8081. You'll need a custom strategy to figure out when the container is up and running:
private readonly ITestcontainersContainer container = new TestcontainersBuilder<TestcontainersContainer>()
  .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
  .WithName("azure-cosmos-emulator")
  .WithExposedPort(8081)
  .WithPortBinding(8081, true)
  .WithPortBinding(10251, true)
  .WithPortBinding(10252, true)
  .WithPortBinding(10253, true)
  .WithPortBinding(10254, true)
  .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
  .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
  .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
  .WithWaitStrategy(Wait.ForUnixContainer()
    .UntilPortIsAvailable(8081))
  .Build();

[Fact]
public async Task Issue()
{
  // TODO: You need to replace this with a proper wait strategy. Port 8081 is accessible before the container is ready.
  await Task.Delay(TimeSpan.FromSeconds(30))
    .ConfigureAwait(false);

  using (var handler = new HttpClientHandler())
  {
    handler.ServerCertificateCustomValidationCallback = (_, _, _, _) => true;

    using (var client = new HttpClient(handler))
    {
      var mappedPort = this.container.GetMappedPublicPort(8081);

      var response = await client.GetAsync($"https://localhost:{mappedPort}/_explorer/emulator.pem")
        .ConfigureAwait(false);

      var pem = await response.Content.ReadAsStringAsync()
        .ConfigureAwait(false);

      Debug.WriteLine(pem);
    }
  }
}

public Task InitializeAsync()
{
  return this.container.StartAsync();
}

public Task DisposeAsync()
{
  return this.container.DisposeAsync().AsTask();
}

You need to expose the port 8081 too. It's not exposed by the Dockerfile.
Then I recommend assigning random ports.
In addition to that, it's not enough to just wait for port 8081. You'll need a custom strategy to figure out when the container is up and running:
private readonly ITestcontainersContainer container = new TestcontainersBuilder<TestcontainersContainer>()
  .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
  .WithName("azure-cosmos-emulator")
  .WithExposedPort(8081)
  .WithPortBinding(8081, true)
  .WithPortBinding(10251, true)
  .WithPortBinding(10252, true)
  .WithPortBinding(10253, true)
  .WithPortBinding(10254, true)
  .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
  .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
  .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
  .WithWaitStrategy(Wait.ForUnixContainer()
    .UntilPortIsAvailable(8081))
  .Build();

[Fact]
public async Task Issue()
{
  // TODO: You need to replace this with a proper wait strategy. Port 8081 is accessible before the container is ready.
  await Task.Delay(TimeSpan.FromSeconds(30))
    .ConfigureAwait(false);

  using (var handler = new HttpClientHandler())
  {
    handler.ServerCertificateCustomValidationCallback = (_, _, _, _) => true;

    using (var client = new HttpClient(handler))
    {
      var mappedPort = this.container.GetMappedPublicPort(8081);

      var response = await client.GetAsync($"https://localhost:{mappedPort}/_explorer/emulator.pem")
        .ConfigureAwait(false);

      var pem = await response.Content.ReadAsStringAsync()
        .ConfigureAwait(false);

      Debug.WriteLine(pem);
    }
  }
}

public Task InitializeAsync()
{
  return this.container.StartAsync();
}

public Task DisposeAsync()
{
  return this.container.DisposeAsync().AsTask();
}

You need to expose the port 8081 too. It's not exposed by the Dockerfile. Then I recommend assigning random ports.
In addition to that, it's not enough to just wait for port 8081. You'll need a custom strategy to figure out when the container is up and running:
private readonly ITestcontainersContainer container = new TestcontainersBuilder<TestcontainersContainer>()
  .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
  .WithName("azure-cosmos-emulator")
  .WithExposedPort(8081)
  .WithPortBinding(8081, true)
  .WithPortBinding(10251, true)
  .WithPortBinding(10252, true)
  .WithPortBinding(10253, true)
  .WithPortBinding(10254, true)
  .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
  .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
  .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
  .WithWaitStrategy(Wait.ForUnixContainer()
    .UntilPortIsAvailable(8081))
  .Build();

[Fact]
public async Task Issue()
{
  // TODO: You need to replace this with a proper wait strategy. Port 8081 is accessible before the container is ready.
  await Task.Delay(TimeSpan.FromSeconds(30))
    .ConfigureAwait(false);

  using (var handler = new HttpClientHandler())
  {
    handler.ServerCertificateCustomValidationCallback = (_, _, _, _) => true;

    using (var client = new HttpClient(handler))
    {
      var mappedPort = this.container.GetMappedPublicPort(8081);

      var response = await client.GetAsync($"https://localhost:{mappedPort}/_explorer/emulator.pem")
        .ConfigureAwait(false);

      var pem = await response.Content.ReadAsStringAsync()
        .ConfigureAwait(false);

      Debug.WriteLine(pem);
    }
  }
}

public Task InitializeAsync()
{
  return this.container.StartAsync();
}

public Task DisposeAsync()
{
  return this.container.DisposeAsync().AsTask();
}

Incredible work, will be looking at implementing this shortly!

Thanks so much!
In case anyone else stumbles across this I managed to get everything working with the following snippet:
[CollectionDefinition(nameof(IntegrationTestCollection))]
public class IntegrationTestCollection : ICollectionFixture<IntegrationTestCollectionFixture> { }

public sealed class IntegrationTestCollectionFixture : IDisposable
{
    private readonly TestcontainersContainer _cosmosContainer;
    public IntegrationTestCollectionFixture()
    {
        var outputConsumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
        var waitStrategy = Wait.ForUnixContainer().UntilMessageIsLogged(outputConsumer.Stdout, "Started");
        _cosmosContainer = new TestcontainersBuilder<TestcontainersContainer>()
            .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
            .WithName("azure-cosmos-emulator")
            .WithExposedPort(8081)
            .WithExposedPort(10251)
            .WithExposedPort(10252)
            .WithExposedPort(10253)
            .WithExposedPort(10254)
            .WithPortBinding(8081, true)
            .WithPortBinding(10251, true)
            .WithPortBinding(10252, true)
            .WithPortBinding(10253, true)
            .WithPortBinding(10254, true)
            .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30")
            .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
            .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
            .WithOutputConsumer(outputConsumer)
            .WithWaitStrategy(waitStrategy)
            .Build();

        _cosmosContainer.StartAsync().GetAwaiter().GetResult();
    }

    public void Dispose()
    {
        _ = _cosmosContainer.DisposeAsync();
    }
}

You need to be aware of the async operations. Your IntegrationTestCollectionFixture is not right. Use the IAsyncLifetime interface of xUnit.net.
public sealed class IntegrationTestCollectionFixture : IAsyncLifetime, IDisposable
{
  private readonly IOutputConsumer consumer = Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());

  private readonly ITestcontainersContainer container;

  public IntegrationTestCollectionFixture()
  {
    this.container = new TestcontainersBuilder<TestcontainersContainer>()
      .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
      .WithName("azure-cosmos-emulator")
      .WithExposedPort(8081)
      .WithExposedPort(10251)
      .WithExposedPort(10252)
      .WithExposedPort(10253)
      .WithExposedPort(10254)
      .WithPortBinding(8081, true)
      .WithPortBinding(10251, true)
      .WithPortBinding(10252, true)
      .WithPortBinding(10253, true)
      .WithPortBinding(10254, true)
      .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30")
      .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
      .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
      .WithOutputConsumer(this.consumer)
      .WithWaitStrategy(Wait.ForUnixContainer()
        .UntilMessageIsLogged(this.consumer.Stdout, "Started"))
      .Build();
  }

  public Task InitializeAsync()
  {
    return this.container.StartAsync();
  }

  public Task DisposeAsync()
  {
    return this.container.StopAsync();
  }

  public void Dispose()
  {
    this.consumer.Dispose();
  }
}

What do you think about a specific project, like DotNet.Testcontainers.CosmosDB?

@HofmeisterAn  Do you mean new project in same repo or in dedicated repo?

I think it makes sense to keep modules in this repository, but don't include them in the Testcontainers dependency anymore. Each module should be an independent package.

Maybe we can avoid using the Cosmos client SDK (at least for now) by using the REST API see: https://docs.microsoft.com/en-us/rest/api/cosmos-db/querying-cosmosdb-resources-using-the-rest-api).
Would this cover the usecase?
Would love to try my hand at this for a bit if noone else is.

OC, we can use the version without client as MVP and enhance it as soon as the new structure is available.

This thread has been incredibly helpful. Thank you!
I've got my test container starting, and I can see the cosmos db emulator container start up in the docker desktop dashboard however I am struggling to get the test to connect to it.
I think it is down to the connection string.
In the example above ...
services.AddSingleton(sp =>
{
    var cosmosClientBuilder = new CosmosClientBuilder(connString);
    cosmosClientBuilder.WithHttpClientFactory(() =>
    {
        HttpMessageHandler httpMessageHandler = new HttpClientHandler()
        {
            ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
        };

        return new HttpClient(httpMessageHandler);
    });
    cosmosClientBuilder.WithConnectionModeGateway();

    return cosmosClientBuilder.Build();
});

The string for connString is mentioned but not constructed. Is this the default dev emulator detail or do I need to update it to point to a different port or something? If it's not this, then I am lost 😕
Any help would be appreciated.
Thanks

one but the port is updated before being provided to the CosmosClientBuilder.

I assume you refer to the random assigned host port? This is a best practice and avoid clashes. If you assign a random host port WithPortBinding(8081, true) (notice the true arg), you need to get the public mapped port first. Like you have done with GetMappedPublicPort(8081). See this example too: https://github.com/testcontainers/testcontainers-dotnet/discussions/438.

Next issue, creating db/containers on the fly for the tests ...

Usually, that's done via a client dependency or a cli command (ExecAsync).

@HofmeisterAn I opened a Draft PR #549 with some progress, managed to get it up and running and create a database in the container at least. I havn't done any OS before so some early-ish feedback is much appreciated!

@Yeseh @HofmeisterAn I got CosmosDB running in a testcontainer but have some issues with performance. I reached out on twitter and was put in contact with a PM at Microsoft on the cosmos db emulator team. I put together a demo repo to show the setup. Just waiting for a reply.
Please take a look at the repo - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test

Found this which has some good examples of constructing the web requests to call CosmosDB directly which could aid with not using the cosmos client - https://github.com/Azure-Samples/cosmos-db-rest-samples/blob/main/Program.cs

In the examples on how to run the cosmosdb linux emulator in docker it specifies setting the memory and cpu levels ...
docker run \
    --publish 8081:8081 \
    --publish 10251-10254:10251-10254 \
    --memory 3g --cpus=2.0 \
    --name=test-linux-emulator \
    --env AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10 \
    --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \
    --env AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=$ipaddr \
    --interactive \
    --tty \
    mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator

How can these be specified in the testcontainer setup? Thanks

Please take a look at the repo - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test

Please notice, I'm not an CosmosDB expert 😃, but OC, I'll take a look at it.

How can these be specified in the testcontainer setup? Thanks

You can use WithCreateContainerParametersModifier, see #503 and #508.

Cool, thanks @HofmeisterAn. Will have a go with the WithCreateContainerParametersModifier 😄

Found this which has some good examples of constructing the web requests to call CosmosDB directly which could aid with not using the cosmos client - https://github.com/Azure-Samples/cosmos-db-rest-samples/blob/main/Program.cs

Good stuff, that is very useful. Thanks! I'll go implemnent a couple of those at least .
What should the user interface of the Testcontainer be? The above example seems pretty complete, would it be desirable to have all these operations available to the user in the final version (with or without client)?

I see CosmosDB is in progress but Azurite isn't.
Are there any objections if I will try work on AzuriteTestcontainer?

@HofmeisterAn  Thanks for your feedback on setting up Testcontainers with CosmosDB emulator.
I'm using EF Core Cosmos provider in my application so in xUnit during the test initialization phase I am calling EnsureDeletedAsync() and EnsureCreatedAsync() methods but I am getting these  errors:
Gateway HttpRequestException Endpoint not reachable. Failed Location: https://<IP_ADDRESS>:8081/; ResourceAddress: dbs/Database
Endpoint https://<IP_ADDRESS>:8081/ unavailable for Write added/updated to unavailableEndpoints with timestamp 09/19/2022 15:39:36
I wonder if I would need to specify other local IP address in override of environment variable? Really weird as I am running in Gateway mode

I wonder if I would need to specify other local IP address in override of environment variable? Really weird as I am running in Gateway mode

Is <IP_ADDRESS> a const value? Try the Hostname property and get the random assigned host port via GetMappedPublicPort.

See the CosmosDb emulator PR for more details.
This should work as well:
`public sealed class CosmosDbFixture : IDisposable, IAsyncLifetime
{
private readonly IOutputConsumer consumer =
Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());
    private readonly ITestcontainersContainer _container;

    public CosmosDbFixture()
    {
        _container = new TestcontainersBuilder<TestcontainersContainer>()
            .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
            .WithExposedPort(8081)
            .WithExposedPort(10251)
            .WithExposedPort(10252)
            .WithExposedPort(10253)
            .WithExposedPort(10254)
            .WithPortBinding(8081, true)
            .WithPortBinding(10251, true)
            .WithPortBinding(10252, true)
            .WithPortBinding(10253, true)
            .WithPortBinding(10254, true)
            .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1")
            .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
            .WithOutputConsumer(consumer)
            .WithWaitStrategy(Wait.ForUnixContainer()
                .UntilMessageIsLogged(consumer.Stdout, "Started"))
            .Build();
    }

    public int Port => _container.GetMappedPublicPort(8081);

    public string Host => _container.Hostname;

    public Task InitializeAsync()
    {
        return _container.StartAsync();
    }

    public Task DisposeAsync()
    {
        return _container.DisposeAsync().AsTask();
    }

    public void Dispose()
    {
        consumer.Dispose();
    }
}`

Together with a DelegatingHandler which rewrites the url:
request.RequestUri = new Uri($"https://{_host}:{_portNumber}{request.RequestUri.PathAndQuery}");

This is the same way I did it with creating the default db and container as well - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/main/tests/Api.Tests/Infrastructure/DataContainerFixture.cs
The key for me was the delegating handler to make sure the updated uri was mapped correctly as the default port seemed hardcoded somewhere in the emulator - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/8c04fd345fc272123f4ac964beaea0d0d690ca41/tests/Api.Tests/Infrastructure/FixRequestLocationHandler.cs

So I've tried the above solution to running CosmosDB emulator but I keep getting the following error:

System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:49174)

This exception is thrown when executing the following line:
var response = await _client.CreateDatabaseAsync("mydatabase", 4000);
Any ideas?
I'm using version 2.2.0-beta.3123645733

@wbuck Can you more information about your host setup (OS, Docker version, etc.)? How do you set up your client?

@wbuck Can you add more information about your host setup (OS, Docker version, etc.)? How do you set up your client?

@HofmeisterAn Yeah of course.
The code I'm using to setup the client is identical to what is shown here except that I've exposed more ports in an attempt to actually connect with the container.
I've also tried different addresses (instead on localhost).
The code for the FixRequestLocationHandler can be found here.
Like I mentioned previously I copied the code that @WestDiscGolf had shown.
[TestFixture]
internal class ProjectServiceTests
{
    private readonly IOutputConsumer _consumer =
        Consume.RedirectStdoutAndStderrToStream(new MemoryStream(), new MemoryStream());

    private const string AccountKey =
        "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";

    private ITestcontainersContainer _container;
    private CosmosClient _client;

    [OneTimeSetUp]
    public async Task InitializeAsync()
    {
        _container = new TestcontainersBuilder<TestcontainersContainer>()
            .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator")
            .WithExposedPort(8081)
            .WithExposedPort(8900)
            .WithExposedPort(8901)
            .WithExposedPort(8902)
            .WithExposedPort(10250)
            .WithExposedPort(10251)
            .WithExposedPort(10252)
            .WithExposedPort(10253)
            .WithExposedPort(10254)
            .WithExposedPort(10255)
            .WithExposedPort(10256)
            .WithExposedPort(10350)
            .WithPortBinding(8081, true)
            .WithPortBinding(8900, true)
            .WithPortBinding(8901, true)
            .WithPortBinding(8902, true)
            .WithPortBinding(10250, true)
            .WithPortBinding(10251, true)
            .WithPortBinding(10252, true)
            .WithPortBinding(10253, true)
            .WithPortBinding(10254, true)
            .WithPortBinding(10255, true)
            .WithPortBinding(10256, true)
            .WithPortBinding(10350, true)
            .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "10")
            .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
            .WithEnvironment("AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
            .WithOutputConsumer(_consumer)
            .WithWaitStrategy(Wait.ForUnixContainer()
                .UntilMessageIsLogged(_consumer.Stdout, "Started"))
            .Build();

        await _container.StartAsync().ConfigureAwait(false);

        var mappedPort = _container.GetMappedPublicPort(8081);
        var updated = $"https://localhost:{mappedPort}";

        var cosmosClientBuilder = new CosmosClientBuilder(updated, AccountKey);
        cosmosClientBuilder.WithHttpClientFactory(() =>
        {
            var handler = new HttpClientHandler
            {
                ServerCertificateCustomValidationCallback =
                    HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
            };

            return new HttpClient(new FixRequestLocationHandler(mappedPort, handler));
        });

        cosmosClientBuilder.WithConnectionModeGateway();
        _client = cosmosClientBuilder.Build();

        var response = await _client.CreateDatabaseAsync("mydatabase", 4000)
            .ConfigureAwait(false);

        var container = await response.Database.CreateContainerAsync("organization", "/organizationId", 4000)
            .ConfigureAwait(false);
    }

    [OneTimeTearDown]
    public async Task TearDownAsync()
    {
        await _container.StopAsync().ConfigureAwait(false);
    }

    [Test]
    public void DoNothing()
    {
        Debugger.Break();
    }
}

OS:  Windows 10 Enterprise, version 21H2, build 19044.2006
Docker: version 20.10.17, build 100c701

I've also tried different addresses (instead of localhost).

Do not use localhost. Use the Hostname property instead. This applies to FixRequestLocationHandler too. localhost is not always right. Can you double-check if the Hostname property resolves to the correct hostname / ip?

The HostName is localhost with an IpAddress of <IP_ADDRESS> which does correspond to the containers IP address.
I've tried using both the HostName and the IpAddress as the host, but I get the below error message (sub out the IP address for localhost when I use HostName instead of the IpAddress).

System.Net.Http.HttpRequestException : A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (<IP_ADDRESS>:49320)

Ok, let's test something. Can you add an empty .dockerenv file to the root directory of your host / OS and test it again?

OK, so after creating the .dockerenv in root the HostName property is now host.docker.internal.
If I use the HostName I get the following exception:

System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (host.docker.internal:49380)

I also want to note that I still have this line: .WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>") specified on the builder. I'm unsure if that should be removed.

If you have some time tomorrow, ping me on Slack. Maybe the Hostname resolves the wrong name or address in your case and we can figure it out together.

@wbuck If you have some time tomorrow, ping me on Slack. Maybe the Hostname resolves the wrong name or address in your case and we can figure it out together.

@HofmeisterAn Maybe release a new beta and use the new cosmosdb testcontainer. It should work directly without any extra config.

@wbuck or just check the code in develop.  Use CosmosDbTestcontainer and related classes.

@ktjn Funny I was thinking the same thing. For my usual day-to-day I do a lot of system driver programming and due to that I have a bunch of VM's and virtual switches set up. I have a suspicion that's what's messing me up. I'll grab the develop branch and give it a go.

I tried the CosmosDbTestcontainer but unfortunately it failed for the same reason.
Tonight, after I put my kids down, I'll do some more digging into the matter and try to figure out what it is about my machine that's giving me grief.

OK, so currently I have the following network adapters:

Ethernet (Realtek USB GbE Family Controller)
vEthernet (Default Switch) Hyper-V
vEthernet (External VM Switch) Hyper-V
vEthernet (nat) Hyper-V
vEthernet (WSL) Hyper-V
Wi-Fi

In order to connect with the test container, I had to first disable the vEthernet (External VM Switch) Hyper-V adaptor.
I then had to enable Internet Protocol Version 4 (TCP/IPv4) for my actual NIC (this was disabled on the actual NIC because I was sharing the network adaptor with the VM's through the external VM switch).
I have some really distant memory that Hyper-V and Docker have had some issues, but I can't remember what that was.
Anyways, the issue seems resolved. Thank-you for everyone's help.

In order to connect with the test container, I had to first disable the vEthernet (External VM Switch) Hyper-V adaptor.
I then had to enable Internet Protocol Version 4 (TCP/IPv4) for my actual NIC (this was disabled on the actual NIC because I was sharing the network adaptor with the VM's through the external VM switch).

@kiview @eddumelendez @mdelapenya how do you deal in other languages with the network settings? Some devs have issues to connect to containers. Usually, I cannot reproduce it. Any idea how to make it more reliable or convenient for the dev to figure out what is going wrong?

I have tried running my example - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/ - with hyper v running as well as docker desktop and the tests are successful. I have tried commenting out the following line in the DataContainerFixture:
//.WithEnvironment("AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", "<IP_ADDRESS>")
The test runs continue to function.
I have the Default Switch Hyper-V NIC and not the "external switch" configured NIC. I am running Windows 10 21H1 if that helps.
Unsure if any of this information is helpful but thought I'd mention it just in case :smile:

I had to create the external switch manually via the virtual switch manager.
You can use the following guide](https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/get-started/create-a-virtual-switch-for-hyper-v-virtual-machines) to set that up.
I think the key is to enable Allow management operating system to share this network adapter.
Also I'm using gen 1 VM's (due to HCK, HLK limitations), but I don't believe that's an issue in this case.
If I re-enable my external VM switch I lose the ability to connect to the container.

If I re-enable my external VM switch I lose the ability to connect to the container.

To any container? Are you able to connect through a different IP or something like that?

I'll test that after I drop my kids off at school.
I have a SQL Server Linux container that I can try and connect to. I'll try connecting to it via this library and manually via Azure Data Studio.

Getting an error while running the test application - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test
System.TypeInitializationException
HResult=0x80131534
Message=The type initializer for 'DotNet.Testcontainers.Configurations.TestcontainersSettings' threw an exception.
Source=Testcontainers
StackTrace:
at DotNet.Testcontainers.Configurations.TestcontainersSettings.get_OS()
at DotNet.Testcontainers.Builders.TestcontainersBuilder`1..ctor()
at Api.Tests.Infrastructure.DataContainerFixture..ctor() in C:\CosmosDB-Testcontainers-Test-main\tests\Api.Tests\Infrastructure\DataContainerFixture.cs:line 21
This exception was originally thrown at this call stack:
[External Code]
Inner Exception 1:
InvalidOperationException: Sequence contains no elements

@rrr-ganesh This looks like Docker is not running or Testcontainers for .NET cannot detect your Docker host configuration. The latest snapshot release should show an error message with more context (see Custom Configuration too).

Think I am having similar issues with the stock CosmosDbTestcontainer (version 2.2.0) - container is running, but connection is refused. Happens on both Windows 11 with Docker installed, and in GitHub Actions Linux environment. Here's the stack trace for Actions:

warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
      Failed to determine the https port for redirect.
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
      Failed to determine the https port for redirect.
[xUnit.net 00:00:05.35]  TESTNAME [FAIL]
  Failed FUNCTIONNAME at CODELOCATION [2 s]
  Error Message:
   System.Net.Http.HttpRequestException : Connection refused (localhost:8081)
---- System.Net.Sockets.SocketException : Connection refused
  Stack Trace:
     at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync(ITrace trace)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request, ITrace trace)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(String resourceUriString, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerCore, FeedRange feedRange, Stream streamPayload, Action`1 requestEnricher, ITrace trace, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosClient.<>c__DisplayClass50_0.<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](ITrace trace, Func`2 task)
   at Microsoft.Azure.Cosmos.ClientContextCore.<>c__DisplayClass31_0`1.<<OperationHelperWithRootTraceWithSynchronizationContextAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.CosmosRepository.Services.DefaultCosmosContainerService.GetContainerAsync(Type itemType, Boolean forceContainerSync)
   at Microsoft.Azure.CosmosRepository.Providers.DefaultCosmosContainerProvider`1.<>c__DisplayClass1_0.<<-ctor>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at FUNCTIONNAME in CODELOCATION
   at FUNCTIONNAME in CODELOCATION
   at FUNCTIONNAME in CODELOCATION
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[xUnit.net 00:00:41.29]     TESTNAME [FAIL]
  Failed TESTNAME [3 s]
  Error Message:
   System.Net.Http.HttpRequestException : Connection refused (localhost:49154)
---- System.Net.Sockets.SocketException : Connection refused
  Stack Trace:
     at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>g__Core|5_0(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync(ITrace trace)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request, ITrace trace)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(String resourceUriString, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerCore, FeedRange feedRange, Stream streamPayload, Action`1 requestEnricher, ITrace trace, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosClient.<>c__DisplayClass50_0.<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](ITrace trace, Func`2 task)
   at Microsoft.Azure.Cosmos.ClientContextCore.<>c__DisplayClass31_0`1.<<OperationHelperWithRootTraceWithSynchronizationContextAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.CosmosRepository.Services.DefaultCosmosContainerService.GetContainerAsync(Type itemType, Boolean forceContainerSync)
   at Microsoft.Azure.CosmosRepository.Providers.DefaultCosmosContainerProvider`1.<>c__DisplayClass1_0.<<-ctor>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.CosmosRepository.DefaultRepository`1.ExistsAsync(String id, PartitionKey partitionKey, CancellationToken cancellationToken)
   at FUNCTIONNAME in /home/runner/work/CODELOCATION
   at FUNCTIONNAME in /home/runner/work/CODELOCATION
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

What can I do to troubleshoot or provide more information?

There is a known problem with the cosmos emulator running in docker. Sometimes it's stuck in starting. This is due to running under a certain intel arch with ubuntu 20 or 22. Can you see in the log that it's actually started?

There is a known problem with the cosmos emulator running in docker. Sometimes it's stuck in starting. This is due to running under a certain intel arch with ubuntu 20 or 22. Can you see in the log that it's actually started?

@ktjn thanks for your reply - below is what I see in Docker Desktop for Windows.
TestContainers container log:
2022/11/03 18:18:02 Pinging Docker...
2022/11/03 18:18:02 Docker daemon is available!
2022/11/03 18:18:02 Starting on port 8080...
2022/11/03 18:18:02 Started!
2022/11/03 18:18:02 New client connected: <IP_ADDRESS>:46316
2022/11/03 18:18:02 Received the first connection
2022/11/03 18:18:02 Adding {"label":{"testcontainers.resource-reaper-session=9002e6f2-ea85-47b1-8858-924a3eff9fa9":true}}

Cosmos Emulator log:
This is an evaluation version.  There are [11] days left in the evaluation period.
Starting
Started 1/3 partitions
Started 2/3 partitions
Started 3/3 partitions
Started

And here is the code I'm using (changing the connection string to my local CosmosDb emulator succeeds):
using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Configurations;
using DotNet.Testcontainers.Containers;
using Microsoft.Azure.CosmosRepository;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using System.Runtime.InteropServices;

namespace IntegrationTests.Features.Thing;

public class CosmosDbThing { public string Id { get; init; } = Guid.NewGuid().ToString(); }

public class CosmosDbThingServiceTests : IAsyncLifetime
{
    private CosmosDbTestcontainer? _dbContainer;
    private readonly List<string> _createdThingIds = new();
    private CosmosDbThingService? _cosmosDbThingService;

    public async Task InitializeAsync()
    {
        _dbContainer =
            new TestcontainersBuilder<CosmosDbTestcontainer>()
            .WithDatabase(new CosmosDbTestcontainerConfiguration())
            .WithWaitStrategy(RuntimeInformation.IsOSPlatform(OSPlatform.Windows) switch
                {
                    true => Wait.ForWindowsContainer(),
                    _    => Wait.ForUnixContainer(),
                })
            .Build();

        await _dbContainer.StartAsync();

        var cosmosThingRepository =
            new ServiceCollection()
                .AddSingleton<IConfiguration>(new ConfigurationBuilder().Build())
                .AddOptions()
                .AddCosmosRepository(
                    options =>
                    {
                        options.CosmosConnectionString = _dbContainer?.ConnectionString ?? "";
                        options.ContainerId = "thing-store";
                        options.DatabaseId = "main";
                        options.ContainerPerItemType = true;
                        options.ContainerBuilder.Configure<CosmosDbThing>(containerOptions => containerOptions
                            .WithContainer("things")
                            .WithPartitionKey("/id")
                            .WithSyncableContainerProperties()
                        );
                    })
                .BuildServiceProvider()
                .GetRequiredService<IRepository<CosmosDbThing>>();

        _cosmosDbThingService = new CosmosDbThingService();
    }

    public async Task DisposeAsync()
    {
        foreach (var createdThingId in _createdThingIds)
        {
            await _cosmosDbThingService!.DeleteAsync(createdThingId);
        }

        await (_dbContainer?.StopAsync() ?? Task.CompletedTask);
    }

    [Fact]
    public async Task CreateCosmosDbThingAsync_CreatesCosmosDbThing()
    {
        // Arrange
        var thing = new CosmosDbThing();

        // Act
        var success = await _cosmosDbThingService!.CreateCosmosDbThingAsync(thing);
        _createdThingIds.Add(thing.Id);

        // Assert
        success.Should().BeTrue();
    }
}

You should use a IClassFixture, othwise the container will be recreated for each test. Anyways...
Use something similar to: https://github.com/testcontainers/testcontainers-dotnet/blob/develop/tests/Testcontainers.Tests/Fixtures/Containers/Unix/Modules/Databases/CosmosDbFixture.cs
I think your waitstategy is the problem. Use the correct one.
See this for how to use: https://github.com/testcontainers/testcontainers-dotnet/blob/develop/tests/Testcontainers.Tests/Unit/Containers/Unix/Modules/Databases/CosmosDbTestcontainerTest.cs

The WithWaitStrategy does not make sense. Your configuration waits until the container is running, but not until the service is ready.
I think you need to configure your HTTP client too, at least the Cosmos DB container offers one.

https://github.com/testcontainers/testcontainers-dotnet/blob/87914f38a67a57e23c61542e30f86842928bfe84/src/Testcontainers/Containers/Modules/Databases/CosmosDbTestcontainer.cs#L37

Thanks for the feedback @ktjn and @HofmeisterAn ! My apologies for the distracting issues. To address them:

Re: WaitStrategy, my fault - this was erroneous logic. I'd played with this to see if I could resolve my original issue, and it was intended to change the strategy based on the characteristics of the container, but that's not what I coded.
Re: HttpClient, I'm using the Microsoft.Azure.CosmosRepository IRepository abstraction - their AddCosmosRepository() configures an HttpClient internally.
Re: IClassFixture - yes, good points

I've factored the code into a stand-alone solution, and placed on my GitHub here, which now surfaces the original SSL connection denied exception I'd observed:
 IntegrationTests.Features.Thing.CosmosDbThingServiceTests.CreateCosmosDbThingAsync_CreatesCosmosDbThing
   Source: CosmosDbThingServiceTests.cs line 56
   Duration: 2.2 sec

  Message:
System.Net.Http.HttpRequestException : The SSL connection could not be established, see inner exception.
---- System.IO.IOException :  Received an unexpected EOF or 0 bytes from the transport stream.

  Stack Trace:
ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
GetAccountPropertiesHelper.GetAccountPropertiesAsync()
GatewayAccountReader.InitializeReaderAsync()
CosmosAccountServiceConfiguration.InitializeAsync()
DocumentClient.InitializeGatewayConfigurationReaderAsync()
DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
DocumentClient.EnsureValidClientAsync(ITrace trace)
RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request, ITrace trace)
RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
RequestInvokerHandler.SendAsync(String resourceUriString, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerCore, FeedRange feedRange, Stream streamPayload, Action`1 requestEnricher, ITrace trace, CancellationToken cancellationToken)
<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](ITrace trace, Func`2 task)
<<OperationHelperWithRootTraceWithSynchronizationContextAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
DefaultCosmosContainerService.GetContainerAsync(Type itemType, Boolean forceContainerSync)
<<-ctor>b__0>d.MoveNext()
--- End of stack trace from previous location ---
DefaultRepository`1.ExistsAsync(String id, PartitionKey partitionKey, CancellationToken cancellationToken)
CosmosDbThingService.CreateCosmosDbThingAsync(CosmosDbThing thing) line 20
CosmosDbThingServiceTests.CreateCosmosDbThingAsync_CreatesCosmosDbThing() line 62
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
SslStream.ReceiveBlobAsync[TIOAdapter](CancellationToken cancellationToken)
SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)

If you have any time to review this further, I'd be greatly appreciative!

I've had some of the issues described in this issue, none of them had anything to do with testcontainers or the options provided by default. All of them also happened running the docker image manually.
See if your issue may be resolved by one of these:

Depending on your OS, you have to trust the certificate used by the emulator before you can do any connection, this process is described here. This page also talks about getting the The TLS/SSL connection couldn't be established error at the Troubleshoot issues section.
I was also getting timeouts/errors due to high CPU. The documentation itself says you should use at least 4 CPUs for the CosmosDb emulator container, so make sure you have at least 4 CPUs available. Even with 4 CPUs I was occasionally getting timeout errors. You can read more about it here. Took me a while to find out about that.

Hope this can help some of you.

Re: HttpClient, I'm using the Microsoft.Azure.CosmosRepository IRepository abstraction - their AddCosmosRepository() configures an HttpClient internally.

Where do you configure SSL? Install the certificate, use your own or configure the ServerCertificateCustomValidationCallback:
https://github.com/testcontainers/testcontainers-dotnet/blob/87914f38a67a57e23c61542e30f86842928bfe84/src/Testcontainers/Containers/Modules/Databases/CosmosDbTestcontainer.cs#L48
I do not know AddCosmosRepository maybe you need to set the right security protocol too.

@dcuccia There are a lot of issues with your code. Some hints:

Always start with something that works and add one thing at a time
Cosmosdb is messy to begin with, the emulator is even messier and needs a lot a special treatment to work
dont check if an item exists before doing an operation, there are status codes for that. The perfomance(and cost) will be double

Here are fixed versions of your code:
Use the cosmosclient directly. It needs a special http client with ssl validation disabled. Also a special http handler to rewrite the endpoints.
using Microsoft.Azure.Cosmos;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Azure.CosmosRepository;
namespace IntegrationTests.Features.Thing;
public class CosmosDbThing : Item { }
public class CosmosDbThingService
{
private readonly Container _container;
public CosmosDbThingService(Container container)
{
    _container = container;
}

public async ValueTask<bool> CreateCosmosDbThingAsync(CosmosDbThing thing)
{
    return await _container.CreateItemAsync(thing) is { };
}

public async ValueTask<bool> UpdateAsync(CosmosDbThing thing)
{
    return await _container.UpsertItemAsync(thing) is { };
}

public async ValueTask<bool> DeleteAsync(string id)
{
    await _container.DeleteItemAsync<CosmosDbThing>(id, new PartitionKey(id));
    return true;
}

public async ValueTask<CosmosDbThing?> GetByIdAsync(string id)
    => await _container.ReadItemAsync<CosmosDbThing>(id, new PartitionKey(id));

}
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Azure.CosmosRepository;
using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Configurations;
using DotNet.Testcontainers.Containers;
using Microsoft.Azure.Cosmos;
namespace IntegrationTests.Features.Thing;
public class CosmosDbThingServiceTests : IClassFixture, IAsyncLifetime
{
private readonly List _createdThingIds = new();
private CosmosDbThingService? _cosmosDbThingService;
private readonly CosmosClient _cosmosClient;

public CosmosDbThingServiceTests(CosmosDbFixture cosmosDbFixture)
{
    var cosmosClientOptions = new CosmosClientOptions { ConnectionMode = ConnectionMode.Gateway, HttpClientFactory = () => cosmosDbFixture.Container.HttpClient };
    _cosmosClient = new CosmosClient(cosmosDbFixture.Container.ConnectionString, cosmosClientOptions);
}
public async Task InitializeAsync()
{
    var db = await _cosmosClient.CreateDatabaseIfNotExistsAsync("thing-store");
    var c = await db.Database.CreateContainerIfNotExistsAsync(new ContainerProperties { Id = "things", PartitionKeyPath = "/id"});
    _cosmosDbThingService = new CosmosDbThingService(c.Container);
}

public async Task DisposeAsync()
{
    foreach (var createdThingId in _createdThingIds)
    {
        await _cosmosDbThingService!.DeleteAsync(createdThingId);
    }
}

[Fact]
public async Task CreateCosmosDbThingAsync_CreatesCosmosDbThing()
{
    // Arrange
    var thing = new CosmosDbThing { Id = Guid.NewGuid().ToString() };

    // Act
    var success = await _cosmosDbThingService!.CreateCosmosDbThingAsync(thing);
    _createdThingIds.Add(thing.Id);

    // Assert
    Assert.True(success);
}

}
public class CosmosDbFixture : DatabaseFixture
{
private readonly CosmosDbTestcontainerConfiguration configuration = new CosmosDbTestcontainerConfiguration();
private readonly CancellationTokenSource cts = new CancellationTokenSource(TimeSpan.FromMinutes(5));

public CosmosDbFixture()
{
    this.Container = new TestcontainersBuilder<CosmosDbTestcontainer>()
      .WithDatabase(this.configuration)
      .Build();
}

public override Task InitializeAsync()
{
    return this.Container.StartAsync(this.cts.Token);
}

public override Task DisposeAsync()
{
    return this.Container.DisposeAsync().AsTask();
}

public override void Dispose()
{
    this.configuration.Dispose();
}

}
public abstract class DatabaseFixture : IAsyncLifetime, IDisposable
where TDockerContainer : ITestcontainersContainer
{
public TDockerContainer Container { get; protected set; }
public abstract Task InitializeAsync();

public abstract Task DisposeAsync();

public abstract void Dispose();

}

Thank you to everyone contributing to this thread, I've been trying to set this up on a project since yesterday, but I cannot figure out what the problem is. It seems as if the CosmosClient is defaulting to port :8081 even though I initialize it with the proper connection string. Here is my code:
public class GetPhraseHandlerTests : IAsyncLifetime
{
    private readonly CosmosDbTestcontainer? _cosmosDbTestContainer;

    public GetPhraseHandlerTests()
    {
        _cosmosDbTestContainer = new TestcontainersBuilder<CosmosDbTestcontainer>()
	        .WithDatabase(new CosmosDbTestcontainerConfiguration())
	        .Build();
    }

    public async Task InitializeAsync()
    {
        if (_cosmosDbTestContainer != null)
            await _cosmosDbTestContainer.StartAsync();

        var cosmosClientBuilder = new CosmosClientBuilder(_cosmosDbTestContainer.ConnectionString);
        cosmosClientBuilder.WithHttpClientFactory(() =>
        {
            HttpMessageHandler httpMessageHandler = new HttpClientHandler()
            {
                ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
            };

            return new HttpClient(httpMessageHandler);
        });
        var cosmosClient = cosmosClientBuilder.WithConnectionModeGateway().Build();

        await cosmosClient.CreateDatabaseAsync("NewDatabase");
        var db = cosmosClient.GetDatabase("NewDatabase");
    }
}

When the code gets to the .CreateDatabaseAsync() method, everything crashes, and this is what I get in the output:

However, just before that it says: "DocDBTrace Information: 0 : DocumentClient with id 1 initialized at endpoint: https://localhost:49159/ with ConnectionMode: Gateway, connection Protocol: Https, and consistency level: null"
If I use .WithPortBinding (8081, 8081), this same code works just fine. This question might be better suited for the azure-cosmos-dotnet repo, apologies if that's the case. :)

Can you compare the ConnectionString and the endpoint the instance of cosmosClient points to? <IP_ADDRESS>:8081 looks wrong. Testcontainers assigns random host ports to avoid port conflicts.

As @HofmeisterAn said testcontainers will associate a new port to it each time. If I recall correctly the emulator hard codes the port to 8081 so you have to manually update it for the requests. Take a look at https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/main/tests/Api.Tests/Infrastructure/FixRequestLocationHandler.cs as an example of how to do this.

Can you compare the ConnectionString and the endpoint the instance of cosmosClient points to? <IP_ADDRESS>:8081 looks wrong. Testcontainers assigns random host ports to avoid port conflicts.

@HofmeisterAn cosmosClient.Endpoint shows the correct random port.

As @HofmeisterAn said testcontainers will associate a new port to it each time. If I recall correctly the emulator hard codes the port to 8081 so you have to manually update it for the requests. Take a look at https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/main/tests/Api.Tests/Infrastructure/FixRequestLocationHandler.cs as an example of how to do this.

Although it's very unintuitive behaviour, it seems @WestDiscGolf is correct - even if you instantiate the Client using the proper ConnectionString, it still tries communicating via :8081.
Thank you both so much for the quick replies, looks like the FixRequestLocationHandler.cs did the trick for me. Funny thing is, I am sure I tried implementing it yesterday, but I must've got something wrong (my brain probably just gave up :) ).

Hi, dose anyone facing below issue
System.Net.Http.HttpRequestException : No connection could be made because the target machine actively refused it. (localhost:49194)
---- System.Net.Sockets.SocketException : No connection could be made because the target machine actively refused it.

@rrr-ganesh Is this just running the repository as it is? Or doing anything different? If you raise an issue with the information in the repository then I would be happy to take a look to see if I can see anything obvious - https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/issues

Can you compare the ConnectionString and the endpoint the instance of cosmosClient points to? <IP_ADDRESS>:8081 looks wrong. Testcontainers assigns random host ports to avoid port conflicts.

@HofmeisterAn cosmosClient.Endpoint shows the correct random port.

As @HofmeisterAn said testcontainers will associate a new port to it each time. If I recall correctly the emulator hard codes the port to 8081 so you have to manually update it for the requests. Take a look at https://github.com/WestDiscGolf/CosmosDB-Testcontainers-Test/blob/main/tests/Api.Tests/Infrastructure/FixRequestLocationHandler.cs as an example of how to do this.

Although it's very unintuitive behaviour, it seems @WestDiscGolf is correct - even if you instantiate the Client using the proper ConnectionString, it still tries communicating via :8081.
Thank you both so much for the quick replies, looks like the FixRequestLocationHandler.cs did the trick for me. Funny thing is, I am sure I tried implementing it yesterday, but I must've got something wrong (my brain probably just gave up :) ).

After long, long search I found this:
https://github.com/Azure/azure-cosmos-dotnet-v3/issues/2784
and after even longer struggle I managed to make it work:

I know it's far from a good solution but it works without too much hacking in cosmos code itself, where you need these options:

Do you guys know how to set "/EnablePreview" flag when spinning up the container? I'm using hierarchical partition keys.
I thought this could be achieved with .WithEnvironment("AZURE_COSMOS_EMULATOR_ARGS","/EnablePreview") but when I access the portal on the container itself I cant see the partition keys columns.

@rf-0 Starting without any argument will add the /EnablePreview flag. From a 'ps axv' inside the container:
19 ?        Sl     0:39    237  1612 28229991 1516060  4.6 ./cosmosdb-emulator -w /tmp/cosmos/appdata -- /Microsoft.Azure.Cosmos.Emulator.exe /enablepreview
From the 'start.sh' entrypoint:
EMULATOR_OTHER_ARGS="${AZURE_COSMOS_EMULATOR_ARGS:-/enablepreview}"

For me
.WithWaitStrategy(Wait.ForUnixContainer().UntilMessageIsLogged("Started"))

wasn't enough as I received
System.Net.Http.HttpRequestException : The SSL connection could not be established, see inner exception.
---- System.IO.IOException : Received an unexpected EOF or 0 bytes from the transport stream.

Figured it out during debugging.
Adding a delay helped
    public async Task InitializeAsync()
    {
        await _container.StartAsync();
        await Task.Delay(TimeSpan.FromSeconds(120));
        await SeedData();
    }

I think you are using an outdated version. This issue should already be addressed in version 3.8.0: https://github.com/testcontainers/testcontainers-dotnet/pull/1109.

Oh, I see. It is encapsulated in 3.8.0. Removing .WithWaitStrategy(...) helped without a need to delay.
Thanks!


health is not None was checked twice

This will now show the hardware unsupported alert when there's no health packet.


Support for Persona 3 Reload

Game name: Persona 3 Reload

Game package name: SEGAofAmericaInc.L0cb6b3aea_s751p9cej88mt

wgs.zip

It seems that the Steam saves are encrypted, so adding support for this seems unlikely at least for now.

You can try to load these unencrypted saves with the Steam version and maybe if we're lucky, it'll work.
These are your extracted saves:
persona_3_reload_2535424010670539_2024-02-02_18_25_04.zip

Tested this, doesn't show up.

Shame :disappointed:
I'll close this for now, as I can't really do anything about the encryption.

Save decrypt tool: https://github.com/illusion0001/P3R-Save-EnDecryptor

Save decrypt tool: https://github.com/illusion0001/P3R-Save-EnDecryptor

How do I use this?

how do you use the extractor above? im kind of a noob to this :C

Save decrypt tool: https://github.com/illusion0001/P3R-Save-EnDecryptor

When I run the program, it just did nothing, any idea why?

Any chance you can decrypt my save files so I can move my save from gp to steam? the program is not working for me. @illusion0001. I appreciate it, thank you!
save.zip

if you want I know how to edit these saves (I know how to do it for steam but it can also work for other versions)

Hey guys! Any news on this one? 😭


Implement Rate Limiting for Messages.

Description
Without rate limiting, users can flood the chat with messages, potentially leading to performance issues and misuse. Rate limiting will ensure that users can send messages at a reasonable pace and prevent abuse.
Screenshots

Additional information
I propose implementing rate limiting for message sending. A rate limit of, for example, 70 messages per minute per user, can be a good starting point .
🥦 Browser
Google Chrome
👀 Have you checked if this issue has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Contributing Guidelines?

[X] I have read and understood the rules in the Contributing Guidelines

Are you willing to work on this issue ?
Yes I am willing to submit a PR!

How about 25, 70 is too much for each person.
Will assign it to you, Make sure you follow the rules 👉🏽here👈🏽, else your PR will not be accepted and will be closed. Good Luck

okay noted 👍 Thank you !

Hello Maintainers ,
On this issue , connected the DB without any errors , but when i opened 2 browsers with the URL, the 2 browsers are not connecting with each other !
Any idea on this ?
Here are the Screen Shots ;

Hello Maintainers ,
On this issue , connected the DB without any errors , but when i opened 2 browsers with the URL, the 2 browsers are not connecting with each other !
Any idea on this ?
Here are the Screen Shots ;

Fill up the env for both backend and frontend

My Bad 😅 , thanks working , but how did you find that out very quickly without even seeing the code ?

Common mistake lately


Making the component buildable

Hello Das,
These are changes I made locally to make it buildable - notice that I also added a TODO to support both content type - I heard that JSON is frequently used with SWF so it is better to support both - if we want finish YAML and we can do JSON later
This PR only makes it buildable, but I notice errors when I used pnpm build:start
I also removed patternfly "page" to make it lightweight and easier to build
Thanks!

I am building using  the following command from root dir:
pnpm  bootstrap -F dashbuilder-component-swf-editor...  && pnpm  -F dashbuilder-component-swf-editor... build:dev


Fixed run_deepbench.sh to run only libxsmm format

Can we add a new script which only runs "A L", right now the script doesn't support the other formats anymore, but within a couple of we will have a performance fix (c.f. #137)

For DeepBench we would want only the best performing variant. I can modify the script to take the format as an argument.

That would be great, perhaps we should also add the last parameter to toggle logical vs, physical padding?!

Codecov Report

Merging #138 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #138   +/-   ##
======================================
  Coverage    12.5%   12.5%
======================================
  Files         157     157
  Lines       23250   23250
  Branches     4424    4424
======================================
  Hits         2907    2907
+ Misses      20037   20022   -15
- Partials      306     321   +15

Impacted Files
Coverage Δ

src/generator_gemm.c
29.35% <ø> (-0.92%)
:x:

src/generator_x86_instructions.c
16.42% <ø> (-0.15%)
:x:

src/libxsmm_main.c
48.01% <ø> (-0.15%)
:x:

src/generator_gemm_sse3_avx_avx2_avx512.c
60% <ø> (ø)
:white_check_mark:

src/libxsmm_malloc.c
52.03% <ø> (+0.54%)
:white_check_mark:

src/generator_gemm_common.c
47.5% <ø> (+0.57%)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d469714...6cbbec6. Read the comment docs.

thanks and merged.

done, updated run script with additional arguments.


add static route for active-standby centralized subnet when use old a…

…ctive gateway node

[ ] Make sure you have followed Kube-OVN Code Style.

What type of this PR

Bug fixes

The static route for centralized subnet would be deleted in some case, but we still do not get the certain condition.
The route will not be added if the subnet still used old active gateway after route is deleted.
So add an operation to add back static route when use old active gateway for centralized subnet.
Which issue(s) this PR fixes:
Fixes #(issue-number)
WHAT

🤖 Generated by Copilot at 9901c12
Add static routes for subnets and improve error handling in subnet.go

🤖 Generated by Copilot at 9901c12

We'll add some static routes for the subnets
With the tunnel IP of the gateway nodes
We'll handle errors and log them as we go
And pull up the sail on the count of three

HOW

🤖 Generated by Copilot at 9901c12

Add static routes for subnets with source IP policy (link,link)

出问题时，看到的静态路由信息

正常的路由信息

在最后存在，src-ip 类型的路由，指定 集中式子网cidr 的下一跳


Add client method new_with_connector

This allow to create HttpsConnector with hyper's default HttpConnector reason behind this is extra configuration for HttpConnector like set_nodelay and new set_local_address https://github.com/hyperium/hyper/blob/master/src/client/connect.rs#L254
cc @seanmonstar

+1 This would also be helpful for using connectors such as SOCKS proxies.

cc @seanmonstar please advise how to proceed with this change, you have released new version with new new_ private function on HttpsConnector.. For my use case I need to pass constructed HttpConnector to this struct because I set additional configuration on it such as
        let mut http_connector: HttpConnector =
            HttpConnector::new_with_executor(executor.clone(), None);
        http_connector.set_keepalive(self.hyper_keep_alive_timeout);
        http_connector.set_nodelay(self.hyper_no_delay);
        http_connector.enforce_http(self.hyper_enforce_http);

Sorry I forgot about this! It seems what you want to do can be done with the from constructor. HttpsConnector::from((http, tls)).

Awesome :+1:


Rainbow artifacts when increasing video resolution - Quest 2

I'm unable to increase the video resolution as when I pick something over 100%, I get the following video artifacts:
https://www.youtube.com/watch?v=uTwGrkk0Q-U
Weirdly enough it only occurs inside the headset, it doesn't appear if I record a video using the headset.
Environement
What hardware do you have: CPU, GPU, Audio:   cpu - i5-8600k.   gpu - RTX 3090.  Audio - AudioQuest Dragonfly Black
What version of ALVR and SteamVR have you tried:  ALVR server v15.1.0. SteamVR 1.17.3
Windows version: 20H2 (OS Build 19042.867)

I have them too, i think it's the Quest decoder that can't keep up.

Odd, got a quest 1 and can push it to 125 without issues. Anything higher and it wants h265, but i prefer h264, so i just keep it at 125, got no weird rainbow artifacts, on older hardware as well.

The Quest 1 is 1440 x 1600 (2.3 million pixel)
The Quest 2 is 1832 x 1920 (3.5 million pixel)
As far as I understand maybe the decoder of the quest 1 was able to do more work and the one in the quest 2 is borderline underpowered ...

except spec wise that would not make sense, the quest two has a chip built for vr, and the quest one had a normal mobile chip in comparison.. sure the decoder might be weaker, but i still doubt that, as it would happen to more things than just alvr streams if that was the case..
but sure, could be the case, even if i doubt it a lot. a simple way to test it is to use a link cable.. it will decode the same, but just do it using a cable instead of wifi, if the rainbow artifacts still appear, it could be a decoder issue, or maybe a defective quest 2. if it stops when using the link cable, the issue is elsewhere, and further testing would be needed.

Link (and VD) use sliced encoding while ALVR does not. Maybe artifacts appear because the decoder is overloaded from a single stream vs. using multiple streams?

Quest 2 user here. I actually get these artifacts in the ALVR grid "waiting area" as well. This makes me wonder if the GPU can't keep up with rendering ALVR at over 100% resolution, because the rainbow tearing affects the Guardian overlay.
Though it is interesting that Oculus VrApi's "Extra Latency Mode" is enabled by default in ALVR, which is supposed to give the GPU more time to render.

Interesting. Yeah, it is very strange how when I had the Quest 1, I could easily run it at 125% resolution. So does the Quest 2 use a different video decoder?

FYI, we've had this conversation before. (discord message link)


Visibility classes aren't updated when moving to next slide

With watchSlidesVisibility & watchSlidesProgress enabled - on initial render, the visibility class (swiper-slide-visible) is added correctly (it is added to visible slides and not to hidden slides).
However, when sliding to the next slide, the visibility class is added to the new slide, but is not removed from the previous slide (which is now hidden from view).
When then sliding to the previous slide, the visibility classes work as expected (gets removed from the now hidden next slide).

Hi @mirjacobsen , can u provide a demo by Codesandbox?


Issues on mobile

I've tried to use the LiaScript docs page as a test for mobile usage. Here is a list of issues I've found there.
Starts with the an open navigation bar
When opening the page the nav sidebar is opened. You have to click on the burger menu to start reading. It would be better if the nav-bar is hidden on mobile by default and only shows up by clicking on the burger menu (or adding the function that you could open it with a swipe from the left border).

Out of range problems
Some elements seem to get out of the view. This is sometimes hard to prevent (e.g. for tables) but also there are some issues with text overflow and such. I tracked it in the docs with some screenshots:

Zooming with two fingers
I accidently zoomed into it with the zoom gesture. I think this is a useful feature on mobile but I wasn't able to reset to 100%. Since the I'm getting problems like cut end of pages (not able to scroll down any further) and so buttons get lost:

I don't know if this problem comes from firefox (that I'm using on the phone) so that the adress bar on the bottom simply isn't substracted from the screen height or it is something that could be done on the websites settings. I also had other troubles with the adress bar unpredictably hiding my settings panel:

Cut radio buttons
I stumbled upon the issue that the radio buttons seem to be cut on the top and bottom of a quiz. This happens only on mobile devices:

Some CSS overflow bugs could be fixed, the screenshot with the large table show a bug, since the quotation was not large enough and the same was true for the quizzes ... but the overlong tables result from an additional styling parameter (width=500px) that was set just as an example. The open navbar on mobile was also fixed, now there is a check, if the screenwidth is larger than 620 pixels... but zooming, cut off radio-buttons, and mobile firefox, will be fixed in the future ...
Thanks a lot ...

Closing this, since the new design solves these issues also on mobile and feature phones ...


Implementation details for CATER dataset

Can you please provide the configuration file for the CATER dataset?

You can have a look at the following fully-unsupervised SAVi implementation in SlotFormer (https://slotformer.github.io/) which successfully trains SAVi on the related CLEVRER dataset: https://github.com/pairlab/SlotFormer/blob/master/slotformer/base_slots/configs/stosavi_clevrer_params.py
You should be able to swap in the CATER dataset from https://github.com/deepmind/multi_object_datasets.
The SAVi implementation above includes a small change that adds noise during the training process to make decomposition more stable in the fully unsupervised case.


Read access violation in device.cpp

Required Info

Camera Model
D400

Firmware Version
<IP_ADDRESS>

Operating System & Version
Win (8.1/10) / Linux (Ubuntu 18/20/22)

Kernel Version (Linux Only)

Platform
PC

SDK Version
2.53.1

Language

Segment

Hi,
with the latest realsense library, sometimes we receive an Exception: "read access violation" in device.cpp, line 181, std::lock_guardstd::mutex lock(_device_changed_mtx).
With a little analysis we think that there is an design issue in constructor of the "device" class (file device.cpp). The issue is that the lambda expression is created  and registered to the librealsense::context in the "device" class constructor. This lambda expression is also accessing member variables of the "device" class.
There is a two issues with this approach:

Immediately after lambda expression is registered to the librealsense::context it can be called, although creation of "device" class object is not completed yet.
Second, de-registration of a lambda expression is performed in destructor of the "device" class. This mean that when "device" object is destroyed there is a window in which lambda expression can still be called (for example from some other thread) and then access to a non-existing member variables will occur which will lead  to an Exception: "read access violation".

How to reproduce:

Start the rs2::pipeline configured with some serial number of a camera while another camera is connected to the computer.
Connect the second camera as the pipeline.start(cfg) runs and looks for the device with the given serial number.

Explaination: callback from the device watcher is called that the new device connected and notifies the device object that might not exist as it is repeatedly created and destroyed in the process of looking for a device with the given serial number.

If timed correctly the exception occurs

Hi @Marenix  Thank you for your question.  I will highlight it to my Intel RealSense colleagues.  Thanks very much for your patience.

Hi again @Marenix  My Intel RealSense collagues have been able to reproduce the 'read access violation in device.cpp' error and have created an official internal report to investigate the issue further.
I have added an *Enhancement label to this case to indicate that it should be kept open whilst the internal report is active, but you do not need to take further action.  Thanks again for highlighting this error!

Thanks for letting me know.


refactor: remove npm downgrade

Overview
This is a testing branch to clean up workflow of build and test

Changelog

Review requests

Risk assessment

This was to test app build, test, and deploy process without downgrading nodejs version.
The result was that the process is no longer downgrading because of packages that are related to electron build precess have been updated by a couple of PRs.
I told this Josh already, so I will closed this PR.


Feature request:math

Github supports the insertion of latex into markdown, and it would be a very useful feature for grip.
Github has a hack for rendering LaTeX that essentially involves using
https://render.githubusercontent.com/render/math?math=<insert latex here>

Something like that would be great in grip.

There is no longer any need for the hack, as GitHub now supports LaTeX equations in Markdown natively using MathJax.  It would be great, however, if grip could display such equations correctly in its preview.

My latex injection won't render through grip, but it will render on actual github
actual github

local host

On GitHub a custom math-renderer HTML element is used for rendering the mathematical expressions. I added support for the math-renderer HTML element in my pull request in a new commit. It only works when the --user-content option is also used as explained in the pull request, since we do not get the math-renderer HTML elements in raw mode. The --export option does not use the math-renderer HTML elements in the implementation from the pull request.

This is already requested in the issue #208


feat: add support for snapshot matchers in concurrent tests

Summary

This implements support for snapshot matchers in concurrent tests. The long-standing issue with this has been rooted in the fact that because concurrent tests run in parallel there is no concept of a single currently running test and so jest-snapshot is not able to get this information from a global state object.
This problem can however be solved with AsyncLocalStorage, which is an object that can hold different values for different async contexts. Each concurrent test runs in its own async context and when asking for a value from the storage, gets a result that is unique to it.
Support is added only to jest-circus.
Test plan

There is a new integration test for this use-case.
Fixes #2180.
Fixes #5801.

Hey! I landed #14045 which aims to solve the same issue, albeit with a new matcher. Do you think this PR adds additional value?

To add my two cents: this PR looks like a better solution than new matcher.

The solution propose here makes the same matcher work in normal and constant modes. This means users can switch back and forth between the modes easily.
This solution does need any additional checks or lint rules. (See https://github.com/jest-community/eslint-plugin-jest/issues/1367)
The toMatchSnapshot() and toMatchNamedSnapshot() are very similar. More API surface and not so clear when to use which.
toMatchNamedSnapshot() requires manually naming each snapshot. Hm.. That’s an extra step in contrary the solution proposed here simply works with toMatchSnapshot().

It is a pity that so much work went into toMatchNamedSnapshot() PR, but I would say it should be reverted. This PR should land instead to solve the issue.

I tend to agree with @mrazauskas. The fact that the same matcher can be used in concurrent and normal tests and that you don't need to give it an explicit name makes it more appealing to me.

Yeah, I'm kinda convinced by that argument 😅

https://github.com/jestjs/jest/releases/tag/v29.6.0

i bumped my jest to 29.6.0 and there are multiple concurrent tests in multiple describes in the same file, I think I'm still getting random snapshot names. Is this a known issue or an expected behaviour?

Ohh I see. Thanks a lot. Can't wait for the release.

@nutchanon-c new release is out - can you confirm it fixes your issue? If not, please open a new issue 🙂

It seems to fix the issue so far! If any new issues come up, I'll open a new issue later. Thank you very much!

Why is @types/node added in this PR?
"dependencies": {
    "@types/node": "*"
}

It just caused my non-node project to explode with type errors since Jest types are global and @types/jest depends on expect. I cannot see any obvious reason why this dependency needs to be added at all.

I added it because expect now imports a type from the Node.js module async_hooks. But you're right, this can probably be abstracted away such that this type import wouldn't be necessary. I'll try to fix this.


Add Wavey Capes please

CurseForge link
https://www.curseforge.com/minecraft/mc-mods/waveycapes
CurseForge Mod Distribution
Allowed
Modrinth link
https://modrinth.com/mod/wavey-capes
Source/other link
https://github.com/tr7zw/WaveyCapes
Mod file size
321.3 KB
License
(any other license)
What it does
It turns the static cape model into a physics based cloth that blows in the wind and moves more like how a cape would in real life
Why should it be in the modpack
It's a very neat and very cool mod that's can be toggled on or off, is compatible with half of every cape mod out there, and It's a client side mod
Why shouldn't it be in the modpack
I'm unaware if it could effect performance or not but it might since it's physics based but it also might not work like that since in first person you're not seeing it
Additional details
No response

Duplicate of #560


fixes for Windows build

See https://github.com/tessel/node-usb/issues/141 for details.

Did this fix (libusb change) not work for you? (remember that you have to git submodule update) to pull the updated submodule. If I remember correctly, defining these variables from node-gyp broke compiler versions that didn't have the TIMESPEC definition.

I've added a test for building on VS2015 and master is passing.
This PR is failing (though appveyor isn't picking up the error) on VS2013, so I'm going to close for now. Do let me know if master is still failing to build for you after a git submodule update.


Is it time to redefine the purpose of ActiveHash?

Has the purpose of ActiveHash changed?
Many of the recent feature requests feel like it is going towards an in-memory thread safe transactional database.
README.md [ref]

ActiveHash is a simple base class that allows you to use a ruby hash as a readonly datasource for an ActiveRecord-like model.

I read that ActiveHash is an alternative to storing reference data in the database or in constants.

ActiveHash assumes that every hash has an :id key, which is what you would probably store in a database. This allows you to seamlessly upgrade from ActiveHash objects to full ActiveRecord objects without having to change any code in your app, or any foreign keys in your database.

It even has a suggested migration plan for when requirements change and data is no longer read only. Then it suggests migrating the model to the standard SQL datastore. (or sqlite with ":memory:")
Question
I don't feel that this library and the definition of this library match.
Is it time to re-evaluate the use cases for this library?
Hopefully update the statement at the top of README.md?

I know this is old, but since it's open i would like to pitch in. I am a happy user, but have gone beyond read-only
With little effort i use activehash as a simple file based db. For very similar reasons that the project was created for.
Ie very small data, with little change
migration of data from dev->production (ie checking files in)
I have written a small cms and use ActiveHash files as store. All edits happen on developer machines and so content changes flow in the same way as code canges.
I would be happy to contribute code if the project maintainers want to go to read/write

@rubydesign is there a reason why you don't want to use active record and SQLite?
If the discussions are any indication, it seems many users want to go in this direction. I was curious for the reason. Understanding the reasoning is the best way to properly meet those needs.
Then we can possibly update the goal in the Readme

On my current project, we use fixture files snd populate Postgres using rake db:seeds.

@kbrock Thanks for asking, i did not expect this to be picked up. But as i said, i am a happy read/write user and would be happy to contribute back. For me the functionality has been very helpful.
My use case is quite simple, the data is generated in development/staging environments. So db is not an option. Files are useful because the whole development process works around files.
Then the question is, why do i generate data in development/staging. I think the answer is very very similar to the original project use case.
Two concrete examples, mine and one i have run into before

in my use case i have content writers create static content. Since they are not devs, i can't give them haml to work on. I built a mini cms that uses ActiveHash as a db. My cms is basically a bespoke yaml editor. The yaml goes into git and ends up on the site, like any other changes.
Translations are another use case. I have built many multi-lingual sites and it is always a pain to keep translations in sync. Again non-devs would do the translation in a dev/staging system. Again edit yamls in a very bespoke way.

My needs are currently met in a single-user, non threaded implementation. Ie upon change i write all files out.
https://github.com/FeenixMakers/merged/blob/main/app/models/merged/active_base.rb#L35
I've tried to keep the AH changes in that file. There are other "models" there that are based on AH, they are close enough to AR to work in basic crud controllers.

@rubydesign thanks.

You run a server locally and people use that server to add/edit values.
Then you go to that server and commit the yaml file into git
git pull in production
Production runs off of the yaml
Production does not change the yaml files.

That makes sense.
Storing yaml text data in git handles merge conflicts better than a bin file. So it would require adding imports and exports and a bunch of complexity.
I've only ever used active hash for developer edited yaml files. My current team uses pot (via gettext) files for translations which are used across many ruby gems, languages, and frameworks.
I just look at the complexities we are adding around mutexes, defaults, or quicker id generation. The new use cases seem like we're duplicating a standard rails database adapter.

@kbrock That's the use, mostly, yes. And it is very useful (for me at least). I would imagine for translations too.
The use cases have make the system usually single threaded, single user.  A little really goes a LONG way.
Conceptually it is a step along the db adapter, yes. But i tried writing one of those once and the basic crud is really just the small tip of the iceberg. I mean in reality it's nothing like a db adapter, at least in terms of effort.
Really a simple save for a collection would be enough. If you look at the method i linked, that's does a lot. I never understood why i have to do a reload there, but that simple save would be a good start that goes a long way.
PS: sorry about the intermittent communication, i am in the mountains of Tuscany and there is surprisingly bad network coverage.

I'm not a very active maintainer at this point, so feel free to discount my opinion; but I think it's a very bad idea to try to turn this gem into a transactional database. That is surely going to be a ton of work to do correctly, and will greatly increase the demands on the maintainers and the support burden (and I'm personally not prepared or willing to take that on).
If people need something more advanced than a yaml file then I would strongly recommend looking into using a sqlite database that can be deployed with the app.
If anyone feels super strongly that they want to go in this direction: would you consider forking this project to try to support those more advanced use cases, rather than introducing that complexity here?

Hey @flavorjones
My issue is that we seem to already be going in this new direction.
And it scares me. I'm curious if that is intentional.
examples:

introducing a mutex #229
quicker id creation #321
creating a json datasource. #198
some default field behavior that doesn't modify base data hash (can't find)

A read only database probably doesn't need a mutex or id creation. Right?
I wanted to know if others felt we were starting to veer in this new direction and see how they felt about this.

Ok, since i have been pro saving here i just want to retract that. I fully understand that open source is collaboration and not making feature requests and don't want to add to anyone's burden.
It just took me a couple of days to rewrite everything that i need, which includes save. Not multi threaded, and no ActiveRecord relations. just 140 lines of dense code that does what i need
https://codeberg.org/FeenixMakers/staged/src/branch/main/app/models/staged/active_yaml.rb
Thanks for the code/ideas i lifted and the code i used before. (torsten out)

@rubydesign sorry I didn't reply sooner. That turned out nicely.

Yes, remember i needed the text files.  For my "developer cms" that integrates into git workflow. I finally managed to publish it, thanks again for the hand up.
https://rubygems.org/gems/breeze_cms
https://www.ruby-toolbox.com/projects/breeze_cms


Web Component: Add Masthead custom typeahead & search redirect - Group 5

User Story

As a [user role below]:
IBM.com Library developer

I need to:
create/change the Masthead custom typeahead & redirect

so that I can:
provide the ibm.com adopter developers components they can use to build ibm.com web pages

Additional information

Story within Storybook with corresponding knobs
Utilize Carbon
Create with Shadow DOM and Custom Elements standards
See the Epic for the Design and Functional specs information
React Storybook link
Visual Designs & Design Specs Box folder link
Prod QA testing issue (#4714)

Acceptance criteria

[ ] Include README for the web component and corresponding styles
[ ] Create Web Components styles in styles package
[ ] Do not create knobs in Storybook that include JSON objects
[ ] Break out Storybook stories into multiple variation stories, if applicable
[ ] Create codesandbox example under /packages/web-components/examples/codesandbox and include in README
[ ] Minimum 80% unit test coverage
[ ] If a design is provided, the Designer is included as a Reviewer in the Pull Request
[ ] A comment is posted in the Prod QA issue, tagging Praveen and Chetan, when development is finished

This feature is supported in Web Components codebase from the beginning, described at here.


Subfolders Support

Hi !
Do you plan to handle subfolders ?

I'm interested too and thanks for your work!

great library! I agree with this, I can't see a way to represent non-trivial 3D scenes without subfolders...

Another (similar) usecase:
To represent the Stage of pixijs:
Stage
| -> Sprite (folder)
|      | -> Position (x,y)
|      | -> Scale (x,y)
| -> Container (folder)
|      | -> Position (x,y)
|      | -> Scale (x,y)
|      | -> Children (folder)
|      |      | -> Sprite (folder)
|      |             | -> Position (x,y)
|      |             | -> Scale (x,y)
|      | -> Container (folder)
|      |      | -> Position (x,y)
|      |      | -> Scale (x,y)
|      |      | -> Children (folder)
|      |             | -> Sprite (folder)
|      |             |      | -> Position (x,y)
|      |             |      | -> Scale (x,y)
|      |             | -> Sprite (folder)
|      |                    | -> Position (x,y)
|      |                    | -> Scale (x,y)
|      | -> Sprite (folder)
|             | -> Position (x,y)
|             | -> Scale (x,y)
| -> ...

Thank you for requesting the feature. Released in 1.4.0!