Writing integration tests with Testcontainers and Cosmos DB Docker emulator for Spring Boot applications

1. OVERVIEW

As part of your organization’s modernization effort, your team is writing Spring Boot applications that store data to, and retrieve data from Azure Cosmos DBs instead of relational databases.

You might have written dynamic Cosmos DB queries using Spring Data Cosmos and ReactiveCosmosTemplate.

Your integration tests might be connecting to a dedicated Cosmos DB hosted in Azure, which increases your organization’s subscription cost.

You could instead be using a dedicated Cosmos DB emulator Docker container you would need to make sure is running, and which ports it listens on, before you seed test data and run every test.

Or you might not have integration tests at all. But you also know integration testing is one of the keys to deploy often and with confidence.

This blog post shows you how to write integration tests with Testcontainers and the Cosmos DB Docker emulator for Spring Boot applications.
Listening on available ports instead of hard-coding them. And importing the Cosmos DB self-signed emulator certificate to a temporal Java truststore your tests could use.
All these automated.

Testcontainers and Cosmos DB Docker emulator Integration tests with Testcontainers and Azure Cosmos DB Docker emulator

Tip: You might want to take a look at the recent Code and Configuration Snippets / Recipes / Tips section, which helps you with common tasks without reading through a full blog post.

1.1. About Testcontainers

Briefly, Testcontainers is an open-source framework with a commercial-friendly license (MIT) that provides throwaway instances of third-party systems like databases, message brokers, etc.; running in Docker containers you can use in your integration tests.

We’ll use Testcontainers for Java library; which supports Jupiter/JUnit 5.

2. MAVEN DEPENDENCIES

This tutorial uses Spring Boot version 2.7.14, but it might work with Spring Boot 2.2+, which is the first version that started using JUnit 5 by default.

I would also expect it to work with Spring Boot 3.x.

Let’s add the relevant Maven dependencies:

pom.xml:

<properties>
  <testcontainers.version>1.19.6</testcontainers.version>
</properties>

<dependencies>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <scope>test</scope>
  </dependency>
...
</dependencies>

<dependencyManagement>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers-bom</artifactId>
    <version>${testcontainers.version}</version>
    <type>pom</type>
    <scope>import</scope>
  </dependency>
</dependencyManagement>
...

You still need spring-boot-starter-test, and rest-assured dependencies, which are not covered in this blog post.

It’s also a follow up on Seeding Cosmos DB Data to run Spring Boot 2 Integration Tests blog post, so I’ll assume you are already using spring-cloud-azure-dependencies, spring-cloud-azure-starter-data-cosmos.

testcontainers-bom manages a number of Testcontainers modules/libraries.

org.testcontainers:junit-jupiter provides support to use Testcontainers libraries with Jupiter/JUnit 5, which is the JUnit version brought in by Spring Boot 2.7.x’s spring-boot-starter-test dependency.

3. CONTAINER DOCUMENTS

User.java:

@Container(containerName = "users")
public class User {

  @Id
  private String id;
  private String firstName;
  private String lastName;
  private String address;
// ...
}

users is the name of the container space in the Cosmos database. It’ll store JSON documents corresponding to User.java POJO instances.

Warning:
If you take a look at the logs after running the integration test shouldRetrieveUserWithIdEqualsTo100() method, you’ll notice this warning:

WARN ... ReactiveCosmosTemplate    : The partitionKey is not id!! Consider using findById(ID id, PartitionKey partitionKey) instead to avoid the need for using a cross partition query which results in higher latency and cost than necessary. See https://aka.ms/PointReadsInSpring for more info.

That’s fine for the purposes of this blog post.
But stay tuned and sign up to the newsletter. I’ll cover modeling CosmosDB data in another blog post.

4. TEST CONFIGURATION

src/test/resources/application-integration-test.yml:

spring:
  cloud:
    azure:
      cosmos:
        endpoint: placeholder-will-be-set-at-runtime
        key: C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
        database: sampleIntegrationTestsDB

As a precaution, make sure the database name is different than the one used for development.

The key will be the same for even for different Cosmos DB emulator Docker containers.

The interesting setting here is spring.cloud.azure.cosmos.endpoint. It’s set to a random value because the real value will come at runtime after the Cosmos DB emulator starts.
This is useful in this case because the data storage system the tests depend on will listen on a random port. This is covered in UserControllerTestContainersIntegrationTest.java.

5. CosmosDBEmulatorContainer IMPLEMENTATION

If you weren’t using Testcontainers, how would you start a Cosmos DB emulator container for local development?

You would run a command similar to:

docker run -d \ 
 --publish 8081:8081 \
 --publish 10250-10255:10250-10255 \ 
 --memory 3g --cpus=4.0 \
 --name=cosmosdb-linux-emulator \ 
 --env AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10 \ 
 --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \ 
 --env AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=$ipaddr \ 
 --interactive \ 
 --tty \ 
 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator

That’s what our CosmosDBEmulatorContainer implementation does.

It takes care of starting a Cosmos DB emulator Docker container and waiting until the emulator endpoint is ready.

AsimioCosmosDBEmulatorContainer.java:

public class AsimioCosmosDBEmulatorContainer extends GenericContainer<AsimioCosmosDBEmulatorContainer> {

  private static final int EMULATOR_ENDPOINT_PORT = 8081;

  public AsimioCosmosDBEmulatorContainer() {
    this("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator");
  }

  private AsimioCosmosDBEmulatorContainer(String dockerImageName) {
    super(dockerImageName);

    this.addFixedExposedPort(EMULATOR_ENDPOINT_PORT, EMULATOR_ENDPOINT_PORT;
    this.addFixedExposedPort(10250, 10250);
    this.addFixedExposedPort(10251, 10251);
    this.addFixedExposedPort(10252, 10252);
    this.addFixedExposedPort(10253, 10253);
    this.addFixedExposedPort(10254, 10254);
    this.addFixedExposedPort(10255, 10255);

    try {
      this.withEnv(Map.of(
        "AZURE_COSMOS_EMULATOR_ARGS", "/Port=" + EMULATOR_ENDPOINT_PORT,
        "AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", InetAddress.getLocalHost().getHostAddress(),
        "AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1",
        "AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
      );
    } catch (UnknownHostException ex) {
      throw new RuntimeException(ex);
    }

    WaitAllStrategy waitStrategies = new WaitAllStrategy(WaitAllStrategy.Mode.WITH_INDIVIDUAL_TIMEOUTS_ONLY)
      .withStrategy(Wait.forLogMessage("(?s).*Started\\r\\n$", 1))
      .withStrategy(Wait.forHttps("/_explorer/index.html")
        .forPort(EMULATOR_ENDPOINT_PORT)
        .allowInsecure()
        .withRateLimiter(RateLimiterBuilder.newBuilder()
          .withRate(6, TimeUnit.MINUTES)  // 6 per minute, one request every 10 seconds 
          .withConstantThroughput()
          .build()
        )
        .withStartupTimeout(Duration.ofSeconds(120))  // 2 minutes
      );
    this.waitingFor(waitStrategies);
  }

  public String getEmulatorEndpoint() {
    return String.format("https://%s:%s", this.getHost(), this.getMappedPort(EMULATOR_ENDPOINT_PORT));
  }

  public int getEmulatorEndpointPort() {
    return EMULATOR_ENDPOINT_PORT;
  }

}

We are implementing our custom container using GenericContainer as the base class.

mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator is the Docker image.

The private constructor executes:

addFixedExposedPort() method binds a host port to a container port.
Similarly to --publish hostPort:containerPort as included in the shell command.
withEnv() method passes environment variables to the container.
Similarly to --env VAR_NAME=VAR_VALUE from the shell command.
You won’t need to keep data between multiple runs. In fact, the container name will be different between runs. That’s why AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE is set to false.
If you use a limited data set to run your tests, you probably need only one partition.
What’s interesting in this case is that we are also passing AZURE_COSMOS_EMULATOR_ARGS='/Port=8081'. EMULATOR_ENDPOINT_PORT is hard-coded to 8081.
I did that on purpose because it’s easy to refactor to bind a container port to a random, and available host port (as included in the downloadable source code your organization can purchase.
You would just need to set EMULATOR_ENDPOINT_PORT to SocketUtils.findAvailableTcpPort().
Not only does this prevent to hard-code ports that might be in use, but you could also run your integration tests concurrently, starting multiple Cosmos DB emulators, possibly decreasing the build time.
The startup process combines two Testcontainers wait strategies before returning the control back to the integration test class.
- forLogMessage() method waits for the log message Started to show up in the standard output when running a Cosmos DB emulator container.
  But this wait strategy is not enough. The emulator endpoint might still be unavailable.
- forHttps().forPort().allowInsecure() configuration sends a dozen requests per minute for up to two minutes until /_explorer/index.html, the Cosmos DB emulator UI home page becomes available, after that it times out.
  We need this page to be available so that we can get the unsigned cert and add it to the truststore, as done below in the integration test class.

docker logs f80
...
Starting
Started 1/11 partitions
Started 2/11 partitions
Started 3/11 partitions
Started 4/11 partitions
Started 5/11 partitions
Started 6/11 partitions
Started 7/11 partitions
Started 8/11 partitions
Started 9/11 partitions
Started 10/11 partitions
Started 11/11 partitions
Started

Cosmos DB Emulator UI Home

Tip: I also wrote a follow up to this blog post with improvements:

Reuse Testcontainers initialization and configuration code with JUnit 5 Extension Callbacks in your Spring Boot Integration Tests.

6. INTEGRATION TEST CLASS

Let’s now the discuss a Spring Boot-based integration test class that uses CosmosDBEmulatorContainer to run tests against.

UserControllerTestContainersIntegrationTest.java:

@ExtendWith(SpringExtension.class)
@SpringBootTest(classes = Application.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ContextConfiguration(classes = { Application.class })
@ActiveProfiles("integration-test")
@Testcontainers
public class UserControllerTestContainersIntegrationTest {

  @Container
  public final static AsimioCosmosDBEmulatorContainer COSMOS_DB_EMULATOR = new AsimioCosmosDBEmulatorContainer();

  @LocalServerPort
  private int port;

  @TempDir
  private static File TEMP_FOLDER;

  @BeforeAll
  public static void setupTruststore() throws Exception {
    String pemCertPath = TEMP_FOLDER.getPath() + File.separator + "cosmos-db-emulator.pem";
    KeyStoreUtils.extractPublicCert(COSMOS_DB_EMULATOR.getHost(), COSMOS_DB_EMULATOR.getEmulatorEndpointPort(), pemCertPath); 
    log.info("Public cert saved to {}", pemCertPath);

    String trustStorePath = TEMP_FOLDER.getPath() + File.separator + "integration-tests-cosmos-emulator.truststore";
    KeyStoreUtils.createKeyStore(
      Lists.newArrayList(pemCertPath),
      trustStorePath,
      "changeit"
    );
    log.info("Truststore set to {}", trustStorePath);

    System.setProperty("javax.net.ssl.trustStore", trustStorePath);
    System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
    System.setProperty("javax.net.ssl.trustStoreType", "PKCS12");
  }

  @DynamicPropertySource
  public static void cosmosDbProperties(DynamicPropertyRegistry registry) throws Exception {
    log.info("Setting spring.cloud.azure.cosmos.endpoint to {}", COSMOS_DB_EMULATOR.getEmulatorEndpoint());
    registry.add("spring.cloud.azure.cosmos.endpoint", COSMOS_DB_EMULATOR::getEmulatorEndpoint);
  }

  @Test
  public void shouldCreateNewUser() throws IOException {
    JsonPath jsonPath = RestAssured
      .given()
        .accept(ContentType.JSON)
        .contentType(ContentType.JSON)
        .body(FileUtils.readFileToString(
          new File(UserControllerIntegrationTest.class.getClassLoader().getResource("stubs/new-user-request.json").getFile()),
          "UTF-8")
        )
      .when()
        .post("/api/users")
      .then()
        .statusCode(HttpStatus.OK.value())
        .contentType(ContentType.JSON)
        .extract()
        .jsonPath();

    Map<String, Object> actualUser = jsonPath.get("$");
    MatcherAssert.assertThat(actualUser.get("id"), Matchers.equalTo("200"));
    MatcherAssert.assertThat(actualUser.get("firstName"), Matchers.equalTo("First-200"));
  }

// ...
}

Remember this project uses Jupiter/JUnit 5.

We need to turn on the integration-test Spring profile so that this test class uses the application-integration-test.yml properties file.

@Testcontainers annotation is a Jupiter/JUnit 5 extension that helps to start up and stop the Docker containers used by the tests.

@Container is a marker annotation specifying the Cosmos DB emulator container lifecycle to be managed by the Testcontainers Jupiter/JUnit extension implementation.

setupTruststore() method extracts the public certificate using the emulator endpoint, creates a truststore in the TEMP_FOLDER, and sets Java truststore system properties for the tests to use.

@DynamicPropertySource-annotated cosmosDbProperties() method updates application properties at runtime.
Instead of hard-coding the emulator endpoint to https://localhost:8081, I ran these tests using random, available ports. Thus, I needed to update spring.cloud.azure.cosmos.endpoint before the Spring context loads.

And the test methods you would run from the IDE, or through the build process.

Integration tests with Testcontainers and Cosmos DB Docker emulator for Spring Boot applications

7. CONCLUSION

This blog post covered how to write integration tests for Spring Boot applications using Testcontainers and the Azure Cosmos DB emulator Docker image.

Writing integration tests with Jupiter/JUnit 5 and Testcontainers helps you to build and deploy Cosmos DB applications often and with confidence.

You can write your own Cosmos DB emulator Testcontainer listening on available ports, which you can then use in your Spring Boot-based integration tests.

Tip: Read on to learn:

How to seed Cosmos DB documents to run Spring Boot 2 Integration Tests.

Reuse Testcontainers initialization and configuration code with JUnit 5 Extension Callbacks in your Spring Boot Integration Tests.

Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.