While cloud technologies like AWS S3, DynamoDB, Lambda, and so forth make it easy to develop powerful cloud-native applications, testing such apps can be quite cumbersome. You want to emulate production-like conditions in your tests for accuracy, but it’s not possible to do so without setting up similar resources on your cloud, which incurs additional costs. A cloud-based test environment that’s away from your dev environment can also cause latency and other network-related issues.
A better solution is to set up production-like test environments locally using Testcontainers and LocalStack. Testcontainers for Java is a library that helps you to run tests with real services using ephemeral Docker containers. It provides lightweight instances of databases, web browsers, and more to facilitate easy and reliable testing of your apps. LocalStack is a cloud service simulator that lets you mock AWS services in your tests to save on costs while enhancing development speed.
Testcontainers and LocalStack help ensure that your cloud-native apps are well tested from day one. Setting them up is super simple, as you’ll see later in this article, and they are great at creating reproducible and reliable test conditions across platforms. This means that they boost both developer experience and productivity.
In this article, you’ll learn how to set up Testcontainers and LocalStack in a Java Spring Boot app to simplify running integration tests.
Project Overview
The example app in this tutorial uses AWS SQS and AWS DynamoDB to set up a payments webhook app that can collect payment updates asynchronously from a third-party payment processor and update them in your database. This will help you understand how to use LocalStack and Testcontainers to test the AWS service integrations.
Many payment processors decouple payment initiation and completion. Since the payment transactions are usually done outside the scope of the payment processor, you need an event-based system to capture and store payment-related async updates from the payment processor. AWS SQS helps you to build an event-based system that listens for payment events (triggered via a webhook), and DynamoDB is used to store the payment records.
You can find the complete code of the sample app for this tutorial in this GitHub repo.
Prerequisites
You need to have the following softwares installed to follow along this article:
- Java 17+
- Any of the supported container runtimes
Setting Up Dependencies and Configuring the Environment
To begin, create a new Maven project in your local system using an IDE of your choice.
Once the project is created, the first step is to add the dependencies for the application. You’ll need to paste the following code snippet in your pom.xml file within the <project>...</project> tags:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.1.3</version>
<relativePath/>
</parent>
<properties>
<java.version>17</java.version>
<awspring.version>3.0.2</awspring.version>
<testcontainers.version>1.19.0</testcontainers.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-sqs</artifactId>
</dependency>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-dynamodb</artifactId>
</dependency>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-sqs</artifactId>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>localstack</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-dependencies</artifactId>
<version>${awspring.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
By doing this, you set up the following prominent dependencies:
spring-cloud-aws-starter-sqs: to set up SQS queuesspring-cloud-aws-starter-dynamodb: to set up DynamoDBorg.testcontainers.localstack: to set up Localstackorg.testcontainers.junit-jupiter: to set up Testcontainersorg.awaitility.awaitility: to help test asynchronous Java code
Make sure to sync your dependencies after updating the pom.xml file.
Next, you will create an Application.java class, which will act as the entry point to your project, and paste the following contents in it:
package com.testcontainers.demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
You will need to create a Payment class to handle the payment details. To do that, store the following code in a file called Payment.java:
package com.testcontainers.demo;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbAttribute;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbBean;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbPartitionKey;
@DynamoDbBean
public class Payment {
private String paymentId;
private String payerId;
private String orderId;
private float paymentAmount;
private String paymentStatus;
private String paymentDateTimeISO;
public Payment() {
}
@JsonCreator
public Payment(String json) throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper();
Payment payment = objectMapper.readValue(json, Payment.class);
this.paymentId = payment.paymentId;
this.payerId = payment.payerId;
this.orderId = payment.orderId;
this.paymentAmount = payment.paymentAmount;
this.paymentStatus = payment.paymentStatus;
this.paymentDateTimeISO = payment.paymentDateTimeISO;
}
@DynamoDbPartitionKey
public String getPaymentId() {
return paymentId;
}
public void setPaymentId(String paymentId) {
this.paymentId = paymentId;
}
@DynamoDbAttribute("payerId")
public String getPayerId() {
return payerId;
}
public void setPayerId(String payerId) {
this.payerId = payerId;
}
@DynamoDbAttribute("orderId")
public String getOrderId() {
return orderId;
}
public void setOrderId(String orderId) {
this.orderId = orderId;
}
@DynamoDbAttribute("paymentAmount")
public float getPaymentAmount() {
return paymentAmount;
}
public void setPaymentAmount(float paymentAmount) {
this.paymentAmount = paymentAmount;
}
@DynamoDbAttribute("paymentStatus")
public String getPaymentStatus() {
return paymentStatus;
}
public void setPaymentStatus(String paymentStatus) {
this.paymentStatus = paymentStatus;
}
@DynamoDbAttribute("paymentDateTimeISO")
public String getPaymentDateTimeISO() {
return paymentDateTimeISO;
}
public void setPaymentDateTimeISO(String paymentDateTimeISO) {
this.paymentDateTimeISO = paymentDateTimeISO;
}
@Override
public String toString() {
return "Payment{" +
"paymentId='" + paymentId + '\'' +
", payerId='" + payerId + '\'' +
", orderId='" + orderId + '\'' +
", paymentAmount=" + paymentAmount +
", paymentStatus='" + paymentStatus + '\'' +
", paymentDateTimeISO='" + paymentDateTimeISO + '\'' +
'}';
}
}
The Payment class makes use of @DynamoDbBean and @DynamoDbAttribute annotations to define the schema and facilitate packing and unpacking of the class when using DynamoDB.
Next, create a new class called PaymentHandler.java to handle payments as following:
package com.testcontainers.demo;
import io.awspring.cloud.sqs.annotation.SqsListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
@Component
public class PaymentHandler {
private static final Logger LOG = LoggerFactory.getLogger(PaymentHandler.class);
private final DynamoDbClient dynamoDbClient;
public PaymentHandler(DynamoDbClient dynamoDbClient) {
this.dynamoDbClient = dynamoDbClient;
}
@SqsListener("payment-queue")
public void handlePayment(Payment payment) {
LOG.info("Payment details received: " + payment.toString());
DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(dynamoDbClient).build();
enhancedClient.table("Payments", TableSchema.fromBean(Payment.class)).putItem(payment);
LOG.info("Payment details saved in table");
}
}
The handlePayment() method listens for events sent to the payment-queue SQS queue. Whenever it receives an event, it stores the event details in the Payments table in DynamoDB.
This completes the setup of your Spring Boot application. You do not need to configure DynamoDB and SQS manually as the Spring Cloud AWS SDK v3 automatically does basic configuration. If you need to customize the configuration some more, you can do so by defining handler classes.
You can now proceed to write the integration test that makes use of Testcontainers and LocalStack.
Setting Up Testcontainers
To get started with integration testing, you will first need to create a new test class in your project. Create a new test with the name PaymentHandlerIntegrationTest.java.
Note: The completed file is shown at the end of this section. You can follow along to understand how the file is built, or you could jump to the end and copy the completed code for the file.
To set up Testcontainers, add the @Testcontainers annotation to the class:
@Testcontainers
@SpringBootTest
public class PaymentHandlerIntegrationTest {
// ...
}
Once this annotation is added, Testcontainers will automatically create ephemeral containers for all resources annotated with @Container in the class, which you’ll define in the next step.
Using LocalStack to Mock AWS Services
Next, you will set up LocalStack to mock AWS SQS and DynamoDB for the test. Create a new LocalStackContainer instance in the test class to set up mock AWS resources:
@Container
static LocalStackContainer localStack =
new LocalStackContainer(DockerImageName.parse("localstack/localstack:2.2"));
Once the localStack container is set up with the mock AWS services, run the AWS CLI commands to set up the message queue and database table by defining the commands in the beforeAll() method of the test:
@BeforeAll
static void beforeAll() throws IOException, InterruptedException {
localStack.execInContainer("awslocal", "dynamodb", "create-table",
"--table-name", "Payments",
"--attribute-definitions", "AttributeName=paymentId,AttributeType=S",
"--key-schema", "AttributeName=paymentId,KeyType=HASH",
"--provisioned-throughput", "ReadCapacityUnits=5,WriteCapacityUnits=5"
);
}
This will run the initialization commands and update the local system properties with the URL of the newly launched DynamoDB instance.
Next, you will create a function to update the configuration from the localStack instance into your Spring Boot app’s system properties so that the app can connect to the mocked AWS resources. You can do it by pasting the following code snippet in the test class:
@DynamicPropertySource
static void overrideConfiguration(DynamicPropertyRegistry registry) {
registry.add("spring.cloud.aws.sqs.endpoint", () -> localStack.getEndpointOverride(SQS));
registry.add("spring.cloud.aws.dynamodb.endpoint", () -> localStack.getEndpointOverride(DYNAMODB));
registry.add("spring.cloud.aws.credentials.access-key", () -> localStack.getAccessKey());
registry.add("spring.cloud.aws.credentials.secret-key", () -> localStack.getSecretKey());
registry.add("spring.cloud.aws.region.static", () -> localStack.getRegion());
}
Write and Run an Integration Test between Two AWS Services in LocalStack
Now you can proceed to write the integration test for the app.
The integration test is simple in itself. It sends a payment object to the SQS queue and then checks if the object was added to the DynamoDB table after a few seconds. Here’s the code for the test:
@Autowired
private DynamoDbClient dynamoDbClient;
@Autowired
private SqsTemplate sqsTemplate;
@Test
public void paymentShouldBeSavedToDBOnceConsumedFromQueue() {
sqsTemplate.send(QUEUE_NAME, new GenericMessage<>("""
{
"paymentId": "payment_3566",
"payerId": "cust_234",
"orderId": "order_809",
"paymentAmount": 4.99,
"paymentStatus": "successful",
"paymentDateTimeISO": "2023-03-22T00:18:26+0000"
}
""", Map.of("contentType", "application/json")));
DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(dynamoDbClient).build();
PageIterable<Payment> paymentList = enhancedClient.table("Payments", TableSchema.fromBean(Payment.class)).scan();;
given()
.await()
.atMost(10, SECONDS)
.ignoreExceptions()
.untilAsserted(() -> assertThat(paymentList.items().stream()).hasSize(1));
given()
.await()
.atMost(10, SECONDS)
.ignoreExceptions()
.untilAsserted(() -> assertThat(paymentList.items().iterator().next().getPaymentId()).isEqualTo("payment_3566"));
}
This is what the PaymentHandlerIntegrationTest.java class should look like once its complete:
package com.testcontainers.demo;
import io.awspring.cloud.dynamodb.DynamoDbTemplate;
import io.awspring.cloud.sqs.operations.SqsTemplate;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.support.GenericMessage;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.localstack.LocalStackContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.Expression;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.model.PageIterable;
import software.amazon.awssdk.enhanced.dynamodb.model.ScanEnhancedRequest;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import static java.util.concurrent.TimeUnit.SECONDS;
import static org.assertj.core.api.Assertions.assertThat;
import static org.awaitility.Awaitility.given;
import static org.testcontainers.containers.localstack.LocalStackContainer.Service.DYNAMODB;
import static org.testcontainers.containers.localstack.LocalStackContainer.Service.SQS;
@Testcontainers
@SpringBootTest
public class PaymentHandlerIntegrationTest {
private static final String QUEUE_NAME = "payment-queue";
@Container
static LocalStackContainer localStack = new LocalStackContainer(DockerImageName.parse("localstack/localstack:2.1.0"));
@BeforeAll
static void beforeAll() throws IOException, InterruptedException {
localStack.execInContainer("awslocal", "dynamodb", "create-table",
"--table-name", "Payments",
"--attribute-definitions", "AttributeName=paymentId,AttributeType=S",
"--key-schema", "AttributeName=paymentId,KeyType=HASH",
"--provisioned-throughput", "ReadCapacityUnits=5,WriteCapacityUnits=5"
);
}
@DynamicPropertySource
static void overrideConfiguration(DynamicPropertyRegistry registry) {
registry.add("spring.cloud.aws.sqs.endpoint", () -> localStack.getEndpointOverride(SQS));
registry.add("spring.cloud.aws.dynamodb.endpoint", () -> localStack.getEndpointOverride(DYNAMODB));
registry.add("spring.cloud.aws.credentials.access-key", () -> localStack.getAccessKey());
registry.add("spring.cloud.aws.credentials.secret-key", () -> localStack.getSecretKey());
registry.add("spring.cloud.aws.region.static", () -> localStack.getRegion());
}
@Autowired
private DynamoDbClient dynamoDbClient;
@Autowired
private SqsTemplate sqsTemplate;
@Test
public void paymentShouldBeSavedToDBOnceConsumedFromQueue() {
sqsTemplate.send(QUEUE_NAME, new GenericMessage<>("""
{
"paymentId": "payment_3566",
"payerId": "cust_234",
"orderId": "order_809",
"paymentAmount": 4.99,
"paymentStatus": "successful",
"paymentDateTimeISO": "2023-03-22T00:18:26+0000"
}
""", Map.of("contentType", "application/json")));
DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(dynamoDbClient).build();
PageIterable<Payment> paymentList = enhancedClient.table("Payments", TableSchema.fromBean(Payment.class)).scan();;
given()
.await()
.atMost(10, SECONDS)
.ignoreExceptions()
.untilAsserted(() -> assertThat(paymentList.items().stream()).hasSize(1));
given()
.await()
.atMost(10, SECONDS)
.ignoreExceptions()
.untilAsserted(() -> assertThat(paymentList.items().iterator().next().getPaymentId()).isEqualTo("payment_3566"));
}
}
Upon running the test via ./mvnw verify, you will notice that the Payment object that you sent to the queue gets printed to the console and then saved in the database:
...
...
2023-03-22 10:52:19.550 INFO 18220 --- [ main] d.d.demo.PaymentHandlerIntegrationTest : Started PaymentHandlerIntegrationTest in 3.954 seconds (JVM running for 246.479)
2023-03-22 10:52:20.219 INFO 18220 --- [enerContainer-2] dev.draft.demo.PaymentHandler : Payment details received: Payment{paymentId='payment_3566', payerId='cust_234', orderId='order_809', paymentAmount=4.99, paymentStatus='successful', paymentDateTimeISO='2023-03-22T00:18:26+0000'}
2023-03-22 10:52:20.433 INFO 18220 --- [enerContainer-2] dev.draft.demo.PaymentHandler : Payment details saved in table
...
...
Tips for Running Tests and Debugging Issues with Testcontainers and LocalStack
As you’ve seen, setting up Testcontainers and LocalStack is very simple. However, you need to be mindful of some of the fine details when configuring them together to ensure that your tests work.
Here are some best practices to follow and tips that might come in handy when troubleshooting your integration tests:
- Make sure either the application.yml (or application.properties) file or AWS environment variables on your local system is configured correctly. Both LocalStack and the AWS SDK rely on the system properties to fetch and share AWS credentials and other metadata.
- Do not assign static names to the containers created by Testcontainers. It can result in clashes with existing resources. Testcontainers will automatically generate random names to avoid such a situation.
- As with names, avoid static ports. Also make sure that the randomly assigned port for your AWS services is correctly updated in your application’s system properties when the LocalStack containers are ready.
- While containers created by Testcontainers cannot be reused between classes, remember that you can always define a static container to reuse between multiple tests in the same class. This is best implemented using the Singleton Containers Pattern and can help you save both resources and time.
Conclusion
Testcontainers is a Java-based testing library that’s growing in popularity due to its lightweight and easy-to-use nature. When it’s coupled with LocalStack, you get a powerful system for running integration tests on cloud-native systems of any scale.
In this tutorial, you learned how to use Testcontainers and LocalStack to mock AWS resources and run an integration test on a simple Java Spring Boot application. You can find the complete code for this tutorial in this GitHub repo.
As next steps, you can try implementing a few of your own test cases in the same application. Or you could take a whole new stack of AWS services and write integration tests for them. Either way, make sure to try out Testcontainers to understand its power and flexibility in testing your cloud-native application.
by Kumar Harsh
Kumar Harsh is an indie software developer and devrel enthusiast. He is a spirited writer who puts together content around popular web technologies like Serverless and JavaScript.
