Microservices – a software architecture style where applications are developed as a suite of small, independently deployable services—are notoriously difficult to test because of their design. Each service runs in its own process and communicates with other services via HTTP or a message broker and queue.
The main idea behind microservices is to break down large, monolithic systems into smaller, more manageable components that can be developed, tested, and deployed independently. This approach allows for greater flexibility, scalability, and resilience and enables teams to work independently on individual services.
However, testing microservices can be challenging due to their highly distributed and independent nature. You have to ensure that each service is functioning correctly in isolation and in collaboration with other services. Testing microservices requires a comprehensive approach that includes automated unit, integration, and end-to-end testing to ensure that the system is reliable, scalable, and resilient.
This guide explains why integration tests are a great way to test the interaction between microservices in the larger system of microservices. It examines a few examples of cross-service integration tests to explain the advantages of integration testing in this context.
Why Integration Testing Is Useful When Building Microservices
Firstly, let’s be clear: all types of tests are good if they tell you whether your software is good or not.
However both unit and end-to-end tests have drawbacks. Unit tests rarely give you enough confidence for automatic decision making of whether a particular commit is good to go to production. And end-to-end tests requires environments too similar in complexity to production to be maintainable.
Integration tests hit the sweet spot because they’re easy to create and maintain, and they’re easy to run on developer machines. This makes them ideal for a single source of truth for quality. By using the same technologies and actually using all the code paths without mocking code, they exercise the system in conditions close enough to production to be a good indication of whether things work or is broken.
Benefits of Integration Testing for Microservices
Let’s look at the major benefits of integration testing for microservices in more detail.
Ensuring Proper Communication between Microservices
Integration testing in a microservices architecture allows developers to detect and address issues early in the development cycle. It helps identify compatibility issues between services, dependencies that need to be resolved, and communication issues. Without integration testing, it’s easy for developers to miss defects that can cause problems when services are combined.
As an example, suppose two developers are each working on an employee and department microservice. If the employee microservice sends a request to the department microservice with the wrong ID, the department microservice would return the wrong department name. More cases where communication between microservices can go wrong include the following:
- When serialization logic differs between service
- When message schemas don’t match on the producer/consumer end
- Incompatibility of broker configuration in each service, for instance, a mismatch due to protocol settings
- A topic/destination configuration mismatch
Unit tests would not catch this defect as they test the functionality of only each microservice in isolation. However, integration testing allows developers to simulate the interaction between the microservices and catch this defect before deploying to production.
This brings us to the next point.
Simulating Real-World Scenarios
Integration testing makes it easier to identify and resolve issues that could occur in a production environment. When multiple services are deployed to a production environment, it can be challenging to identify issues quickly. Integration testing helps identify issues in a continuous integration environment by simulating real-world scenarios, which is a far less costly process than detecting them in production.
Improved Reliability
Integration testing helps reduce the risk of downtime and improves the reliability of microservices. As more services are added, the interactions between these services become more complex, and the probability of failure and the potential impact on performance increases. Without integration testing, it becomes increasingly challenging to identify the root cause of an issue, resulting in a poor application experience or long downtime.
For example, suppose you have an e-commerce system that consists of four microservices—order, customer, product, and payment. When a customer places an order through the system, all four of these microservices need to be available and coordinate together for an order to be successfully placed. Even one service that’s unavailable or not returning a response on time will result in a system availability or reliability issue. Performing integration testing to simulate such scenarios in advance would help to mitigate such issues.
Improved Maintainability
Integration testing facilitates the maintenance of microservices by reducing the complexity of the system. It helps to identify dependencies and communication channels between services, making it easier to make changes to the microservices. Also, when a change is made to one service, the impact on the other services can be determined, easing up maintenance.
For example, to return to the scenario of the department and employee microservices, suppose you need to make a change to the department microservice that affects how it communicates with the employee microservice. Integration testing would allow you to identify the dependencies between these two services and test the changes in the context of the entire system. This would help you understand the impact between the two services and ensure that the new changes do not break the communication channel between them. Integration testing can help you verify that the department microservice can still process information from the employee microservice correctly.
How Integration Tests Work in Microservices?
Now that you understand the importance of integration testing in microservices, let’s examine how it works in practice by examining a demo project.
Demo Project Overview
The demo microservices project mirrors our earlier example of two microservices—an employee service and a department service.
The department service holds only the department name and ID information whereas the employee service deals with employee information stored in the PostgreSQL database. The department service serves an API (example URI: http://localhost:8081/department/1) to fetch the department details for a given department ID. The employee service serves an API (example URI: http://localhost:8080/department/1) to fetch the details of all employees for a given department ID. This particular API will, in turn, call the department service API to retrieve the department details (especially the department name, which is available only in the department service). It merges it with the employee information to return the overall gathered employees’ details of that particular department ID.
In this example, the department service is dependent on the employee service to receive the endpoint’s response with consolidated information. In a real application, you will have similar dependencies, but it won’t be limited to one other microservice and one endpoint—it’s more likely to be with multiple microservices and multiple endpoints. As discussed in the earlier sections, this is why integration testing that simulates real-world test scenarios is vital. It ensures that microservices are reliable and communicate properly for a smooth application experience in production.
So how do you perform this type of integration testing for a microservices setup like this? This demo project makes use of Testcontainers by AtomicJar to spin up a containerized database. This setup lets you perform a real-world simulation to verify if your microservices are communicating correctly with the database and that the database is configured correctly.
By simulating failure scenarios—for example, incorrect configuration or network failures —you can plan and develop code for how to handle failures in this communication between the microservices and the database gracefully. You can clearly see a way to improve the error-handling mechanism of your microservices by drawing out such test scenarios.
You can also use Testcontainers to spin up a containerized dependent service, which, in this example, is a department service for cross-service integration testing. This ensures that both the employee and department microservices are communicating correctly with each other over the desired protocol like HTTP and that the services are configured correctly.
Let’s take a closer look at some code snippets from this demo project. For a complete picture of how the project is built and chained together, you can review the source code from this GitHub repo.
Setting Up Testcontainers in the Employee Microservice
To add Testcontainers to your Java project, you have to add Testcontainers as a dependency in your project’s build file (pom.xml in the case of Maven users or build.gradle in the case of Gradle users). You can refer to the official documentation of Testcontainers for more details on how to do this. Just ensure that you have a corresponding database-related Testcontainers library also added as a dependency.
In the context of this demo project, PostgreSQL is the database, so the following code snippet should be part of your project’s pom.xml file:
<dependency> <groupId>org.testcontainers</groupId> <artifactId>postgresql</artifactId> <scope>test</scope> </dependency>
Testing Code Using Testcontainers
Once you’ve set up Testcontainers dependencies for your project, the next step is to understand how to make use of Testcontainers classes, methods, and annotations to use them for your microservices integration testing.
The test package of the employee service has a class file named EmployeeServiceApplicationTests.java. This class uses Testcontainers for integration testing demonstration, as shown in the code snippet that follows:
@Testcontainers
@SpringBootTest
class EmployeeServiceApplicationTests {
@Container
public static PostgreSQLContainer<?> postgresContainer = new PostgreSQLContainer<>("postgres:15");
@DynamicPropertySource
static void properties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", postgresContainer::getJdbcUrl);
registry.add("spring.datasource.password", postgresContainer::getPassword);
registry.add("spring.datasource.username", postgresContainer::getUsername);
}
@Container
public static GenericContainer<?> departmentService =
new GenericContainer<>(DockerImageName.parse("department-service:latest"))
.withExposedPorts(8081)
.waitingFor(Wait.forHttp("/department/1"));
@Autowired
EmployeeRepository employeeRepository;
@Autowired
EmployeeDetailsService employeeDetailsService;
@Test
void assertEmployeeCountByDepartmentId() {
List<Employee> employeeList = employeeRepository.findByDepartmentId(1);
assert employeeList.size() == 3;
}
@Test
void assertDepartmentServiceCall(){
RestTemplate restTemplate = new RestTemplate();
String departmentRestEndpoint = "http://"
+ departmentService.getHost()
+ ":" + departmentService.getMappedPort(8081)
+ "/department/1";
Department department =
restTemplate.getForObject(departmentRestEndpoint, Department.class);
assert Objects.equals(Objects.requireNonNull(department).getName(), "Engineering");
}
}
In this code, @Testcontainers is a Java annotation that lets you start and manage containers easily during unit and integration testing. Using Testcontainers, the test defines the PostgreSQL container and the department service container. These containers are created based on images specified in the PostgreSQLContainer and GenericContainer definition in the test code. They’re configured with specific settings, such as database connection URL, access credentials for the database, and port bindings. When you initiate the test of the employee service application, Testcontainers will automatically take care of starting these containers before the test and stop them after the test has finished.
The following diagram shows how the employee service makes use of Testcontainers for testing:
The test assertEmployeeCountByDepartmentId makes a connection request to the PostgreSQL database and retrieves the employee records by the given department ID for assertion. The test assertDepartmentServiceCall makes an external call to the department service container to get the department details and then makes an assertion on the retrieved department details.
You use your IDE to start the tests of the employee service. Once the tests are executed successfully, you should get as result that two out of two tests passed:
Testcontainers is helpful here because it ensures that the resources required for cross-service integration testing are used efficiently. Containers can be spun up and down as required, which reduces the overall resource consumption of the system. Testcontainers can be easily configured with its built-in class methods to meet the specific requirements of the microservices architecture.
Other Best Practices for Designing Integration Tests in Microservices
Designing effective integration tests for microservices requires careful consideration of various factors. Best practices include the following:
- Use a representative data set to test the microservice. This can help in identifying potential performance issues, bottlenecks, or errors that may not appear with a smaller data set.
- Test all endpoints and use cases. This can help in identifying any gaps in functionality, and it helps ensure that the microservice meets all requirements.
- Test with real-world scenarios. It ensures that the microservice works as expected in a production environment and helps identify issues before deployment.
- Automate testing wherever possible. It increases efficiency and ensures that tests are repeatable. Frameworks such as JUnit, TestNG, or Selenium help here.
- Continuously improve testing. Incorporate feedback from testing as well as project stakeholders and other sources of information like logs, metrics collected from monitoring tools, and so on. Doing so can help you identify areas for improvement and ensure that the microservice is performing optimally.
Conclusion
Integration testing is crucial for building microservices because it ensures that each service works seamlessly with others in the system.
Microservices architecture involves building software applications as a collection of small, independent services, each serving a specific function. These services need to interact with each other to provide the expected result. Especially as microservices scale, ensuring that they integrate correctly with one another is essential.
Testcontainers makes it easier to achieve your integration testing goals. As you saw in the demo project, instead of you having to manage the external services lifecycle during your integration testing, Testcontainers manages them automatically. Testcontainers can also help you speed up the testing process by reducing the amount of time it takes to set up and tear down testing environments. Using lightweight containers lets you spin up and tear down testing environments quickly as needed, allowing you to iterate and test code quicker.
Visit Testcontainers for more about how it can help you test your microservices.
by Rajkumar Venkatasamy
Rajkumar has nearly sixteen years of experience in the software industry as a developer, data modeler, tester, project lead, product consultant, data architect, ETL specialist, and technical architect. Currently he is a principal architect at an MNC. He has hands-on experience in various technologies, tools, and libraries.
