Integration tests have been growing in popularity the past few years. Development teams are increasingly realizing that despite them writing extensive unit tests covering all their code, their applications fail when they deploy them to production. They are gradually limiting unit tests to critical components and focusing more on writing integration tests.
Integration tests are software tests where multiple components in an application are combined (or integrated) and tested. Integration tests focus on how well individual components communicate and interact when working together. They form an intermediate stage in a broad spectrum of testing based on code coverage per test, coming in between unit tests and end-to-end (E2E) tests.
The shift in focus away from unit tests towards integration tests is resulting in better tools that make integration testing easier. This, in turn, is increasing the popularity of integration tests in the software development ecosystem, creating an upward spiral of adoption. For example, the Java repository of Testcontainers, the most popular integration testing library, has close to 1,500 forks and more than 7,000 stars, and its .NET and Go versions are trending in the same direction. Testcontainers-related software shows a total of more than 100 million pulls on Docker Hub with around 7 million per month and tens of thousands of downloads on Maven Central.
In this article, you’ll learn more about why integration tests are growing in popularity and see examples of how well-known open source projects are using them.
The Downsides of Unit Tests and End-to-End Tests
As mentioned before, integration tests lie in the middle of a software testing spectrum between unit tests and E2E tests.
Unit tests are typically the most commonly seen and implemented software tests. They involve testing an individual component, or unit, in isolation and creating mocks—simplified artificial models that mimic the behavior of real objects in a controlled manner—for any integrations with that component. Every major software framework or library comes with unit testing functionality included.
The limitation of unit tests is that they don’t give enough information on how the application will work in the real world. While they’re helpful in testing the internal behavior of components, they don’t offer insights into how those components work with others.
Software applications increasingly rely on the interaction between numerous components and services that must work together as a whole. It is especially true in a distributed architecture—like one based on microservices—where different components might not share a common machine, memory, disk space, or other resources. This results in greater complexity: higher latency, network errors, and hardware failure can occur.
On the other end of the spectrum are end-to-end tests, which test the whole system working together. They can be seen as the ultimate integration test where every component in the application is connected and tested together.
It is easy to assume that everything else should be ignored to focus on end-to-end tests. However, it takes a lot of work to implement end-to-end tests in cloud environments. Entire user journeys must be mapped, and every associated component and service in the application is involved. This leads to a massive set of dependencies that must be set up in the test environment every time a test has to run, which can take an impractical amount of time. The delays in deployment would be antithetical to the faster time to market for which businesses choose a distributed architecture in the first place.
Another downside of E2E testing is that it can be very expensive to replicate the production system on cloud-based staging or test environments. Because such a large number of resources are involved, it can be close to impossible to set up on a development machine. The alternative—a scaled-down version of the production environment—will offer different results and might render the end-to-end tests pointless.
An alternative way to think about tests for microservices is in terms of integrated, integration, and implementation detail tests, also known as the honeycomb testing framework.
In this framework, you can think of a microservice as a single unit. A typical unit test will translate to an implementation detail test, while a test whose output depends on another system’s correctness is an integrated test.
The third type of test is the integration test, which helps you verify a single microservice in isolation and focus on any communication points between other services. The honeycomb methodology recommends focusing more on integration tests than integrated tests or implementation detail tests.
The Sweet Spot: Integration Tests
In cloud environments, integration tests land in the sweet spot between unit tests and end-to-end tests. They offer better assurance than unit tests that the system will work as the integration between multiple components is tested. However, they are more practical than running end-to-end tests on the full system.
In integration tests, real connections between components are tested with real technologies instead of mocking them with oversimplified models. Suppose one of your application’s components makes an API call. In that case, the call will actually be made during an integration test instead of a mock value being returned under a unit test.
For instance, if you have a grocery shopping app, the components that handle the login screen, cart, and order payments will differ. Unit tests test these functionalities individually. Integration tests test all these components and how they interact with each other together. Having tests like these that scope multiple interacting components increases confidence that the code will work in production, where all of these components are going to interact at scale.
Because you don’t have to recreate full production systems in integration testing (you’re not testing the full application), you can run integration tests on your local machine as well as on continuous integration (CI) systems in the cloud. Test containers don’t need to be as resource rich as the production environment. Identical test containers can be easily and cheaply created on your local machine as well as programmatically in your CI pipeline without additional manual intervention.
Such container-based environments are isolated from other environments on the same cloud. This means that any external process is not allowed to affect the test results, and the tests don’t create a bottleneck in any other system. Different test environments are also not allowed to interfere with each other. The test containers can be deleted as soon as their need is finished—right after a successful test or after debugging a failed test—so that resources are not wasted needlessly.
Examples of Open Source Projects Using Integration Tests
While it’s difficult to gauge the usage rate of integration tests in private repositories, the use of integration tests in open source tools offers some insight. The communities and broad user base that drive the development of open source tools often require higher standards of resilience and, therefore, testing.
Integration tests are especially useful for tools that involve a large variety of integrations. Therefore, telemetry and database tools use integration tests heavily. Let’s examine a few of them.
Telemetry Tools
Telemetry tools have to support integrations with a large number of other tools and services. It makes them ideal candidates for integration testing, where real connections between components are checked. It’s not surprising then that many popular telemetry tools in the Cloud Native Computing Foundation (CNCF) use Testcontainers for integration tests.
Prometheus is a popular open source system monitoring and alerting service. It supports adding instrumentation to your application code through a wide variety of client libraries in over ten different languages.
Prometheus uses integration tests in many of its applications, including its Java client and JMX Exporter projects.
Below is a trimmed-down snippet of Prometheus’ Java client that is trying to spin up a Testcontainers environment with an OpenTelemetry image for running a smoke test on a fake backend:
package io.opentelemetry.smoketest;
import java.io.IOException;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.function.Consumer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testcontainers.containers.Container;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
import org.testcontainers.containers.output.Slf4jLogConsumer;
import org.testcontainers.containers.output.ToStringConsumer;
import org.testcontainers.utility.DockerImageName;
import org.testcontainers.utility.MountableFile;
public class LinuxTestContainerManager extends AbstractTestContainerManager {
private static final Logger backendLogger = LoggerFactory.getLogger("Backend");
private final Network network = Network.newNetwork();
private GenericContainer<?> backend = null;
@Override
protected void startEnvironment() {
backend =
new GenericContainer<>(
DockerImageName.parse(
"ghcr.io/open-telemetry/opentelemetry-java-instrumentation/smoke-test-fake-backend:20221127.3559314891"))
.withExposedPorts(BACKEND_PORT)
.withEnv("JAVA_TOOL_OPTIONS", "-Xmx128m")
.waitingFor(Wait.forHttp("/health").forPort(BACKEND_PORT))
.withNetwork(network)
.withNetworkAliases(BACKEND_ALIAS)
.withLogConsumer(new Slf4jLogConsumer(backendLogger));
backend.start();
}
...
...
}
Smoke testing is one of the simplest examples of integration testing. Smoke tests involve interaction between multiple components, so they cannot be done with unit tests, which test only one component of your application at a time. Testcontainers lets you use various components of your system as a whole with actual dependencies, all deployed in disposable Docker containers.
Another example of a telemetry tool that uses integration tests is OpenTelemetry (OTel), a vendor-agnostic open-source framework for managing telemetry data for cloud-based projects that also supports numerous libraries and frameworks in multiple languages. OTel integrates with Testcontainers to provision containers for integration tests in its instrumentation tooling and underlying SDKs in Java, Go, and .NET so that you can test your applications across traces and spans.
Database Tools
Integration tests are also useful for database tools since they have to support many possible integrations.
Databases are at the heart of application architecture. Most application components talk to the database layer to read and write data, so it’s critical to ensure that database code such as stored procedures, user-defined functions, triggers, and more work as they should. You could test some of these database-related entities with unit tests, but since these tests wouldn’t emulate a real-world scenario, it doesn’t instill as much confidence in the database setup.
Testcontainers’ libraries offer modules—file-based tool-specific configurations with a standard set of parameters that ease the creation of test containers for that tool—for the most popular database tools, including InfluxDB, MySQL, MariaDB, Postgres, MongoDB, Cassandra, QuestDB, and more.
Spinning up a container using Testcontainers modules is very easy. Here’s a snippet defining a method that creates an InfluxDB container for testing and returns the connection string to the database:
public static InfluxDB createInfluxDBWithUrl(final InfluxDBContainer<?> container) {
InfluxDB influxDB = InfluxDBFactory.connect(
container.getUrl(),
container.getUsername(),
container.getPassword()
);
influxDB.setDatabase(container.getDatabase());
return influxDB;
}
Using database-specific Testcontainer modules prevents the need for complex setups on the developers’ computers and preserves the sanctity of the tests by always starting them from a known database state.
Cloud-Native Applications Testing
Cloud-Native applications may leverage the services provided by the Cloud Platforms. This brings some challenges in terms of local development and testing.
Testcontainers enables the development and testing of Cloud-Native application by providing modules such as LocalStack, Google Cloud, Azure etc.
Conclusion
Integration testing is becoming more important in the development workflow of complex applications, especially as the cloud-native paradigm grows. Open source projects and associated companies increasingly use tools like Testcontainers that help teams run integration tests more conveniently.
Integration tests allow you to test a combination of components in your application and their interactions with each other. This matches how your app would work in the real world more closely than unit tests, which makes integration tests more reliable in most cases. Integration tests for complex applications in cloud environments are also more practical than end-to-end tests, which test your entire application together.
To give integration tests a shot on your project, try Testcontainers. And if you want to use integration tests without having to set up your own testing infrastructure, be sure to check out Testcontainers Cloud.
by Kovid Rathee
Kovid Rathee is a data and infrastructure engineer working as a senior consultant at Servian in Melbourne. Before moving into the data space, he was an assistant professor at an engineering college and a full-stack developer. Kovid likes to write about data engineering, infrastructure-as-code, DevOps, and SRE.
