For sure you know that Testcontainers is THE library for doing integration tests right, don’t you? And most likely you already know that it’s a big thing in the Java ecosystem, where Spring Boot, Quarkus and Micronaut use it internally for integration testing.
But did you know that there is a strong Testcontainers community in the Go ecosystem, using its Go implementation in companies and organizations of any size? A quick search for “testcontainers-go” in Github returns a list of repositories owned by great companies such as Microsoft, Elastic, Confluent, Hashicorp, Newrelic, Alibaba, Influxdata, Snyk, Timescale, SAP, Adobe and many more.
In this blog post, we want to share two things: AtomicJar is now sponsoring the development and maintenance of the Go implementation of the Testcontainers project: https://github.com/testcontainers/testcontainers-go! And to put a cherry on top, the v0.14.0 release for testcontainers-go is finally out. A great release that is also the first testcontainers-go release sponsored by AtomicJar since our launch. Besides that, we also want to take the chance to present the amazing community around the Go project.
Community
We’d like to introduce you to the Go community using testcontainers-go. As we mentioned before, there are big companies using the Go library, and we at AtomicJar are super proud of it. There is also a great number of gophers that consider Testcontainers already as the way to Go for integration testing, and for that reason we’d like to share with you a great community story.
Back in May ’22, a company approached the testcontainers-go maintainers because they wanted to organize a hackathon for an entire month in June! They had chosen testcontainers-go as the repository to send their contributions to during that time, because it’s a library they love and use on a daily basis. This is the true spirit of Open Source: giving back. During this past June they were triaging issues, creating pull requests, and even commenting on the existing issues/pulls, which resulted in adding a lot of value to the project. Because of that, we’d like to say that a great part of the beauty and value of this release comes from them, so thank you Anna and team for your time and dedication here.
Continuing with the community, we want to share with you some interesting facts about it:
- Gianluca Arbezzano submitted the initial commit in July, 2018, so the project is 4 years old at this moment.
- 851 commits have been merged in total since its inception.
- There are 110 different authors, 44 last year.
- 98 different developers submitted an issue, 25 last year.
- 120 different developers submitted a review, 44 last year.
- The average number of lines per commit is 44.43.
- The average number of lines per commit and file is 0.03.
If you are into OSS and Stats and want to dig deeper for yourself, you can have a look at the following query.
testcontainers-go v0.14.0 is out!
We are pleased to announce the release of testcontainers-go v0.14.0!
Before going into the major highlights on the release, we want to share certain numbers about this release:
- v0.13.0 was released 5 months ago
- 239 commits have been merged since the previous release. However, we are not squashing commits yet, this is something we want to address for the following releases.
- 19 different contributors have participated.
- A company hackathon happened last June!
- 52 PRs were merged: 13 new features; 12 housekeeping PRs, or tasks that are not user facing, such as CI tasks, or improving the build system; 5 documentation PRs; 3 bug fixes; 2 breaking changes; 17 as dependency PRs.
Major highlights
There’s some great stuff that has been contributed by our beloved Community in this release, and we would like to highlight some of the major things. For a more comprehensive list of what’s contained in the release, please take a look at the changelog.
Reusable containers!
Have a long running test suite with multiple tests hitting a database? Do you need to keep the container running for the entire test execution? Now it’s possible with reusable containers. Just set the name of the container in the ContainerRequest struct for the container that needs to be reused, then indicate that following usages of the container will reuse it, and the container will be reused for those container requests.
Reusing a container will only work if you pass an existing container name via req.Name field. If the name is not in the list of the existing containers retrieved by the Docker client, filtered by container name, the function will create a new generic container. If Reuse is true and Name is empty, you will get an error.
The following snippet creates an Nginx container, adds a file into it and then reuses the container again, checking the file is already present in the container:
const (
reusableContainerName = "my_reusable_container"
)
ctx := context.Background()
n1, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "nginx:1.17.6",
ExposedPorts: []string{"80/tcp"},
WaitingFor: wait.ForListeningPort("80/tcp"),
Name: reusableContainerName,
},
Started: true,
})
if err != nil {
log.Fatal(err)
}
defer n1.Terminate(ctx)
copiedFileName := "hello_copy.sh"
err = n1.CopyFileToContainer(ctx, "./testresources/hello.sh", "/"+copiedFileName, 700)
if err != nil {
log.Fatal(err)
}
n2, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "nginx:1.17.6",
ExposedPorts: []string{"80/tcp"},
WaitingFor: wait.ForListeningPort("80/tcp"),
Name: reusableContainerName, // reusing container name
},
Started: true,
Reuse: true, // marked to reuse the existing container
})
if err != nil {
log.Fatal(err)
}
code, _, err := n2.Exec(ctx, []string{"bash", copiedFileName})
if err != nil {
log.Fatal(err)
}
fmt.Printf("exit code is %d\n", code) // exit code should be zero, as the script was already in the reused container
You can inspect the running containers with docker ps -a and notice that there is only one container running in your system.
Copy files before a container is created
Before this amazing feature you could only copy files to a running container. This is fine, but what if you need copying a file into the container before it is in the Running state? Or even worse? Say you want to use Testcontainers Cloud – which you should – and want to mount a file into a container that runs in a remote Docker host? You simply cannot.
But it’s possible now to copy files into the container right after it’s created but not yet started, declaring the paths for the file and its mode, as an int64.
ctx := context.Background()
nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "nginx:1.17.6",
ExposedPorts: []string{"80/tcp"},
WaitingFor: wait.ForListeningPort("80/tcp"),
Files: []ContainerFile{
{
HostFilePath: "./testresources/hello.sh",
ContainerFilePath: "/copies-hello.sh",
FileMode: 700,
},
},
},
Started: false,
})
Parallel containers!
It’s now possible to create a collection of containers that start in parallel using the testcontainers.ParallelContainers exposed method. It takes a testcontainers.ParallelContainerRequest struct, which is backed by an array of ContainerRequest structs.
ctx := context.Background()
requests := testcontainers.ParallelContainerRequest{
{
ContainerRequest: testcontainers.ContainerRequest{
Image: "nginx",
ExposedPorts: []string{
"10080/tcp",
},
},
Started: true,
},
{
ContainerRequest: testcontainers.ContainerRequest{
Image: "nginx",
ExposedPorts: []string{
"10081/tcp",
},
},
Started: true,
},
}
res, err := testcontainers.ParallelContainers(ctx, requests, testcontainers.ParallelContainersOptions{})
if err != nil {
e, ok := err.(testcontainers.ParallelContainersError)
if !ok {
log.Fatalf("unknown error: %v", err)
}
for _, pe := range e.Errors {
fmt.Println(pe.Request, pe.Error)
}
return
}
for _, c := range res {
defer c.Terminate(ctx)
}
The way it’s implemented is by using a pool of 8 workers by default, which will handle the concurrent creation of the containers. If for any reasons you need a different number of concurrent workers, please set it up using the testcontainers.ParallelContainersOptions struct, which sets the initial number of parallel workers (goroutines) that a sync.WaitGroup will handle:
res, err := testcontainers.ParallelContainers(context.Background(), tc.reqs, testcontainers.ParallelContainersOptions{
WorkersCount: 2,
})
Custom SQL query when waiting for a SQL database
When dealing with desired states before running a test, it’s very convenient using the existing built-in wait strategies that testcontainers-go offer, i.e. an HTTP endpoint being available, a TCP port is listening, a string entry in a log file, etc. But, when working with SQL databases, it is handy to wait for a SQL result.
Before this release, the project used a fairly simple and hardcoded “SELECT 1” SQL query, but it could be the case it’s not what the tests need to wait for. For those cases, the wait.ForSQL strategy supports passing a custom SQL query in the WithQuery(string) method, so you are able to define the right query for your own database. Here you can find an example:
req := testcontainers.ContainerRequest{
Image: "postgres:14.1-alpine",
ExposedPorts: []string{port},
Cmd: []string{"postgres", "-c", "fsync=off"},
Env: env,
WaitingFor: wait.ForSQL(nat.Port(port), "postgres", dbURL).
Timeout(time.Second * 5).
WithQuery("SELECT 10"),
}
container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Fatal(err)
}
defer container.Terminate(ctx)
Bugs
Here you can find a list of the fixed bugs in this release:
- Container IP now inspects the NetworkSettings struct from the container response, and if it belongs to one network, it is retrieved (#491) by @leszko
- Container logs are now stripped (#454) by @funvit
- Compose v2 wait strategy now generates proper service names (#426) by @oriser
And there are more! Please visit the entire changelog here.
Do you want to contribute to the project? Please open an issue, submit a pull request, or interact with the community on our Slack. We are more than happy to help.
Beyond small fixes
Nobody wants to deal with breaking changes. None of us. But because the project has not reached a v1.0.0 version yet (which is on our radar), we considered taking some liberties to see progress when reviewing contributions.
There are exactly two breaking changes in this release, which are documented in the changelog. Let me shine some light on them:
First, the NewDockerClient method returns a new value representing the Testcontainers configuration (testcontainers.TestContainersConfig). This struct is obtained from the .testcontainers.properties file that lives inside the user’s home directory. This properties file contains specific configuration for testcontainers-go, including Docker daemon specifics, and will be automatically applied to all projects in your system that use testcontainers-go. If you are interested in the properties configuration, please check out the spec from the Java project.
- func NewDockerClient() (cli *client.Client, host string, err error) {
+ func NewDockerClient() (cli *client.Client, host string, tcConfig TestContainersConfig, err error) {
The properties configuration in the Go project has limited support for reading certain properties, including exactly two environment variables: TESTCONTAINERS_RYUK_CONTAINER_PRIVILEGED and TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE. But we as an OSS team are trying to align all the different Testcontainers implementation libraries in terms of configuration and, eventually, features.
The second and last breaking change relates to the way the library handles the execution of commands within containers. There is an Exec method that will contain the output logs of the command execution. Its signature has changed:
- Exec(ctx context.Context, cmd []string) (int, error) + Exec(ctx context.Context, cmd []string) (int, io.Reader, error)
