It has been some time since we published an update on Testcontainers for Go, and it’s time to share the recent updates in terms of Community and functionality with you.
We strongly believe in Open Source; and with that, community comes first.
Community
I’ll start by telling you a story about building a Community across different OSS projects. What would happen if a well-known OSS project such as OpenTelemetry Collector discovered they were missing some feature in the Testcontainers for Go library when they started experiencing flakiness in their test suite?
Because OSS is great, they only needed to ping us in an issue in their repository, and we discussed how to resolve it, resulting in a pull request to our library that has been available since our latest 0.18.0 release. This is collaboration at its finest; OpenTelemetry Collector receives the feature, our whole community benefits, and everybody is happy.
In another community highlight, we’d like to emphasize the addition of two example modules, MongoDB and Consul, authored by two community members as their first contribution to the project. These are especially important because just recently we created a single-purpose tool for simplifying the lives of our users when adding new modules. And the results are pretty amazing — two contributions of modules for very popular technologies!
Another Community topic that we as an Open Source project value highly, is the presence at conferences and meetups. It’s an opportunity to interact with other like-minded folks and where we can share our love for testing done-right. So, if you are in Madrid-Spain around April 21st, Manu will be talking about integration testing and Go with Testcontainers at CommitConf.
And finally let’s now mention a few metrics from our amazing Community, obtained using the following query:
- 166 commits have been merged in total since the 0.14.0 release, back in September 2022.
- 29 different authors have committed to the repository.
- 29 different developers submitted an issue.
- 31 different developers submitted a review.
- The average number of lines per commit is 338.10 (it was 44.43 back in September).
- The average number of lines per commit and file is 0.34 (it was 0.03 back in September).
- 53 issues were created vs. 48 that were closed. There are still 36 open.
- 251 pull requests were created vs. 262 that were closed. There are still 10 open.
- 1 Testcontainers Live! session about Testcontainers for Go was recorded. You can check it out here.
testcontainers-go v0.18.0 is out!
Let’s talk about some of the most important things and new features you’ll may discover when using the latest release of Testcontainers for Go.
Waiting for SQL query results now support remote Docker host (since v0.15.0)
It’s now possible for the SQL wait strategy to work correctly even if you are using a remote Docker host (like you know from other strategies such as the HTTP wait strategy). This was a highly requested feature, and it was finally implemented by a community member.
Integration with Testcontainers Cloud (since v0.15.0)
Testcontainers Cloud is one of the easiest ways to run Testcontainers tests at any scale, both in development environments and CI. We recently announced the public beta, and if you are using Testcontainers for Go you can, since v0.15.0, use it with Testcontainers Cloud. For example, it allows you to forget about the containers running on your machine, as they are instead transparently run in the Cloud, without consuming your local CPU and memory.
Improvements on deadlines and timeouts when using multiple wait strategies (since v0.16.0)
We are very proud of this new feature as it was entirely contributed by a community member using some great OSS best practices. He created a design doc in the form of a Github discussion beforehand, validated the ideas with the comments there, submitted a pull request, and after a few iterations, we merged it into the project! 🚀
Docker Compose support using the native Go bindings (since v0.16.0)
Do you have a very old legacy support for docker-compose files? With Testcontainers for Go you can control its lifecycle with a separate Go module, directly using Go and without shelling out to the local Docker Compose binary that might (or might not) be installed in your machine.
Since v0.17.0, Compose code is treated as a separated Go module that you must import separately. Check out the docs for the examples and how to use Docker compose support with Testcontainers.
Modules support and examples! (since v0.17.0)
Loving a technology that is used in your tests? Now it’s possible to create a Go module and thereby sharing code on how to easily use that technology with Testcontainers for Go. You’ll be able to share it in two different manners: 1) By creating a simple example module, which does not expose an API but provides shareable code snippets, or 2) by creating a real Go module that exposes a public API in order to be consumed as a dependency.
We have provided a code generation tool that creates all the scaffolding and boiler plate for you, including integration with Dependabot, a Github action workflow for running the tests for the module (because we know tests are important!), the API of the module to start a container, and much more. You can find more on this topic in the docs for the code generation tool.
We have already created a few examples for you to just copy and use them including, among others: Google Cloud emulators (BigTable, Datastore, Firestore, PubSub and Spanner), Apache Pulsar, or the amazing Toxiproxy example module, which will allow you to inject errors in the networks your services live in and indulge in the chaos engineering madness. Please check them out and share your feedback with us 🙏
LocalStack module (since v0.18.0)
As a result of the above, we have created the LocalStack module as a real Go module. Therefore, it exposes a public API that allows you to start a LocalStack container for tests that require AWS services… but locally. Yes, locally! All the benefits of testing against a Cloud provider, but on your own machine, without increasing the cloud bill or messing around with staging environments.
With the LocalStack module creating and configuring a LocalStack container becomes a one-liner:
container, err := localstack.StartContainer(ctx, localstack.NoopOverrideContainerRequest)
If you use AWS or just you want to know more about this module, please check out its documentation.
ContainerRequest modifiers (since v0.18.0)
The ContainerRequest abstraction originally represented the first and only layer of customisation of containers, and it’s super helpful for defining how the containers should look like, but over the course of time, it grew in a non-organic manner, thereby almost creating a 1:1 mapping with the Docker types. At the same time, defining more advanced configurations, such as specific network settings (see this interesting setup for using a fixed IP, with a specific context in which it was required) was not possible without changing how the library worked.
In response to that, and following in the footsteps of what the other languages already do (.NET and Java) we have added a second layer of customisation in the form of “modifiers” to the container request. These modifiers, which are applied right before calling the Docker API itself, allow direct access to the Docker types in order to craft those advanced configurations. Therefore, users of Testcontainers for Go will now be able to customise the Config, HostConfig and EndpointSettings Docker types for more advanced use cases using this second layer of customisation. We are deprecating the existing fields in the container request that were exposing these advanced settings to the first layer of customisation.
Now that the modifiers are in place, we are reducing the size of this first layer, reducing the possible misuses of the public API for the majority of our users. At the same time, the second layer of customisation that the modifiers provide, opens the gate to those users who need more advanced configurations, and perfectly know what they are doing.
Go modules and LocalStack
We want to dedicate a special section for the new LocalStack Go module to demonstrate how simple it is creating tests against LocalStack. First, you have to import it into your project:
go get github.com/testcontainers/testcontainers-go go get github.com/testcontainers/testcontainers-go/modules/localstack
Then, you have to start a container for LocalStack using the following function, which is exported by the new module:
// 1. passing a NOOP override using nil container, err := localstack.StartContainer(ctx, nil) // 2. passing a NOOP override using types container, err := localstack.StartContainer(ctx, localstack.NoopOverrideContainerRequest)
And voilà! You have a regular container in the same way as using Testcontainers for Go using the more generic mechanisms.
In those cases where you need to add specific configuration to LocalStack, such as the services you are interested in, or you need to change the AWS region to work on, you can leverage the OverrideContainerRequest type, which provides you a way to override the original container request:
container, err := localstack.StartContainer(
ctx,
localstack.OverrideContainerRequest(testcontainers.ContainerRequest{
Image: "localstack/localstack:1.3.1", // changing the version is easy
Env: map[string]string{
"AWS_ACCESS_KEY_ID": accesskey,
"AWS_SECRET_ACCESS_KEY": secretkey,
"AWS_SESSION_TOKEN": token,
"AWS_DEFAULT_REGION": region, // override AWS region
"SERVICES": "s3,sqs,cloudwatchlogs,kms", // define AWS services
}},
),
)
Then you can start writing tests agains the different AWS services you enabled. As an example, you can test creating a S3 bucket and putting an object into it:
// Create Bucket
outputBucket, err := s3Client.CreateBucket(ctx, &s3.CreateBucketInput{
Bucket: aws.String(bucketName),
})
require.Nil(t, err)
assert.NotNil(t, outputBucket)
// put object
s3Key1 := "key1"
body1 := []byte("Hello from localstack 1")
outputObject, err := s3Client.PutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(s3Key1),
Body: bytes.NewReader(body1),
ContentLength: int64(len(body1)),
ContentType: aws.String("application/text"),
ContentDisposition: aws.String("attachment"),
})
require.Nil(t, err)
assert.NotNil(t, outputObject)
Depending on which version of the AWS Go SDKs you are using, v1 or v2, you’ll acquire the S3 client in a different manner. We explain how to get both clients here.
Finally, I’d like to share that, like we can use LocalStack within Testcontainers for Go, you can do exactly the same with the .NET, Java implementations of Testcontainers! While working on the module, we have experienced a great team collaboration, aligning all implementations in terms of behaviour and expectations. For example, the default wait strategy originally was different for each language, and we all agreed on checking for the health HTTP endpoint of LocalStack simplifying the experience for the whole Testcontainers community!
All in all, the recent improvements in Testcontainers for Go highlight it as an active open source project focusing not only on features and quality of life improvements, but also a larger ecosystem of modules.
From a recently-introduced tool simplifying creating new modules, to community and AtomicJar’s contributions of modules for popular technologies, like Consul, MongoDB, LocalStack, the number of technologies you can use in your tests with just a few lines of code grows every week! If you’re interested in a particular integration, join us!
