As an operations strategists, I work with teams across Discover to help them creatively solve their problems. As teams at Discover move their products and services onto a modern infrastructure, I have heard success stories about how this move to modern tooling has removed manual effort from employees' day-to-day work.
Integrations testing is an area where the benefit of modernization is seen and felt. This article shares a common pain point—sharing a single database instance between teams— and some tools that could be used to solve this challenge.
Single database challenges
An existing pain point for engineers is the usage of a single database instance by multiple teams which contributes to pipeline failures and workflow disruptions due to high rates of instability. Multiple teams running test cases against a single database instance can create unexpected changes in data, data types, schemas, stored procedures, and the like. The result is that engineers are spending excessive time troubleshooting vague historical patterns and errors. Sound familiar?
Tools to meet and progress past database challenges
The following collection of tools and methods can help alleviate these pain points.
Embedded Kafka
Embedded Kafka is an in-memory Kafka instance you can run tests against locally. An in-memory Kafka broker's life span is limited to the life of a test, eliminating the need to depend on any external Kafka server or external services.
- Check out the Embedded Kafka documentation
- Read more about Embedded Kafka in the article, Testing with Embedded Kafka.
Embedded Postgres
Embedded Postgres is a fast-processing, standalone, and isolated database for running tests against which eliminates dependencies on real-time environmental databases. This tool enables teams to shift testing left, ensuring bugs are caught earlier in the software development lifecycle. Read more about Embedded Postgres in Database Testing for Spring Boot Apps using Embedded Postgres.
Flyway
Flyway helps you implement automated and version-based database migrations. It allows you to define the required update operations in a SQL script or as Java code. You can then run the migration from a command line client or automatically as part of your build process or integrated into your Java application. Read more about Flyway.
Conclusion
By taking advantage of modern tooling and infrastructure, engineering teams can reduce manual effort and innovate and simplify everyday processes.
