Challenges with end-to-end testing in microservices
More applications are moving towards a microservices architecture which offer a short lead time and fast, independent deployments of components. These components are typically owned by small, empowered autonomous teams using fully automated CI/CD pipelines. Because there are more components to test in a microservices architecture, it adds new complexity if teams continue to rely on too many end-to-end tests.
End-to-end tests can help you find potential bugs and give you a high level of confidence. But running end-to-end tests on every component can hamper your deployment frequency and lead time significantly, create coupling between services and teams, and make teams less autonomous.
Services are distributed and owned by different teams with their own backlogs and priorities. Who writes, implements, and maintains end-to-end tests that connect all these services together? Who manages testing environments shared by all these services and teams? Responsibilities can sometimes be unclear.
Scaling creates bottlenecks. And trying to add more teams or components to the system would result in exponential increase in the number of testing environments, build queue time, developers' idle time, and complexity in test data and environment management.
Contract testing to the rescue
Contract testing enables you to reduce reliance on expensive end-to-end tests and complex test environments.
It is a technique for testing an integration point by checking each application in isolation to ensure the messages it sends or receives conform to a shared understanding that is documented in a "contract".
For applications that communicate via HTTP, these "messages" would be the HTTP request and response, and for an application that used queues, this would be the message that goes on the queue.
Contracts are agreed between consumer and provider and are regularly verified against a real instance of the provider component. This effectively partitions a larger system into smaller pieces that can be tested individually in isolation of each other. This leads to simple, fast, and stable tests that also give the tester confidence to release.
Some end-to-end tests may still be required to verify the system when deployed in a real environment, but they should be kept to a minimum and preferably run in a staging environment which mirrors production.
Consumer-driven contract testing
Consumer-driven contract testing ensures that a provider is compatible with the consumer's expectations of it.
For an HTTP API and other synchronous protocols, this involves checking that the provider accepts the expected requests, and that it returns the expected responses. For a system that uses message queues, this involves checking that the provider generates the expected message.
Pact is an example of a code-first, consumer-driven contract testing tool used by developers. In Pact, the contract is generated during the execution of the automated consumer tests. A major advantage of this pattern is that only parts of the communication used by the consumer(s) get tested. This in turn means that any provider behavior not used by current consumers is free to change without breaking tests.
Unlike a schema or specification, which is a static artefact that describes all states of a resource, a Pact contract is enforced by executing a collection of test cases, each of which describes a single concrete request/response pair. Pact is, in effect, "contract by example". There are other consumer-driven contract testing tools, of course, but I wanted to give an example of how one works.
Provider contract testing
Sometimes the term contract testing is used to describe a standalone provider application instead of describing the integration context. In this instance, contract testing describes a method that dictates that a provider’s output adheres to the documented contract. One such case would be an OpenAPI doc.
This approach to contract testing helps avoid integration failures by syncing the provider code and documentation. In a standalone environment, it fails to give test-based assurance that the end user is calling the provider correctly. It also means that the provider may not be able to match consumers’ expectations. For these reasons, this approach is ineffective when it comes to avoiding integration bugs.
Conclusion
When managing a microservices architecture with numerous components—and their integration points— to test, consider how contract testing might help your team scale to meet the demands of having more components to test.
