Data Management Patterns In Microservice Architecture

This block of templates describes possible options for the interaction of microservices with databases.

Database Per Service Template

The main recommendation when moving to microservices is to provide each service with its data store so that there are no strong dependencies at the data layer. In this case, it is precisely the logical separation of data that is meant; that is, microservices can share the same physical database, but they must interact with a separate schema, collection or table.

Based on these principles, the Database Per Service pattern increases the autonomy of microservices and reduces communication between teams developing individual services.

The pattern also has disadvantages: it complicates the exchange of data between services and the provision of ACID transactional guarantees. The pattern should not be used in small applications – it is intended for large-scale projects with many microservices. Each team needs full ownership of resources to increase development speed and scale better.

The Database Per Service pattern is often contrasted with another pattern – Shared Database. It is an anti-pattern and implies the use of one data store by several microservices. It can be used at the initial stages of migration to a microservice architecture or in minimal applications developed by one team (2-3 microservices).

API Composition Template

This pattern is one of the possible options for obtaining data from multiple services after applying the Database Per Service pattern. He suggests creating a separate API that will call the necessary services that own the data and connect the results from them in memory. The pattern can be considered a variant of using another pattern – API Gateway, which we will discuss below.

The Composition API is the simplest way to get data from multiple sources, but it can lead to inefficient combining of large datasets in memory. An alternative solution is the following CQRS template.

Command Query Responsibility Segregation (CQRS) Pattern

This pattern suggests separating data change (Command) from data reading (Query). The CQRS template has two forms: simple and advanced.

In a simple form, different Object-Relational Mapping (ORM) models are used for reading and writing, but a common data store.

The extended form uses different data stores that are optimized for writing and reading data. Data is copied from storage for writing to storage for reading asynchronously. As a result, read storage lags behind write storage but is ultimately consistent. Extended CQRS is often used in conjunction with the Event Sourcing pattern discussed below.

The CQRS pattern provides high data availability, independent scaling of reading/write systems, and faster data reads in event-driven microservices. However, its use increases the complexity of the system and results in poor data consistency. The pattern is suitable for complex systems where reading data requires several storage requests or read and write operations have different loads.

Event Sourcing Template

Microservice architecture applications often use asynchronous communication methods: messages or events and ensure the atomicity of operations in such systems; it is recommended to use the Event Sourcing pattern.

In traditional databases, the object with the current state is stored directly. When using the Event Sourcing template, events that change their states are saved instead of objects. The final state can be obtained by reprocessing a series of events that have come over a specific time. Various services can replay events from the event store to calculate the appropriate state of their data stores. The CQRS pattern is usually used to implement an event store.

The pattern is recommended for use in highly scalable, event-driven transactional systems. It is not suitable for simple applications where microservices can synchronize data (for example, via an API).

Saga Template

This pattern is intended for managing distributed transactions in a microservice architecture, where the traditional Two-phase commit protocol (2PC) becomes difficult to implement.

When using the pattern, each local transaction updates data in the store within one microservice and publishes an event or message, which, in turn, starts the next local transaction, and so on. If the local transaction fails, a series of compensating transactions are performed that undo the changes of the previous transactions.

There are two main ways to coordinate transactions:

• Choreography. Decentralized coordination where each microservice listens for events/messages from another microservice, decides whether an action should be taken or not.
• Orchestration. Centralized coordination, in which a separate component (orchestrator) tells microservices what action to take next.

Using a template ensures transaction consistency in loosely coupled distributed systems but increases the difficulty of debugging. Saga is excellent for event-driven systems and NoSQL databases without 2PC support but is not recommended for SQL databases and systems with circular dependencies between services.

Also Read: Problems in Migration to Managed Kubernetes