How Microservices and Event-Driven Architectures Are Related
When starting with Microservices, one of the first questions is how to maintain consistency of the overall systems despite all Microservices being segregated from each other. Depending on the requirements, the segregation can sometimes be omitted at the persistence level.
However, if the goal is to have different teams working on specific microservices so that each one be independently deployable, a good way to keep consistency is to implement an Event-Driven Architecture.
Microservices and Persistence
Deciding how to persist the data in all microservices, it’s not only about if all microservices will share the same database or if each one will have private tables or a separate schema/database. It’s much more than that.
In some occasions, the data will not be accessed the same way because the volume or the complexity might be too high or only consists in reads or in heavy writes or both.
Identifying how the data should be accessed will help you make an informed decision on which kind of persistence best fits the need of each specific microservice. An example could be retrieving the historic data of all purchases by a customer. You can have a NoSQL database that can supply that information quickly, instead of using a relational database that is being used by another microservice that joins information from multiple tables. This approach of having different data sources will lead us to have a Polyglot persistence.
As you can see, there are a lot of benefits in letting each microservice manage its own persistence. But what happens in the bounded context when different microservices need to share the same data? Can we ensure Atomic transactions?
Considering everything so far, how can we keep our microservice persistence independent and also keep the data consistent? There is when Event-Driven architectures come in to play.
Event-Driven Architectures
The idea behind event driven architecture is to publish an event when something important happens. Then a subscriber to that event can take the information needed to update their state, which allows the data to stay consistent and synchronize. It’s important to specify that you will have eventual consistency as time passes between the moment when the event is published and when the subscriber gets it.
There are different ways to do this:
- Transaction log tailing
- Application created events
- Event sourcing
Transaction Log Tailing
This approach is on the database level. It basically mines the transaction log of the database to publish events. With this approach, it’s not needed to make changes at the application level, but it is limited by the type of database it can use. With this approach, it is also impossible to have a Polyglot persistence.
Application created events
In this case, the events are at an application level. After a change in the database and an event is published, no notification about the change is sent to their subscribers. Then, the subscriber gets the event and updates their state. In this case, a problem may arise with the atomicity of the transactions. Let’s imagine a scenario where a change was committed to the database, then an event was published to the broker. However, for some reason it got lost, which will cause an inconsistency, because the changes in the microservice persistence were not propagated. This can be solved by keeping track of the event to know if was propagated or not and act accordingly.
Event sourcing
As Martin Fowler wrote it, “The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application state itself.”
With this approach, it’s possible instead of persisting the entity, to persist only the events, so that the state of the entities can be obtained by replaying all events.
In this case, an Event Store will act as a broker in charge of communicate when a new state is persisted to the appropriated subscribers.
With event sourcing, you have a lot of benefits because it allows you to track the changes in real time, which will let you avoid using a traditional audit. As is expected, it has its drawbacks. Imagine a system that will generate a lot of events, that will force to generate snapshots to allow easily recreate the current state.
You may want to only use event sourcing on a part of your system. Use it only where it makes sense instead of making your entire system rely on it.
Conclusion
There is not an exact recipe to keeping consistency in a systems based on Microservices, but Event-Driven architectures can help accomplish an eventual consistency at the bottom line. The decision of which approach to follow will depend on the system’s requirements and business logic.
