One of the better outcomes of Digital Business has been that organizations are becoming outward-facing and more inclusive of their customers’ needs, reactions and feedback. As a consequence, their customer facing solutions are transforming into fully inclusive, engagement-driven ecosystems. This shift is driving a transition from retrospective data-centric thinking to event-centric thinking. Gartner comments in their May 2017 report, Champion Event Thinking to Excel at Digital Business:
“ By 2020, participation in 80% of new business ecosystems will require support for event processing.”
The concept of event-centric thinking came about in the early 2000’s initially around Message Queues and has evolved as the need for speed and immediate responsiveness drove growing interest in Event-Driven Architecture (EDA).
At the Core of EDA is Event Stream Processing
With EDA, event flows are implemented. Flows consist of a set of steps through which the events are collected from event sources, enhanced, analyzed and dispatched to event handlers. See the graphic at the top of this post to visualize how event flows work.
To meet the needs of Digital Business, EDA requires sophisticated, fast and scalable event analysis not found in traditional message-oriented middleware or traditional software architectures around the batch-oriented Data Warehouse. To overcome the speed limitations of these older approaches, the concept of Event Stream Processing (ESP) emerges as central to EDA. Think of Event Stream Processing as more of an umbrella term, comprised of both Event Stream Analytics and Data Integration. Within ESP, Stream Analytics products are primarily used to trigger responses. Stream Data Integration is typically used to accumulate database repositories of events to support analytics queries. The following table gives you a breakdown of corresponding functionality.
Inherently there is more interest in Stream Analytics—triggering responses as an enabler of real-time responsiveness and situational awareness—since these are key components of Digital Business.
EDA Enables Real-Time Actionable Information
At Daitan, what we like about EDA is that it enables companies to leverage events into real-time actionable information.
An “ Event-Driven Architecture (EDA), is a software architecture pattern promoting the production, detection, consumption of, and reaction to events.” [1]
EDA supports critical trends in the transformation to Digital Business, which produces tangible business value. For example, enabling companies to efficiently operate in complex ecosystems (including customers, partners, vendors and other third parties); provide real-time responsiveness; make context-informed decisions; be agile and scale. Examples highlighted by Gartner:
- Responsive Customer Engagement: Events with added situational awareness enable context enriched customer experiences.
- Capitalizing on Business Moments: Coalesced groups of events indicate significant business moments. Companies can respond in (near) real time.
- Internet of Things: Events from sensors are the single most important foundation of IoT. Event-Driven Architecture is the best option.
- Artificial Intelligence and Machine Learning: Many of the applications involve loops of event collection, analysis and response.
Our latest white paper titled (no surprise), “Event-Driven Architecture: How to Prepare, Migrate and Scale EDA” takes a deep dive into the business drivers, adoption and best practices, which could be helpful if you’re considering EDA as part of your development roadmap.
Are There Times When EDA Does Not Fit?
Yes, there are. Although EDA is widely applicable and can become a business differentiator in many cases, it is not recommended for all types of problems:
- In some situations, REST-based, synchronous communication is faster to implement and less expensive to maintain – this usually applies to simpler environments, with very few producers and consumers, not expected to change much or grow
- When on-the-fly data processing is not needed, but message routing can be complex, traditional Message Queue Middleware (MQM) tools can do the job of facilitating communication between multiple producers and consumers without the infrastructure needed for a full EDA implementation.
- When data needs to be quickly distributed to multiple consumers, but without complex routing rules, a scalable pub-sub tool such as Kafka alone can do the job. Consider EDA when you need to do some processing on events, at high speed, and distribute it to a large or even unanticipated list of consumers.
EDA Adoption is Occurring in Multiple Industries
In May 2017, Gartner published, Champion Event Thinking to Excel at Digital Business which emphasizes the value of adopting an event-driven mindset and the shift toward event-centric architecture with projections such as, “by 2020, 50% of new user-facing applications will be event-triggered.”
The movement described by Gartner is actively occurring across multiple industries, including ride sharing, communications, ore mining, food delivery, and financial services to name a few. We take a closer look at each of these in the paper to explain their approach.
- Uber – Ride sharing
- Deliveroo – Restaurant food delivery for high-end dining-in
- Monzo Bank — The Cloud-first, mobile bank
- Centrica — Energy services
We decided to publish Event-Driven Architecture, because our client inquiries led to many discussions around solving for both, data and event processing architecture needs. In the paper, we coalesced our experience and expertise into a discussion around the design considerations essential to migrating toward an event-driven architecture, several use cases across multiple industries and guidelines for migration based on best practices and our own experience.
We invite you to check out our paper on Event-Driven Architecture, and contact us if you are considering migrating to EDA and would like to discuss its implications on your roadmap and business. Our Innovation Group can help bring insights around adoption, tools, and best practices, so you put a solid plan in place to move forward.
Footnotes
[1] K. Mani Chandy Event-Driven Applications: Costs, Benefits and Design Approaches, California Institute of Technology, 2006
