Use Case Reference Architecture - #5: Event-Driven Reference Architecture - Kappa

[Anand_Rao]

Invest in less expensive hardware and solve multi-layered, Lambda architecture redundancy by replaying data instead of maintaining two code bases (batch and speed layers) to process unique events continuously in real-time while meeting standard quality of service.

The Kappa architecture solves the redundant part of the Lambda architecture. It is designed with the idea of replaying data. Kappa architecture avoids maintaining two different code bases for the batch and speed layers. The key idea is to handle real-time data processing, and continuous data reprocessing using a single stream processing engine and avoid a multi-layered Lambda architecture while meeting the standard quality of service. The Kappa architecture is used with less expensive hardware to process unique events occurring continuously in real-time.

[Anand_Rao]

The architecture comprises the following components:

Data is ingested in real-time using change data capture for real-time data replication without impairing production system performance.

Analytics is used to discover, interpret, and communicate meaningful patterns in data to apply toward effective decision-making.

Monitor data ingestion tasks with a single pane of glass view.

Orchestrate data ingestion tasks based on pre-set conditions or calculations.

APIs to automate and integrate with other monitoring and orchestration applications.

Event-Driven Architecture - Kappa