In modern event driven microservice based applications where services are loosely coupled, communication between the decoupled services requires integration code. Maintaining the integration code is a challenge. Amazon EventBridge Pipes is a new feature of Amazon EventBridge that makes it easier to build event-driven applications. It provides a simple & and cost-effective way to create point-to-point integrations between event producers and consumers. Amazon EventBridge Pipes helps to reduce the amount of integration code that needs to be maintained for event-driven applications.
An EventBridge pipe setup consists of 4 steps – Source, Filtering, Enrichment & Target. Filtering and Enrichment are optional.
The source can receive events from various AWS services like SQS, Kinesis, DynamoDB, etc… In the filtering step filter patterns can be configured to filter the events that are passed on to the enrichment or target step. This helps to reduce the cost by filtering out unnecessary events that need not be processed. In the enrichment step, data from the source can be enhanced before sending it to the target. Built-in transformations are available to enrich data or AWS Lambda, API Gateway, or Step Functions can be used to perform advanced transformations on the data. Enrichments are invoked synchronously. Pipes can send the event/data to targets such as AWS service (AWS Lambda, API Gateway, ECS Cluster, CloudWatch Log, Kinesis Stream, etc..) or an API destination. Transformers can be written in the Target step to define how data needs to be sent to the targets. EventBridge supports invoking the targets synchronously or asynchronously based on the target.
Pipes can be activated or deactivated to process the events based on the need. CloudTrail and CloudWatch can be used to monitor the EventBridge Pipes. CloudTrail tracks the EventBridge Pipe invocations & their details. The health status of the Pipes can be monitored using various metrics supported by CloudWatch. Overall, EventBridge Pipes provides a simple, fast, cost-efficient way to configure advanced integrations with enhanced security, reliability, and scalability out of the box.