Enterprise Event-Driven Data Platform

Zero-loss, real-time event infrastructure processing enterprise data streams at sub-minute latency.

The problem

Legacy batch pipelines introduced hours of latency between event capture and downstream availability, and failure handling risked silent data loss at enterprise scale.

Impact

Data latency: Hours → sub-minute
Event loss: Zero-loss processing
Production DAGs managed: 70+

Approach

1
Built RESTful APIs for inventory, cart, and order management secured with JWT/OAuth2.
2
Integrated Stripe and PayPal for multi-method transactions.
3
Used Celery and Redis for asynchronous order confirmation, refunds, and inventory updates.
4
Shipped an ML-powered recommendation engine driven by browsing and purchase history.

Stack

GCP
Pub/Sub
Dataflow
Cloud Composer
Airflow
BigQuery