13:40 - 14:40
Talk (60 min)
The Metamorphosis: How Kafka helped us migrate our 10Gb/s ingest pipeline to the cloud
Akamai’s CSI (Cloud Security Intelligence) group is responsible for processing massive amounts of security events arriving from our edge network, making it accessible by various internal consumers powering customer-facing products.
In this talk, we’ll visit the reasons for migrating our 10Gb/s ingest pipeline to the cloud, examine our new architecture and its benefits, and focus on how Kafka played a key role in this transformation.
Whereas Kafka is mostly used either for operational messaging or as a streaming storage for actual events, this session will provide you with a unique way of leveraging many of Kafka’s advantages in a big data pipeline.