tags : Distributed Systems, Infrastructure, Message Passing, Zookeeper
What
- Publishing with Apache Kafka at The New York Times | Confluent
- Kafka is not a Database
- Can Apache Kafka Replace a Database? - Kai Waehner
Gotchas
- Kafka doesn’t scale well with partitions, throughput falls over quickly especially if using replication and acks.
- If each user has a topic your partitions are unbounded. You have at least one partition per user.
- Instead, I’d use a single notification topic, set a reasonable number of partitions on it and partition by user id.
Resources
- Webcomic: https://www.gentlydownthe.stream/#/3
- Windowing in Kafka Streams
- Apache Kafka Beyond the Basics: Windowing | Hacker News : See comments, has good learning roadmap