Article

LinkedIn has transitioned its service discovery platform from Apache ZooKeeper to a new architecture utilizing Apache Kafka and the xDS protocol. This upgrade addresses capacity limits reached by 2025 due to scaling issues, including high latency and session timeouts from write spikes and read storms.

The new system employs an eventual consistency model, separating the write path through Kafka from the read path via an Observer service. The Observer can maintain 40k client streams and process 10k updates per second, operating independently in data centers while allowing client connections for failover.