Đức Nguyễn • 8/6/2022

[KAFKA] How to chose the number of partitions?

This article explains how to determine the number of partitions for a Kafka topic by considering factors like expected write/read throughput, consumer processing limits per partition, broker resources (disk and network), and the challenges of adding partitions later when using key-based messaging. It provides a practical formula: divide target throughput by expected consumer throughput per partition to estimate the minimum partitions needed, using a 1 GB/sec example requiring 20 partitions for 50 MB/sec consumers.

0 comments

#throughput #Kafka #Partitions