在前面的文章《Kafka集群扩展以及重新分布分区》中,我们将介绍如何通过Kafka自带的工具来增加Topic的分区数量。本文将简单地介绍如何通过Kafka自带的工具来动态增加Tpoic的副本数。首先来看看我们操作的Topic相关的信息
[iteblog@www.iteblog.com ~]$ kafka-topics.sh --topic iteblog --describe --zookeeper www.iteblog.com:2181 Topic:iteblog PartitionCount:2 ReplicationFactor:1 Configs: Topic: iteblog Partition: 0 Leader: 2 Replicas: 2 Isr: 2 Topic: iteblog Partition: 1 Leader: 3 Replicas: 3 Isr: 3
从上面的输出信息可以看出,iteblog主题有两个分区,副本数为1。假如我们需要将这个主题的副本数增加到3,以便提供高可用性。
动态地增加相关主题的副本数非常的简单,同样是使用kafka-reassign-partitions.sh
工具来执行,所以我们在操作之前得准备一个Json文件,里面详细地记录着每个分区的副本所在机器等信息,具体如下:
{ "version": 1, "partitions": [ { "topic": "iteblog", "partition": 0, "replicas": [ 2, 1, 3 ] }, { "topic": "iteblog", "partition": 1, "replicas": [ 3, 2, 1 ] } ] }
上面已经把主题的副本数变成3了,我们把这个Json数据存放在名为replication.json的文件中。现在我们需要执行这个配置文件,如下:
[iteblog@www.iteblog.com ~]$ kafka-reassign-partitions.sh --zookeeper www.iteblog.com:2181 --reassignment-json-file replication.json --execute Current partition replica assignment {"version":1,"partitions":[{"topic":"iteblog","partition":0,"replicas":[2]},{"topic":"iteblog","partition":1,"replicas":[3]}]} Save this to use as the --reassignment-json-file option during rollback Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"iteblog","partition":0,"replicas":[2,1,3]},{"topic":"iteblog","partition":1,"replicas":[3,2,1]}]}
同样我们可以使用verify参数来检查上面的执行计划是否完成,如下:
[iteblog@www.iteblog.com ~]$ kafka-reassign-partitions.sh --zookeeper www.iteblog.com:2181 --reassignment-json-file replication.json --verify Status of partition reassignment: Reassignment of partition [iteblog,0] is still in progress Reassignment of partition [iteblog,1] is still in progress [iteblog@www.iteblog.com ~]$ kafka-reassign-partitions.sh --zookeeper www.iteblog.com:2181 --reassignment-json-file replication.json --verify Status of partition reassignment: Reassignment of partition [iteblog,0] completed successfully Reassignment of partition [iteblog,1] completed successfully
好了,我们可以看出上面的执行计划已经完成。我们来查看一下现在iteblog主题的相关信息:
[iteblog@www.iteblog.com ~]$ kafka-topics.sh --topic iteblog --describe --zookeeper www.iteblog.com:2181 Topic:iteblog PartitionCount:2 ReplicationFactor:3 Configs: Topic: iteblog Partition: 0 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic: iteblog Partition: 1 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1
可以看出,iteblog主题的副本数已经变成了3,好了收工。
本博客文章除特别声明,全部都是原创!原创文章版权归过往记忆大数据(过往记忆)所有,未经许可不得转载。
本文链接: 【Kafka如何动态增加Topic的副本(Replication)】(https://www.iteblog.com/archives/1614.html)