自定義分區分配和遷移
分區重新分配工具還可用於選擇性地將分區的副本移動到特定的代理集。當以這種方式使用時,假設用戶知道重新分配計劃並且不需要工具生成候選重新分配,有效地跳過 - 生成步驟並直接移動到--execute步驟
例如,以下示例將主題foo1的分區0移動到代理5,6,將主題foo2的分區1移動到代理2,3:
第一步是在json文件中手工製作自定義重新分配計劃:
1
2
|
> cat custom-reassignment.json
{ "version" :1, "partitions" :[{ "topic" : "foo1" , "partition" :0, "replicas" :[5,6]},{ "topic" : "foo2" , "partition" :1, "replicas" :[2,3]}]}
|
然後,使用帶有--execute選項的json文件來啓動重新分配過程:
1
2
3
4
五
6
7
8
9
10
11
12
13
14
|
> bin /kafka-reassign-partitions .sh --zookeeper localhost:2181 --reassignment-json- file custom-reassignment.json --execute
Current partition replica assignment
{ "version" :1,
"partitions" :[{ "topic" : "foo1" , "partition" :0, "replicas" :[1,2]},
{ "topic" : "foo2" , "partition" :1, "replicas" :[3,4]}]
}
Save this to use as the --reassignment-json- file option during rollback
Successfully started reassignment of partitions
{ "version" :1,
"partitions" :[{ "topic" : "foo1" , "partition" :0, "replicas" :[5,6]},
{ "topic" : "foo2" , "partition" :1, "replicas" :[2,3]}]
}
|
--verify選項可與該工具一起使用,以檢查分區重新分配的狀態。請注意,相同的expand-cluster-reassignment.json(與--execute選項一起使用)應與--verify選項一起使用:
1
2
3
4
|
> bin /kafka-reassign-partitions .sh --zookeeper localhost:2181 --reassignment-json- file custom-reassignment.json --verify
Status of partition reassignment:
Reassignment of partition [foo1,0] completed successfully
Reassignment of partition [foo2,1] completed successfully
|
增加複製因子
增加現有分區的複製因子很容易。只需在自定義重新分配json文件中指定額外副本,並將其與--execute選項一起使用,以增加指定分區的複製因子。
例如,以下示例將主題foo的分區0的複製因子從1增加到3.在增加複製因子之前,分區的唯一副本存在於代理5上。作爲增加複製因子的一部分,我們將添加更多副本經紀人6和7。
第一步是在json文件中手工製作自定義重新分配計劃:
1
2
3
|
> cat increase-replication-factor.json
{ "version" :1,
"partitions" :[{ "topic" : "foo" , "partition" :0, "replicas" :[5,6,7]}]}
|
然後,使用帶有--execute選項的json文件來啓動重新分配過程:
1
2
3
4
五
6
7
8
9
10
|
> bin /kafka-reassign-partitions .sh --zookeeper localhost:2181 --reassignment-json- file increase-replication-factor.json --execute
Current partition replica assignment
{ "version" :1,
"partitions" :[{ "topic" : "foo" , "partition" :0, "replicas" :[5]}]}
Save this to use as the --reassignment-json- file option during rollback
Successfully started reassignment of partitions
{ "version" :1,
"partitions" :[{ "topic" : "foo" , "partition" :0, "replicas" :[5,6,7]}]}
|
--verify選項可與該工具一起使用,以檢查分區重新分配的狀態。請注意,相同的increase-replication-factor.json(與--execute選項一起使用)應與--verify選項一起使用:
1
2
3
|
> bin /kafka-reassign-partitions .sh --zookeeper localhost:2181 --reassignment-json- file increase-replication-factor.json --verify
Status of partition reassignment:
Reassignment of partition [foo,0] completed successfully
|
您還可以使用kafka-topics工具驗證複製因子的增加:
1
2
3
|
> bin /kafka-topics .sh --zookeeper localhost:2181 --topic foo --describe
Topic:foo PartitionCount:1 ReplicationFactor:3 Configs:
Topic: foo Partition: 0 Leader: 5 Replicas: 5,6,7 Isr: 5,6,7
|