ureplicator複製topic

1.首先你得去git拉replicator的代碼。然後編譯。編譯之後啓動。

1.首先我有2套zk和kafka

源kafka 是172.30.3.120

目標kafka是 172.30.3.210

a.啓動第一步:啓動controller

/home/datacanvas/huml/uReplicator/uReplicator-Distribution/target/uReplicator-Distribution-pkg/bin/start-controller.sh
-mode haha 
-enableAutoWhitelist false 
-port 9000 -refreshTimeInSeconds 10
-srcKafkaZkPath 172.30.3.120:2181 
-zookeeper 172.30.3.120:2181  //存儲ureplicator的信息
-destKafkaZkPath 172.30.3.210:2181 
-helixClusterName uReplicatorDev01

b.啓動 work 

   需要修改配置文件。

  1>修改 helix.properties

zkServer=172.30.3.120:2181  //與controller啓動時填寫的zk信息相同
instanceId=testHelixMirrorMaker01
helixClusterName=uReplicatorDev01  //需要與controller啓動的clusterName一樣
enableAutowhitelist=false
port=9000
srckafkaZkpath=172.30.3.120:2181
destKafkaZkpath=172.30.3.210:2181

2> consumer.properties

zookeeper.connect=172.30.3.120:2181  //源kafka
//source zk
zookeeper.connection.timeout.ms=30000
zookeeper.session.timeout.ms=30000
#consumer group id
group.id=kloak-mirrormaker-test
consumer.id=kloakmms01-sjc1
partition.assignment.strategy=roundrobin
socket.receive.buffer.bytes=1048576
fetch.message.max.bytes=8388608
queued.max.message.chunks=5
auto.offset.reset=smallest

3> product.properties

//源ip:host,目標 ip:host
bootstrap.servers=172.30.3.120:9092,172.30.3.210:9092

target kafka
client.id=kloak-mirrormaker-test
//source kafka 和producer.type=async
compression.type=none
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer

啓動:

/home/datacanvas/huml/uReplicator/uReplicator-Distribution/target/uReplicator-Distribution-pkg/bin/start-worker.sh
--consumer.config
/home/datacanvas/huml/uReplicator/config/consumer.properties
--producer.config
/home/datacanvas/huml/uReplicator/config/producer.properties 
--helix.config
/home/datacanvas/huml/uReplicator/config/helix.properties

官方文檔例子:

https://github.com/uber/uReplicator/wiki/uReplicator-User-Guide

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章