HBase跨集羣遷移

背景：假設有hbase1和hbase2兩個集羣，現在需要將hbase1的表遷移到hbase2，而且不能停止服務的運行。

準備工作

1. 開啓hbase.replication功能
   設置hbase-site.xml中hbase.replication爲true
2. 開啓待複製集羣的Yarn計算，後續拷貝數據基於MR

開始遷移

1.hbase2集羣建表(click-count’在hbase1是已經存在的)

create table 'click-count','info'

2.建立兩個集羣的replication

在hbase1集羣執行

hbase(main):002:0> alter 'click-count',{NAME=>'info', REPLICATION_SCOPE=>'1'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 2.1320 seconds

hbase(main):003:0> add_peer '1',"172.16.201.7,172.16.201.8,172.16.201.9:2181:/hbase2"
0 row(s) in 0.0510 seconds

hbase(main):052:0> list_peers
 PEER_ID CLUSTER_KEY STATE TABLE_CFS
 1 172.16.201.7,172.16.201.8,172.16.201.9:2181:/hbase2 ENABLED
1 row(s) in 0.0080 seconds

3.暫停同步

在hbase1集羣執行
hbase(main):052:0> disable_peer '1'

4.建立快照

在hbase1集羣執行
hbase(main):006:0> snapshot 'click-count','click-count-snp'

hbase(main):007:0> list_snapshots
SNAPSHOT                                         TABLE + CREATION TIME
 click-count-snp                                 click-count (Mon Jul 08 16:20:46 +0800 2019)
1 row(s) in 0.0100 seconds

=> ["click-count-snp"]

5.跨集羣複製

在集羣hbase1執行（需要開啓MR，因爲hbase集羣一般都不開MR）:
[hadoopuser@nn1 bin]$ ./hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -Dmapreduce.map.memory.mb=4096 -Dmapreduce.map.java.opts=-Xmx3686m -snapshot click-count-snp -copy-to hdfs://172.16.201.42:8020/hbase -mappers 10 -bandwidth 10
2019-07-09 16:24:02,886 INFO  [main] snapshot.ExportSnapshot: Copy Snapshot Manifest
2019-07-09 16:24:03,193 WARN  [main] mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present.  Continuing without it.
2019-07-09 16:24:04,217 INFO  [main] snapshot.ExportSnapshot: Loading Snapshot 'click-count-snp' hfile list
2019-07-09 16:24:04,475 INFO  [main] mapreduce.JobSubmitter: number of splits:1
2019-07-09 16:24:04,603 INFO  [main] mapreduce.JobSubmitter: Submitting tokens for job: job_1562659148784_0006
2019-07-09 16:24:05,024 INFO  [main] impl.YarnClientImpl: Submitted application application_1562659148784_0006
2019-07-09 16:24:05,054 INFO  [main] mapreduce.Job: The url to track the job: http://nn1:8088/proxy/application_1562659148784_0006/
2019-07-09 16:24:05,054 INFO  [main] mapreduce.Job: Running job: job_1562659148784_0006
2019-07-09 16:24:11,179 INFO  [main] mapreduce.Job: Job job_1562659148784_0006 running in uber mode : false
2019-07-09 16:24:11,180 INFO  [main] mapreduce.Job:  map 0% reduce 0%
2019-07-09 16:24:16,233 INFO  [main] mapreduce.Job:  map 100% reduce 0%
2019-07-09 16:24:16,241 INFO  [main] mapreduce.Job: Job job_1562659148784_0006 completed successfully
2019-07-09 16:24:16,338 INFO  [main] mapreduce.Job: Counters: 37
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=158069
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=6998
		HDFS: Number of bytes written=6797
		HDFS: Number of read operations=5
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters
		Launched map tasks=1
		Other local map tasks=1
		Total time spent by all maps in occupied slots (ms)=11704
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=2926
		Total vcore-seconds taken by all map tasks=2926
		Total megabyte-seconds taken by all map tasks=11984896
	Map-Reduce Framework
		Map input records=1
		Map output records=0
		Input split bytes=201
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=71
		CPU time spent (ms)=740
		Physical memory (bytes) snapshot=325951488
		Virtual memory (bytes) snapshot=5925187584
		Total committed heap usage (bytes)=300941312
	org.apache.hadoop.hbase.snapshot.ExportSnapshot$Counter
		BYTES_COPIED=6797
		BYTES_EXPECTED=6797
		BYTES_SKIPPED=0
		COPY_FAILED=0
		FILES_COPIED=1
		FILES_SKIPPED=0
		MISSING_FILES=0
	File Input Format Counters
		Bytes Read=0
	File Output Format Counters
		Bytes Written=0
2019-07-09 16:24:16,341 INFO  [main] snapshot.ExportSnapshot: Finalize the Snapshot Export
2019-07-09 16:24:16,348 INFO  [main] snapshot.ExportSnapshot: Verify snapshot integrity
2019-07-09 16:24:16,378 INFO  [main] snapshot.ExportSnapshot: Export Completed: click-count-snp

6.目標集羣hbase2恢復

在集羣hbase2執行:
hbase(main):140:0> disable 'click-count'
0 row(s) in 2.2270 seconds

hbase(main):141:0> restore_snapshot 'click-count-snp'
0 row(s) in 0.7720 seconds

hbase(main):142:0> enable 'click-count'
0 row(s) in 1.3340 seconds

7.遷移完成，通知業務方擇期遷移