Goldengate之Mysql到Mysql的同步

使用Goldengate完成Mysql到Mysql的數據同步，源庫的o2m-mc庫同步到目標庫的gmdc庫，表名GMQDSJSP

1、初始化

1）目標庫創建數據庫

mysql> create database gmdc;

2）導入數據 

將源庫表GMQDSJSP數據導出，然後導入到目標庫中

mysqldump -uroot -p123456  --single-transaction --flush-logs --master-data=2 -B o2m-mc --tables GMQDSJSP > /tmp/GMQDSJSP20151021.sql
mysql -uroot -p123456 gmdc <  /tmp/GMQDSJSP20151021.sql

2、源庫

1）修改參數

在/etc/my.cnf的[mysqld]下添加如下參數

binlog_format=row

重啓

service mysqld restat

2）創建用戶並授權

mysql> GRANT ALL PRIVILEGES ON `o2m-mc`.* TO 'ogg'@localhost IDENTIFIED BY 'ogg;
mysql> GRANT ALL PRIVILEGES ON `o2m-mc`.* TO 'ogg'@'%'IDENTIFIED BY 'ogg;

3）解壓安裝創建相應目錄

unzip ggs_121210_Linux_x64_MySQL_64bit.zip -d /home/goldengate
cd /home/goldengate
tar xvf ggs_Linux_x64_MySQL_64bit.tar 
./ggsci 

GGSCI (hadooptest05) 1> create subdirs

Creating subdirectories under current directory /home/goldengate

Parameter files                /home/goldengate/dirprm: already exists
Report files                   /home/goldengate/dirrpt: created
Checkpoint files               /home/goldengate/dirchk: created
Process status files           /home/goldengate/dirpcs: created
SQL script files               /home/goldengate/dirsql: created
Database definitions files     /home/goldengate/dirdef: created
Extract data files             /home/goldengate/dirdat: created
Temporary files                /home/goldengate/dirtmp: created
Credential store files         /home/goldengate/dircrd: created
Masterkey wallet files         /home/goldengate/dirwlt: created
Dump files                     /home/goldengate/dirdmp: created

4）配置管理進程

GGSCI (hadooptest05) 2>  edit params mgr

port 7809
dynamicportlist 7840-7939
autorestart er *, retries 5, waitminutes 3
purgeoldextracts /home/goldengate/dirdat/*,usecheckpoints, minkeepdays 2
GGSCI (hadooptest05) 3> start mgr
        Manager started.

5）配置抽取進程

GGSCI (hadooptest05) 4> edit param mce1
extract mce1
sourcedb [email protected]:3306 userid ogg password ogg
exttrail /home/goldengate/dirdat/me
discardfile  /home/goldengate/dirrpt/mce1.dsc,append
TranLogOptions AltLogDest /home/mariadb/data/binlogs.index
table o2m-mc.GMQDSJSP;
GGSCI (hadooptest05) 5> add extract mce1,tranlog,begin now
GGSCI (hadooptest05) 6> add exttrail /home/goldengate/dirdat/me,extract mce1
GGSCI (hadooptest05) 7> start mce1

6）配置傳遞進程

GGSCI (hadooptest05) 8> edit param mcp1
extract mcp1
passthru
sourcedb [email protected]:3306 userid ogg password ogg
rmthost  192.168.7.221,mgrport 7809,compress
rmttrail  /home/goldengate/dirdat/mp

dynamicresolution
numfiles 3000
table o2m-mc.GMQDSJSP;
GGSCI (hadooptest05) 9> add extract mcp1,exttrailsource /home/goldengate/dirdat/me
GGSCI (hadooptest05) 10> add rmttrail /home/goldengate/dirdat/mp,extract mcp1
GGSCI (hadooptest05) 11> start mc1

7）創建表的定義文件

#創建參數文件
GGSCI (hadooptest05) 10>  edit param defgen

defsfile ./dirdef/gmqdsjsjp.def
sourcedb [email protected]:3306 userid ogg, password ogg
table o2m-mc.GMQDSJSP;
#生成表定義文件
 ./defgen paramfile ./dirprm/defgen.prm
#傳至目標庫目錄
scp /home/mysql/goldengate/dirdef/gmqdsjsjp.def 192.168.7.221:/home/goldengate/dirdef/

3、目標庫

1）創建用戶

mysql> GRANT ALL PRIVILEGES ON `gmdc`.* TO 'ogg'@localhost  IDENTIFIED BY 'ogg';
mysql> GRANT ALL PRIVILEGES ON `gmdc`.* TO 'ogg'@'%'  IDENTIFIED BY 'ogg';

2）同樣解壓安裝並創建目錄

unzip ggs_121210_Linux_x64_MySQL_64bit.zip -d /home/goldengate
cd /home/goldengate
tar xvf ggs_Linux_x64_MySQL_64bit.tar 
./ggsci 

GGSCI (db) 1> create subdirs

Creating subdirectories under current directory /home/goldengate

Parameter files                /home/goldengate/dirprm: already exists
Report files                   /home/goldengate/dirrpt: created
Checkpoint files               /home/goldengate/dirchk: created
Process status files           /home/goldengate/dirpcs: created
SQL script files               /home/goldengate/dirsql: created
Database definitions files     /home/goldengate/dirdef: created
Extract data files             /home/goldengate/dirdat: created
Temporary files                /home/goldengate/dirtmp: created
Credential store files         /home/goldengate/dircrd: created
Masterkey wallet files         /home/goldengate/dirwlt: created
Dump files                     /home/goldengate/dirdmp: created

3）配置管理進程

GGSCI (db) 3>  edit params mgr

port 7809
dynamicportlist 7840-7939
autorestart er *, retries 5, waitminutes 3
purgeoldextracts /home/goldengate/dirdat/*,usecheckpoints, minkeepdays 2
GGSCI (db) 4> start mgr
        Manager started.

4）添加檢查點表

GGSCI (db) 5>  dblogin sourcedb [email protected]:3306 userid ogg password ogg
GGSCI (db DBLOGIN as ogg) 6>  add checkpointtable gmdc.checkpoint
GGSCI (db DBLOGIN as ogg) 7>  edit params ./GLOBALS
checkpointtable gmdc.checkpoint

5）配置應用進程

GGSCI (db) 1>  edit params mcr1
replicat mcr1
targetdb [email protected]:3306 userid ogg password ogg
handlecollisions
sourcedefs /home/goldengate/dirdef/gmqdsjsp.def
discardfile /home/goldengate/dirrpt/mcr1.dsc,purge
map o2m-mc.GMQDSJSP, target gmdc.GMQDSJSP;

GGSCI (db) 2> add replicat mcr1,exttrail /home/goldengate/dirdat/mp,checkpointtable gmdc.checkpoint
GGSCI (db) 3> start dcr1

4、測試數據同步

注意：mysql中表的大小注意區分