一、前提工作
- 安裝mysql(5.5版本)，5.7的版本我試了，接收不到binlog，原因後續在排查。參看：mac用brew 安裝mysql5.5
- 安裝gradle(4.7版本)。因爲databus源碼是用gradle構建的；安裝方法參考：gradle安裝
- 從githun下載databus源碼。https://github.com/linkedin/databus
- 下載ojdbc6.jar，重命名爲：ojdbc6-11.2.0.2.0.jar。copy到databus/sandbox-repo/com/oracle/ojdbc6/11.2.0.2.0路徑下
二、準備mysql
- 修改my.cnf
# Default Homebrew MySQL server config
[mysqld]
# Only allow connections from localhost
bind-address = localhost
log_bin=on
log-bin=/usr/local/opt/[email protected]/logs/mysql-bin
expire-logs-days = 14
max-binlog-size = 500M
server-id=1
binlog_format=ROW
- 重啓mysql，檢查配置是否生效
show variables like '%log_bin%';
show variables like 'binlog_format';
- 創建database ： or_test
- 創建tabal：person（建表語句位於：databus2-example/databus2-example-relay-pkg/sql/create_person.sql）
三、修改代碼
需要修改databus2-example模塊下的代碼，共3處需要修改。
1.修改 databus2-example/databus2-example-relay-pkg/conf/sources-or-person.json 的url；用戶名爲root，密碼爲root，端口爲3306,server-id爲1，binlog的前綴爲mysql-bin;server-id和binlog前綴在my.cnf中配置的。注意%2F爲分隔符不能刪。
{
"name" : "person",
"id" : 1,
"uri" : "mysql://root%2Froot@localhost:3306/1/mysql-bin",
"slowSourceQueryThreshold" : 2000,
"sources" :
[
{
"id" : 40,
"name" : "com.linkedin.events.example.or_test.Person",
"uri": "or_test.person",
"partitionFunction" : "constant:1"
}
]
}
2.修改 com.linkedin.databus.client.example.PersonClientMain的PERSON_SOURCE="com.linkedin.events.example.or_test.Person"
public class PersonClientMain
{
static final String PERSON_SOURCE = "com.linkedin.events.example.or_test.Person";
public static void main(String[] args) throws Exception
{
DatabusHttpClientImpl.Config configBuilder = new DatabusHttpClientImpl.Config();
//Try to connect to a relay on localhost
configBuilder.getRuntime().getRelay("1").setHost("localhost");
configBuilder.getRuntime().getRelay("1").setPort(11115);
configBuilder.getRuntime().getRelay("1").setSources(PERSON_SOURCE);
//Instantiate a client using command-line parameters if any
DatabusHttpClientImpl client = DatabusHttpClientImpl.createFromCli(args, configBuilder);
//register callbacks
PersonConsumer personConsumer = new PersonConsumer();
client.registerDatabusStreamListener(personConsumer, null, PERSON_SOURCE);
client.registerDatabusBootstrapListener(personConsumer, null, PERSON_SOURCE);
//fire off the Databus client
client.startAndBlock();
}
}
3.在com.linkedin.databus2.core.container.netty.ServerContainer的initializeContainerJmx方法中添加：LocateRegistry.createRegistry(_containerStaticConfig.getJmx().getRmiRegistryPort()); 不然client會報Cannot bind to URL rmi://localhost:1099 ServiceUnavailableException錯誤。親測。
protected void initializeContainerJmx()
{
if (_containerStaticConfig.getJmx().isRmiEnabled())
{
try
{
JMXServiceURL jmxServiceUrl =
new JMXServiceURL("service:jmx:rmi://" +
_containerStaticConfig.getJmx().getJmxServiceHost() + ":" +
_containerStaticConfig.getJmx().getJmxServicePort() +"/jndi/rmi://" +
_containerStaticConfig.getJmx().getRmiRegistryHost() + ":" +
_containerStaticConfig.getJmx().getRmiRegistryPort() + "/jmxrmi" +
_containerStaticConfig.getJmx().getJmxServicePort());
_jmxConnServer = JMXConnectorServerFactory.newJMXConnectorServer(jmxServiceUrl, null,
getMbeanServer());
LocateRegistry.createRegistry(_containerStaticConfig.getJmx().getRmiRegistryPort());
}
catch (Exception e)
{
LOG.warn("Unable to instantiate JMX server", e);
}
}
}
四、編譯代碼
在databus目錄下執行：gradle -Dopen_source=true assemble 打包項目。
打包好的relay位於：databus/build/databus2-example-relay-pkg/distributions/
打包好的client位於：databus/build/databus2-example-client-pkg/distributions/
五、啓動relay
cd到目錄 databus/build/databus2-example-relay-pkg/distributions/
解壓jia包：tar -zxvf databus2-example-relay-pkg-2.0.0.tar.gz
執行啓動relay腳本：./bin/start-example-relay.sh or_person -Y ./conf/sources-or-person.json
啓動後訪問：http://localhost:11115/sources ，如果返回一下內容說明啓動relay成功。
訪問：http://localhost:11115/containerStats/inbound/events/total，往person表insert4條數據。insert腳本位於：databus2-example/databus2-example-relay-pkg/sql/insert_person_test_data_1.sql。如果numDataEvents 從0變爲4說明已經relay已經能獲取了binlog變更。
如果啓動失敗可以通過logs目錄下的 databus2-relay-or_person.out 和 relay.log 日誌排查原因
六、啓動client
cd 到目錄：databus/build/databus2-example-client-pkg/distributions/
解壓databus2-example-client-pkg-2.0.0.tar.gz : tar -zxvf databus2-example-client-pkg-2.0.0.tar.gz
執行client啓動腳本：./bin/start-example-client.sh person
查看日誌：tail -f logs/client.log
這時候往person表insert數據，可以在日誌打印了出來，說明client已經能收到數據。
處理數據的代碼可以寫到com.linkedin.databus.client.example.PersonConsumer#processEvent方法中，可以看到example程序只是通過日誌打印出來。
參考：