databus安裝 for mysql

一、前提工作

二、準備mysql

  • 修改my.cnf
# Default Homebrew MySQL server config
[mysqld]
# Only allow connections from localhost
bind-address = localhost
log_bin=on
log-bin=/usr/local/opt/[email protected]/logs/mysql-bin
expire-logs-days = 14
max-binlog-size = 500M
server-id=1
binlog_format=ROW
  • 重啓mysql,檢查配置是否生效
    show variables like '%log_bin%';


     show variables like 'binlog_format';

  • 創建database :  or_test
  • 創建tabal:person(建表語句位於:databus2-example/databus2-example-relay-pkg/sql/create_person.sql)

三、修改代碼

需要修改databus2-example模塊下的代碼,共3處需要修改。

1.修改 databus2-example/databus2-example-relay-pkg/conf/sources-or-person.json 的url;用戶名爲root,密碼爲root,端口爲3306,server-id爲1,binlog的前綴爲mysql-bin;server-id和binlog前綴在my.cnf中配置的。注意%2F爲分隔符不能刪。

{
    "name" : "person",
    "id"  : 1,
    "uri" : "mysql://root%2Froot@localhost:3306/1/mysql-bin",
    "slowSourceQueryThreshold" : 2000,
    "sources" :
    [
        {
        "id" : 40,
        "name" : "com.linkedin.events.example.or_test.Person",
        "uri": "or_test.person",
        "partitionFunction" : "constant:1"
         }
    ]
}

2.修改 com.linkedin.databus.client.example.PersonClientMain的PERSON_SOURCE="com.linkedin.events.example.or_test.Person"

public class PersonClientMain
{
  static final String PERSON_SOURCE = "com.linkedin.events.example.or_test.Person";

  public static void main(String[] args) throws Exception
  {
    DatabusHttpClientImpl.Config configBuilder = new DatabusHttpClientImpl.Config();

    //Try to connect to a relay on localhost
    configBuilder.getRuntime().getRelay("1").setHost("localhost");
    configBuilder.getRuntime().getRelay("1").setPort(11115);
    configBuilder.getRuntime().getRelay("1").setSources(PERSON_SOURCE);

    //Instantiate a client using command-line parameters if any
    DatabusHttpClientImpl client = DatabusHttpClientImpl.createFromCli(args, configBuilder);

    //register callbacks
    PersonConsumer personConsumer = new PersonConsumer();
    client.registerDatabusStreamListener(personConsumer, null, PERSON_SOURCE);
    client.registerDatabusBootstrapListener(personConsumer, null, PERSON_SOURCE);

    //fire off the Databus client
    client.startAndBlock();
  }

}

3.在com.linkedin.databus2.core.container.netty.ServerContainer的initializeContainerJmx方法中添加:LocateRegistry.createRegistry(_containerStaticConfig.getJmx().getRmiRegistryPort()); 不然client會報Cannot bind to URL rmi://localhost:1099 ServiceUnavailableException錯誤。親測。

protected void initializeContainerJmx()
  {

    if (_containerStaticConfig.getJmx().isRmiEnabled())
    {
      try
      {
        JMXServiceURL jmxServiceUrl =
            new JMXServiceURL("service:jmx:rmi://" +
                              _containerStaticConfig.getJmx().getJmxServiceHost() + ":" +
                              _containerStaticConfig.getJmx().getJmxServicePort() +"/jndi/rmi://" +
                              _containerStaticConfig.getJmx().getRmiRegistryHost() + ":" +
                              _containerStaticConfig.getJmx().getRmiRegistryPort() + "/jmxrmi" +
                              _containerStaticConfig.getJmx().getJmxServicePort());

        _jmxConnServer = JMXConnectorServerFactory.newJMXConnectorServer(jmxServiceUrl, null,
                                                                         getMbeanServer());
        LocateRegistry.createRegistry(_containerStaticConfig.getJmx().getRmiRegistryPort());
      }
      catch (Exception e)
      {
        LOG.warn("Unable to instantiate JMX server", e);
      }
    }
  }

四、編譯代碼

在databus目錄下執行:gradle -Dopen_source=true assemble 打包項目。 

打包好的relay位於:databus/build/databus2-example-relay-pkg/distributions/

打包好的client位於:databus/build/databus2-example-client-pkg/distributions/

五、啓動relay

cd到目錄 databus/build/databus2-example-relay-pkg/distributions/

解壓jia包:tar -zxvf databus2-example-relay-pkg-2.0.0.tar.gz

執行啓動relay腳本:./bin/start-example-relay.sh or_person -Y ./conf/sources-or-person.json

啓動後訪問:http://localhost:11115/sources ,如果返回一下內容說明啓動relay成功。

訪問:http://localhost:11115/containerStats/inbound/events/total,往person表insert4條數據。insert腳本位於:databus2-example/databus2-example-relay-pkg/sql/insert_person_test_data_1.sql。如果numDataEvents 從0變爲4說明已經relay已經能獲取了binlog變更。

如果啓動失敗可以通過logs目錄下的 databus2-relay-or_person.out 和 relay.log 日誌排查原因

六、啓動client

cd 到目錄:databus/build/databus2-example-client-pkg/distributions/

解壓databus2-example-client-pkg-2.0.0.tar.gz : tar -zxvf databus2-example-client-pkg-2.0.0.tar.gz 

執行client啓動腳本:./bin/start-example-client.sh person

查看日誌:tail -f logs/client.log  

這時候往person表insert數據,可以在日誌打印了出來,說明client已經能收到數據。

處理數據的代碼可以寫到com.linkedin.databus.client.example.PersonConsumer#processEvent方法中,可以看到example程序只是通過日誌打印出來。

參考:

  1. https://github.com/linkedin/databus
  2. https://github.com/linkedin/databus/wiki/Databus-2.0-Example
  3. https://github.com/linkedin/databus/wiki/Databus-for-MySQL
  4. https://sq.163yun.com/blog/article/173554725500456960
  5. https://blog.csdn.net/liuyeshennai/article/details/91992802
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章