druid學習-安裝

最近因爲業務需要，需要部署druid來做olap分析，關於選型可以看下知乎裏大神的分析
https://www.zhihu.com/question/41541395?sort=created
(此druid非阿里的數據庫連接池druid)
步入正題，druid安裝，可以選擇imply ( https://imply.io/ )，我這兒選擇的是druid-0.10.1
1. 解壓 tar zxvf druid-0.10.1-bin.tar.gz
2. cd conf/druid
關於druid各組件的配置都在該目錄下
broker
_common
coordinator
historical
middleManager
overlord
3. vim _common/common.runtime.properties
設置需要加載的擴展插件，zk，源數據存儲，業務數據存儲，metric監控

druid.extensions.loadList=["druid-kafka-eight", "druid-hdfs-storage", "druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "mysql-metadata-storage", "druid-protobuf-extensions"]

我這兒源數據存儲選擇mysql，deep-storage選擇HDFS，所以需要將mysql和hdfs的擴展加入到druid.extensions.loadList中，並且確認druid-0.10.1/extensions 目錄下有相關插件，其中mysql需要自己下載放在該目錄下。
4. 啓動每個組件
bin/init
bin/coordinator.sh start
bin/broker.sh start
bin/historical.sh start
bin/overlord.sh start
bin/middleManager.sh start
觀察log目錄下日誌
注意：默認配置下historical，broker都會報錯，報錯如下

1) Not enough direct memory.  Please adjust -XX:MaxDirectMemorySize, druid.processing.buffer.sizeBytes, druid.processing.numThreads, or druid.processing.numMergeBuffers: maxDirectMemory[4,294,967,296], memoryNeeded[5,368,709,120] = druid.processing.buffer.sizeBytes[536,870,912] * (druid.processing.numMergeBuffers[2] + druid.processing.numThreads[7] + 1)

需要修改下配置滿足報錯中的表達式

druid.processing.buffer.sizeBytes=25600000
druid.processing.numThreads=2

5.查看進程，5個組件都成功啓動。
打開ui http://localhost:7081/#/
http://localhost:8090/console.html

6.跑下druid提供的示例

$ curl -X 'POST' -H 'Content-Type:application/json' -d @quickstart/wikiticker-index.json http://overloadhost:8090/druid/indexer/v1/task
{"task":"index_hadoop_wikiticker_2017-09-11T16:26:56.251+08:00"}

在http://localhost:8090/console.html可以看到執行日誌和執行情況，待狀態成功後 hdfs可以看到數據

$ hadoop fs -ls /druid/segments/wikiticker
Found 1 items
drwxr-xr-x   - ad supergroup          0 2017-08-29 20:47 /druid/segments/wikiticker/20150912T000000.000Z_20150913T000000.000Z

在http://localhost:7081/#/也可以看到wikiticker datasource