感謝uber開源了presto pinot connector
Download and build Pinot:
git clone https://github.com/apache/incubator-pinot.git
cd incubator-pinot/
git checkout tags/release-0.1.0
mvn install package -DskipTests -Pbin-dist
Start Pinot:
cd pinot-distribution/target/apache-pinot-incubating-0.1.0-bin/apache-pinot-incubating-0.1.0-bin
bin/start-controller.sh > ~/tmp/pinot_controller_cmd.log &
bin/start-broker.sh > ~/tmp/pinot_broker_cmd.log &
bin/start-server.sh > ~/tmp/pinot_server_cmd.log &
Prepare Pinot Data
Create some Pinot segments:
mkdir airlineStatsData
cp sample_data/airlineStats_data.avro airlineStatsData/.
bin/pinot-admin.sh CreateSegment -schemaFile sample_data/airlineStats_schema.json -dataDir airlineStatsData -tableName airlineStats -segmentName airlineStatsSeg_0 -outDir airlineStatsData/airlineStatsSeg_0
bin/pinot-admin.sh CreateSegment -schemaFile sample_data/airlineStats_schema.json -dataDir airlineStatsData -tableName airlineStats -segmentName airlineStatsSeg_1 -outDir airlineStatsData/airlineStatsSeg_1
bin/pinot-admin.sh CreateSegment -schemaFile sample_data/airlineStats_schema.json -dataDir airlineStatsData -tableName airlineStats -segmentName airlineStatsSeg_2 -outDir airlineStatsData/airlineStatsSeg_2
Create Table:
bin/pinot-admin.sh AddSchema -schemaFile sample_data/airlineStats_schema.json -exec
bin/pinot-admin.sh AddTable -filePath sample_data/airlineStats_offline_table_config.json -controllerPort 9000 -exec
Upload Segments:
bin/pinot-admin.sh UploadSegment -controllerPort 9000 -segmentDir airlineStatsData/airlineStatsSeg_0
bin/pinot-admin.sh UploadSegment -controllerPort 9000 -segmentDir airlineStatsData/airlineStatsSeg_1
bin/pinot-admin.sh UploadSegment -controllerPort 9000 -segmentDir airlineStatsData/airlineStatsSeg_2
Query:
http://localhost:9000/query#
Build Presto:
git clone [email protected]:prestodb/presto.git
mvn clean install -DskipTests -T 4
Running Presto in IDE
Use the following options to create a run configuration:
Main Class: com.facebook.presto.server.PrestoServer
VM Options: -ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties
Working directory: $MODULE_DIR$
Use classpath of module: presto-main
The working directory should be the presto-main subdirectory. In IntelliJ, using $MODULE_DIR$ accomplishes this automatically.
Running Presto CLI:
presto-cli/target/presto-cli-*-executable.jar --catalog pinot
presto> use airlinestats;
Query Pinot table through Presto
presto:airlinestats> select count(AirlineID) from airlinestats;
_col0
-------
29238
(1 row)
Query 20190828_224004_00003_3wqm2, FINISHED, 1 node
Splits: 20 total, 20 done (100.00%)
0:04 [29.2K rows, 114KB] [8.29K rows/s, 32.4KB/s]
OLAP System Union Query:
