利用ycsb測試hbase性能

java 、maven、ycsb 的安裝及配置見這篇博客：
http://blog.csdn.net/hs794502825/article/details/17309845

本篇博客主要介紹 hbase 的安裝，以及利用 ycsb 對 hbase 進行基本的測試

步驟一：
在http://mirrors.hust.edu.cn/apache/hbase/下載hbase穩定版：
hbase-0.94.14.tar.gz 存放於 /home/hs/program 目錄下
cd /home/hs/program
tar -zxvf hbase-0.94.14.tar.gz


步驟二：
修改/home/hs/program/YCSB/hbase目錄下的pom.xml文件（根據你所使用的hbase版本和hadoop版本，注意紅色部分）：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.yahoo.ycsb</groupId>
    <artifactId>root</artifactId>
    <version>0.1.4</version>
  </parent>


  <artifactId>hbase-binding</artifactId>
  <name>HBase DB Binding</name>


  <dependencies>
    <dependency>
      <groupId>org.apache.hbase</groupId>
      <artifactId>hbase</artifactId>
      <!--<version>${hbase.version}</version>-->
      <version>0.94.14</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-core</artifactId>
      <!--<version>1.0.0</version>-->
      <version>1.0.4</version>
    </dependency>
    <dependency>
      <groupId>com.yahoo.ycsb</groupId>
      <artifactId>core</artifactId>
      <version>${project.version}</version>
    </dependency>
  </dependencies>


  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>${maven.assembly.version}</version>
        <configuration>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
          <appendAssemblyId>false</appendAssemblyId>
        </configuration>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>


</project>


步驟三：
因爲更改了步驟二中的文件，所以需要重新編譯YCSB項目
cd /home/hs/program/YCSB
mvn clean package | tee mvn-clean-package.dat //將輸出同時輸出到屏幕和文件
顯示與下類似信息，則說明編譯成功：
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] YCSB Root ............................................. SUCCESS [7.540s]
[INFO] Core YCSB ............................................. SUCCESS [13.293s]
[INFO] Cassandra DB Binding .................................. SUCCESS [15.861s]
[INFO] HBase DB Binding ...................................... SUCCESS [2:21.346s]
[INFO] Hypertable DB Binding ................................. SUCCESS [5.296s]
[INFO] DynamoDB DB Binding ................................... SUCCESS [7.347s]
[INFO] ElasticSearch Binding ................................. SUCCESS [18.140s]
[INFO] JDBC DB Binding ....................................... SUCCESS [5.495s]
[INFO] Mapkeeper DB Binding .................................. SUCCESS [4.054s]
[INFO] Mongo DB Binding ...................................... SUCCESS [3.368s]
[INFO] OrientDB Binding ...................................... SUCCESS [4.199s]
[INFO] Redis DB Binding ...................................... SUCCESS [3.261s]
[INFO] Voldemort DB Binding .................................. SUCCESS [4.113s]
[INFO] YCSB Release Distribution Builder ..................... SUCCESS [13.764s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 minutes 8 seconds
[INFO] Finished at: Mon Dec 16 12:43:12 CST 2013
[INFO] Final Memory: 94M/227M

步驟四：
成功編譯YCSB之後，在/home/hs/program/YCSB/distribution/target目錄下會有ycsb-0.1.4.tar.gz
將該文件解壓到/home/hs/program目錄下：
tar -zxvf /home/hs/program/YCSB/distribution/target -C /home/hs/program

步驟五：
將/home/hs/program/hbase-0.94.14/lib中的以下文件：
slf4j-api-*.jar
slf4j-log4j12-*.jar
zookeeper-*.jar
拷貝到/home/hs/program/ycsb-0.1.4/hbase-binding/lib中

步驟六：
使用/home/hs/program/hbase-0.94.14/conf中的hbase-site.xml
替換掉/home/hs/program/ycsb-0.1.4/hbase-binding/conf中的hbase-site.xml

步驟七：
執行測試
終端1：
進入hbase的安裝目錄：
執行：./bin/start-hbase.sh
顯示：
starting master, logging to /home/hs/program/hbase-0.94.14/bin/../logs/hbase-hs-master-hs-virtual-machine.out

接着執行：
./bin/hbase shell
create 'usertable','cf'

終端2中：
cd /home/hs/program/ycsb-0.1.4

hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb load hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/load-hbase-a

hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb run hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/run-hbase-a

得到如下結果：
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p hosts=localhost -p columnfamily=cf -t
[OVERALL], RunTime(ms), 5886.0
[OVERALL], Throughput(ops/sec), 169.89466530750934
[UPDATE], Operations, 487
[UPDATE], AverageLatency(us), 9366.180698151951
[UPDATE], MinLatency(us), 33
[UPDATE], MaxLatency(us), 4459758
[UPDATE], 95thPercentileLatency(ms), 0
[UPDATE], 99thPercentileLatency(ms), 0
[UPDATE], Return=0, 486
......
[READ], Operations, 514
[READ], AverageLatency(us), 2482.817120622568
[READ], MinLatency(us), 707
[READ], MaxLatency(us), 176125
[READ], 95thPercentileLatency(ms), 2
[READ], 99thPercentileLatency(ms), 29
[READ], Return=0, 514