benchMarkSQL5.0 （TPCC）測試

測試環境

測試機
- 10.37.64.52 ： RHEL 6.3 (64 Bit)
Master
- 10.37.64.53 ： RHEL 6.3 (64 Bit)
Slave
- 10.37.64.54 ： RHEL 6.3 (64 Bit)
數據庫
- postgresql
- mysql

使用說明

本說明手冊是在RHEL 6.3 (64 Bit)，jdk-7u79環境下編譯過的，如果直接使用該文件進行測試，請直接跳到“benchmarksql5.0測試步驟”。因爲該文件中已增加對mysql的支持與優化。

注：針對mysql5.5的優化沒有反應到其中。如果測試mysql5.5，請參考後面的章節，加入mysql5.5的優化。

軟件下載與安裝

安裝benchmarkSQL5.0

下載地址：https://sourceforge.net/projects/benchmarksql/
安裝：直接解壓，假設解壓路徑 /data/benchmarksql-5.0

安裝jdk7

yum install jdk-7u79-linux-x64.rpm

安裝Apache ant

用於編譯benchmarkSQL, 5.0版本是需要編譯的。

下載地址：https://www.baidu.com/#ie=utf-8&f=8&rsv_bp=1&rsv_idx=1&tn=baidu&wd=txkbps&oq=rxpktsps&rsv_pq=fda53d7e00006c93&rsv_t=95daqeDHwjUCPg%2F53yzOfJJcalfg0MQo0jTBW3kJTzUaRK2hjG0RqTOhbpI&rqlang=cn&rsv_enter=0&inputT=1380&rsv_n=2&rsv_sug3=12&rsv_sug4=1380&rsv_sug=2

安裝：直接解壓apache-ant-1.9.1-bin.tar.gz包即可。編譯benchmarksql時在環境變量PATH中添加上解壓包的bin路徑即可。

注：編譯ant的jdk版本不能高於上面安裝的JDK版本（本次爲jdk7），ant.jar包中的MANIFEST.MF文件中記載了jdk版本。如“Created-By: 1.8.0_121-b13 (Oracle Corporation)”表示jdk8。

下載postgresql JDBC驅動(與jdk版本一致）

下載地址：https://jdbc.postgresql.org/download.html
本次下載postgresql-42.0.0.jre7.jar

將jar包放入benchmarksql的lib目錄下的postgres目錄下:

#cp postgresql-42.0.0.jre7.jar /data/benchmarksql-5.0/lib/postgres/

下載mysql的JDBC驅動

下載地址：https://dev.mysql.com/downloads/connector/j/

本次下載的是mysql-connector-java-5.1.42.tar.gz

解壓後在解壓目錄會有mysql-connector-java-5.1.42-bin.jar包。

由於benchmarksql5.0不支持mysql，所以需要在benchmarksql的lib目錄下創建mysql目錄，並將jar包放入該目錄：

# mkdir -p /data/benchmarksql-5.0/lib/mysql
# cp mysql-connector-java-5.1.42-bin.jar /data/benchmarksql-5.0/lib/mysql/

修改benchmarksql腳本，解決與OS不兼容問題

跑測試腳本時，由於benchmarksql與OS不兼容，導致如下錯誤：

Traceback (most recent call last):
 File "<stdin>", line 299, in <module>
 File "<stdin>", line 62, in main
 File "<stdin>", line 166, in initSystemUsage
 Exception: cpu line in /proc/stat too short
 10:45:36,966 [Thread-0] ERROR  OSCollector$CollectData : OSCollector, unexpected EOF while reading from external helper process

解決方法：

根據測試機器中 /proc/stat文件第一行的列數（假設爲N），修改benchmarksql-5.0/run/misc/os_collector_linux.py腳本。

# vi /proc/stat
cpu  2924404 2878 881591 4758076392 305979 20 28284 74916 0
......
// 共10列，所以N爲10

# vi run/misc/os_collector_linux.py
......
if len(lastStatData) != 10:       //此處的10修改爲N-1的值，即9
raise Exception("cpu line in /proc/stat too short");
......
return [
'cpu_user', 'cpu_nice', 'cpu_system',

      'cpu_idle', 'cpu_iowait', 'cpu_irq',

      'cpu_softirq', 'cpu_steal',

      'cpu_guest', 'cpu_guest_nice',

    'vm_nr_dirty',

    ]
//這裏多返回1列，根據/proc/stat第一行各列的含義，此處多了'cpu_guest_nice'，刪除'cpu_guest_nice'即可。

修改benchmarksql5.0源碼，增加對mysql的支持

修改funcs.sh腳本

# vim run/funcs.sh
...
function setCP()
{
    ...
	postgres)
	    cp="../lib/postgres/*:../lib/*"
	    ;;
	mysql)
	    cp="../lib/mysql/*:../lib/*"
	    ;;
    ...
}
...
case "$(getProp db)" in
	firebird|oracle|postgres|mysql)
...

修改jTPCC.java

# vim src/client/jTPCC.java
...
if (iDB.equals("firebird"))
    dbType = DB_FIREBIRD;
else if (iDB.equals("oracle"))
    dbType = DB_ORACLE;
else if (iDB.equals("postgres"))
    dbType = DB_POSTGRES;
else if (iDB.equals("mysql"))
    dbType = DB_UNKNOWN;
else
{
    log.error("unknown database type '" + iDB + "'");
    return;
}

修改jTPCCConnection.java

# vim src/client/jTPCCConnection.java
...
// PreparedStatements for STOCK_LEVEL
switch (dbType)
{
    case jTPCCConfig.DB_POSTGRES:
	stmtStockLevelSelectLow = dbConn.prepareStatement(
	    "SELECT count(*) AS low_stock FROM (" +
	    "    SELECT s_w_id, s_i_id, s_quantity " +
	    "        FROM bmsql_stock " +
	    "        WHERE s_w_id = ? AND s_quantity < ? AND s_i_id IN (" +
	    "            SELECT ol_i_id " +
	    "                FROM bmsql_district " +
	    "                JOIN bmsql_order_line ON ol_w_id = d_w_id " +
	    "                 AND ol_d_id = d_id " +
	    "                 AND ol_o_id >= d_next_o_id - 20 " +
	    "                 AND ol_o_id < d_next_o_id " +
	    "                WHERE d_w_id = ? AND d_id = ? " +
	    "        ) " +
	    "    ) AS L");
	break;

    default:
	stmtStockLevelSelectLow = dbConn.prepareStatement(
	    "SELECT count(*) AS low_stock FROM (" +
	    "    SELECT s_w_id, s_i_id, s_quantity " +
	    "        FROM bmsql_stock " +
	    "        WHERE s_w_id = ? AND s_quantity < ? AND s_i_id IN (" +
	    "            SELECT ol_i_id " +
	    "                FROM bmsql_district " +
	    "                JOIN bmsql_order_line ON ol_w_id = d_w_id " +
	    "                 AND ol_d_id = d_id " +
	    "                 AND ol_o_id >= d_next_o_id - 20 " +
	    "                 AND ol_o_id < d_next_o_id " +
	    "                WHERE d_w_id = ? AND d_id = ? " +
	    "        ) " +
	    "    ) AS aliasA");  /* mysql的子查詢必須添加一個別名 */
	break;
}
...

修改benchmarksql-5.0\run\sql.common目錄下的.sql文件

爲mysql創建sql的目錄：

# mkdir -p benchmarksql-5.0\run\sql.mysql

將要修改的.sql文件先拷貝到sql.mysql目錄，然後再進行修改。

# cp run\sql.common\buildFinish.sql run\sql.mysql\buildFinish.sql
# cp run\sql.common\tableCreates.sql run\sql.mysql\tableCreates.sql
# cp run\sql.common\tableDrops.sql run\sql.mysql\tableDrops.sql

修改上面的3個.sql文件：

1）修改run\sql.mysql\buildFinish.sql，在文件末尾追加以下內容：

ANALYZE TABLE bmsql_config;
ANALYZE TABLE bmsql_warehouse;
ANALYZE TABLE bmsql_district;
ANALYZE TABLE bmsql_customer;
ANALYZE TABLE bmsql_history;
ANALYZE TABLE bmsql_oorder;
ANALYZE TABLE bmsql_new_order;
ANALYZE TABLE bmsql_order_line;
ANALYZE TABLE bmsql_stock;
ANALYZE TABLE bmsql_item;

2）修改run\sql.mysql\tableCreates.sql，刪除以下內容：

create sequence bmsql_hist_id_seq;

3）修改run\sql.mysql\tableDrops.sql，刪除以下內容：

drop sequence bmsql_hist_id_seq;

針對mysql，優化腳本建庫步驟

爲mysql複製建庫腳本

cp benchmarksql-5.0/run/runDatabaseBuild.sh benchmarksql-5.0/run/runDatabaseBuild_mysql.sh

優化mysql建庫腳本步驟(先建索引再插數據）,修改內容如下：

# vi benchmarksql-5.0/run/runDatabaseBuild_mysql.sh
...
BEFORE_LOAD="tableCreates indexCreates foreignKeys"
AFTER_LOAD="buildFinish"
...

針對mysql5.5的優化

測試mysql5.5時，總會遇到死鎖或者鎖超時的錯誤，而且測試的性能（tpm）很低。通過調查發現，有一條測試SQL語句執行時間太長，導致了死鎖或鎖超時的發生。而在mysql5.6中，該語句的執行時間很短（mysql5.6對查詢有性能優化）。因此，爲了解決該問題，在測試mysql5.5時，將對該查詢語句進行優化修改，使測試正常進行。修改方法如下：

修改benchmarksql-5.0/src/client/jTPCCConnection.java

修改前：

...
default:
          stmtStockLevelSelectLow = dbConn.prepareStatement(
                "SELECT count(*) AS low_stock FROM (" +
                "    SELECT s_w_id, s_i_id, s_quantity " +
                "        FROM bmsql_stock " +
                "        WHERE s_w_id = ? AND s_quantity < ? AND s_i_id IN (" +
                "            SELECT ol_i_id " +
                "                FROM bmsql_district " +
                "                JOIN bmsql_order_line ON ol_w_id = d_w_id " +
                "                 AND ol_d_id = d_id " +
                "                 AND ol_o_id >= d_next_o_id - 20 " +
                "                 AND ol_o_id < d_next_o_id " +
                "                WHERE d_w_id = ? AND d_id = ? " +
                "        )  " +
                "    ) AS aliasA");
...

修改後：

...
default:
		stmtStockLevelSelectLow = dbConn.prepareStatement(
                "SELECT count(*) AS low_stock FROM (" +
                "    SELECT distinct( s_i_id),s_w_id, s_quantity " +
                "        FROM bmsql_stock,bmsql_district " +
                "             JOIN bmsql_order_line ON ol_w_id = d_w_id " +
                "             AND ol_d_id = d_id AND ol_o_id >= d_next_o_id - 20" +
                "        WHERE s_w_id = ? " +
                "              AND s_quantity < ? " +
                "              AND s_i_id = ol_i_id " +
                "              AND d_w_id = ? " +
                "              AND d_id = ?" +

                "    ) AS aliasA");
...

編譯benchmarksql5.0

# cd benchmarksql5.0
# ant

benchmarksql5.0測試步驟

設置測試機（）可以免密碼登錄master（10.37.64.53）

以下全是使用root用戶，若以其他用戶登錄（如postgres用戶），則先切換到該用戶環境下在執行以下步驟。

1）測試機上

cd /root/.ssh/
rm -f ./*
ssh-keygen            //隨後按3次ENTER鍵

此時在.ssh目錄下生成了id_rsa和id_rsa.pub。其中id_rsa是私鑰，本機要用；id_rsa.pub是公鑰，給其他的機器用。

2）將公鑰上傳到master上

# scp ./id_rsa.pub [email protected]:/root/.ssh/

3）將公匙寫入master中的/root/.ssh/authorized_keys文件

# cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys

使用benchmarksql5.0進行測試

以下步驟以postgreSQL數據庫爲例，mysql的操作步驟與之類似。

1）在master上創建benchmarksql的測試用戶和數據庫

[postgres#localhost ~] $ psql postgres

psql (9.5.2)

Type "help" for help.
postgres=# CREATE USER benchmarksql WITH ENCRYPTED PASSWORD 'changeme';

postgres=# CREATE DATABASE benchmarksql OWNER benchmarksql;
postgres=# \q

2）回到測試機的benchmarksql目錄，修改配置文件prop.pg（可以先備份配置文件）

# cp props.pg props.pg.bak
# vi props.pg
db=postgres
driver=org.postgresql.Driver
conn=jdbc:postgresql://localhost:5432/benchmarksql_100
user=benchmarksql
password=benchmarksql12345

warehouses=100            //測試因子，本次取值100，500
loadWorkers=8

terminals=1               //測試因子，本次取值1,10,30,100,300,500,1000
//To run specified transactions per terminal- runMins must equal zero
runTxnsPerTerminal=0
//To run for specified minutes- runTxnsPerTerminal must equal zero
runMins=20
//Number of total transactions per minute
limitTxnsPerMin=0

//Set to true to run in 4.x compatible mode. Set to false to use the
//entire configured database evenly.
terminalWarehouseFixed=false

//The following five values must add up to 100
//The default percentages of 45, 43, 4, 4 & 4 match the TPC-C spec
newOrderWeight=45
paymentWeight=43
orderStatusWeight=4
deliveryWeight=4
stockLevelWeight=4

// Directory name to create for collecting detailed result data.
// Comment this out to suppress.
resultDirectory=my_result_%tY-%tm-%td_%tH%tM%tS
osCollectorScript=./misc/os_collector_linux.py
osCollectorInterval=1
[email protected]  //使用ssh訪問master，前面必須配置免密碼登錄
osCollectorDevices=net_eth0 blk_vda blk_vdb     //這裏的塊設備（如vda、vdb）應該在 /sys/block/ 目錄下存在，需要改成本地數據目錄所在的塊名稱

建立測試庫，並加載數據

$ ./runDatabaseBuild.sh props.pg

注：數據庫爲mysql時，請使用runDatabaseBuild_mysql.sh腳本。

4）備份數據目錄，這樣以後測試時就不需要刪除表再重新加載數據了。

5）跑測試，並生成結果

$ ./runBenchmark.sh props.pg

6）多次測試時，爲了測試的準確性，最好刪除當前數據目錄，然後從4）中備份的數據目錄中拷貝一份用於新的測試。或者執行以下步驟，清空數據庫，再重建數據庫（這樣比較耗時間，500個warehouses的庫可能需要4個小時以上，因此不推薦）。

$ ./runDatabaseDestroy.sh props.pg
$ ./runDatabaseBuild.sh props.pg

生成html以及圖形彙總

1）安裝R語言

下載安裝R需要依賴的rpm包，本次缺少2個

ftp://fr2.rpmfind.net/linux/centos/6.8/os/x86_64/Packages/texinfo-tex-4.13a-8.el6.x86_64.rpm

http://mirror.ox.ac.uk/sites/mirror.centos.org/6/os/x86_64/Packages/libjpeg-turbo-1.2.1-3.el6_5.x86_64.rpm

安裝

yum install texinfo-tex-4.13a-8.el6.x86_64.rpm
yum install libjpeg-turbo-1.2.1-3.el6_5.x86_64.rpm
yum install R

2）生成html彙總結果

$ ./generateReport.sh 結果路徑（runBenchmark.sh測試結果的路徑）