利用ycsb測試redis性能

1、redis的安裝

在http://redis.io/download上面下載了redis-2.8.3.tar.gz存放在/home/hs/program目錄下

cd /home/hs/program
tar -zxvf redis-2.8.3.tar.gz
cd ./redis-2.8.3/
make 32bit
make test   //（用於測試安裝是否成功）

結果顯示:沒有安裝tcl或者需要比8.5新的版本的tcl（tool command language）。

在http://www.linuxfromscratch.org/blfs/view/svn/general/tcl.html上面下載了tcl8.6.1-src.tar.gz存放在/home/hs/program目錄下
tar -zxvf tcl8.6.1-src.tar.gz
cd ./tcl8.6.1/unix/
執行如下命令（參考上面的網址）：

./configure --prefix=/usr --without-tzdata --mandir=/usr/share/man
make
sed -e "s@^\(TCL_SRC_DIR='\).*@\1/usr/include'@"     -e "/TCL_B/s@='\(-L\)\?.*unix@='\1/usr/lib@" \
sed -e "s@^\(TCL_SRC_DIR='\).*@\1/usr/include'@" -e "/TCL_B/s@='\(-L\)\?.*unix@='\1/usr/lib@" -i tclConfig.sh
make test
sudo make install
sudo make install-private-headers
sudo ln -v -sf tclsh8.6 /usr/bin/tclsh
sudo chmod -v 755 /usr/lib/libtcl8.6.so

然後重新返回到/home/hs/program/redis-2.8.3

make distclean   //（清除上次編譯信息）
make 32bit
make test

顯示安裝成功之後，就可以使用redis-server/redis-cli進行一些簡單的測試。

2、jdk的安裝

在http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html上面
下載了jdk-7u45-linux-i586.tar.gz存放在/home/hs/program目錄下

cd /home/hs/program
tar -zxvf jdk-7u45-linux-i586.tar.gz

得到jdk1.7.0_45

然後設置java的環境變量
sudo gedit /etc/profile

在最後加上了如下代碼：

#set java environment
export JAVA_HOME=/home/hs/program/jdk1.7.0_45
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

然後. /etc/profile（使對/etc/profile裏面的更改生效）

接着執行如下代碼：java -version
能夠得到如下結果：

java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) Client VM (build 24.45-b08, mixed mode)

3、maven的安裝

在http://maven.apache.org/download.cgi上面下載了
apache-maven-3.1.1-bin.tar.gz也存放在/home/hs/program目錄下

cd /home/hs/program
tar -zxvf apache-maven-3.1.1-bin.tar.gz -C /usr/local/

然後ls /usr/local得到如下結果：
apache-maven-3.1.1 bin etc games include lib man sbin share src

然後設置maven的環境變量
sudo gedit /etc/profile

在最後加上如下代碼：

#set maven environment
export M2_HOME=/usr/local/apache-maven-3.1.1
export MAVEN_OPTS="-Xms256m -Xmx512m"
export PATH=$M2_HOME/bin:$PATH

執行 mvn -version也可以得到如下結果：

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /usr/local/apache-maven-3.1.1
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /home/hs/program/jdk1.7.0_45/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.0.0-32-generic", arch: "i386", family: "unix"

4、YCSB的安裝

在/home/hs/program目錄下執行如下命令：

git clone git://github.com/brianfrankcooper/YCSB.git
cd YCSB

然後mvn clean package > build-result.dat（爲了方便查看信息，我將輸出重定向了）
接着去查看build-result.dat裏面的內容就發現了好多錯誤，幾個典型錯誤如下：
a、Downloading: http://snapshots.repository.codehaus.org/org/mortbay/jetty/jetty-util/6.1.5/jetty-util-6.1.5.jar
[INFO] Unable to find resource 'org.mortbay.jetty:jetty-util:jar:6.1.5' in repository codehaus.org (http://snapshots.repository.codehaus.org)

b、[INFO] Starting audit...
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:0: Missing package-info.java file.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:1: Line is longer than 80 characters.

c、/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:14: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:15: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:23: Missing a Javadoc comment.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:24: member def type at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:24:1: File contains tab characters (this is the first instance).
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:24:16: Variable 'str' must be private and have accessor methods.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:25: member def type at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:25:13: Variable 'off' must be private and have accessor methods.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:31: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:31: method def modifier at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:32: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:32: for at indentation level 15 not at correct indentation, 4
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:33: method def rcurly at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:39: Line is longer than 80 characters.

d、/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:75:25: '{' should be on the previous line.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:76: if at indentation level 32 not at correct indentation, 8
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:77: if lcurly at indentation level 32 not at correct indentation, 8
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:77:33: '{' should be on the previous line.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:78: if child at indentation level 40 not at correct indentation, 10
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:79: if rcurly at indentation level 32 not at correct indentation, 8

e、[INFO] There are 444 checkstyle errors.
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/hs/program/YCSB/cassandra/src/main/resources

f、[INFO] [INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] null
[INFO] ------------------------------------------------------------------------
[INFO] Trace
java.lang.StackOverflowError
at sun.nio.cs.UTF_8.updatePositions(UTF_8.java:77)
at sun.nio.cs.UTF_8$Encoder.encodeArrayLoop(UTF_8.java:564)
at sun.nio.cs.UTF_8$Encoder.encodeLoop(UTF_8.java:619)
at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:561)
at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:271)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
at java.io.PrintStream.write(PrintStream.java:526)
at java.io.PrintStream.print(PrintStream.java:669)
at org.codehaus.plexus.logging.console.ConsoleLogger.info(ConsoleLogger.java:40)
at org.codehaus.plexus.logging.AbstractLogger.info(AbstractLogger.java:56)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:478)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)

裏面有好多的語法錯誤，這一點我很納悶，git得到的源文件，我並沒有做任何的修改

在github/ycsb上提問得到了別人的回答如下：
需要將YCSB的top目錄下的pom.xml文件中的<module>infinispan</module>註釋掉即可。

然後重新mvn clean package,得到如下結果：

[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] YCSB Root ............................................. SUCCESS [8.624s]
[INFO] Core YCSB ............................................. SUCCESS [21.317s]
[INFO] Cassandra DB Binding .................................. SUCCESS [13.588s]
[INFO] HBase DB Binding ...................................... SUCCESS [35.079s]
[INFO] Hypertable DB Binding ................................. SUCCESS [5.871s]
[INFO] DynamoDB DB Binding ................................... SUCCESS [7.593s]
[INFO] ElasticSearch Binding ................................. SUCCESS [18.131s]
[INFO] JDBC DB Binding ....................................... SUCCESS [36.329s]
[INFO] Mapkeeper DB Binding .................................. SUCCESS [11.661s]
[INFO] Mongo DB Binding ...................................... SUCCESS [8.293s]
[INFO] OrientDB Binding ...................................... SUCCESS [32.434s]
[INFO] Redis DB Binding ...................................... SUCCESS [7.217s]
[INFO] Voldemort DB Binding .................................. SUCCESS [15.579s]
[INFO] YCSB Release Distribution Builder ..................... SUCCESS [13.252s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 minutes 56 seconds
[INFO] Finished at: Sat Dec 14 12:07:06 CST 2013
[INFO] Final Memory: 64M/205M

注：上面的一些語法還是有一些，比如 Line is longer than 80 characters.等等。但是文件最後面的BUILD SUCCESSFUL確實不爭的事實。

但是在重新執行./bin/ycsb shell redis，又出現了新問題：

Exception in thread "main" java.lang.IllegalArgumentException: hostname can't be null
	at java.net.InetSocketAddress.checkHost(InetSocketAddress.java:149)
	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:216)
	at redis.clients.jedis.Connection.connect(Connection.java:119)
	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:54)
	at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1657)
	at com.yahoo.ycsb.db.RedisClient.init(RedisClient.java:49)
	at com.yahoo.ycsb.CommandLine.main(CommandLine.java:210)

在ycsb的郵件列表中找到解決方法如下：
如果你是在同一臺機器上跑ycsb和redis，並且都是在默認的配置下，那麼你在ycsb上測redis時需要加上參數：
-p redis.host=localhost -p redis.port=6379
重新執行./bin/ycsb shell redis -p redis.host=localhost -p redis.port=6379
得到結果如下：
Connected.
>

5、在確認與redis連接成功之後，就可以進行測試

1、load

./bin/ycsb load redis -P workloads/workloada -p redis.host=localhost -p redis.port=6379 > ./my-results/redis-load-a

結果如下：

YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.RedisClient -P workloads/workloada -p redis.host=localhost -p redis.port=6379 -load
[OVERALL], RunTime(ms), 752.0
[OVERALL], Throughput(ops/sec), 1329.787234042553
[INSERT], Operations, 1000
[INSERT], AverageLatency(us), 461.042
[INSERT], MinLatency(us), 216
[INSERT], MaxLatency(us), 59507
[INSERT], 95thPercentileLatency(ms), 0
[INSERT], 99thPercentileLatency(ms), 2
[INSERT], Return=0, 1000

2、run

./bin/ycsb run redis -P workloads/workloada -p redis.host=localhost -p redis.port=6379 > ./my-results/redis-run-a

結果如下：

YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.RedisClient -P workloads/workloada -p redis.host=localhost -p redis.port=6379 -t
[OVERALL], RunTime(ms), 291.0
[OVERALL], Throughput(ops/sec), 3436.426116838488
[UPDATE], Operations, 513
[UPDATE], AverageLatency(us), 179.85185185185185
[UPDATE], MinLatency(us), 99
[UPDATE], MaxLatency(us), 8500
[UPDATE], 95thPercentileLatency(ms), 0
[UPDATE], 99thPercentileLatency(ms), 0
[UPDATE], Return=0, 513
......
......
......


[READ], Operations, 487
[READ], AverageLatency(us), 252.36139630390144
[READ], MinLatency(us), 101
[READ], MaxLatency(us), 27355
[READ], 95thPercentileLatency(ms), 0
[READ], 99thPercentileLatency(ms), 0
[READ], Return=0, 487

利用ycsb測試redis性能

Python 潮流週刊#50：我最喜歡的 Python 3.13 新特性！

測試系統可供malloc（）使用的內存空間大小

利用ycsb測試redis性能

Linux CFS 進程調度算法

flashcache中應用device mapper機制

打印堆棧中的初始化信息

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結