最快的http hello world服務器調優指南 （C20Khttp短鏈接請求/S每桌面CPU)

erl的虛擬機有2種方式 plain版本的和smp版本的。 smp版本由於鎖的開銷相比要比plain版本的慢很多。 而32位機器由於內存訪問比64位的少，也會快出很多。所有我選擇在32位的linux系統下調優這個httpd服務器。 這個服務器就是實現個簡單的功能，在browser下返回hello world。 以下我們會先編譯我們的優化版本的虛擬機，然後再分別測試R13B02的標準版本的和我們優化版的性能：

root@nd-desktop:/build_opt_plain# uname -a
Linux nd-desktop 2.6.31-14-generic #3 SMP Sun Nov 1 23:03:10 CST 2009 i686 GNU/Linux

#準備開發環境
root@nd-desktop:/# apt-get build-dep erlang

#下載otp R13B02-1源碼包
root@nd-desktop:/# wget http://www.erlang.org/download/otp_src_R13B02-1.tar.gz

#解開patch包
root@nd-desktop:/# tar xzvf build_opt_plain.tgz

#解開源碼包
root@nd-desktop:/# tar xzf otp_src_R13B02-1.tar.gz

#打補丁
root@nd-desktop:/# cd otp_src_R13B02-1
root@nd-desktop:/otp_src_R13B02-1# patch -p1 <../build_opt_plain/otp_src_R13B02-1_patch_by_yufeng
patching file erts/emulator/beam/erl_binary.h
patching file erts/emulator/beam/erl_process.c
patching file erts/emulator/beam/sys.h
patching file erts/emulator/drivers/common/inet_drv.c
patching file erts/preloaded/src/Makefile
patching file erts/preloaded/src/prim_inet.erl
patching file lib/asn1/src/Makefile
patching file lib/hipe/Makefile
patching file lib/parsetools/src/Makefile
root@nd-desktop:/otp_src_R13B02-1# ../build_opt_plain/build.plain
。。。
如果編譯都沒有任何錯誤的話, 就大功告成了。

好 現在我們開始性能比較：

#先加大文件句柄數
root@nd-desktop:/otp_src_R13B02-1# cd ../build_opt_plain
root@nd-desktop:/build_opt_plain# ulimit -n 99999

#標準發佈版本
root@nd-desktop:/build_opt_plain# erlc ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1 erl +K true +h 99999 +P 99999 -smp enable +S 2:1 -s ehttpd
Erlang R13B03 (erts-5.7.4) [source] [smp:2:1] [rq:2] [async-threads:0] [hipe] [kernel-poll:true]

ehttpd ready with 2 schedulers on port 8888
Eshell V5.7.4 (abort with ^G)
1>

#在另外的一臺機器上發動ab測試
# ab -c 60 -n 100000 http://192.168.235.147:8888/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.235.147 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:
Server Hostname: 192.168.235.147
Server Port: 8888

Document Path: /
Document Length: 12 bytes

Concurrency Level: 60
Time taken for tests: 8.925945 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Total transferred: 5100051 bytes
HTML transferred: 1200012 bytes
Requests per second: 11203.29 [#/sec] (mean)
Time per request: 5.356 [ms] (mean)
Time per request: 0.089 [ms] (mean, across all concurrent requests)
Transfer rate: 557.92 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 65.7 0 3001
Processing: 0 3 1.5 4 7
Waiting: 0 2 1.8 4 6
Total: 0 4 65.8 4 3007
WARNING: The median and mean for the waiting time are not within a normal deviation
These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
50% 4
66% 4
75% 4
80% 4
90% 5
95% 5
98% 5
99% 5
100% 3007 (longest request)

標準smp版本1個CPU的結果是： 11203.29 [#/sec] (mean)

#啓用hipe的標準版本
root@nd-desktop:/build_opt_plain# erlc +native +"{hipe, [o3]}" ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1 erl +K true +h 99999 +P 99999 -smp enable +S 2:1 -s ehttpd
Erlang R13B03 (erts-5.7.4) [source] [smp:2:1] [rq:2] [async-threads:0] [hipe] [kernel-poll:true]

ehttpd ready with 2 schedulers on port 8888
Eshell V5.7.4 (abort with ^G)
1>

標準smp hipe版本1個CPU結果是： 12390.32 [#/sec] (mean)


#我們的優化版本
root@nd-desktop:/build_opt_plain# ../otp_src_R13B02-1/bin/erlc ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1 ../otp_src_R13B02-1/bin/erl +K true +h 99999 +P 99999 -s ehttpd
Erlang R13B02 (erts-5.7.3) [source] [rq:1] [hipe] [kernel-poll:true]

ehttpd ready with 1 schedulers on port 8888
Eshell V5.7.3 (abort with ^G)
1>

優化版本單個cpu: 19662.37 [#/sec] (mean)


#啓用hipe的優化版本
root@nd-desktop:/build_opt_plain# ../otp_src_R13B02-1/bin/erlc +native +"{hipe, [o3]}" ehttpd.erl
root@nd-desktop:/build_opt_plain# taskset -c 1 ../otp_src_R13B02-1/bin/erl +K true +h 99999 +P 99999 -s ehttpd
Erlang R13B02 (erts-5.7.3) [source] [rq:1] [hipe] [kernel-poll:true]

ehttpd ready with 1 schedulers on port 8888
Eshell V5.7.3 (abort with ^G)
1>

優化版本啓用hipe單個cpu:20090.83 [#/sec] (mean)

附上我們的最小的高性能的http echo 服務器：

root@nd-desktop:/build_opt_plain# cat ehttpd.erl
-module(ehttpd).
-compile(export_all).


start() ->
start(8888).
start(Port) ->
N = erlang:system_info(schedulers),
listen(Port, N),
io:format("ehttpd ready with ~b schedulers on port ~b~n", [N, Port]),

register(?MODULE, self()),
receive Any -> io:format("~p~n", [Any]) end. %% to stop: ehttpd!stop.

listen(Port, N) ->
Opts = [{active, false},
binary,
{backlog, 256},
{packet, http_bin},
{raw,6,9,<<1:32/native>>}, %defer accept
%%{delay_send,true},
%%{nodelay,true},
{reuseaddr, true}],

{ok, S} = gen_tcp:listen(Port, Opts),
Spawn = fun(I) ->
register(list_to_atom("acceptor_" ++ integer_to_list(I)),
spawn_opt(?MODULE, accept, [S, I], [link, {scheduler, I}]))
end,
lists:foreach(Spawn, lists:seq(1, N)).

accept(S, I) ->
case gen_tcp:accept(S) of
{ok, Socket} -> spawn_opt(?MODULE, loop, [Socket], [{scheduler, I}]);
Error -> erlang:error(Error)
end,
accept(S, I).

loop(S) ->
case gen_tcp:recv(S, 0) of
{ok, http_eoh} ->
Response = <<"HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\nhello world!">>,
gen_tcp:send(S, Response),
gen_tcp:close(S),
ok;

{ok, _Data} ->
loop(S);

Error ->
Error
end.


這個服務器是最小的，單是在多處理器和單cpu上都有非常好的性能。

root@nd-desktop:/build_opt_plain# cat /proc/cpuinfo
model name : Pentium(R) Dual-Core CPU E5200 @ 2.50GHz

注：這個http服務器基本上是在c的程序跑，erlang的代碼執行的很少， 所以hipe的提升效果不是很明顯。對於複雜的業務，應該是有很大的幫助的。

附件裏是用到的腳本和補丁。


我們可以得出結論：
[color=red]hipe啓用要比不啓用快。
優化版本的和標準版本的 20090:11203， 性能提高了將近80% 還是非常可觀的。[/color]