基於Varnish 3.0.4的網站靜態加速

Varnish 的一些特點:

(1)、是基於內存緩存,重啓後數據將消失;

(2)、利用虛擬內存方式,I/O 性能好;

(3)、支持設置 0~60 秒內的精確緩存時間;

(4)、VCL 配置管理比較靈活;

(5)、32位機器上緩存文件大小爲最大2G;

(6)、具有強大的管理功能,例如 top、stat、admin、list 等;

(7)、狀態機設計巧妙,結構清晰;

(8)、利用二叉堆管理緩存文件,達到積極刪除目的。

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 的 Storage 方式可分爲兩種:

1)、Malloc 通過 malloc 獲取內存;

2)、Mmap file 創建大文件,通過二分法分段映射成 1G 以內的大塊。


以 Mmap file 的緩存方式啓動 I/O 也會形成瓶頸,原因主要是 Varnish 緩存的數據先會刷到磁盤上,然後在一次性讀到內存中,這在訪問量大的時候同時也會對 I/O 造成很大的壓力。Malloc 緩存方式雖然對 I/O 沒有壓力,因所有緩存數據都寫到內存中。


Malloc 方式啓動:

/usr/local/varnish/sbin/varnishd -u nobody -g nogroup -f /usr/local/varnish/etc/varnish.vcl -s malloc,4G -w 50,51200,120 -a 192.168.10:80 -T 127.0.0.1:8080


Mmap file 方式啓動:

/usr/local/varnish/sbin/varnishd -u nobody -g nogroup -f /usr/local/varnish/etc/varnish.vcl -s file,/data/varnish/varnish_storage.bin,4G -w 50,51200,120 -a 192.168.10:80 -T 127.0.0.1:8080

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 進程的工作模式:

Varnish 啓動或有2個進程 master(management) 進程和 child(worker) 進程。master 讀入存儲配置命令,進行初始化,然後 fork,監控 child。child 則分配線程進行 cache 工作,child 還會做管理線程和生成很多 worker 線程。


child 進程主線程初始化過程中,將存儲大文件整個加載到內存中,如果該文件超出系統的虛擬內存,則會減少原來配置 mmap 大小,然後繼續加載,這時候創建並初始化空閒存儲結構體,放在存儲管理的 struct 中,等待分配。


接着 Varnish 某個負責接受新 http 連接的線程開始等待用戶,如果有新的 http 連接,但是這個線程只負責接收,然後喚醒等待線程池中的 work 線程,進行請求處理。


worker 線程讀入 uri 後,將會查找已有的 object,命中直接返回,沒有命中,則會從後端服務器中取出來,放到緩存中。如果緩存已滿,會根據 LRU 算法,釋放舊的 object。對於釋放緩存,有一個超時線程會檢測緩存中所有 object 的生命週期,如果緩存過期 (ttl),則刪除,釋放相應的存儲內存。

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 的優點:

(1)、Varnish 的穩定性很高,兩者在完成相同負荷的工作時,Squid 服務器發生故障的機率要高於 Varnish,因爲使用Squid 要經常重啓;

(2)、Varnish 訪問速度更快,Varnish 採用了Visual Page Cache技術,所有緩存數據都直接從內存讀取,而 Squid 是從硬盤讀取,因而 Varnish 在訪問速度方面會更快;

(3)、Varnish 可以支持更多的併發連接,因爲 Varnish 的 TCP 連接釋放要比 Squid 快。因而在高併發連接情況下可以支持更多 TCP 連接;

(4)、Varnish 可以通過管理端口,使用正則表達式批量的清除部分緩存,而 Squid 是做不到的;

(5)、Squid 屬於是單進程使用單核 CPU,但 Varnish 是通過 fork 形式打開多進程來做處理,所以是合理的使用所有核來處理相應的請求。


與傳統的 Squid 相比,Varnish 也有缺點:

1)、Varnish 進程一旦 Hang、Crash 或者重啓,緩存數據都會從內存中完全釋放,此時所有請求都會發送到後端服務器,在高併發情況下,會給後端服務器造成很大壓力;

2)、在 Varnish 使用中,如果單個 url 的請求通過 HA/F5(負載均衡)、每次請求不同的 Varnish 服務器中,被請求Varnish 服務器都會被穿透到後端,而且同樣的請求會在多臺服務器上緩存,也會造成 Varnish 的緩存的資源浪費,也會造成性能下降。


解決方案:

1)、綜上所述在訪問量很大的情況下推薦使用 Varnish 的內存緩存方式啓動,而且後面需要跟多臺 Squid 服務器。主要爲了防止前面的 Varnish 服務被重啓的情況下,前期肯定會有很多的穿透,這樣 Squid 可以擔當第二層 CACHE,而且也彌補了 Varnish 緩存在內存中重啓都會釋放的問題;

2)、這樣的問題可以在負載均衡上做 url 哈希,讓單個 url 請求固定請求到一臺 Varnish 服務器上,可以解決該問題。


注:以上內容是在借鑑別人總結的基礎上,做了一些精簡、修正,並加入了一些自己的總結。


-----------------------------------------------------------------------------------------------------------------------------------------------

系統環境:SUSE Linux Enterprise Server 10 SP1 (x86_64)

硬件配置:8G內存、8核CPU


1、編譯安裝

ftp://invisible-island.net/ncurses/ncurses-5.9.tar.gz

# tar xvzf ncurses-5.9.tar.gz

# ./configure --prefix=/usr/local

# make LAGS=-fPIC

# make LAGS=-fPIC install


http://downloads.sourceforge.net/project/pcre/pcre/8.32/pcre-8.32.tar.gz

# tar xvzf  pcre-8.32.tar.gz

# cd pcre-8.32

# ./configure --prefix=/usr/local

# make

# make install


ftp://ftp.gnu.org/gnu/readline/readline-6.2.tar.gz

# tar xvzf readline-6.2.tar.gz

# ./configure --prefix=/usr/local

# make

# make install


http://repo.varnish-cache.org/source/varnish-3.0.4.tar.gz

# tar xvzf varnish-3.0.4.tar.gz

# ./configure --prefix=/usr/local/varnish

# make

# make install

-----------------------------------------------------------------------------------------------------------------------------------------------

2、後端WEB服務健康檢測

# vim /usr/local/varnish/etc/health_check.vcl

probe backend_healthcheck {
    .interval = 5s;
    .timeout = 3s;
    .window = 10;
    .threshold = 8;
         
    .request =
    "GET /index.html HTTP/1.1"
    "Host: mycheckweb.mytest.com"
    "Connection: close"
    "Accept-Encoding: foo/bar";
}

-----------------------------------------------------------------------------------------------------------------------------------------------

3、後端WEB服務定義

# vim /usr/local/varnish/etc/hosts/10.160.22.88_8080.conf

backend WEBSRV_10_160_22_88_8080 {
    .host = "10.160.22.88";
    .port = "8080";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
    .connect_timeout = 50s;
    .between_bytes_timeout = 30s;
    .first_byte_timeout = 30s;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
    .probe = backend_healthcheck;
}

# vim /usr/local/varnish/etc/hosts/10.173.146.35_8080.conf

backend WEBSRV_10_173_146_35_8080 {
    .host = "10.173.146.35";
    .port = "8080";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
    .connect_timeout = 50s;
    .between_bytes_timeout = 30s;
    .first_byte_timeout = 30s;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
    .probe = backend_healthcheck;
}

-----------------------------------------------------------------------------------------------------------------------------------------------

4、集羣定義

# vim /usr/local/varnish/etc/cluster.vcl

include "/usr/local/varnish/etc/health_check.vcl";
include "/usr/local/varnish/etc/hosts/10.160.22.88_8080.conf";
include "/usr/local/varnish/etc/hosts/10.173.146.35_8080.conf";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
director CLUSTER_BACKEND_SERVER round-robin {
    { .backend = WEBSRV_10_160_22_88_8080; }
    { .backend = WEBSRV_10_173_146_35_8080; }
}

-----------------------------------------------------------------------------------------------------------------------------------------------

5、Varnish主配置文件

# vim /usr/local/varnish/etc/varnish.vcl

include "/usr/local/varnish/etc/cluster.vcl";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
acl allow_purge_cache {
    "127.0.0.1";
    "10.0.0.0"/8;
    "172.0.0.0"/8;
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_recv {
    if (req.request == "PURGE") {
        if (!client.ip ~ allow_purge_cache) {
            error 405 "Not Allowed.";
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
        return (lookup);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.host ~ "^(.*).mytest.com") {
        set req.backend = CLUSTER_BACKEND_SERVER;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 動態資源直接拋到後端服務器
    if (req.url ~ "\.(php|asp|aspx|jsp|do|ashx|shtml)($|\?)") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 靜態資源需要去除cookie信息
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        unset req.http.cookie;
        return (lookup);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Cache-Control ~ "no-cache") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
        return (pipe);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Authorization || req.http.Cookie) {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 靜態資源壓縮
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)$") {
            remove req.http.Accept-Encoding;
        } elseif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elseif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            remove req.http.Accept-Encoding;
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 防盜鏈設置
    if (req.http.referer ~ "http://.*") {
        if (!(req.http.referer ~ "http://.*\.qq\.com" ||
            req.http.referer ~ "http://.*\.baidu\.com" ||
            req.http.referer ~ "http://.*\.google\.com.*" ||
            req.http.referer ~ "http://.*\.sogou\.com" ||
            req.http.referer ~ "http://.*\.soso\.com" ||
            req.http.referer ~ "http://.*\.so\.com")) {
            set req.http.host = "www.mytest.com";
            set req.url = "/";
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (!req.backend.healthy) {
        unset req.http.Cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 跳過緩存大文件
    if (req.http.x-pipe && req.restarts > 0) {
        unset req.http.x-pipe;
        return (pipe);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若backend是健康的,則僅grace 5s,如果backend不健康,則grace 1m,主要用於提高併發時的吞吐率
    if (req.backend.healthy) {
        set req.grace = 5s;
    } else {
        set req.grace = 1m;
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_pipe {
    return (pipe);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_pass {
    if (req.request == "PURGE") {
        error 502 "PURGE on a passed object";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_hash {
    hash_data(req.url);
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Accept-Encoding ~ "gzip") {
        hash_data("gzip");
    } elseif (req.http.Accept-Encoding ~ "deflate") {
        hash_data("deflate");
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    return (hash);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_hit {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged.";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_miss {
    if (req.request == "PURGE") {
        purge;
        error 404 "Purged.";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_fetch {
    ## 確保所有Cache中的內容在TTL過期後5分鐘內不被刪除,以應對高併發的場合
    set beresp.grace = 5m;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (beresp.http.Set-Cookie) {
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 如果返回頭有Cache-Control,則刪除Set-Cookie頭
    if (beresp.http.Cache-Control && beresp.ttl > 0s) {
        set beresp.grace = 1m;
        unset beresp.http.Set-Cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 不緩存大於10MB的資源文件
    if (beresp.http.Content-Length ~ "[0-9]{8,}") {
        set req.http.x-pipe = "1";
        return (restart);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.url ~ "\.(php|asp|aspx|jsp|do|ashx|shtml)($|\?)") {
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        unset beresp.http.set-cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 如果返回頭沒有Cache-Control,則標記爲hit_for_pass,強制後續請求回源
    if ((!beresp.http.Cache-Control && !beresp.http.Expires) ||
         beresp.http.Pragma ~ "no-cache" ||
         beresp.http.Cache-Control ~ "(no-cache|no-store|private)") {
        set beresp.ttl = 120s;
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {
        set beresp.ttl = 120s;
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 對不同類型靜態資源進行緩存時間設置
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico)($|\?)") {
        set beresp.ttl = 15m;
    } elseif (req.request == "GET" && req.url ~ "\.(gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        set beresp.ttl = 30m;
    } else {
        set beresp.ttl = 10m;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    return (deliver);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT from " + req.http.host;
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS from " + req.http.host;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 去掉不必要的頭信息
    unset resp.http.X-Powered-By;
    unset resp.http.Server;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    unset resp.http.Via;
    unset resp.http.X-Varnish;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    unset resp.http.Age;
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_error {
    if (obj.status == 503 && req.restarts < 5) {
        set obj.http.X-Restarts = req.restarts;
        return (restart);
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_init {
    return (ok);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_fini {
    return (ok);
}

-----------------------------------------------------------------------------------------------------------------------------------------------

6、Varnish啓動參數配置文件

# vim /usr/local/varnish/etc/varnish.conf

# Configuration file for varnish
#
# /etc/init.d/varnishd expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Maximum number of open files (for ulimit -n)
NFILES=131072
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Set this to 1 to make init script reload try to switch vcl without restart.
# To make this work, you need to set the following variables
# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,
# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE, or in short,
# use Alternative 3, Advanced configuration, below
RELOAD_VCL=1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Main configuration file. You probably want to change it :)
VARNISH_VCL_CONF=/usr/local/varnish/etc/varnish.vcl
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=0.0.0.0
VARNISH_LISTEN_PORT=80
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=8080
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Shared secret file for admin interface
#VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # The minimum number of worker threads to start
VARNISH_MIN_THREADS=10
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # The Maximum number of worker threads to start
VARNISH_MAX_THREADS=5000
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Cache file location
VARNISH_STORAGE_FILE=/data/varnish/varnish_storage.bin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=3G
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Backend storage specification
#VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Default TTL used when the backend does not specify one
VARNISH_TTL=120
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Other optimization parameter
HTTP_RESP_HDR_LEN="http_resp_hdr_len=8192"
HTTP_MAX_HDR="http_max_hdr=256"
HTTP_REQ_HDR_LEN="http_req_hdr_len=8192"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
THREAD_POOLS="thread_pools=8"
THREAD_POOL_MIN="thread_pool_min=50"
THREAD_POOL_MAX="thread_pool_max=5120"
THREAD_POOL_TIMEOUT="thread_pool_timeout=10"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
LRU_INTERVAL="lru_interval=20"
LISTEN_DEPTH="listen_depth=1024"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # DAEMON_OPTS is used by the init script.  If you add or remove options, make
# # sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -t ${VARNISH_TTL} \
             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
             -u nobody -g nogroup \
             -s ${VARNISH_STORAGE} \
             -p ${HTTP_RESP_HDR_LEN} \
             -p ${HTTP_MAX_HDR} \
             -p ${HTTP_REQ_HDR_LEN} \
             -p ${THREAD_POOLS} \
             -p ${THREAD_POOL_MIN} \
             -p ${THREAD_POOL_MAX} \
             -p ${THREAD_POOL_TIMEOUT} \
             -p ${LRU_INTERVAL} \
             -p ${LISTEN_DEPTH}"

-----------------------------------------------------------------------------------------------------------------------------------------------

7、Varnish 配置重載腳本

# vim /usr/local/varnish/sbin/varnish_reload_vcl

#!/bin/bash
#
# reload vcl revisited
# A script that loads new vcl based on data from /usr/local/varnish/etc/varnish.conf
# Ingvar Hagelund <[email protected]>
#
# This is free software, distributed under the standard 2 clause BSD license,
# see the LICENSE file in the Varnish documentation directory
#
# The following environment variables have to be set:
# RELOAD_VCL, VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_PORT
# The following are optional:
# VARNISH_SECRET_FILE, VARNISH_ADMIN_LISTEN_ADDRESS
#
# Requires GNU bash and GNU date
#
                                                                                                                                                                                                                                                                                                                                                                                                                                          
debug=false
                                                                                                                                                                                                                                                                                                                                                                                                                                          
missing() {
    echo "Missing configuration variable: $1"
    exit 2
}
                                                                                                                                                                                                                                                                                                                                                                                                                                          
print_debug() {
    echo "
Parsed configuration:
RELOAD_VCL=\"$RELOAD_VCL\"
VARNISH_VCL_CONF=\"$VARNISH_VCL_CONF\"
VARNISH_ADMIN_LISTEN_ADDRESS=\"$VARNISH_ADMIN_LISTEN_ADDRESS\"
VARNISH_ADMIN_LISTEN_PORT=\"$VARNISH_ADMIN_LISTEN_PORT\"
VARNISH_SECRET_FILE=\"$VARNISH_SECRET_FILE\"
"
}
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Read configuration
. /usr/local/varnish/etc/varnish.conf
                                                                                                                                                                                                                                                                                                                                                                                                                                          
$debug && print_debug
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Check configuration
if [ ! "$RELOAD_VCL" = "1" ]; then
    echo "Error: RELOAD_VCL is not set to 1"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_VCL_CONF" ]; then
    echo "Error: VARNISH_VCL_CONF is not set"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ ! -s "$VARNISH_VCL_CONF" ]; then
    echo "Eror: VCL config $VARNISH_VCL_CONF is unreadable or empty"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_ADMIN_LISTEN_ADDRESS" ]; then
    echo "Warning: VARNISH_ADMIN_LISTEN_ADDRESS is not set, using 127.0.0.1"
    VARNISH_ADMIN_LISTEN_ADDRESS="127.0.0.1"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_ADMIN_LISTEN_PORT" ]; then
    echo "Error: VARNISH_ADMIN_LISTEN_PORT is not set"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
#elif [ -z "$VARNISH_SECRET_FILE" ]; then
#   echo "Warning: VARNISH_SECRET_FILE is not set"
#   secret=""
                                                                                                                                                                                                                                                                                                                                                                                                                                          
#elif [ ! -s "$VARNISH_SECRET_FILE" ]; then
#   echo "Error: varnish secret file $VARNISH_SECRET_FILE is unreadable or empty"
#   exit 2
else
#   secret="-S $VARNISH_SECRET_FILE"
    echo
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Done parsing, set up command
#VARNISHADM="varnishadm $secret -T $VARNISH_ADMIN_LISTEN_ADDRESS:$VARNISH_ADMIN_LISTEN_PORT"
VARNISHADM="/usr/local/varnish/bin/varnishadm -T $VARNISH_ADMIN_LISTEN_ADDRESS:$VARNISH_ADMIN_LISTEN_PORT"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Now do the real work
new_config="reload_$(date +%FT%H:%M:%S)"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Check if we are able to connect at all
if $VARNISHADM vcl.list > /dev/null; then
    $debug && echo vcl.list succeeded
else
    echo "Unable to run $VARNISHADM vcl.list"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.list | awk ' { print $3 } ' | grep -q $new_config; then
    echo Trying to use new config $new_config, but that is already in use
    exit 2
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
current_config=$( $VARNISHADM vcl.list | awk ' /^active/ { print $3 } ' )
                                                                                                                                                                                                                                                                                                                                                                                                                                          
echo "Loading vcl from $VARNISH_VCL_CONF"
echo "Current running config name is $current_config"
echo "Using new config name $new_config"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.load $new_config $VARNISH_VCL_CONF; then
    $debug && echo "$VARNISHADM vcl.load succeded"
else
    echo "$VARNISHADM vcl.load failed"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.use $new_config; then
    $debug && echo "$VARNISHADM vcl.use succeded"
else
    echo "$VARNISHADM vcl.use failed"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
$VARNISHADM vcl.list
echo Done
                                                                                                                                                                                                                                                                                                                                                                                                                                          
exit 0

-----------------------------------------------------------------------------------------------------------------------------------------------

8、Varnish啓動腳本

# vim /etc/init.d/varnishd

#!/bin/bash
#
# varnish Control the Varnish Cache
#
# chkconfig: - 90 10
# description: Varnish is a high-perfomance HTTP accelerator
# processname: varnishd
# config: /usr/local/varnish/etc/varnish.conf
# PIDFILE: /var/run/varnishd.pid
#
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
                                                                                                                                                                                                                                                                                                                                                                                                              
BINFILE="/usr/local/varnish/sbin/varnishd"
PROG="varnishd"
                                                                                                                                                                                                                                                                                                                                                                                                              
RETVAL=0
                                                                                                                                                                                                                                                                                                                                                                                                              
GFILE="/usr/local/varnish/etc/varnish.conf"
PIDFILE="/var/run/varnish.pid"
LOCKFILE="/var/lock/varnish.lock"
RELOAD_EXEC="/usr/local/varnish/sbin/varnish_reload_vcl"
                                                                                                                                                                                                                                                                                                                                                                                                              
[[ -e ${GFILE} ]] && . ${GFILE}
                                                                                                                                                                                                                                                                                                                                                                                                              
start() {
    IS_EXIST=`ps -A -oppid,pid,cmd | grep sbin/${PROG} | grep -v grep`
    [[ -n "${IS_EXIST}" ]] && echo "The process of ${PROG} has been running." && exit 1
                                                                                                                                                                                                                                                                                                                                                                                                                 
    [[ ! -x ${BINFILE} ]] && echo ${BINFILE} has no found && exit 5
    [[ ! -f ${GFILE} ]] && echo ${GFILE} has no found && exit 6
                                                                                                                                                                                                                                                                                                                                                                                                                 
    echo -n "Starting Varnish Cache......"
                                                                                                                                                                                                                                                                                                                                                                                                                 
    ulimit -n ${NFILES:-131072}
    ulimit -l ${MEMLOCK:-82000}
    ulimit -u ${NPROCS:-unlimited}
                                                                                                                                                                                                                                                                                                                                                                                                                 
    if [[ "${DAEMON_OPTS}X" == "X" ]]; then
        echo -n "Please setting DAEMON_OPTS options in ${GFILE}"
        RETVAL=6
    else
        VARNISH_CACHE_DIR=`dirname $VARNISH_STORAGE_FILE`
        RETVAL=`grep -w '^VARNISH_STORAGE' ${GFILE} | grep malloc`
        if [[ "${RETVAL}X" = "X" ]]; then
            mkdir -p ${VARNISH_CACHE_DIR} && chown -R nobody:nogroup ${VARNISH_CACHE_DIR}
        else
            [[ -e ${VARNISH_CACHE_DIR} ]] && rm -rf ${VARNISH_CACHE_DIR}
        fi
                                                                                                                                                                                                                                                                                                                                                                                                                     
        ${BINFILE} -P ${PIDFILE} $DAEMON_OPTS >/dev/null 2>&1
        RETVAL=$?
        echo
        [[ $RETVAL -eq 0 ]] && touch ${LOCKFILE}
    fi
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
stop() {
    echo -n "Stopping Varnish Cache......"
    /sbin/killproc -QUIT ${PROG}
    RETVAL=$?
    echo
    [[ $RETVAL -eq 0 ]] && rm -f ${LOCKFILE}
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
restart() {
    stop
    sleep 1
    start
}
                                                                                                                                                                                                                                                                                                                                                                                                              
reload() {
    [[ "$RELOAD_VCL" = "1" ]] && $RELOAD_EXEC || restart
                                                                                                                                                                                                                                                                                                                                                                                                                 
    RETVAL=$?
    echo
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
configtest() {
    if [[ -f ${GFILE} ]]; then
        ${BINFILE} -f ${GFILE} -C -n /tmp >/dev/null 2>&1
        RETVAL=$?
        [[ $? -eq 0 ]] && echo "The syntax is ok." || echo "The syntax is error."
    else
        RETVAL=$?
        echo "The config file: ${GFILE} is no exists."
    fi
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
case "$1" in
start|stop|restart|reload|configtest)
    $1
    ;;
                                                                                                                                                                                                                                                                                                                                                                                                                 
*)
    echo "Usage: $0 {start|stop|restart|reload|configtest}"
    exit 2
esac
                                                                                                                                                                                                                                                                                                                                                                                                              
exit $RETVAL

# chmod +x /etc/init.d/varnishd

# chkconfig --add varnishd

-----------------------------------------------------------------------------------------------------------------------------------------------

9、Varnish服務健康檢查腳本

# vim /usr/local/varnish/sbin/check_varnish_health.sh

#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
                                                                                                                                                                                                                                                                                                                                                                                                       
## varnish啓動參數配置文件
VARNISH_CONFIG="/usr/local/varnish/etc/varnish.conf"
                                                                                                                                                                                                                                                                                                                                                                                                       
IS_DEBUG=false
                                                                                                                                                                                                                                                                                                                                                                                                       
[[ -e ${VARNISH_CONFIG} ]] && . ${VARNISH_CONFIG}
                                                                                                                                                                                                                                                                                                                                                                                                       
${IS_DEBUG} && echo "監聽地址:${VARNISH_LISTEN_ADDRESS}  監聽端口:${VARNISH_LISTEN_PORT}"
LAN_IPADDR=`/sbin/ifconfig eth1 | awk -F ':' '/inet addr/{print $2}' | sed 's/[a-zA-Z ]//g'`
                                                                                                                                                                                                                                                                                                                                                                                                       
RETVAL=`nmap --system-dns -sT -p ${VARNISH_LISTEN_PORT} ${LAN_IPADDR} | grep open`
[[ -z ${RETVAL} ]] && /sbin/service varnishd restart >/dev/null 2>&1

# crontab -e

*/5 * * * * /usr/local/varnish/sbin/check_varnish_health.sh >/dev/null 2>&1



發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章