ip 独立ip 特定时间内统计到的不通ip
squid统计方法
[root@localhost etc]# cut -d " " -f1 /usr/local/squid/var/logs/access.log|sort|uniq|wc -l
2823
当用到awk时
awk '{print $1}'|sort|uniq test_8.5.2016
10.185.130.78 - - [04/Aug/2016:23:59:29 +0800] "GET http://swcdn.apple.com/content/downloads/52/05/041-9986/25o8fo3gq5m75ylgi9h1vgf4b0r5fovzhr/041-9986.zh_TW.dist HTTP/1.1" 200 53879 TCP_MISS:HIER_DIREC
以上应是cut与awk 的不同之处
pv page view,页面流浪次数
在squid中每一条log就是一个页面的pv，但这个统计是没有意义的嚒（统计那么多的网站的总共pv，有啥用，倒是可以用grep抓取同一域名网站的进行统计）
笼统的统计所有：
[root@localhost etc]# wc -l /usr/local/squid/var/logs/access.log|awk '{print $1}'
3874236
根据不通域名统计
（通过for循环实现）
[root@localhost etc]# for i in `cut -d " " -f7 test_8.5.2016 |cut -d "/" -f3|sort|uniq`;do echo "${i} is visited (times)"; grep $i test_8.5.2016 -c ;done
cn180156 is visited (times)
1
dungcoivb.googlepages.com is visited (times)
1
lh-hn-505 is visited (times)
8
swcdn.apple.com is visited (times)
4
www.whatismyip.com is visited (times)
1
下面来到郑州服务器实践
!/bin/bash
#Version 1.0
#Author Scott
#Mail [email protected]
#Introduction This is for count website that was visited a few hours ago in special log
cut -d " " -f7 /bash/script/log|sort|uniq \
>/bash/script/website.log
for i in `grep -v ^htt /bash/script/website.log`
do
echo -ne "$i is visited\n"
grep -c $i /bash/script/log
done
for j in `cut -d "/" -f3 /bash/script/website.log|uniq`
do
echo "$j is visited"
grep $j /bash/script/log -c
done
以上程序跑了1.5个小时还没出结果，log共有300W个
改进程序
#!/bin/bash
#Version 1.1
#Author Scott
#Mail [email protected]
#Introduction This is for count website that was visited a few hours ago in special log
cut -d " " -f7 /bash/scripts/log|sort\
>/bash/scripts/website.log
grep -v ^htt /bash/scripts/website.log|uniq -c >loged.log &&\
cut -d "/" -f3 /bash/scripts/website.log|uniq -c >loged.log
再改进
[root@hadphost scripts]# wc -l log
2805719 log
[root@hadphost scripts]# cat pv_test.sh
#!/bin/bash
#Version 1.2
#Author Scott
#Mail [email protected]
#Introduction This is for count website that was visited a few hours ago in special log
cut -d " " -f7 /bash/scripts/log|sort\
>/bash/scripts/website.log
grep -v ^htt /bash/scripts/website.log|uniq -c >loged.log &&\
cut -d "/" -f3 /bash/scripts/website.log|uniq -c >>loged.log &&\
sort -rbn loged.log >queue.log
[root@hadphost scripts]# wc -l queue.log
18408 queue.log
[root@hadphost scripts]# fg
time sh pv_test.sh
real 1m40.616s
user 1m37.560s
sys 0m1.807s
uv 统计不通客服端个数
squid的log信息占时没有相关数据，故不做研究了