这几天想做用现在比较流行的HtmlAgilityPack重写下采集的功能，看了一些HtmlAgilityPack 的介绍，感觉非常好用，还内置了htmlWeb用来http请求。但是发现有的开启了gzip压缩的网页请求时会报错。
原来的代码如下：
|
1 2 |
HtmlWeb webClient = new HtmlWeb(); HtmlDocument doc = webClient.Load(this.getUrl()); |
报错信息为“gzip”不是受支持的编码名。
在谷歌上搜索了半天，终于找到解决方案，而且不必更换HttpRequest或WebClient进行请求。同时还可以用此方法设置cookie，render伪装等等。。。
解决后代码如下：
HtmlAgilityPack.HtmlWeb.PreRequestHandler handler = delegate(HttpWebRequest request)
{
request.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
request.CookieContainer = new System.Net.CookieContainer();
return true;
};
webClient.PreRequest += handler;
HtmlDocument doc = webClient.Load(this.getUrl());
更多参考代码：
string url = "https://kaijiang.500.com/qxc.shtml";
HtmlWeb web = new HtmlWeb();
web.PreRequest += delegate (HttpWebRequest request)
{
request.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
request.CookieContainer = new System.Net.CookieContainer();
return true;
};
HtmlDocument doc = web.Load(url);
HtmlNode table = doc.DocumentNode.SelectSingleNode("/html/body/div[5]/div[3]/div[2]/div[1]/div[2]/table[1]");
table = HtmlNode.CreateNode(table.OuterHtml);
HtmlNode strong = table.SelectSingleNode("//strong");
if (strong == null)
throw new Exception("获取期数失败，位置strong");
string timeStr = strong.InnerText;
timeStr = Regex.Replace(timeStr, @"[^0-9]+", "");
HtmlNode ul = table.SelectSingleNode("//ul");
if (ul == null)
throw new Exception("获取开奖记录失败，位置ul");
string val = ul.InnerText;
//获取字符串中的数字
val = Regex.Replace(val, @"[^0-9]+", "");
更多：
C# HTML解析工具HtmlAgilityPack XPath 模糊查询not()函数和contains()函数