浅谈httpclient

做java的对httpclient应该是不会陌生的吧，应该算比较常见的包了，这两天项目中有同学遇到了httpclient的问题，从团队同学解决问题的思路、对这块知识的了解程度上看，发现很多人都停留在用的层面。其实，对这块我也是一知半解。有一知半解的地方就可以做出，所以浅谈一下，一方面是自己对这块理解更深，另外一方面发现很久没总结，没写文章了，要坚持下去。

httpclient算是神器了，当年大学时候就喜欢用这玩意,给自己的博客刷个访问量，比如：当年的网易博客，不以uid为维度计数，只以页面count计数，当年用的还是VC（因为VC写出来的exe，直接双击就可以运行了，逼格高，话说大学写代码的动力不就是装逼么），用VC自己socket+http协议封装了一个类似c版的httpclient。还做过很多，比如当年创业时候从人人网拉流量，申请一堆大波妹账号，然后遍历留言，出现验证码就请求回来做验证码破解（之前有文章讲过），总之用这类似的玩意做了挺多。

牛皮吹完了，回归正题，用过java的httpclient的应该都大致了解这玩意有俩jar包，并且都是apache的，尼玛好多困惑有么有，是不是每次用这玩意就1.从老项目中拷贝一个前人已经写好的httpclientUtil.java然后自己就用了，能通就对了！2.百度httpclient，拷贝一个例子，写个测试用例访问下百度，返回成功就哦了。但是好像大家都不清楚这俩httpclient之间啥区别。

httpclient:org.apache.httpcomponents » httpclient

commons-httpclient:commons-httpclient » commons-httpclient

apache提供了以上两个jar，都有httpclient，他们之前的关系是这样的。

1. 你可以认为他们都是Httpclient，commons-httpclient是3.x以前版本，httpclient是4.x以后的版本

2. commons-httpclient后续是不会在做维护了，apache给的建议也是用新的httpclient

3. 从功能上来讲，httpclient比commons-httpclient的功能要强大很多，比如：常见的HTTPS的支持、代理等。。。

其实我也是有点不理解的，因为从编码的易用性上来讲commons-httpclient貌似对研发者更友好，httpclient虽然功能很强大，但是使用方法上不是太友好

如下示例：

commons-httpclient

get示例

public static String get(String url, Map<String, String> queryParams) {
        if (StringUtils.isBlank(url)) {
            return null;
        }
        HttpClient httpclient = new HttpClient();
        GetMethod getMethod = new GetMethod(url);
        //组装参数
        if (queryParams != null && !queryParams.isEmpty()) {
            NameValuePair[] pairs = new NameValuePair[queryParams.size()];
            int i = 0;
            for (Map.Entry<String, String> entry : queryParams.entrySet()) {
                pairs[i] = new NameValuePair(entry.getKey(), entry.getValue());
                i++;
            }
            getMethod.setQueryString(pairs);
        }

        String resultString = null;
        try {
            int statusCode = httpclient.executeMethod(getMethod);
            if (statusCode == HttpStatus.SC_OK) {
                //获取请求响应内容
                resultString = getMethod.getResponseBodyAsString();
            } else {
                throw new EmcooperateException("网络访问异常,错误状态：" + statusCode);
            }
        } catch (HttpException e) {
            throw new EmcooperateException("网络访问异常", e);
        } catch (IOException e) {
            throw new EmcooperateException("网络访问异常", e);
        }
        return resultString;
    }

post示例

public static String post(String url, Map<String, String> queryParams) {
        if (StringUtils.isBlank(url)) {
            return null;
        }
        HttpClient httpclient = new HttpClient();
        PostMethod postMethod = new PostMethod(url);
        //组装参数
        if (queryParams != null && !queryParams.isEmpty()) {
            NameValuePair[] pairs = new NameValuePair[queryParams.size()];
            int i = 0;
            for (Map.Entry<String, String> entry : queryParams.entrySet()) {
                pairs[i] = new NameValuePair(entry.getKey(), entry.getValue());
                i++;
            }
            postMethod.setQueryString(pairs);
        }

        String resultString = null;
        try {
            int statusCode = httpclient.executeMethod(postMethod);
            if (statusCode == HttpStatus.SC_OK) {
                //获取请求响应内容
                resultString = postMethod.getResponseBodyAsString();
                if (logger.isInfoEnabled()) {
                    logger.info("http返回值 :" + resultString);
                }
            } else {
                if (logger.isInfoEnabled()) {
                    logger.info("http返回状态值 :" + statusCode);
                }
                throw new EmcooperateException("网络访问异常,错误状态：" + statusCode);
            }
        } catch (HttpException e) {
            throw new EmcooperateException("网络访问异常", e);
        } catch (IOException e) {
            throw new EmcooperateException("网络访问异常", e);
        }
        return resultString;
    }

如上：post、get使用都分几步

1. new httpcliet

2. new xxxMethod

3. 构造参数NameValuePair

4. 执行httpclient.executeMethod(xxxMethod)

使用步骤很明确，就是Httpclient、xxxMethod、NameValuePair几个关键的类，并且post和get采用类似的编码步骤，基本上看了就懂，并且现在网上大部分的示例都是这么个用法

接下来看看httpclient的使用：（http://hc.apache.org/httpcomponents-client-4.5.x/quickstart.html）

CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse response1 = httpclient.execute(httpGet);

try {
    System.out.println(response1.getStatusLine());
    HttpEntity entity1 = response1.getEntity();
    EntityUtils.consume(entity1);
} finally {
    response1.close();
}

HttpPost httpPost = new HttpPost(url);
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("username", "vip"));
nvps.add(new BasicNameValuePair("password", "secret"));
httpPost.setEntity(new UrlEncodedFormEntity(nvps));

CloseableHttpResponse response2 = httpclient.execute(httpPost);

try {
    HttpEntity entity2 = response2.getEntity();
    EntityUtils.consume(entity2);
} finally {
    response2.close();
}

可以看到httpget并没有设置参数的地方，那么参数是如何设置呢？原来参数默认是在url中包装好的。为了能快速的包装url，httpcliet提供了URIBuilder来实现参数的封装

URIBuilder builder = new URIBuilder();
            
URI uri=builder.setScheme("http")
               .setHost("www.google.com")
               .setPath("/search")
               .setParameter("q", "httpclient")
               .setParameter("btnG", "Google Search")
               .setParameter("aq", "f")
               .setParameter("oq", "").build();
HttpGet httpget = new HttpGet(uri);

最后通过setEntity来做数据的传递，这里有各种类型的数据支持参见：http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/fundamentals.html#d5e43 （1.7.1章节）

从上可以看出，httpclient4x开始基本上很少做通用的封装了，更多的是类似http协议的编程实现方式，带来的优势是可以保证各种http协议的支持，缺点就是不理解http协议（head、cookie、response）的同学会觉得用起来很麻烦，毕竟没有像commons-httpclient那么套路了，但是功能确实强大了很多，随便操作。

最后，commons-httpclient做普通的http都没太大问题，但是在做https的访问时候可能就会遇到ssl的签名验证相关问题，这个是因为jdk的

X509TrustManagerImpl会坐checkServerTrusted的证书验证于是就会报错。那么如何能通过httpclient实现https的访问呢。参考下面示例

public class HttpClientUtil2 {
    /** logger */
    private final static Logger logger = LoggerFactory.getLogger(HttpClientUtil2.class);

    public  static void ignoreHttps(HttpClient httpClient) {//设置httpclient自己的x509验证，忽略证书
        
        // 创建TrustManager
        X509TrustManager xtm = new X509TrustManager() {
          public void checkClientTrusted(X509Certificate[] chain,
              String authType) throws CertificateException {
              System.out.println("1");
          }
          
          public void checkServerTrusted(X509Certificate[] chain,
              String authType) throws CertificateException {
              System.out.println("2");

          }
          
          public X509Certificate[] getAcceptedIssuers() {
              System.out.println("3");

            return new X509Certificate[] {};
          }
        };
        try {
          SSLContext ctx = SSLContext.getInstance("SSL");
          
          // 使用TrustManager来初始化该上下文，TrustManager只是被SSL的Socket所使用
          ctx.init(null, new TrustManager[] { xtm }, null);
          
          SSLSocketFactory sf = new SSLSocketFactory(
              ctx,
              SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
          Scheme sch = new Scheme("https", 443, sf);
          httpClient.getConnectionManager().getSchemeRegistry().register(sch);

        } catch (Exception e) {
        } finally {
          // 关闭连接,释放资源
//              httpClient.getConnectionManager().shutdown(); 
        }
      }
    /**
     * http get请求 
     * @param url 请求的url
     * @param queryParams 请求参数
     * @return 响应文本
     */
    public static String get(String url, Map<String, String> queryParams) {
        if (StringUtils.isBlank(url)) {
            return null;
        }
        HttpClient httpclient = new DefaultHttpClient();      
        //ignoreHttps(httpclient);  //这里默认不忽略，那么访问12306就会报错
        String params="?";
        //组装参数
        if (queryParams != null && !queryParams.isEmpty()) {
            List <NameValuePair> nvps = new ArrayList <NameValuePair>();
            for (Map.Entry<String, String> entry : queryParams.entrySet()) {
                nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
                params=params+entry.getKey()+"="+ entry.getValue()+"&";
            }
        }
        HttpGet getMethod = new HttpGet(url+params);
       


        String resultString = null;
        try {
            
            
            HttpResponse statusCode = httpclient.execute(getMethod);
            if (statusCode.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
                //获取请求响应内容
                resultString = EntityUtils.toString(statusCode.getEntity());
            } else {
                throw new EmcooperateException("网络访问异常,错误状态：" + statusCode);
            }
        } catch (Exception e) {
            throw new EmcooperateException("网络访问异常", e);
        } 
        return resultString;
    }

    /**
     * http post请求 
     * @param url 请求的url
     * @param queryParams 请求参数
     * @return 响应文本
     */
    public static String post(String url, Map<String, String> queryParams) {
        if (StringUtils.isBlank(url)) {
            return null;
        }
        HttpClient httpclient = new DefaultHttpClient();
        buildHttps(httpclient);

        HttpPost postMethod = new HttpPost(url);
        
        String resultString = null;
        try {
          //组装参数
            if (queryParams != null && !queryParams.isEmpty()) {
                List <NameValuePair> nvps = new ArrayList <NameValuePair>();
                for (Map.Entry<String, String> entry : queryParams.entrySet()) {
                    nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
                }
                postMethod.setEntity(new UrlEncodedFormEntity(nvps));
            }

            HttpResponse statusCode = httpclient.execute(postMethod);
            if (statusCode.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
                //获取请求响应内容
                resultString = EntityUtils.toString(statusCode.getEntity());
                if (logger.isInfoEnabled()) {
                    logger.info("http返回值 :" + resultString);
                }
            } else {
                if (logger.isInfoEnabled()) {
                    logger.info("http返回状态值 :" + statusCode);
                }
                throw new EmcooperateException("网络访问异常,错误状态：" + statusCode);
            }
        } catch (Exception e) {
            throw new EmcooperateException("网络访问异常", e);
        } 
        return resultString;
    }
    
    public static void main(String args[]) { 
       Map<String, String> queryParams =new HashMap<String,String>();
       System.out.println(HttpClientUtil2.get("https://www.12306.cn/mormhweb/", queryParams));
        
    } 
}

lightdb hash index的性能和限制

diy面向請求的java框架

scala安裝 scala eclipse插件安裝 scala簡單示例

java字節碼

String和Xml之間的轉換、String轉InputStream

java調用https的webservice，https的wsdl

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結