OkHttp與Cookie及Cookie的持久化

參考：OkkHttp3之Cookie管理、模擬登錄知乎並抓取用戶信息

一、OkHttp3下的Cookie的使用

①、OkHttpClient取消了setCookieHandler(CookieHandler cookieHandler);

改而使用：

setCookieJar(CookieJar cookieJar);

CookieJar是一個接口，需要自己實現CookieJar的定義。

CookieJar cookieJar = new CookieJar() {
	//存儲數據		
	@Override
	public void saveFromResponse(HttpUrl arg0, List<Cookie> arg1) {
		// TODO Auto-generated method stub
			
<span style="white-space:pre">	</span>}
<span style="white-space:pre">	</span>//讀取數據	
	@Override
	public List<Cookie> loadForRequest(HttpUrl arg0) {
		// TODO Auto-generated method stub
		return null;
	}
};

並且使用OkHttp的Cookie類來存儲Cookie，而不使用自帶的HttpCookie類。

②、OkHttp推薦使用Builder來創建OkHttpClient對象，來配置OkHttpClient

OkHttpClient.Builder builder = new OkHttpClient.Builder();
//設置超時時常
builder.connectTimeout(timeout, unit);
//設置Cookie管理器
builder.cookieJar(cookieJar);
//等等。。查看文檔

二、使用OkHttpClient獲取知乎關注的人的數據（暱稱和簡介）非持久化

①、初始化CookieJar

//初始化Cookie管理器
		CookieJar cookieJar = new CookieJar() {
			//Cookie緩存區
			private final Map<String, List<Cookie>> cookiesMap = new HashMap<String, List<Cookie>>();
			@Override
			public void saveFromResponse(HttpUrl arg0, List<Cookie> arg1) {
				// TODO Auto-generated method stub
				//移除相同的url的Cookie
				String host = arg0.host();
				List<Cookie> cookiesList = cookiesMap.get(host);
				if (cookiesList != null){
					cookiesMap.remove(host);
				}
				//再重新天添加
				cookiesMap.put(host, <span style="font-family: Arial, Helvetica, sans-serif;">arg1</span><span style="font-family: Arial, Helvetica, sans-serif;">);</span>
			}
			
			@Override
			public List<Cookie> loadForRequest(HttpUrl arg0) {
				// TODO Auto-generated method stub
				List<Cookie> cookiesList = cookiesMap.get(arg0.host());
				//注：這裏不能返回null，否則會報NULLException的錯誤。
				//原因：當Request 連接到網絡的時候，OkHttp會調用loadForRequest()
				return cookiesList != null ? cookiesList : new ArrayList<Cookie>();
			}
		};

②、登陸知乎

1、查看知乎的Post請求

首先通過F12打開瀏覽器的開發者工具(本人用的Chrome)

點擊到紅色的位置，並勾選藍色位置的選項。

2、登陸知乎：https://www.zhihu.com/

登陸完成後，會在開發者工具發現該文件(本人是手機登陸所以是phone_num，如果是郵箱登陸是emila文件)

然後我們查看我們要上傳的數據和上傳的地址

這裏有個_xsfr：需要在登陸的html中的form表單中查找其真正的value

本人的值是：bf284aba4cc706ebfc5ebcba1c4f97fc。據測試，該值在同一個瀏覽器中提交是不會變的。

③、使用OkHttp登陸，並獲取Cookie

//創建OkHttpClient
		OkHttpClient client = new OkHttpClient.Builder()
						.connectTimeout(5000, TimeUnit.MILLISECONDS)
						.cookieJar(cookieJar)
						.build();
		//創建登陸的表單
		FormBody loginBody = new FormBody.Builder()
		  			.add("_xsrf", "bf284aba4cc706ebfc5ebcba1c4f97fc")
		  			.add("password", "cay1314159")
		  			.add("captcha_type", "cn")
		  			.add("remember_me", "true")
		  			.add("phone_num", "15520762775")
		  			.build();//賬號密碼自己填
		//創建Request請求
		Request loginRequest = new Request.Builder()
						.url("https://www.zhihu.com/login/phone_num")
						.post(loginBody)
						.build();
		//上傳
		Call loginCall = client.newCall(loginRequest);
		
		try {
			//非異步執行
			Response loginResponse = loginCall.execute();
			//測試是否登陸成功
			System.out.println(loginResponse.body().string());
			//獲取返回數據的頭部
			Headers headers = loginResponse.headers();
			HttpUrl loginUrl = loginRequest.url();
			//獲取頭部的Cookie,注意：可以通過Cooke.parseAll()來獲取
			List<Cookie> cookies = Cookie.parseAll(loginUrl, headers);
			//防止header沒有Cookie的情況
			if (cookies != null){
				//存儲到Cookie管理器中
				client.cookieJar().saveFromResponse(loginUrl, cookies);//這樣就將Cookie存儲到緩存中了
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

④、獲取關注的人

本人獲取數據的地址：https://www.zhihu.com/people/chen-yan-xiang-83/followees

		//獲取需要提交的CookieStr
		StringBuilder cookieStr = new StringBuilder();
		//從緩存中獲取Cookie
		List<Cookie> cookies = client.cookieJar().loadForRequest(loginRequest.url());
		//將Cookie數據弄成一行
		for(Cookie cookie : cookies){
			cookieStr.append(cookie.name()).append("=").append(cookie.value()+";");
		}
		System.out.println(cookieStr.toString());
		//設置提交的請求
		Request attentionRequest = new Request.Builder()
							.url("https://www.zhihu.com/people/chen-yan-xiang-83/followees")
							.header("Cookie", cookieStr.toString())
							.build();
		Call attentionCall = client.newCall(attentionRequest);
		try {
			//連接網絡
			Response attentionResponse = attentionCall.execute();
			if (attentionResponse.isSuccessful()){
				//獲取返回的數據
				String data = attentionResponse.body().string();
				//測試
				System.out.println(data);
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

⑤、通過Jsoup獲取html文件的數據

Document document = Jsoup.parse(data);
Elements attentions = document.select("div.zm-profile-card");
for(Element attention : attentions){
	System.out.println("name："+attention.select("h2").text()+"  簡介："+attention.select("span").text());
}

⑥、完整代碼

public static void main(String[]args){
		//初始化Cookie管理器
		CookieJar cookieJar = new CookieJar() {
			//Cookie緩存區
			private final Map<String, List<Cookie>> cookiesMap = new HashMap<String, List<Cookie>>();
			@Override
			public void saveFromResponse(HttpUrl arg0, List<Cookie> arg1) {
				// TODO Auto-generated method stub
				//移除相同的url的Cookie
				String host = arg0.host();
				List<Cookie> cookiesList = cookiesMap.get(host);
				if (cookiesList != null){
					cookiesMap.remove(host);
				}
				//再重新天添加
				cookiesMap.put(host, arg1);
			}
			
			@Override
			public List<Cookie> loadForRequest(HttpUrl arg0) {
				// TODO Auto-generated method stub
				List<Cookie> cookiesList = cookiesMap.get(arg0.host());
				//注：這裏不能返回null，否則會報NULLException的錯誤。
				//原因：當Request 連接到網絡的時候，OkHttp會調用loadForRequest()
				return cookiesList != null ? cookiesList : new ArrayList<Cookie>();
			}
		};
		//創建OkHttpClient
		OkHttpClient client = new OkHttpClient.Builder()
						.connectTimeout(5000, TimeUnit.MILLISECONDS)
						.cookieJar(cookieJar)
						.build();
		//創建登陸的表單
		FormBody loginBody = new FormBody.Builder()
		  			.add("_xsrf", "bf284aba4cc706ebfc5ebcba1c4f97fc")
		  			.add("password", "cay1314159")
		  			.add("captcha_type", "cn")
		  			.add("remember_me", "true")
		  			.add("phone_num", "15520762775")
		  			.build();//賬號密碼自己填
		//創建Request請求
		Request loginRequest = new Request.Builder()
						.url("https://www.zhihu.com/login/phone_num")
						.post(loginBody)
						.build();
		//上傳
		Call loginCall = client.newCall(loginRequest);
		
		try {
			//非異步執行
			Response loginResponse = loginCall.execute();
			//測試是否登陸成功
			System.out.println(loginResponse.body().string());
			//獲取返回數據的頭部
			Headers headers = loginResponse.headers();
			HttpUrl loginUrl = loginRequest.url();
			//獲取頭部的Cookie,注意：可以通過Cooke.parseAll()來獲取
			List<Cookie> cookies = Cookie.parseAll(loginUrl, headers);
			//防止header沒有Cookie的情況
			if (cookies != null){
				//存儲到Cookie管理器中
				client.cookieJar().saveFromResponse(loginUrl, cookies);//這樣就將Cookie存儲到緩存中了
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		
		
		//獲取需要提交的CookieStr
		StringBuilder cookieStr = new StringBuilder();
		//從緩存中獲取Cookie
		List<Cookie> cookies = client.cookieJar().loadForRequest(loginRequest.url());
		//將Cookie數據弄成一行
		for(Cookie cookie : cookies){
			cookieStr.append(cookie.name()).append("=").append(cookie.value()+";");
		}
		System.out.println(cookieStr.toString());
		//設置提交的請求
		Request attentionRequest = new Request.Builder()
							.url("https://www.zhihu.com/people/chen-yan-xiang-83/followees")
							.header("Cookie", cookieStr.toString())
							.build();
		Call attentionCall = client.newCall(attentionRequest);
		try {
			//連接網絡
			Response attentionResponse = attentionCall.execute();
			if (attentionResponse.isSuccessful()){
				//獲取返回的數據
				String data = attentionResponse.body().string();
				//測試
				System.out.println(data);
				//解析數據
				Document document = Jsoup.parse(data);
				Elements attentions = document.select("div.zm-profile-card");
				for(Element attention : attentions){
					System.out.println("name："+attention.select("h2").text()+"  簡介："+attention.select("span").text());
				}
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

⑦、殘留的問題

有些時候登陸的時候會要求填寫驗證碼，由於驗證碼需要手動數據，還要使用到GUI交互，所以就沒寫。所以改代碼只能在不需要提交驗證碼的情況下使用。之後會用Android的方式補上

⑧、封裝CookieJar

根據上面的使用情況，我們可以封裝三個部分：

1、當獲取到Response類的時候，我們將獲取到的Cookie放入CookieJar，是可封裝的

2、從CookieJar中獲取Cookie，並連成一個CookieString是可以封裝了。

3、設定Request請求頭的Cookie是可以封裝的。

將其封裝在HttpEngine類中

1、第一類封裝

public void receiveHeaders(Headers headers) throws IOException {
    if (client.cookieJar() == CookieJar.NO_COOKIES) return;

    List<Cookie> cookies = Cookie.parseAll(userRequest.url(), headers);
    if (cookies.isEmpty()) return;

    client.cookieJar().saveFromResponse(userRequest.url(), cookies);
  }

2、第二類封裝

private String cookieHeader(List<Cookie> cookies) {
    StringBuilder cookieHeader = new StringBuilder();
    for (int i = 0, size = cookies.size(); i < size; i++) {
      if (i > 0) {
        cookieHeader.append("; ");
      }
      Cookie cookie = cookies.get(i);
      cookieHeader.append(cookie.name()).append('=').append(cookie.value());
    }
    return cookieHeader.toString();
  }

3、第三類封裝

private Request networkRequest(Request request) throws IOException {
    Request.Builder result = request.newBuilder();
    
    List<Cookie> cookies = client.cookieJar().loadForRequest(request.url());
    if (!cookies.isEmpty()) {
      result.header("Cookie", cookieHeader(cookies));
    }

    return result.build();
}

三、Cookie的持久化

參考android-async-http的PersistentCookieStore類與SerializableHttpCookie類

*PersistentCookieStore類：該類用的是SharePreference存儲，也可以換成外置的文件存儲


public class PersistentCookieStore {
    private static final String LOG_TAG = "PersistentCookieStore";
    private static final String COOKIE_PREFS = "Cookies_Prefs";

    private final Map<String, ConcurrentHashMap<String, Cookie>> cookies;
    private final SharedPreferences cookiePrefs;


    public PersistentCookieStore(Context context) {
        cookiePrefs = context.getSharedPreferences(COOKIE_PREFS, 0);
        cookies = new HashMap<>();

        //將持久化的cookies緩存到內存中 即map cookies
        Map<String, ?> prefsMap = cookiePrefs.getAll();
        for (Map.Entry<String, ?> entry : prefsMap.entrySet()) {
            String[] cookieNames = TextUtils.split((String) entry.getValue(), ",");
            for (String name : cookieNames) {
                String encodedCookie = cookiePrefs.getString(name, null);
                if (encodedCookie != null) {
                    Cookie decodedCookie = decodeCookie(encodedCookie);
                    if (decodedCookie != null) {
                        if (!cookies.containsKey(entry.getKey())) {
                            cookies.put(entry.getKey(), new ConcurrentHashMap<String, Cookie>());
                        }
                        cookies.get(entry.getKey()).put(name, decodedCookie);
                    }
                }
            }
        }
    }

    protected String getCookieToken(Cookie cookie) {
        return cookie.name() + "@" + cookie.domain();
    }

    public void add(HttpUrl url, Cookie cookie) {
        String name = getCookieToken(cookie);

        //將cookies緩存到內存中 如果緩存過期 就重置此cookie
        if (!cookie.persistent()) {
            if (!cookies.containsKey(url.host())) {
                cookies.put(url.host(), new ConcurrentHashMap<String, Cookie>());
            }
            cookies.get(url.host()).put(name, cookie);
        } else {
            if (cookies.containsKey(url.host())) {
                cookies.get(url.host()).remove(name);
            }
        }

        //講cookies持久化到本地
        SharedPreferences.Editor prefsWriter = cookiePrefs.edit();
        prefsWriter.putString(url.host(), TextUtils.join(",", cookies.get(url.host()).keySet()));
        prefsWriter.putString(name, encodeCookie(new SerializableOkHttpCookies(cookie)));
        prefsWriter.apply();
    }

    public List<Cookie> get(HttpUrl url) {
        ArrayList<Cookie> ret = new ArrayList<>();
        if (cookies.containsKey(url.host()))
            ret.addAll(cookies.get(url.host()).values());
        return ret;
    }

    public boolean removeAll() {
        SharedPreferences.Editor prefsWriter = cookiePrefs.edit();
        prefsWriter.clear();
        prefsWriter.apply();
        cookies.clear();
        return true;
    }

    public boolean remove(HttpUrl url, Cookie cookie) {
        String name = getCookieToken(cookie);

        if (cookies.containsKey(url.host()) && cookies.get(url.host()).containsKey(name)) {
            cookies.get(url.host()).remove(name);

            SharedPreferences.Editor prefsWriter = cookiePrefs.edit();
            if (cookiePrefs.contains(name)) {
                prefsWriter.remove(name);
            }
            prefsWriter.putString(url.host(), TextUtils.join(",", cookies.get(url.host()).keySet()));
            prefsWriter.apply();

            return true;
        } else {
            return false;
        }
    }

    public List<Cookie> getCookies() {
        ArrayList<Cookie> ret = new ArrayList<>();
        for (String key : cookies.keySet())
            ret.addAll(cookies.get(key).values());

        return ret;
    }

    /**
     * cookies 序列化成 string
     *
     * @param cookie 要序列化的cookie
     * @return 序列化之後的string
     */
    protected String encodeCookie(SerializableOkHttpCookies cookie) {
        if (cookie == null)
            return null;
        ByteArrayOutputStream os = new ByteArrayOutputStream();
        try {
            ObjectOutputStream outputStream = new ObjectOutputStream(os);
            outputStream.writeObject(cookie);
        } catch (IOException e) {
            Log.d(LOG_TAG, "IOException in encodeCookie", e);
            return null;
        }

        return byteArrayToHexString(os.toByteArray());
    }

    /**
     * 將字符串反序列化成cookies
     *
     * @param cookieString cookies string
     * @return cookie object
     */
    protected Cookie decodeCookie(String cookieString) {
        byte[] bytes = hexStringToByteArray(cookieString);
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
        Cookie cookie = null;
        try {
            ObjectInputStream objectInputStream = new ObjectInputStream(byteArrayInputStream);
            cookie = ((SerializableOkHttpCookies) objectInputStream.readObject()).getCookies();
        } catch (IOException e) {
            Log.d(LOG_TAG, "IOException in decodeCookie", e);
        } catch (ClassNotFoundException e) {
            Log.d(LOG_TAG, "ClassNotFoundException in decodeCookie", e);
        }

        return cookie;
    }

    /**
     * 二進制數組轉十六進制字符串
     *
     * @param bytes byte array to be converted
     * @return string containing hex values
     */
    protected String byteArrayToHexString(byte[] bytes) {
        StringBuilder sb = new StringBuilder(bytes.length * 2);
        for (byte element : bytes) {
            int v = element & 0xff;
            if (v < 16) {
                sb.append('0');
            }
            sb.append(Integer.toHexString(v));
        }
        return sb.toString().toUpperCase(Locale.US);
    }

    /**
     * 十六進制字符串轉二進制數組
     *
     * @param hexString string of hex-encoded values
     * @return decoded byte array
     */
    protected byte[] hexStringToByteArray(String hexString) {
        int len = hexString.length();
        byte[] data = new byte[len / 2];
        for (int i = 0; i < len; i += 2) {
            data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4) + Character.digit(hexString.charAt(i + 1), 16));
        }
        return data;
    }
}

*SerializableHttpCookie類

public class SerializableOkHttpCookies implements Serializable {

    private transient final Cookie cookies;
    private transient Cookie clientCookies;

    public SerializableOkHttpCookies(Cookie cookies) {
        this.cookies = cookies;
    }

    public Cookie getCookies() {
        Cookie bestCookies = cookies;
        if (clientCookies != null) {
            bestCookies = clientCookies;
        }
        return bestCookies;
    }

    private void writeObject(ObjectOutputStream out) throws IOException {
        out.writeObject(cookies.name());
        out.writeObject(cookies.value());
        out.writeLong(cookies.expiresAt());
        out.writeObject(cookies.domain());
        out.writeObject(cookies.path());
        out.writeBoolean(cookies.secure());
        out.writeBoolean(cookies.httpOnly());
        out.writeBoolean(cookies.hostOnly());
        out.writeBoolean(cookies.persistent());
    }

    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        String name = (String) in.readObject();
        String value = (String) in.readObject();
        long expiresAt = in.readLong();
        String domain = (String) in.readObject();
        String path = (String) in.readObject();
        boolean secure = in.readBoolean();
        boolean httpOnly = in.readBoolean();
        boolean hostOnly = in.readBoolean();
        boolean persistent = in.readBoolean();
        Cookie.Builder builder = new Cookie.Builder();
        builder = builder.name(name);
        builder = builder.value(value);
        builder = builder.expiresAt(expiresAt);
        builder = hostOnly ? builder.hostOnlyDomain(domain) : builder.domain(domain);
        builder = builder.path(path);
        builder = secure ? builder.secure() : builder;
        builder = httpOnly ? builder.httpOnly() : builder;
        clientCookies =builder.build();
    }
}

OkHttp與Cookie及Cookie的持久化

參考：OkkHttp3之Cookie管理、模擬登錄知乎並抓取用戶信息

一、OkHttp3下的Cookie的使用

二、使用OkHttpClient獲取知乎關注的人的數據（暱稱和簡介）非持久化

①、初始化CookieJar

②、登陸知乎

③、使用OkHttp登陸，並獲取Cookie

④、獲取關注的人

⑤、通過Jsoup獲取html文件的數據

⑥、完整代碼

⑦、殘留的問題

⑧、封裝CookieJar

三、Cookie的持久化

MySQL 核心模塊揭祕 | 18 期 | 鎖在內存里長什麼樣*

使用perf工具生成火焰圖

HttpSecurity 是如何組裝過濾器鏈的

數說海南——近6年海南各市縣人口簡單看

長序列中Transformers的高級注意力機制總結

大齡程序員思考

響應式界面控件DevExtreme * 更強的數據分析和可視化功能

第二十一條：函數對象表示策略

網絡——Cookie的使用

第二十條：類層次由於標籤類

第二十二條：優先考慮靜態成員類

Http網絡框架的構建

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結

OkHttp與Cookie及Cookie的持久化

參考：OkkHttp3之Cookie管理 、 模擬登錄知乎並抓取用戶信息

一、OkHttp3下的Cookie的使用

二、使用OkHttpClient獲取知乎關注的人的數據（暱稱和簡介） 非持久化

①、初始化CookieJar

②、登陸知乎

③、使用OkHttp登陸，並獲取Cookie

④、獲取關注的人

⑤、通過Jsoup獲取html文件的數據

⑥、完整代碼

⑦、殘留的問題

⑧、封裝CookieJar

三、Cookie的持久化

參考：OkkHttp3之Cookie管理、模擬登錄知乎並抓取用戶信息

二、使用OkHttpClient獲取知乎關注的人的數據（暱稱和簡介）非持久化