presto代理層實現

用過presto的同學都知道，客服端訪問presto集羣，其實是通過http協議訪問的presto的coordinator節點；大多數情況是沒啥問題；可是服務器直接暴露給用戶，或多或少還是有些安全顧慮；所以我們在客戶端和presto集羣之間加了一層訪問代理，如下圖所示

所有的用戶請求通過代理訪問presto集羣；過程如下
1.客戶端可以使用jdbc，Python等方式訪問代理，使用方式和直接訪問集羣方式一樣，只不過以前的host是coordinator，而現在是presot-proxy
2.代理獲得客戶端的請求後封裝post請求訪問presto集羣，presto集羣響應請求的報文如下，代理獲得presto集羣的響應報文後，返回給客戶端

{
    "id":"20181107_132829_00014_u28jg",
    "infoUri":"http://192.168.120.4:8099/ui/query.html?20181107_132829_00014_u28jg",
    "nextUri":"http://192.168.120.4:8099/v1/statement/20181107_132829_00014_u28jg/1",
    "stats":{
        "state":"QUEUED",
        "queued":true,
        "scheduled":false,
        "nodes":0,
        "totalSplits":0,
        "queuedSplits":0,
        "runningSplits":0,
        "completedSplits":0,
        "cpuTimeMillis":0,
        "wallTimeMillis":0,
        "queuedTimeMillis":0,
        "elapsedTimeMillis":0,
        "processedRows":0,
        "processedBytes":0,
        "peakMemoryBytes":0
    }
}

3.客戶端獲得報文後，根據報文中nextUri直接發送GET請求給presto集羣的coordinator節點
4.coordinator節點負責分發任務給node，計算並返回結果

我們從如上過程，可以看到，其實presto-proxy只是處理client的第一次請求，並返回報文；剩下的所有請求都是client根據報文中的nextUri直接發送GET請求給presto集羣；對於用戶來說，他們能看到的只是步驟1，其他的步驟對用戶來說都是透明的

現在來實現一下這個proxy，其實很簡單，核心代碼也就十幾行

package com.fan.prestoproxy.controller;

import org.apache.commons.io.IOUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import javax.servlet.http.HttpServletRequest;
import java.io.BufferedReader;
import java.io.InputStream;
import java.util.Calendar;
import java.util.Enumeration;
import java.util.Locale;

@Controller
@RequestMapping("/v1")
public class QueryController {

    @ResponseBody
    @RequestMapping(value= "statement",method = RequestMethod.POST,produces="application/json;charset=UTF-8")
    public String getResponse(HttpServletRequest request ) throws Exception {
        DefaultHttpClient hc = new  DefaultHttpClient();  //初始化一個HTTP的客戶端對象
        HttpPost httppost = new HttpPost("http://192.168.120.4:8099/v1/statement");
        BufferedReader reader = request.getReader();
        String statement = reader.readLine();
        httppost.setHeader("X-Presto-Source", request.getHeader("x-presto-source"));
        httppost.setHeader("X-Presto-User", request.getHeader("x-presto-user"));
        httppost.setHeader("User-Agent", request.getHeader("user-agent"));
        httppost.setHeader("X-Presto-Time-Zone", Calendar.getInstance().getTimeZone().getID());
        httppost.setHeader("X-Presto-Language", Locale.CHINA.getLanguage());

        Enumeration<String> headerNames = request.getHeaderNames();
        while(headerNames.hasMoreElements()){
            String key = headerNames.nextElement();
            System.out.println(key+"   "+request.getHeader(key));
        }

        httppost.setEntity(new StringEntity(statement));
        HttpResponse httpresponse = hc.execute(httppost);
        InputStream is = httpresponse.getEntity().getContent();
        String body = IOUtils.toString(is, "utf-8");
        System.out.println(body);
        return body;
    }

}

http://192.168.120.4:8099/v1/statement也就是presto集羣對外提供的接口；測試代碼如下，就是普通的jdbc代碼，只不過jdbc的host指向的是proxy的host

 public static void jdbc()throws  Exception{
        Class.forName("com.facebook.presto.jdbc.PrestoDriver");
        String url = "jdbc:presto://localhost:8080";
        Connection connection = DriverManager.getConnection(url, "user", null);
        Statement statement = connection.createStatement();
        long begin = System.currentTimeMillis();
        String sql = "select *  from mysql.shiro.sh_user";
        ResultSet rs = statement.executeQuery(sql) ;

        long end = System.currentTimeMillis();
        System.out.println((end - begin) + " ms ");

        while(rs.next()){
            System.out.println(rs.getString("name")+ "," + rs.getString("passwd"));
        }
        statement.close();
        connection.close();
    }

代理的作用不侷限於此，其實在代理層我們可以實現自己的數據訪問權限，如果用戶的請求不合規，我們也可以直接在代理層拋出異常

end