Java封裝Azkaban相關API

Java封裝Azkaban相關API

版本說明：

azkaban：3.43.0

jdk：1.8

項目地址：https://github.com/shirukai/azkaban-java-api.git

1 前言

之前在項目開發記錄中，寫到過兩篇文章《利用AOP對Azkaban進行登錄控制》和 《Java調用Azkaban相關服務》，記錄了在開發過程中使用spring的aop對azkaban進行了登錄控制，以及使用Java請求azkaban相關的服務。總的來說，能夠完成基本的需求，但是還是存在一些問題，比如：

• 深度依賴Spring，利用AOP切面進行的登錄控制以及RestTemplate的HTTP請求
• Azkaban本身REST API缺陷，響應風格不統一，導致在之前的Java調用Azakban相關服務時候，請求響應統一被當做字符串處理，後期根據需要單獨處理，不友好。
• 登錄控制不精確

爲解決如上問題，進行了代碼重構。對於Azkaban API的請求脫離Spring的RestTemplate使用http-client的fluent，統一了請求響應，使用動態代理替換之前Spring的AOP，使用構建器使封裝的Azkaban API支持可插拔，可以脫離Spring使用也可以整合Spring0。

2 統一Azkaban響應

這裏爲什麼要統一Azkaban響應呢？我們先看一下官網關於API的說明文檔。

上圖截了一個登錄響應描述的圖，看這個描述，參數是error，描述如果登錄失敗會返回錯誤信息，參數session.id 如果登錄成功返回session.id。響應內容不定，還有條件語句，有錯誤也不報個錯誤碼，也沒有個狀態描述。如果單純是這樣的邏輯就算了，看如下的官網給的簡單實例

{
  "status" : "success",
  "session.id" : "c001aba5-a90f-4daf-8f11-62330d034c0a"
}

what?怎麼還有個status？意義何在，響應成功，我就給你返回一個"status":“success”，失敗了，直接返回"error":“info”，難道不應該返回一個"status":"error"嗎？對於其它的接口，響應也有不同的呈現，難道對響應的處理邏輯還要一個API一個嘛。所以這裏爲了解決響應不統一的問題，對Azkaban的響應結果進行了一層封裝。

2.1 創建響應實體類

如下所示爲響應實體類，其中BaseResponse爲響應基類。

BaseResponse內容如下，其中爲了映射Azkaban的響應，既包含了"status"又包含了"error"，最後通過correction會更正信息到"status"，所以我們可以統一對"status"進行判斷是否執行成功。內容如下：

package com.azkaban.response;

import java.util.Objects;

/**
 * Created by shirukai on 2019-06-01 15:03
 */
public class BaseResponse {
    public static final String SUCCESS = "success";
    public static final String ERROR = "error";

    /**
     * 響應狀態
     */
    private String status;
    /**
     * 錯誤類型(單純爲了映射Azkaban)
     */
    private String error;
    /**
     * 詳細信息
     */
    private String message;

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }

    public String getError() {
        return error;
    }

    public void setError(String error) {
        this.error = error;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    /**
     * 更正信息
     */
    public void correction() {
        if (!ERROR.equals(this.status) && Objects.isNull(this.error)) {
            this.status = SUCCESS;
        } else {
            this.status = ERROR;
            if (Objects.isNull(this.error)) {
                this.error = this.message;
            } else if (Objects.isNull(this.message)) {
                this.message = this.error;
            }
        }
    }
}

2.2 響應處理器

爲了統一響應，這裏使用響應處理器，對Azkaban響應進行統一處理，內容如下：

package com.azkaban.response;

import com.alibaba.fastjson.JSONObject;
import org.apache.http.HttpEntity;
import org.apache.http.client.fluent.Request;
import org.apache.http.client.fluent.Response;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Objects;

/**
 * Created by shirukai on 2019-06-01 14:58
 * 響應處理器
 */
public class ResponseHandler {
    private static Logger log = LoggerFactory.getLogger(ResponseHandler.class);

    public static <T extends BaseResponse> T handle(Request request, Class<T> tClass) {
        T response = null;
        try {
            Response res = request.execute();
            HttpEntity entity  = res.returnResponse().getEntity();
            response = handle(entity, tClass);
        } catch (Exception e) {
            try {
                response = tClass.newInstance();
                response.setStatus(T.ERROR);
                response.setError(e.getMessage());
                response.correction();
            } catch (Exception ea) {
                log.warn(ea.getMessage());
            }
        }
        return response;
    }

    public static BaseResponse handle(Request request) {
        return handle(request, BaseResponse.class);
    }


    public static BaseResponse handle(String content) {
        return handle(content, BaseResponse.class);
    }

    public static <T extends BaseResponse> T handle(HttpEntity entity, Class<T> tClass) {
        T response = null;
        try {
            String content = EntityUtils.toString(entity);
            response = handle(content, tClass);
        } catch (Exception e) {
            log.warn(e.getMessage());
        }
        return response;
    }

    public static <T extends BaseResponse> T handle(String content, Class<T> tClass) {
        T response = null;
        try {
            response = JSONObject.parseObject(content, tClass);
            if (Objects.nonNull(response.getError())) {
                response.setStatus(T.ERROR);
            }
            response.correction();
        } catch (Exception e) {
            try {
                response = tClass.newInstance();
                response.setStatus(T.ERROR);
                response.setError(content);
                response.correction();
            } catch (Exception ea) {
                log.warn(ea.getMessage());
            }
        }
        return response;
    }
}

3 API 接口及實現

這裏整理了常用的14種常見的Azkaban的API，在使用Java實現之前，使用Postman測試過，並生成了一份postman的接口文檔，可以訪問https://documenter.getpostman.com/view/2759292/S1TbUaeU查看。

3.1 創建API接口類

在com.azkaban.api包下創建AzkabanApi接口類，提供Azkaban相對應API的接口，內容如下所示：

package com.azkaban.api;

import com.azkaban.response.*;


/**
 * Created by shirukai on 2019-06-01 20:12
 * azkaban api 接口
 */
public interface AzkabanApi {

    /**
     * 創建項目 API
     *
     * @param name 項目名稱
     * @param desc 項目描述
     * @return BaseResponse
     */
    BaseResponse createProject(String name, String desc);

    /**
     * 刪除項目 API
     *
     * @param name 項目名稱
     * @return BaseResponse
     */
    BaseResponse deleteProject(String name);

    /**
     * 上傳Zip API
     *
     * @param filePath    zip文件路徑
     * @param projectName 項目名稱
     * @return ProjectZipResponse
     */
    ProjectZipResponse uploadProjectZip(String filePath, String projectName);

    /**
     * 查詢項目Flows
     *
     * @param projectName 項目名稱
     * @return FetchFlowsResponse
     */
    FetchFlowsResponse fetchProjectFlows(String projectName);

    /**
     * 執行flow
     *
     * @param projectName 項目名稱
     * @param flowName    flow名稱
     * @return ExecuteFlowResponse
     */
    ExecuteFlowResponse executeFlow(String projectName, String flowName);

    /**
     * 取消執行flow
     *
     * @param execId 執行ID
     * @return BaseResponse
     */
    BaseResponse cancelFlow(String execId);

    /**
     * 查詢執行Flow信息
     *
     * @param execId 執行ID
     * @return FetchExecFlowResponse
     */
    FetchExecFlowResponse fetchExecFlow(String execId);


    /**
     * 查詢執行Job的日誌
     *
     * @param execId 執行ID
     * @param jobId  JobID
     * @param offset 起始位置
     * @param length 長度
     * @return FetchExecJobLogs
     */
    FetchExecJobLogs fetchExecJobLogs(String execId, String jobId, int offset, int length);

    /**
     * 查詢Flow的執行記錄
     *
     * @param projectName 項目名稱
     * @param flowName    flow名稱
     * @param start       開始位置
     * @param length      查詢條數
     * @return FetchFlowExecutionsResponse
     */
    FetchFlowExecutionsResponse fetchFlowExecutions(String projectName, String flowName, int start, int length);

    /**
     * 查詢所有項目
     *
     * @return FetchAllProjectsResponse
     */
    FetchAllProjectsResponse fetchAllProjects();


    /**
     * 設置定時任務
     *
     * @param projectName    項目名稱
     * @param flowName       Flow名稱
     * @param cronExpression cron表達式
     * @return ScheduleCronFlowResponse
     */
    ScheduleCronFlowResponse scheduleCronFlow(String projectName, String flowName, String cronExpression);

    /**
     * 查詢定時任務
     *
     * @param projectId 項目ID
     * @param flowId    Flow ID
     * @return FetchScheduleResponse
     */
    FetchScheduleResponse fetchSchedule(String projectId, String flowId);

    /**
     * 移除定時任務
     *
     * @param scheduleId schedule ID
     * @return BaseResponse
     */
    BaseResponse removeSchedule(String scheduleId);
}

3.2 創建接口實現類

接口有了，接下來就是實現接口，創建AzkabanApiImpl實現類，其中請求主要使用了http-client的fluent，內容如下：

package com.azkaban.api;

import com.azkaban.response.*;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpEntity;
import org.apache.http.client.fluent.Form;
import org.apache.http.client.fluent.Request;
import org.apache.http.client.fluent.Response;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.util.EntityUtils;

import java.io.File;
import java.io.IOException;
import java.text.MessageFormat;

/**
 * Created by shirukai on 2019-05-31 16:46
 * Azkaban 操作相關API實現類
 */
public class AzkabanApiImpl implements AzkabanApi {
    private String username;
    private String password;
    private String uri;
    private String sessionId = "b1d4f665-f4b9-4e7d-b83a-b928b41cc323";
    private static final String DELETE_PROJECT = "{0}/manager?delete=true&project={1}&session.id={2}";
    private static final String FETCH_PROJECT_FLOWS = "{0}/manager?ajax=fetchprojectflows&session.id={1}&project={2}";
    private static final String EXECUTE_FLOW = "{0}/executor?ajax=executeFlow&session.id={1}&project={2}&flow={3}";
    private static final String CANCEL_FLOW = "{0}/executor?ajax=cancelFlow&session.id={1}&execid={2}";
    private static final String FETCH_EXEC_FLOW = "{0}/executor?ajax=fetchexecflow&session.id={1}&execid={2}";
    private static final String FETCH_EXEC_JOB_LOGS = "{0}/executor?ajax=fetchExecJobLogs&session.id={1}&execid={2}" +
            "&jobId={3}&offset={4}&length={5}";
    private static final String FETCH_FLOW_EXECUTIONS = "{0}/manager?ajax=fetchFlowExecutions&session.id={1}" +
            "&project={2}&flow={3}&start={4}&length={5}";
    private static final String FETCH_ALL_PROJECTS = "{0}/index?ajax=fetchuserprojects&session.id={1}";
    private static final String SCHEDULE_CRON_FLOW = "{0}/schedule?ajax=scheduleCronFlow&session.id={1}&" +
            "projectName={2}&flow={3}&cronExpression={4}";
    private static final String FETCH_SCHEDULE = "{0}/schedule?ajax=fetchSchedule&session.id={1}&projectId={2}&flowId={3}";

    public AzkabanApiImpl(String uri, String username, String password) {
        this.uri = uri;
        this.username = username;
        this.password = password;
    }

    /**
     * 登錄 API
     *
     * @return LoginResponse
     */
    public LoginResponse login() throws IOException {
        Response res = Request.Post(uri)
                .bodyForm(Form.form()
                        .add("action", "login")
                        .add("username", username)
                        .add("password", password).build())
                .execute();
        HttpEntity entity = res.returnResponse().getEntity();
        String content = EntityUtils.toString(entity).replace("session.id", "sessionId");
        LoginResponse response = ResponseHandler.handle(content, LoginResponse.class);
        if (StringUtils.isNotEmpty(response.getSessionId())) {
            this.sessionId = response.getSessionId();
        }
        return response;
    }

    @Override
    public BaseResponse createProject(String name, String desc) {
        Request res = Request.Post(uri + "/manager")
                .bodyForm(Form.form()
                        .add("session.id", sessionId)
                        .add("action", "create")
                        .add("name", name)
                        .add("description", desc).build());
        return ResponseHandler.handle(res);
    }

    @Override
    public BaseResponse deleteProject(String name) {
        Request res = Request.Get(MessageFormat.format(DELETE_PROJECT, uri, name, sessionId));
        return ResponseHandler.handle(res);
    }

    @Override
    public ProjectZipResponse uploadProjectZip(String filePath, String projectName) {
        HttpEntity entity = MultipartEntityBuilder
                .create()
                .addBinaryBody("file", new File(filePath))
                .addTextBody("session.id", sessionId)
                .addTextBody("ajax", "upload")
                .addTextBody("project", projectName)
                .build();
        Request res = Request.Post(uri + "/manager")
                .body(entity);
        return ResponseHandler.handle(res, ProjectZipResponse.class);
    }

    @Override
    public FetchFlowsResponse fetchProjectFlows(String projectName) {
        Request res = Request.Get(MessageFormat.format(FETCH_PROJECT_FLOWS, uri, sessionId, projectName));
        return ResponseHandler.handle(res, FetchFlowsResponse.class);
    }

    @Override
    public ExecuteFlowResponse executeFlow(String projectName, String flowName) {
        Request res = Request.Post(MessageFormat.format(EXECUTE_FLOW, uri, sessionId, projectName, flowName));
        return ResponseHandler.handle(res, ExecuteFlowResponse.class);
    }

    @Override
    public BaseResponse cancelFlow(String execId) {
        Request res = Request.Post(MessageFormat.format(CANCEL_FLOW, uri, sessionId, execId));
        return ResponseHandler.handle(res);
    }

    @Override
    public FetchExecFlowResponse fetchExecFlow(String execId) {
        Request res = Request.Get(MessageFormat.format(FETCH_EXEC_FLOW, uri, sessionId, execId));
        return ResponseHandler.handle(res, FetchExecFlowResponse.class);
    }

    @Override
    public FetchExecJobLogs fetchExecJobLogs(String execId, String jobId, int offset, int length) {
        Request res = Request.Get(
                MessageFormat.format(FETCH_EXEC_JOB_LOGS, uri, sessionId, execId, jobId, String.valueOf(offset),
                        String.valueOf(length))
        );
        return ResponseHandler.handle(res, FetchExecJobLogs.class);
    }

    @Override
    public FetchFlowExecutionsResponse fetchFlowExecutions(String projectName,
                                                           String flowName,
                                                           int start,
                                                           int length) {
        Request res = Request.Get(
                MessageFormat.format(FETCH_FLOW_EXECUTIONS, uri, sessionId, projectName, flowName,
                        String.valueOf(start), String.valueOf(length))
        );
        return ResponseHandler.handle(res, FetchFlowExecutionsResponse.class);
    }


    @Override
    public FetchAllProjectsResponse fetchAllProjects() {
        Request res = Request.Get(MessageFormat.format(FETCH_ALL_PROJECTS, sessionId));
        return ResponseHandler.handle(res, FetchAllProjectsResponse.class);
    }

    @Override
    public ScheduleCronFlowResponse scheduleCronFlow(String projectName, String flowName, String cronExpression) {
        Request res = Request.Post(
                MessageFormat.format(SCHEDULE_CRON_FLOW, uri, sessionId, projectName, flowName, cronExpression)
        );
        return ResponseHandler.handle(res, ScheduleCronFlowResponse.class);
    }

    @Override
    public FetchScheduleResponse fetchSchedule(String projectId, String flowId) {
        Request res = Request.Get(MessageFormat.format(FETCH_SCHEDULE, uri, sessionId, projectId, flowId));
        return ResponseHandler.handle(res, FetchScheduleResponse.class);
    }

    @Override
    public BaseResponse removeSchedule(String scheduleId) {
        Request res = Request.Post(uri + "/schedule")
                .bodyForm(Form.form()
                        .add("session.id", sessionId)
                        .add("action", "removeSched")
                        .add("scheduleId", scheduleId).build());
        return ResponseHandler.handle(res);
    }
}

4 利用動態代理實現登錄控制

思想與之前的AOP一樣，在代理方法執行完成後，判斷執行結果是否異常，如果異常，調用login方法，進行Azkaban登錄，然後重新執行代理方法。

4.1 創建ApiInvocationHandler

ApiInvocationHander繼承java.lang.reflect.InvocationHandler，重寫invoke()方法，是動態代理的調用處理器。其中主要實現了三個功能，一個是代理AzkabanApiImpl提供的方法，二是判斷執行結果是否異常，如果異常進行登錄，三是進行統一的異常處理。代碼如下：

package com.azkaban.proxy;


import com.azkaban.api.AzkabanApiImpl;
import com.azkaban.response.BaseResponse;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.lang.reflect.Field;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;

/**
 * Created by shirukai on 2019-06-01 20:17
 * API 動態代理處理類
 */
public class ApiInvocationHandler implements InvocationHandler {
    /**
     * api 接口實現類實例
     */
    private AzkabanApiImpl azkabanApi;
    /**
     * 重試次數
     */
    private static final Integer RETRY = 2;
    /**
     * object默認方法
     */
    private List<String> defaultMethods;
    private static Logger log = LoggerFactory.getLogger(ApiInvocationHandler.class);

    ApiInvocationHandler(AzkabanApiImpl azkabanApi) {
        this.azkabanApi = azkabanApi;
        this.defaultMethods = new ArrayList<>(16);
        for (Method method : Object.class.getMethods()) {
            this.defaultMethods.add(method.getName());
        }
    }

    /**
     * 判斷是否爲默認方法
     *
     * @param methodName 方面名稱
     * @return boolean
     */
    private boolean isDefault(String methodName) {
        return this.defaultMethods.contains(methodName);
    }

    /**
     * 重寫動態代理invoke方法
     *
     * @param proxy  代理實例
     * @param method 方法
     * @param args   參數
     * @return 執行結果
     */
    @Override
    public Object invoke(Object proxy, Method method, Object[] args) {
        Object result = null;
        int tryTime = 1;
        try {
            while (tryTime <= RETRY) {
                // 指定代理方法
                result = method.invoke(azkabanApi, args);
                // 判斷是否執行成功
                if (Objects.nonNull(result) && !this.isDefault(method.getName())) {
                    Class superClass = result.getClass().getSuperclass();
                    if (Object.class.equals(superClass)) {
                        superClass = result.getClass();
                    }
                    Field field = superClass.getDeclaredField("status");
                    field.setAccessible(true);
                    if (BaseResponse.SUCCESS.equals(field.get(result))) {
                        log.info("Execute the azkaban's API {} successfully.", method.getName());
                        return result;
                    }
                    azkabanApi.login();
                }

                tryTime++;
            }
        } catch (Exception e) {
            // 如果返回結果爲null,捕獲異常並實重新生成結果實例，設置異常信息
            if (Objects.isNull(result)) {
                Class returnType = method.getReturnType();
                try {
                    Object response = returnType.newInstance();
                    if (response instanceof BaseResponse) {
                        BaseResponse baseResponse = (BaseResponse) response;
                        baseResponse.setStatus(BaseResponse.ERROR);
                        if (e instanceof InvocationTargetException) {
                            baseResponse.setMessage(((InvocationTargetException) e).getTargetException().getMessage());
                        } else {
                            baseResponse.setMessage(e.getMessage());
                        }
                        baseResponse.correction();
                        result = response;
                    }

                } catch (InstantiationException | IllegalAccessException ex) {
                    ex.printStackTrace();
                }
            }
        }
        return result;
    }

}

4.2 創建AzkabanApiProxyBuilder

AzkabanApiProxyBuilder是動態代理的構建器，通過構建器能夠構建出代理的AzkabanApi實例。構建器需要傳入azkaban的服務地址、用戶名、密碼。代碼如下：

package com.azkaban.proxy;


import com.azkaban.api.AzkabanApi;
import com.azkaban.api.AzkabanApiImpl;

import java.lang.reflect.Proxy;

/**
 * Created by shirukai on 2019-06-01 22:26
 * azkaban api Builder
 */
public class AzkabanApiProxyBuilder {
    private String uri;
    private String username;
    private String password;

    private AzkabanApiProxyBuilder() {
    }


    public static AzkabanApiProxyBuilder create() {
        return new AzkabanApiProxyBuilder();
    }

    public AzkabanApiProxyBuilder setUri(String uri) {
        this.uri = uri;
        return this;
    }

    public AzkabanApiProxyBuilder setUsername(String username) {
        this.username = username;
        return this;
    }

    public AzkabanApiProxyBuilder setPassword(String password) {
        this.password = password;
        return this;
    }

    public AzkabanApi build() {
        AzkabanApiImpl impl = new AzkabanApiImpl(this.uri, this.username, this.password);
        ApiInvocationHandler handler = new ApiInvocationHandler(impl);
        return (AzkabanApi) Proxy.newProxyInstance(
                impl.getClass().getClassLoader(),
                impl.getClass().getInterfaces(),
                handler);
    }

}

5 兩種方式調用API

5.1 普通方式調用

無論在什麼時候使用，只要使用代理構建器構建出AzkabanApi實例即可。如下代碼所示：

    @Test
    public void builder() {
        AzkabanApi apis = AzkabanApiProxyBuilder.create()
                .setUri("http://localhost:8666")
                .setUsername("azkaban")
                .setPassword("azkaban")
                .build();
    }

5.2 整合Spring

通過AzkabanApiConfig類，創建Bean註冊到Spring裏，代碼如下所示：

package com.azkaban.config;


import com.azkaban.api.AzkabanApi;
import com.azkaban.proxy.AzkabanApiProxyBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Created by shirukai on 2019-06-03 11:05
 * 配置API，創建Bean，並注入Spring
 */
@Configuration
public class AzkabanApiConfig {
    @Value("${azkaban.url}")
    private String uri;

    @Value("${azkaban.username}")
    private String username;

    @Value("${azkaban.password}")
    private String password;

    @Bean
    public AzkabanApi azkabanApi() {
        return AzkabanApiProxyBuilder.create()
                .setUri(uri)
                .setUsername(username)
                .setPassword(password)
                .build();
    }
}

調用

    @Autowired
    private AzkabanApi azkabanApi;

6 總結

之前寫過兩篇關於Azkaban的文章，都是關於使用Java調用Azkaban的API。其中有同學問具體代碼，這次把重構後的代碼提交到了github上了，項目地址：https://github.com/shirukai/azkaban-java-api.git。