使用java Future模式異步調用詳細實例展示

    java Future模式想必大家都比較熟悉，大體實現起來也比較簡單，因爲模式單一，我先介紹一下一般步驟，再講一下，目前項目中遇到具體問題的解決方式

    一般來說，使用java Future模式實現多線程，具體步驟如下，

    1.新建一個異步任務類，如 xxxTask 實現 Callable<xxxTask.Result>(或者Runnable<xxx>)

    2.重寫call方法，返回值爲xxxTask.Result

    3.調用時，使用線程池調用，如TaskExxcutor.exe()即可

具體場景可能比較複雜，我說一下目前我使用比較規範的寫法，

準備工作：

    1.寫好線程池調用：

import com.google.common.util.concurrent.ThreadFactoryBuilder;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Future;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * Created on 2019/1/9
 * 使用線程池異步處理任務
 *
 * @author lvyunxiao
 */
@Data
@Slf4j
@NoArgsConstructor
public class TaskExecutor {

    /**
     * 線程池核心池的大小
     */
    private static final int CORE_POOL_SIZE = 50;

    /**
     * 線程池的最大線程數
     */
    private static final int MAXIMUM_POOL_SIZE = 100;

    /**
     * 當線程數大於核心時，此爲終止前多餘的空閒線程等待新任務的最長時間。
     */
    private static final long KEEP_ALIVE_TIME = 60;

    /**
     * keepAliveTime 的時間單位
     */
    private static final TimeUnit TIME_UNIT = TimeUnit.SECONDS;

    /**
     * 用來儲存等待執行任務的隊列
     */
    private static final BlockingQueue<Runnable> WORK_QUEUE = new LinkedBlockingDeque<>(1024);

    private static final String NAME_FORMATE = "alpha-thread-factory-%d";

    /**
     * 線程工廠
     */
    private static final ThreadFactory THREAD_FACTORY = new ThreadFactoryBuilder()
            .setNameFormat(NAME_FORMATE).build();

    /**
     * 拒絕策略
     */
    private static final RejectedExecutionHandler HANDLER = new ThreadPoolExecutor.DiscardOldestPolicy();

    /**
     * 創建線程池
     */
    private static final ThreadPoolExecutor THREAD_POOL_EXECUTOR = new ThreadPoolExecutor(
            CORE_POOL_SIZE,
            MAXIMUM_POOL_SIZE,
            KEEP_ALIVE_TIME,
            TIME_UNIT,
            WORK_QUEUE,
            THREAD_FACTORY,
            HANDLER
    );

    /**
     * 使用執行器
     */
    private static final ExecutorService EXEC = THREAD_POOL_EXECUTOR;

    /**
     * 執行不返回參數的任務
     *
     * @param task 任務
     */
    public static void submit(Runnable task) {
        EXEC.submit(task);
    }

    /**
     * 執行返回參數的任務
     *
     * @param task 任務
     * @param <V>  參數
     * @return 返回值
     */
    public static <V> Future<V> submit(Callable<V> task) {
        return EXEC.submit(task);
    }

}

具體參數解釋的很清楚了

2.具體場景的bean類

如，我遇到了一個展示歷史任務狀態展示的列表，數據需要從mesos/marathon中取，實時展示

樣例如圖：

爲此，編寫bean：

/**
 * Created on 2019/1/27
 *
 * @author lvyunxiao
 */
@Data
@NoArgsConstructor
public class ResourceStatus {

    private int id;
    private String type;
    private String role;
    private String appId;
    private String startTime;
    private String endTime;
    private String state;
    private String image;
    private String host;
    private double cpu;
    private double mem;
    private String dockerName;
    private String logPath;
    private String basePath;
    private double cpuUtilization;
    private double memUtilization;
    private String memUse;
    private String cluster;
    private String date;

    @Override
    public String toString() {
        return FastJsonUtils.toJSONString(this);
    }

}

然後，編寫任務Task類，實現Callable接口，順便，需要什麼方法和參數也加進去

/**
 * Created on 2019/1/29
 *
 * @author lvyunxiao
 */
@Data
@Slf4j
public class HistoryResTask implements Callable<HistoryResTask.HistoryResTaskResult> {

    private final String keyword;
    private final String cluster;
    private final String mesosSlavesHost;

    public HistoryResTask(final String keyword, final String cluster, final String mesosSlavesHost) {
        this.cluster = cluster;
        this.keyword = keyword;
        this.mesosSlavesHost = mesosSlavesHost;
    }

    /**
     * get role by app's id
     *
     * @param appId app id
     * @return role
     */
    private String getRoleByAppId(String appId) {
        if (appId.contains(SCHEDULER)) {
            return SCHEDULER;
        } else {
            return SWORKER;
        }
    }

    /**
     * get state map from list of Statues
     *
     * @param statuses list of statues
     * @return state map
     * eg:
     * {
     * "state":"TASK_xxx",
     * "start":"2018-01-01 xx:xx:xx",
     * "end":"2019-12-31 xx:xx:xx",
     * }
     */
    private ResourceMagServiceImpl.StateMap extractStatues(List<Statuses> statuses) {
        ResourceMagServiceImpl.StateMap stateMap = new ResourceMagServiceImpl.StateMap();
        for (Statuses status : statuses) {
            String state = status.getState();
            final double timestamp = status.getTimestamp();
            final String timestampFormat = DateUtils.getDateFromDoubleTimeStamp(timestamp);
            if (TASK_RUNNING.equals(state)) {
                stateMap.setStart(timestampFormat);
            } else {
                stateMap.setState(state);
                stateMap.setEnd(timestampFormat);
            }
        }
        if (StringUtils.isNotEmpty(stateMap.getStart()) && StringUtils.isEmpty(stateMap.getState())) {
            stateMap.setState(TASK_RUNNING);
            stateMap.setEnd(DEFAULT_END_TIME);
        }
        if (StringUtils.isEmpty(stateMap.getStart())) {
            stateMap.setStart(DEFAULT_START_TIME);
        }
        return stateMap;
    }

    /**
     * get log path from completedtask
     *
     * @param task completed task
     * @return log path
     */
    private String getLogPathFromTask(CompletedTasks task) {
        String logPath = "";
        Container container = task.getContainer();
        if (ObjectUtils.notEqual(container, null)) {
            List<Volumes> volumes = container.getVolumes();
            if (CollectionUtils.isNotEmpty(volumes)) {
                for (Volumes volume : volumes) {
                    logPath = volume.getHostPath();
                }
            }
        }
        return logPath;
    }

    @Override
    public HistoryResTaskResult call() throws Exception {
        List<ResourceStatus> resourceStatuses = Lists.newArrayList();
        ResourceStatus resourceStatus;
        // async
        MesosSlaveState mesosSlaveState = ClusterInfoUtils.getMesosSlaveState(mesosSlavesHost);
        List<Frameworks> frameworks = mesosSlaveState.getFrameworks();
        for (Frameworks framework : frameworks) {
            List<CompletedExecutors> completedExecutors = framework.getCompletedExecutors();
            for (CompletedExecutors completedExecutor : completedExecutors) {
                // base path
                String directory = completedExecutor.getDirectory();
                String id = completedExecutor.getId();
                if (!id.contains(keyword)) {
                    continue;
                }
                List<CompletedTasks> completedTasks = completedExecutor.getCompletedTasks();
                for (CompletedTasks completedTask : completedTasks) {
                    for (CompletedTasks task : completedTasks) {
                        resourceStatus = new ResourceStatus();
                        // role
                        resourceStatus.setRole(getRoleByAppId(task.getId()));
                        resourceStatus.setCluster(cluster);
                        // host basePath logPath
                        resourceStatus.setHost(mesosSlavesHost);
                        resourceStatus.setLogPath(getLogPathFromTask(task));
                        resourceStatus.setBasePath(directory);
                        // cpu memory
                        Resources resources = task.getResources();
                        resourceStatus.setCpu(resources.getCpus());
                        resourceStatus.setMem(resources.getMem());
                        // type
                        resourceStatus.setType(ResourceMagType.history.getType());
                        List<Statuses> statuses = task.getStatuses();
                        // start end state
                        ResourceMagServiceImpl.StateMap stateMap = extractStatues(statuses);
                        resourceStatus.setState(stateMap.getState());
                        resourceStatus.setStartTime(stateMap.getStart());
                        resourceStatus.setEndTime(stateMap.getEnd());
                        // add it
                        resourceStatuses.add(resourceStatus);
                    }
                }
            }
        }
        return new HistoryResTaskResult(resourceStatuses);
    }

    @NoArgsConstructor
    @AllArgsConstructor
    @Data
    public static class HistoryResTaskResult {
        private List<ResourceStatus> resourceStatuses;
    }

}

最後，service層寫方法調用

    /**
     * get history status
     *
     * @param
     * @return list of ResourceStatus
     * @throws Exception
     */
    private List<ResourceStatus> getHistoryStatusFromClusterAsync(String keyword, String cluster) throws Exception {
        List<ResourceStatus> ret = Lists.newArrayList();
        ResourceStatus resourceStatus;
        String mesosUrl = MESOS_MAP.getOrDefault(cluster, WANGJING_MARATHONS);
        Set<String> mesosSlavesHosts = ClusterInfoUtils.getMesosSlavesHosts(mesosUrl);
        List<Future<HistoryResTask.HistoryResTaskResult>> futureList = Lists.newArrayListWithExpectedSize(mesosSlavesHosts.size());
        for (final String mesosSlavesHost : mesosSlavesHosts) {
            // new history resource mag task
            HistoryResTask historyResTask = new HistoryResTask(keyword, cluster, mesosSlavesHost);
            // submit task
            Future<HistoryResTask.HistoryResTaskResult> future = TaskExecutor.submit(historyResTask);
            futureList.add(future);
        }
        for (final Future<HistoryResTask.HistoryResTaskResult> future : futureList) {
            HistoryResTask.HistoryResTaskResult resTaskResult = future.get();
            ret.addAll(resTaskResult.getResourceStatuses());
        }
        return ret;
    }

這樣異步多線程調用，由於異步多線程的使用，使得之前需要從多臺機器節點上的取信息異步進行，節省了很多時間

編寫測試執行時間的用例：

    /**
     * get resource management information by app id
     *
     * @param id appid
     * @return resource management information
     * @throws Exception
     */
    @Override
    public List<ResourceStatus> getResourceMagByAppId(String id) throws Exception {
        AppInfo appInfo = appInfoService.findById(id);
        String keyword = appInfo.getKeyword();
        String cluster = appInfo.getCluster().toString();
        //List<ResourceStatus> ret = getCurrentRunningStatus(keyword, cluster);
        List<ResourceStatus> ret = getCurrentRunningStatusFromCluster(keyword, cluster);
        //List<ResourceStatus> historyStatus = getHistoryStatus(id);
        long start = System.currentTimeMillis();
        //List<ResourceStatus> historyStatus = getHistoryStatusFromCluster(keyword, cluster);
        List<ResourceStatus> historyStatus = getHistoryStatusFromClusterAsync(keyword, cluster);
        long end = System.currentTimeMillis();
        log.info("get history status use time {} ms", end - start);
        ret.addAll(historyStatus);
        return ret;
    }

本次任務需要從200多臺服務器組成的mesos集羣取任務狀態信息，比較異步調用前後的使用時間：

使用前：

使用後：

這個是本地IDE測試，上傳服務器調用後，時間更短：

由此看出，在數據量比較大，或者需要分佈式調用時，還是很有必要這樣寫的