淘寶動態配置diamond-server 源碼講解

1.diamond-server工程裏使用了我們常用的一些技術比如spring springmvc jsbctamplate 等等這些技術

jar包如下

2.首先我們先從啓動tomcat的時候加載bean的時候會有三個bean進行初始化數據的bean開始分析

2.1加載diamond-server其他實例的ip地址和端口號

com.taobao.diamond.server.service.NotifyService

@PostConstruct
    public void loadNodes() {
        InputStream in = null;
        try {
            in = ResourceUtils.getResourceAsStream("node.properties");
            nodeProperties.load(in);
        }
        catch (IOException e) {
            log.error("加載節點配置文件失敗");
        }
        finally {
            try {
                if (in != null)
                    in.close();
            }
            catch (IOException e) {
                log.error("關閉node.properties失敗", e);
            }
        }
        log.info("節點列表:" + nodeProperties);
    }

node配置文件裏面的內容(在本地部署兩個tomcat實例)

A=http://10.144.35.250:8080/diamond-server/notify.do
B=http://10.144.35.250:9090/diamond-server1/notify.do

2.2加載數據庫信息,創建BasicDataSource，JdbcTemplate對象

com.taobao.diamond.server.service.PersistService

@PostConstruct
    public void initDataSource() throws Exception {
        // 讀取jdbc.properties配置, 加載數據源
        Properties props = ResourceUtils.getResourceAsProperties("jdbc.properties");
        BasicDataSource ds = new BasicDataSource();
        ds.setDriverClassName(JDBC_DRIVER_NAME);
        ds.setUrl(ensurePropValueNotNull(props.getProperty("db.url")));
        ds.setUsername(ensurePropValueNotNull(props.getProperty("db.user")));
        ds.setPassword(ensurePropValueNotNull(props.getProperty("db.password")));
        ds.setInitialSize(Integer.parseInt(ensurePropValueNotNull(props.getProperty("db.initialSize"))));
        ds.setMaxActive(Integer.parseInt(ensurePropValueNotNull(props.getProperty("db.maxActive"))));
        ds.setMaxIdle(Integer.parseInt(ensurePropValueNotNull(props.getProperty("db.maxIdle"))));
        ds.setMaxWait(Long.parseLong(ensurePropValueNotNull(props.getProperty("db.maxWait"))));
        ds.setPoolPreparedStatements(Boolean.parseBoolean(ensurePropValueNotNull(props
            .getProperty("db.poolPreparedStatements"))));

        this.jt = new JdbcTemplate();
        this.jt.setDataSource(ds);
        // 設置最大記錄數，防止內存膨脹
        this.jt.setMaxRows(MAX_ROWS);
        // 設置JDBC執行超時時間
        this.jt.setQueryTimeout(QUERY_TIMEOUT);
    }

2.3 定時任務服務(每600s 從數據庫拉取數據到本地磁盤和本地緩存)

com.taobao.diamond.server.service.TimerTaskService

@PostConstruct
    public void init() {
        this.scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactory() {

            public Thread newThread(Runnable r) {
                Thread t = new Thread(r);
                t.setName(THREAD_NAME);
                t.setDaemon(true);
                return t;
            }

        });

        DumpConfigInfoTask dumpTask = new DumpConfigInfoTask(this);
        dumpTask.run();
        this.scheduledExecutorService.scheduleWithFixedDelay(dumpTask, SystemConfig.getDumpConfigInterval(),
            SystemConfig.getDumpConfigInterval(), TimeUnit.SECONDS);
    }

3.以添加一條數據走一遍diamond-server的流程

3.1 添加數據的頁面

3.2提交數據進入controller

com.taobao.diamond.server.controller.AdminController

@RequestMapping(params = "method=postConfig", method = RequestMethod.POST)
    public String postConfig(HttpServletRequest request, HttpServletResponse response,
            @RequestParam("dataId") String dataId, @RequestParam("group") String group,
            @RequestParam("content") String content, ModelMap modelMap) {
        response.setCharacterEncoding("GBK");

        boolean checkSuccess = true;
        String errorMessage = "參數錯誤";
        if (StringUtils.isBlank(dataId) || DiamondUtils.hasInvalidChar(dataId.trim())) {
            checkSuccess = false;
            errorMessage = "無效的DataId";
        }
        if (StringUtils.isBlank(group) || DiamondUtils.hasInvalidChar(group.trim())) {
            checkSuccess = false;
            errorMessage = "無效的分組";
        }
        if (StringUtils.isBlank(content)) {
            checkSuccess = false;
            errorMessage = "無效的內容";
        }
        if (!checkSuccess) {
            modelMap.addAttribute("message", errorMessage);
            return "/admin/config/new";
        }

        dataId = dataId.trim();
        group = group.trim();

        <span style="color:#FF0000;">this.configService.addConfigInfo(dataId, group, content);</span>

        modelMap.addAttribute("message", "提交成功!");
        return listConfig(request, response, dataId, group, 1, 20, modelMap);
    }

3.3進入添加數據的service層

com.taobao.diamond.server.service.ConfigService

public void addConfigInfo(String dataId, String group, String content) {
        checkParameter(dataId, group, content);
        ConfigInfo configInfo = new ConfigInfo(dataId, group, content);
        // 保存順序：先數據庫，再磁盤
        try {
            persistService.addConfigInfo(configInfo);
            // 切記更新緩存
            this.contentMD5Cache.put(generateMD5CacheKey(dataId, group), configInfo.getMd5());
            diskService.saveToDisk(configInfo);
            // 通知其他節點
            this.notifyOtherNodes(dataId, group);
        }
        catch (Exception e) {
            log.error("保存ConfigInfo失敗", e);
            throw new ConfigServiceException(e);
        }
    }

到此爲止，已經把數據添加到數據裏，並且其他diamond-server實例也已經更新到最新的數據。

4.我們現在看看diamond是如何通知到其他的diamond實例的，如果通知不到其他diamond實例的話，該如何同步

4.1 通知其他的diamond實例

我們是通過node配置文件進行配置的

現在我們看一下代碼

com.taobao.diamond.server.service.NotifyService

public void notifyConfigInfoChange(String dataId, String group) {
        Enumeration<?> enu = nodeProperties.propertyNames();
        while (enu.hasMoreElements()) {
            String address = (String) enu.nextElement();
            if (address.contains(SystemConfig.LOCAL_IP)) {
                continue;
            }
            String urlString = generateNotifyConfigInfoPath(dataId, group, address);
            final String result = invokeURL(urlString);
            log.info("通知節點" + address + "分組信息改變：" + result);
        }
    }


    String generateNotifyConfigInfoPath(String dataId, String group, String address) {
        String specialUrl = this.nodeProperties.getProperty(address);
        String urlString = PROTOCOL + address + URL_PREFIX;
        // 如果有指定url，使用指定的url
        if (specialUrl != null && StringUtils.hasLength(specialUrl.trim())) {
            urlString = specialUrl;
        }
        urlString += "?method=notifyConfigInfo&dataId=" + dataId + "&group=" + group;
        return urlString;
    }


    /**
     * http get調用
     * 
     * @param urlString
     * @return
     */
    private String invokeURL(String urlString) {
        HttpURLConnection conn = null;
        URL url = null;
        try {
            url = new URL(urlString);
            conn = (HttpURLConnection) url.openConnection();
            conn.setConnectTimeout(TIMEOUT);
            conn.setReadTimeout(TIMEOUT);
            conn.setRequestMethod("GET");
            conn.connect();
            InputStream urlStream = conn.getInputStream();
            StringBuilder sb = new StringBuilder();
            BufferedReader reader = null;
            try {
                reader = new BufferedReader(new InputStreamReader(urlStream));
                String line = null;
                while ((line = reader.readLine()) != null) {
                    sb.append(line);
                }
            }
            finally {
                if (reader != null)
                    reader.close();
            }
            return sb.toString();

        }
        catch (Exception e) {
        	e.printStackTrace();
            log.error("http調用失敗,url=" + urlString, e);
        }
        finally {
            if (conn != null) {
                conn.disconnect();
            }
        }
        return "error";
    }

4.2 如果通過http請求並沒有通知到其他實例，那該如何同步數據。

diamond本身有個定時任務進行查詢數據庫

現在我們看一下代碼

com.taobao.diamond.server.service.TimerTaskService

@PostConstruct
    public void init() {
        this.scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactory() {

            public Thread newThread(Runnable r) {
                Thread t = new Thread(r);
                t.setName(THREAD_NAME);
                t.setDaemon(true);
                return t;
            }

        });

        DumpConfigInfoTask dumpTask = new DumpConfigInfoTask(this);
        dumpTask.run();
        this.scheduledExecutorService.scheduleWithFixedDelay(dumpTask, SystemConfig.getDumpConfigInterval(),
            SystemConfig.getDumpConfigInterval(), TimeUnit.SECONDS);
    }

到這diamond-server基本的業務就差不多了，對於更改數據和刪除數據這些跟添加數據也是一樣業務，必須得同步數據。

存儲到磁盤的業務是一個組一個文件夾，組文件夾相應下面是存儲的不同的dataId，一個dataId是一個文件。

希望關於diamond-server的內容對大家有所幫助。

淘寶動態配置diamond-server 源碼講解

微服務實踐之使用 Visual Studio 2022 調試Dapr 應用程序

wpf附加屬性理解 WPF附加屬性

淘寶動態配置diamond-server 源碼講解

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結