zookeeper源碼解析

zokeeper源碼執行流程分析 1.主流程 >org.apache.zookeeper.server.quorum.QuorumPeerMain.main zookeeper入口方法 >org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun 初始化zookeeper並且運行 >org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse 解析zoo.cfg，將zookeeper的配置映射爲org.apache.zookeeper.server.quorum.QuorumPeerConfig中的屬性 >org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties >org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig 運行集羣模式 >org.apache.zookeeper.server.ZooKeeperServerMain.main 運行單例模式 1.1運行集羣模式流程 >org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig 運行集羣模式 # 默認創建一個org.apache.zookeeper.server.NIOServerCnxnFactory，創建一個java NIO工廠，繼承Runnable；也可以通過配置創建org.apache.zookeeper.server.NettyServerCnxnFactory >org.apache.zookeeper.server.ServerCnxnFactory.createFactory() # 創建一個法定人數（QuorumPeer)，QuorumPeer中包含配置文件中的信息，該類繼承Thread； # # 該類的英文doc如下： # This class manages the quorum protocol. There are three states this server can be in:--翻譯：這個類管理法定人數協議。服務可能存在三種狀態 # Leader election - each server will elect a leader (proposing itself as a leader initially) --翻譯：leader選舉 - 每一個服務回選舉一個leader（首先會選舉自己） # Follower - the server will synchronize with the leader and replicate any transactions -- 翻譯：follower - follow會與leader同步，並且會複製leader所有的事務 # Leader - the server will process requests and forward them to followers.A majority of followers must log the request before it can be accepted. -- 翻譯：leader - leader會處理請求並且轉發給follower。每一個follower在接收leader的請求前必須日誌記錄下該請求 # This class will setup a datagram socket that will always respond with its view of the current leader. The response will take the form of: -- 翻譯：這個類會建立一個socket套接字用於及時響應leader。響應內容將包含以下內容： # int xid; long myid; long leader_id; long leader_zxid; # The request for the current leader will consist solely of an xid: int xid; -- 翻譯：當前leader的請求將只包含xid >org.apache.zookeeper.server.quorum.QuorumPeerMain.getQuorumPeer >org.apache.zookeeper.server.quorum.QuorumPeer.start #啓動QuorumPeer >org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase #加載數據 >org.apache.zookeeper.server.ServerCnxnFactory.start #實際上是運行org.apache.zookeeper.server.NIOServerCnxnFactory.start方法 >org.apache.zookeeper.server.NIOServerCnxnFactory.run #運行 >org.apache.zookeeper.server.quorum.QuorumPeer.startLeaderElection #開始選舉 >org.apache.zookeeper.server.quorum.QuorumPeer.run # 1.1.1 加載數據 >org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase #加載數據 # 委託給ZKDatabase加載數據，ZKDatabase實際上是zookeeper存儲節點數據的一個內存數據庫，實際上數據是存儲在org.apache.zookeeper.server.DataTree類中 # ZKDatabase的doc文檔如下： # This class maintains the in memory database of zookeeper server states that includes the sessions, - 翻譯：這個類管理zookeeper狀態的內存數據庫，包含session、datatree、committed logs # datatree and the committed logs. # It is booted up after reading the logs and snapshots from the disk. - 翻譯：在讀取完硬盤中的日誌和快照文件後啓動 # loadDataBase()方法的doc文檔如下： # load the database from the disk onto memory and also add the transactions to the committedlog in memory. - 翻譯：從硬盤中加載數據，同時將事務添加到提交內存的提交日誌中 >org.apache.zookeeper.server.ZKDatabase.loadDataBase # restore()方法的doc文檔如下： # this function restores the server database after reading from the snapshots and transaction logs.- 翻譯：這個函數在讀取快照和事務日誌後恢復服務器內存數據庫 >org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore #deserialize()doc文檔如下： #deserialize a data tree from the most recent snapshot - 翻譯：從最近的快照中反序列化一個數據樹（data tree） >org.apache.zookeeper.server.persistence.FileSnap.deserialize >org.apache.zookeeper.server.persistence.FileSnap.deserialize >org.apache.zookeeper.server.util.SerializeUtils.deserializeSnapshot # 將數據解析到DataTree中 # dataTree的doc文檔如下： # This class maintains the tree data structure. - 翻譯：這個類維護者一個樹形數據結構 # It doesn't have any networking or client connection code in it so that it can be tested in a stand alone way. - 翻譯：這個類不包含任何網絡或客戶端連接代碼，以便於可以以一種獨立的方式被測試 # The tree maintains two parallel data structures: a hashtable that maps from full paths to DataNodes and a tree of DataNodes. - 翻譯：這個樹管理兩個平行的數據結構：一個hashtable映射全局路徑以及對應該路徑的DataNodes，一個樹形的dataNodes # All accesses to a path is through the hashtable. The tree is traversed only when serializing to disk. - 翻譯：所有訪問path均通過這個hashtable。這個樹會被序列化到硬盤上 # DataTree存儲節點的兩個hashtable： # nodes的doc:his hashtable provides a fast lookup to the datanodes. The tree is the source of truth and is where all the locking occurs # ConcurrentHashMap<String, DataNode> nodes # # ephemerals的doc This hashtable lists the paths of the ephemeral nodes of a session. # Map<Long, HashSet<String>> ephemerals >org.apache.zookeeper.server.DataTree.deserialize 1.1.2 運行ServerCnxnFactory.start；通過我們前面得知實際上不指定則默認zookeeper使用的是NIOServerCnxnFactory實現類，我們通過查看該類的源代碼可以得知使用的是java nio進行操作，該類中默認爲維護一個線程，在org.apache.zookeeper.server.NIOServerCnxnFactory.configure方法中進行賦值，實際使用的是org.apache.zookeeper.server.ZooKeeperThread，並以守護線程的方式運行 >org.apache.zookeeper.server.ServerCnxnFactory.start #實際上是運行org.apache.zookeeper.server.NIOServerCnxnFactory.start方法，在該類中啓動該類的Thread，即調用該類的Thread.start方法，實際上該線程啓動後調用的是本類的run方法 >org.apache.zookeeper.server.NIOServerCnxnFactory.run # 執行線程run方法 >org.apache.zookeeper.server.NIOServerCnxn.doIO # 處理讀或寫事件 >org.apache.zookeeper.server.NIOServerCnxn.readPayload # 當時讀事件時，讀取數據 >org.apache.zookeeper.server.NIOServerCnxn.readConnectRequest 讀取建立連接的請求 >org.apache.zookeeper.server.ZooKeeperServer.processConnectRequest 解析建立連接的讀請求，包含建立和保存session信息 >org.apache.zookeeper.server.ZooKeeperServer.reopenSession 重新打開session >org.apache.zookeeper.server.ZooKeeperServer.createSession 創建session >org.apache.zookeeper.server.NIOServerCnxn.readRequest 讀取正常請求 >org.apache.zookeeper.server.ZooKeeperServer.processPacket 解析請求包 1.1.3 開始選舉，確定選舉算法 >org.apache.zookeeper.server.quorum.QuorumPeer.startLeaderElection #開始選舉 >org.apache.zookeeper.server.quorum.QuorumPeer.getLastLoggedZxid 獲取日誌記錄的最後的zxid >org.apache.zookeeper.server.quorum.QuorumPeer.getCurrentEpoch 獲取當前的紀元（epoch） >currentVote = new Vote(myid, getLastLoggedZxid(), getCurrentEpoch()); 創建選票，其中currentVote的doc如下：This is who I think the leader currently is.表示當前需要選舉的leader >org.apache.zookeeper.server.quorum.QuorumPeer.createElectionAlgorithm 選擇選舉算法，有LeaderElection、AuthFastLeaderElection、FastLeaderElection三種選舉算法，其中前兩種都標記過期，不推薦使用，默認使用FastLeaderElection這種選舉算法 1.1.4 >org.apache.zookeeper.server.quorum.QuorumPeer.run 真正開始選舉，選舉業務邏輯在run方法中 >case LOOKING: # 當時LOOKING狀態 # 選擇leader過程 doc如下： # Starts a new round of leader election. - 翻譯：開啓新一輪的選舉 # Whenever our QuorumPeer changes its state to LOOKING, this method is invoked, - 翻譯：無論何時，我們的法人狀態轉變成爲LOOKING狀態，這個方法就會被調用 # and it sends notifications to all other peers. - 翻譯：並且發送通知到其他法人 >org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader >case OBSERVING: 當時OBSERVING狀態 # 跟隨leader，doc如下： # the main method called by the observer to observe the leader - 翻譯：當observer跟隨leader(observe the leader),這個方法將會被調用 >org.apache.zookeeper.server.quorum.Observer.observeLeader # 跟隨leader >org.apache.zookeeper.server.quorum.Learner.findLeader 查找leader地址 # doc如下： # Establish a connection with the Leader found by findLeader. Retries 5 times before giving up.- 翻譯：於findLeader()方法查詢到的leader建立連接，若未連接上會進行5次重試 >org.apache.zookeeper.server.quorum.Learner.connectToLeader 連接leader # 建立完連接後，註冊到leader中，doc如下： # Once connected to the leader, perform the handshake protocol to establish a following / observing connection. - 翻譯：一旦與leader建立連接，執行握手協議，建立一個following/observing連接 >org.apache.zookeeper.server.quorum.Learner.registerWithLeader # 同步leader信息，doc： # Finally, synchronize our history with the Leader. 最後同步leader信息 >org.apache.zookeeper.server.quorum.Learner.syncWithLeader case FOLLOWING: # 當時FOLLOWING狀態 # doc如下 # the main method called by the follower to follow the leader - 翻譯：follower跟隨leader的被調用的主方法 >org.apache.zookeeper.server.quorum.Follower.followLeader >org.apache.zookeeper.server.quorum.Learner.findLeader 查找leader地址(參見上方OBSERVING狀態中說明) >org.apache.zookeeper.server.quorum.Learner.connectToLeader 連接leader(參見上方OBSERVING狀態中說明) >org.apache.zookeeper.server.quorum.Learner.registerWithLeader 建立完連接後，註冊到leader中(參見上方OBSERVING狀態中說明) >org.apache.zookeeper.server.quorum.Learner.syncWithLeader 同步leader信息(參見上方OBSERVING狀態中說明) case LEADING: # 當時LEADING狀態是 >org.apache.zookeeper.server.quorum.QuorumPeer.setLeader 設置leader # doc如下： # This method is main function that is called to lead 被選舉稱爲leader後需要調用的主方法 >org.apache.zookeeper.server.quorum.Leader.lead >org.apache.zookeeper.server.ZooKeeperServer.loadData 加載數據 # 開始運行leaderServer，doc如下： # Start up Leader ZooKeeper server and initialize zxid to the new epoch - 翻譯：啓動leader Server，使用一個新的epoch初始化一個zxid，實際上一代leader都有一個獨一無二的epoch，代表一個朝代 >org.apache.zookeeper.server.quorum.Leader.startZkServer 通過查看org.apache.zookeeper.server.quorum.QuorumPeer源碼，我們可以得知zookeeper選舉過程中狀態分爲四種：LOOKING, FOLLOWING, LEADING, OBSERVING;