1.註冊中心作用
從官網摘的圖,我們可以簡單瞭解其流程;
- 服務提供者在註冊中心進行註冊(本質是存放一些關鍵數據:提供者IP,Port,serviceKey,method,version,group等等信息);
- 服務消費者進行訂閱(消費者獲取提供者的關鍵數據);
- 消費者與註冊中心通過監聽器對數據進行同步(如果服務提供者的信息修改,銷燬,新增,監聽器來同步);
- 服務消費者調用服務提供者(根據獲取的信息,進行調用,簡單理解爲:給服務提供者Socket端口發送數據,等待響應);
- 調用服務的狀態在監控中心進行記錄,檢測服務質量(記錄服務調用信息,多少次,成功,失敗等次數,用於其他操作);
如果對Dubbo熟悉的同學,肯定知道經常使用zookeeper,nacos等來作爲註冊中心;
2.Dubbo-Register-API源碼解讀
我們看下主要類結構,可以清楚看到Dubbo支持的註冊Zookeeper,Redis,Nacos,Etcd等
我們針對每一個類進行分析與導讀,API大多作爲抽象層,具體的ZookeeperRegister,RedisZookeeper等後續文章導讀;主要關心RegisterService,AbstractRegister,FackRegister等
RegisterService
/**
* 註冊中心的註冊,取消註冊,訂閱,取消訂閱,查詢訂閱列表等動作定義
*/
public interface RegistryService {
/**
註冊URL,注意URL肯定是唯一的,對應存放數據到註冊中心
* @param url Registration information , is not allowed to be empty, e.g: dubbo://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
*/
void register(URL url);
/**
* Unregister
取消註冊,對於刪除對於的數據
*
* @param url Registration information , is not allowed to be empty, e.g: dubbo://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
*/
void unregister(URL url);
/**
* Subscribe to eligible registered data and automatically push when the registered data is changed.
訂閱註冊的數據,如果數據變更,進行通知,Listener就是用來通知數據變更
* @param url Subscription condition, not allowed to be empty, e.g. consumer://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
* @param listener A listener of the change event, not allowed to be empty
*/
void subscribe(URL url, NotifyListener listener);
/**
* Unsubscribe
取消訂閱
* @param url Subscription condition, not allowed to be empty, e.g. consumer://10.20.153.10/org.apache.dubbo.foo.BarService?version=1.0.0&application=kylin
* @param listener A listener of the change event, not allowed to be empty
*/
void unsubscribe(URL url, NotifyListener listener);
/**
查詢註冊列表,通過url進行條件查詢所匹配的所有URL集合。
*/
List<URL> lookup(URL url);
}
Register&Node比較簡單不描述
public interface Registry extends Node, RegistryService {
}
public interface Node {
/**
* get url.
*
* @return url.
*/
URL getUrl();
/**
* is available.
*
* @return available.
*/
boolean isAvailable();
/**
* destroy.
*/
void destroy();
}
AbstractRegistry
abstractRegister實現了對基本方法的實現,導讀之前我們先了解下一些Dubbo的特點;
- Dubbo會在本地文件存儲一些服務註冊中心的數據,減少對註冊中心的訪問壓力;
- 我們知道註冊的本質是存放數據到註冊中心,因此真正的創建數據的邏輯在對應的實現類中如ZookeeperRegister;
基本屬性
public abstract class AbstractRegistry implements Registry {
// URL的地址分隔符,在緩存文件中使用,服務提供者的URL分隔
private static final char URL_SEPARATOR = ' ';
// URL地址分隔正則表達式,用於解析文件緩存中服務提供者URL列表
private static final String URL_SPLIT = "\\s+";
// 存儲數據到文件的最大嘗試次數
private static final int MAX_RETRY_TIMES_SAVE_PROPERTIES = 3;
// Log記錄
protected final Logger logger = LoggerFactory.getLogger(getClass());
// 存放配置信息
private final Properties properties = new Properties();
// 緩存調度器
// File cache timing writing
private final ExecutorService registryCacheExecutor = Executors.newFixedThreadPool(1, new NamedThreadFactory("DubboSaveRegistryCache", true));
// 是否異步存放數據
private final boolean syncSaveFile;
// 數據版本號
private final AtomicLong lastCacheChanged = new AtomicLong();
private final AtomicInteger savePropertiesRetryTimes = new AtomicInteger();
// 存放已經註冊的URL
private final Set<URL> registered = new ConcurrentHashSet<>();
// 存放已經訂閱的URL
private final ConcurrentMap<URL, Set<NotifyListener>> subscribed = new ConcurrentHashMap<>();
// 某個消費者被通知的某一類型的 URL 集合
// 第一個key是消費者的URL,對應的就是哪個消費者。
// value是一個map集合,該map集合的key是分類的意思,例如providers、routes等,value就是被通知的URL集合
private final ConcurrentMap<URL, Map<String, List<URL>>> notified = new ConcurrentHashMap<>();
// 註冊中心URL
private URL registryUrl;
//本地存放文件對象
private File file;
*
*
*
}
public AbstractRegistry(URL url)初始化方法;
public AbstractRegistry(URL url) {
// 存儲註冊中心地址
setUrl(url);
// 獲取syncSave值,默認false
syncSaveFile = url.getParameter(REGISTRY_FILESAVE_SYNC_KEY, false);
// 文件的默認路徑 user.home+/.dubbo/dubbo-register&application&address 唯一區分
String defaultFilename = System.getProperty("user.home") + "/.dubbo/dubbo-registry-" + url.getParameter(APPLICATION_KEY) + "-" + url.getAddress().replaceAll(":", "-") + ".cache";
String filename = url.getParameter(FILE_KEY, defaultFilename);
File file = null;
// 如果是此一次需要創建文件
if (ConfigUtils.isNotEmpty(filename)) {
file = new File(filename);
if (!file.exists() && file.getParentFile() != null && !file.getParentFile().exists()){
if (!file.getParentFile().mkdirs()) {
throw new IllegalArgumentException("Invalid register cache file");
}
}
}
this.file = file;
// When starting the subscription center,
// we need to read the local cache file for future Registry fault tolerance processing.
// 加載配置
loadProperties();
// 監聽數據變化
notify(url.getBackupUrls());
}
LoadProperties()方法
// 加載文件的配置到properties
private void loadProperties() {
if (file != null && file.exists()) {
InputStream in = null;
try {
// 獲取文件流
in = new FileInputStream(file);
// 核心方法,加載配置到properties
properties.load(in);
// 如果日誌級別可以輸出info
if (logger.isInfoEnabled()) {
logger.info("Load registry cache file " + file + ", data: " + properties);
}
} catch (Throwable e) {
logger.warn("Failed to load registry cache file " + file, e);
} finally {
// 關閉流
if (in != null) {
try {
in.close();
} catch (IOException e) {
logger.warn(e.getMessage(), e);
}
}
}
}
}
protected static List<URL> filterEmpty(URL url, List<URL> urls) ;保證
protected static List<URL> filterEmpty(URL url, List<URL> urls) {
// 如果urls爲空
if (CollectionUtils.isEmpty(urls)) {
List<URL> result = new ArrayList<>(1);
// 添加一個空協議進去
result.add(url.setProtocol(EMPTY_PROTOCOL));
return result;
}
return urls;
}
public void doSaveProperties(long version);存放數據到本地文件中與loadProperties相反
public void doSaveProperties(long version) {
// 確定版本號最後
if (version < lastCacheChanged.get()) {
return;
}
if (file == null) {
return;
}
// Save
try {
// 獲取文件絕對路徑.lock文件
File lockfile = new File(file.getAbsolutePath() + ".lock");
// 不存在則創建
if (!lockfile.exists()) {
lockfile.createNewFile();
}
// 獲取文件rw權限
try (RandomAccessFile raf = new RandomAccessFile(lockfile, "rw");
// 獲取管道
FileChannel channel = raf.getChannel()) {
// 鎖住
FileLock lock = channel.tryLock();
if (lock == null) {
throw new IOException("Can not lock the registry cache file " + file.getAbsolutePath() + ", ignore and retry later, maybe multi java process use the file, please config: dubbo.registry.file=xxx.properties");
}
// Save
try {
if (!file.exists()) {
file.createNewFile();
}
try (FileOutputStream outputFile = new FileOutputStream(file)) {
//寫入配置
properties.store(outputFile, "Dubbo Registry Cache");
}
} finally {
lock.release();
}
}
} catch (Throwable e) {
savePropertiesRetryTimes.incrementAndGet();
if (savePropertiesRetryTimes.get() >= MAX_RETRY_TIMES_SAVE_PROPERTIES) {
logger.warn("Failed to save registry cache file after retrying " + MAX_RETRY_TIMES_SAVE_PROPERTIES + " times, cause: " + e.getMessage(), e);
savePropertiesRetryTimes.set(0);
return;
}
if (version < lastCacheChanged.get()) {
savePropertiesRetryTimes.set(0);
return;
} else {
registryCacheExecutor.execute(new SaveProperties(lastCacheChanged.incrementAndGet()));
}
logger.warn("Failed to save registry cache file, will retry, cause: " + e.getMessage(), e);
}
}
我們可以看到對應的file和file.lock文件哈
public List<URL> getCacheUrls(URL url) :獲取內存緩存的URL
public List<URL> getCacheUrls(URL url) {
for (Map.Entry<Object, Object> entry : properties.entrySet()) {
// 獲取分類
String key = (String) entry.getKey();
// 分類value
String value = (String) entry.getValue();
if (key != null && key.length() > 0 && key.equals(url.getServiceKey())
&& (Character.isLetter(key.charAt(0)) || key.charAt(0) == '_')
&& value != null && value.length() > 0) {
// 分割
String[] arr = value.trim().split(URL_SPLIT);
List<URL> urls = new ArrayList<>();
for (String u : arr) {
urls.add(URL.valueOf(u));
}
// 返回截取後的數據
return urls;
}
}
return null;
}
public List<URL> lookup(URL url) 獲取消費者訂閱的URL列表
@Override
public List<URL> lookup(URL url) {
List<URL> result = new ArrayList<>();
Map<String, List<URL>> notifiedUrls = getNotified().get(url);
// 如果監聽的URL列表不爲控
if (notifiedUrls != null && notifiedUrls.size() > 0) {
for (List<URL> urls : notifiedUrls.values()) {
for (URL u : urls) {
// 防止加入空協議
if (!EMPTY_PROTOCOL.equals(u.getProtocol())) {
result.add(u);
}
}
}
} else {
// 原子類 避免在獲取註冊在註冊中心的服務url時能夠保證是最新的url集合
final AtomicReference<List<URL>> reference = new AtomicReference<>();
NotifyListener listener = reference::set;
subscribe(url, listener); // Subscribe logic guarantees the first notify to return
List<URL> urls = reference.get();
if (CollectionUtils.isNotEmpty(urls)) {
for (URL u : urls) {
if (!EMPTY_PROTOCOL.equals(u.getProtocol())) {
result.add(u);
}
}
}
}
return result;
}
register,unregister,subscribe,unsubscribe比較簡單,都是對變量的增加修改,子類會對其重寫,調用;
@Override
public void register(URL url) {
if (url == null) {
throw new IllegalArgumentException("register url == null");
}
if (logger.isInfoEnabled()) {
logger.info("Register: " + url);
}
registered.add(url);
}
@Override
public void unregister(URL url) {
if (url == null) {
throw new IllegalArgumentException("unregister url == null");
}
if (logger.isInfoEnabled()) {
logger.info("Unregister: " + url);
}
registered.remove(url);
}
@Override
public void subscribe(URL url, NotifyListener listener) {
if (url == null) {
throw new IllegalArgumentException("subscribe url == null");
}
if (listener == null) {
throw new IllegalArgumentException("subscribe listener == null");
}
if (logger.isInfoEnabled()) {
logger.info("Subscribe: " + url);
}
Set<NotifyListener> listeners = subscribed.computeIfAbsent(url, n -> new ConcurrentHashSet<>());
listeners.add(listener);
}
@Override
public void unsubscribe(URL url, NotifyListener listener) {
if (url == null) {
throw new IllegalArgumentException("unsubscribe url == null");
}
if (listener == null) {
throw new IllegalArgumentException("unsubscribe listener == null");
}
if (logger.isInfoEnabled()) {
logger.info("Unsubscribe: " + url);
}
Set<NotifyListener> listeners = subscribed.get(url);
if (listeners != null) {
listeners.remove(listener);
}
}
recover:註冊中心重連,恢復操作;對已經註冊,訂閱的URL再來一次;
protected void recover() throws Exception {
// register
Set<URL> recoverRegistered = new HashSet<>(getRegistered());
if (!recoverRegistered.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover register url " + recoverRegistered);
}
for (URL url : recoverRegistered) {
register(url);
}
}
// subscribe
Map<URL, Set<NotifyListener>> recoverSubscribed = new HashMap<>(getSubscribed());
if (!recoverSubscribed.isEmpty()) {
if (logger.isInfoEnabled()) {
logger.info("Recover subscribe url " + recoverSubscribed.keySet());
}
for (Map.Entry<URL, Set<NotifyListener>> entry : recoverSubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
subscribe(url, listener);
}
}
}
}
protected void notify(List<URL> urls)
- 發起訂閱後,會獲取全量數據,此時會調用notify方法。即Registry 獲取到了全量數據
- 每次註冊中心發生變更時會調用notify方法雖然變化是增量,調用這個方法的調用方,已經進行處理,傳入的urls依然是全量的。
- listener.notify,通知監聽器,例如,有新的服務提供者啓動時,被通知,創建新的 Invoker 對象。
protected void notify(List<URL> urls) {
if (urls == null || urls.isEmpty()) return;
// 遍歷訂閱URL的監聽器集合,通知他們
for (Map.Entry<URL, Set<NotifyListener>> entry : getSubscribed().entrySet()) {
URL url = entry.getKey();
// 匹配
if (!UrlUtils.isMatch(url, urls.get(0))) {
continue;
}
// 遍歷監聽器集合,通知他們
Set<NotifyListener> listeners = entry.getValue();
if (listeners != null) {
for (NotifyListener listener : listeners) {
try {
notify(url, listener, filterEmpty(url, urls));
} catch (Throwable t) {
logger.error("Failed to notify registry event, urls: " + urls + ", cause: " + t.getMessage(), t);
}
}
}
}
}
protected void notify(URL url, NotifyListener listener, List<URL> urls) {
if (url == null) {
throw new IllegalArgumentException("notify url == null");
}
if (listener == null) {
throw new IllegalArgumentException("notify listener == null");
}
if ((urls == null || urls.isEmpty())
&& !Constants.ANY_VALUE.equals(url.getServiceInterface())) {
logger.warn("Ignore empty notify urls for subscribe url " + url);
return;
}
if (logger.isInfoEnabled()) {
logger.info("Notify urls for subscribe url " + url + ", urls: " + urls);
}
Map<String, List<URL>> result = new HashMap<String, List<URL>>();
// 將urls進行分類
for (URL u : urls) {
if (UrlUtils.isMatch(url, u)) {
// 按照url中key爲category對應的值進行分類,如果沒有該值,就找key爲providers的值進行分類
String category = u.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY);
List<URL> categoryList = result.get(category);
if (categoryList == null) {
categoryList = new ArrayList<URL>();
// 分類結果放入result
result.put(category, categoryList);
}
categoryList.add(u);
}
}
if (result.size() == 0) {
return;
}
// 獲得某一個消費者被通知的url集合(通知的 URL 變化結果)
Map<String, List<URL>> categoryNotified = notified.get(url);
if (categoryNotified == null) {
// 添加該消費者對應的url
notified.putIfAbsent(url, new ConcurrentHashMap<String, List<URL>>());
categoryNotified = notified.get(url);
}
// 處理通知監聽器URL 變化結果
for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
String category = entry.getKey();
List<URL> categoryList = entry.getValue();
categoryNotified.put(category, categoryList);
// 保存到文件
saveProperties(url);
//通知監聽器
listener.notify(categoryList);
}
}
saveProperties(),存放數據的方法
private void saveProperties(URL url) {
if (file == null) {
return;
}
try {
StringBuilder buf = new StringBuilder();
// key value形式,key:provider,config value 一長串
Map<String, List<URL>> categoryNotified = notified.get(url);
if (categoryNotified != null) {
for (List<URL> us : categoryNotified.values()) {
for (URL u : us) {
// 用分隔符割開
if (buf.length() > 0) {
buf.append(URL_SEPARATOR);
}
buf.append(u.toFullString());
}
}
}
// properties存放對應的key aluev,後續存入文件用
properties.setProperty(url.getServiceKey(), buf.toString());
// 獲取版本號
long version = lastCacheChanged.incrementAndGet();
if (syncSaveFile) {
// 調用核心邏輯,前面的方法
doSaveProperties(version);
} else {
// 如果異步,交給調度器執行一個任務
registryCacheExecutor.execute(new SaveProperties(version));
}
} catch (Throwable t) {
logger.warn(t.getMessage(), t);
}
}
銷燬方法:去笑註冊和訂閱就好了
// 取消所有註冊和訂閱
@Override
public void destroy() {
if (logger.isInfoEnabled()) {
logger.info("Destroy registry:" + getUrl());
}
Set<URL> destroyRegistered = new HashSet<>(getRegistered());
if (!destroyRegistered.isEmpty()) {
for (URL url : new HashSet<>(getRegistered())) {
if (url.getParameter(DYNAMIC_KEY, true)) {
try {
unregister(url);
if (logger.isInfoEnabled()) {
logger.info("Destroy unregister url " + url);
}
} catch (Throwable t) {
logger.warn("Failed to unregister url " + url + " to registry " + getUrl() + " on destroy, cause: " + t.getMessage(), t);
}
}
}
}
Map<URL, Set<NotifyListener>> destroySubscribed = new HashMap<>(getSubscribed());
if (!destroySubscribed.isEmpty()) {
for (Map.Entry<URL, Set<NotifyListener>> entry : destroySubscribed.entrySet()) {
URL url = entry.getKey();
for (NotifyListener listener : entry.getValue()) {
try {
unsubscribe(url, listener);
if (logger.isInfoEnabled()) {
logger.info("Destroy unsubscribe url " + url);
}
} catch (Throwable t) {
logger.warn("Failed to unsubscribe url " + url + " to registry " + getUrl() + " on destroy, cause: " + t.getMessage(), t);
}
}
}
}
}
異步保存數據到文件的任務類
// 異步保存文件方式
private class SaveProperties implements Runnable {
private long version;
private SaveProperties(long version) {
this.version = version;
}
@Override
public void run() {
doSaveProperties(version);
}
}
好,到此就分析完了,abstractRegister;具體如何存儲還得看後續文章,大概抽象導讀完畢;
FailbackRegistry
這個類,實現了對我們註冊,訂閱,取消註冊,取消訂閱的失敗嘗試機制實現;
基本屬性:失敗集合維護,間隔時間等;
public abstract class FailbackRegistry extends AbstractRegistry {
/* retry task map */
private final ConcurrentMap<URL, FailedRegisteredTask> failedRegistered = new ConcurrentHashMap<URL, FailedRegisteredTask>();
private final ConcurrentMap<URL, FailedUnregisteredTask> failedUnregistered = new ConcurrentHashMap<URL, FailedUnregisteredTask>();
private final ConcurrentMap<Holder, FailedSubscribedTask> failedSubscribed = new ConcurrentHashMap<Holder, FailedSubscribedTask>();
private final ConcurrentMap<Holder, FailedUnsubscribedTask> failedUnsubscribed = new ConcurrentHashMap<Holder, FailedUnsubscribedTask>();
private final ConcurrentMap<Holder, FailedNotifiedTask> failedNotified = new ConcurrentHashMap<Holder, FailedNotifiedTask>();
/**
* The time in milliseconds the retryExecutor will wait
*/
private final int retryPeriod;
// Timer for failure retry, regular check if there is a request for failure, and if there is, an unlimited retry
private final HashedWheelTimer retryTimer;
Register方法:我們看下他是如何對父類abstractRegister進行擴展的
// 註冊方法,真實
@Override
public void register(URL url) {
if (!acceptable(url)) {
logger.info("URL " + url + " will not be registered to Registry. Registry " + url + " does not accept service of this protocol type.");
return;
}
// 調用父類接口
super.register(url);
/**
* 移除失敗註冊,失敗取消註冊
*/
removeFailedRegistered(url);
removeFailedUnregistered(url);
try {
// 核心的交給子類去實現,zookeeper,redis等實現方法不同
doRegister(url);
} catch (Exception e) {
Throwable t = e;
// If the startup detection is opened, the Exception is thrown directly.
boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
&& url.getParameter(Constants.CHECK_KEY, true)
&& !CONSUMER_PROTOCOL.equals(url.getProtocol());
boolean skipFailback = t instanceof SkipFailbackWrapperException;
if (check || skipFailback) {
if (skipFailback) {
t = t.getCause();
}
throw new IllegalStateException("Failed to register " + url + " to registry " + getUrl().getAddress() + ", cause: " + t.getMessage(), t);
} else {
logger.error("Failed to register " + url + ", waiting for retry, cause: " + t.getMessage(), t);
}
// 執行重試邏輯
addFailedRegistered(url);
}
}
AddFailedRegister(Url)方法:重試註冊
private void addFailedRegistered(URL url) {
FailedRegisteredTask oldOne = failedRegistered.get(url);
if (oldOne != null) {
return;
}
// 創建一個任務task,我們關心run就好
FailedRegisteredTask newTask = new FailedRegisteredTask(url, this);
oldOne = failedRegistered.putIfAbsent(url, newTask);
if (oldOne == null) {
// never has a retry task. then start a new task for retry.
retryTimer.newTimeout(newTask, retryPeriod, TimeUnit.MILLISECONDS);
}
}
public final class FailedRegisteredTask extends AbstractRetryTask {
private static final String NAME = "retry register";
public FailedRegisteredTask(URL url, FailbackRegistry registry) {
super(url, registry, NAME);
}
// 重試的本質
@Override
protected void doRetry(URL url, FailbackRegistry registry, Timeout timeout) {
registry.doRegister(url);
registry.removeFailedRegisteredTask(url);
}
}
// AbstractRetryTask的方法,run
@Override
public void run(Timeout timeout) throws Exception {
if (timeout.isCancelled() || timeout.timer().isStop() || isCancel()) {
// other thread cancel this timeout or stop the timer.
return;
}
if (times > retryTimes) {
// reach the most times of retry.
logger.warn("Final failed to execute task " + taskName + ", url: " + url + ", retry " + retryTimes + " times.");
return;
}
if (logger.isInfoEnabled()) {
logger.info(taskName + " : " + url);
}
try {
doRetry(url, registry, timeout);
} catch (Throwable t) { // Ignore all the exceptions and wait for the next retry
logger.warn("Failed to execute task " + taskName + ", url: " + url + ", waiting for again, cause:" + t.getMessage(), t);
// reput this task when catch exception.
reput(timeout, retryPeriod);
}
}
其餘比如unregister,subscribe,unsubscribe都是一個套路;
同時我們也看到核心的邏輯交給不同的註冊中心實現,留給子類實現;
到此Dubbo-Register-API就分析完了,後續展開ZookeeperRegister等源碼,探索真正的註冊實現;
總結:
Dubbo-Register-API實現了對基本流程方法的抽象,以及高可用的重試機制;
個人覺得源碼值得仔細品味,有很多設計思想,沒有講,可以多探索下;