目錄導航
前言
關於Dubbo，本系列文章主要講三方面內容。前面我們已經瞭解到Dubbo的基本特性，常用配置、自適應擴展點與服務發佈，服務註冊的過程。接下來我們主要講服務端接收消息處理過程。
- 解開Dubbo的神祕面紗
- Dubbo常用配置
- Dubbo源碼分析(上篇)
- Dubbo源碼分析(中篇)
- Dubbo源碼分析(下篇)
本節我們講5、6、7、8
- Dubbo Extension擴展點
- 服務發佈過程
- 消費端初始化過程
- 什麼時候建立和服務端的連接
- 服務端接收消息處理過程
- Directory
- Cluster
- LoadBalance
Tips：本節文末有Dubbo中文註釋版的源碼哦~
服務端接收消息處理過程
服務端底層也是基於netty通信
NettyHandler. messageReceived
接收消息的時候，通過NettyHandler.messageReceived作爲入口。
@Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) throws Exception {
NettyChannel channel = NettyChannel.getOrAddChannel(ctx.getChannel(), url, handler);
try {
handler.received(channel, e.getMessage());
} finally {
NettyChannel.removeChannelIfDisconnected(ctx.getChannel());
}
}
handler.received
這個handler是什麼呢？還記得在服務發佈的時候，組裝了一系列的handler嗎？代碼如下
HeaderExchanger.bind
public ExchangeServer bind(URL url, ExchangeHandler handler) throws RemotingException {
return new HeaderExchangeServer(Transporters.bind(url, new DecodeHandler(new HeaderExchangeHandler(handler))));
}
received這裏實際調用的是MultiMessageHandler.received
MultiMessageHandler.received
將複合消息轉化爲單獨消息處理
public class MultiMessageHandler extends AbstractChannelHandlerDelegate {
public MultiMessageHandler(ChannelHandler handler) {
super(handler);
}
@SuppressWarnings("unchecked")
@Override
public void received(Channel channel, Object message) throws RemotingException {
if (message instanceof MultiMessage) {
MultiMessage list = (MultiMessage)message;
// deal message
for(Object obj : list) {
handler.received(channel, obj);
}
} else {
handler.received(channel, message);
}
}
}
這裏的received實際上調用的是HeartbeatHandler.received
HeartbeatHandler.received
心跳機制
public void received(Channel channel, Object message) throws RemotingException {
setReadTimestamp(channel);
//如果當前消息是心跳包
if (isHeartbeatRequest(message)) {
Request req = (Request) message;
if (req.isTwoWay()) {
Response res = new Response(req.getId(), req.getVersion());
res.setEvent(Response.HEARTBEAT_EVENT);
channel.send(res);
if (logger.isInfoEnabled()) {
int heartbeat = channel.getUrl().getParameter(Constants.HEARTBEAT_KEY, 0);
if(logger.isDebugEnabled()) {
logger.debug("Received heartbeat from remote channel " + channel.getRemoteAddress()
+ ", cause: The channel has no data-transmission exceeds a heartbeat period"
+ (heartbeat > 0 ? ": " + heartbeat + "ms" : ""));
}
}
}
return;
}
if (isHeartbeatResponse(message)) {
if (logger.isDebugEnabled()) {
logger.debug(
new StringBuilder(32)
.append("Receive heartbeat response in thread ")
.append(Thread.currentThread().getName())
.toString());
}
return;
}
handler.received(channel, message);
}
接下來的 received實際上調用的是AllChannelHandler.received
AllChannelHandler.received
起一個線程處理任務
public void received(Channel channel, Object message) throws RemotingException {
ExecutorService cexecutor = getExecutorService();
try {
cexecutor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
} catch (Throwable t) {
throw new ExecutionException(message, channel, getClass() + " error when process received event .", t);
}
}
既然是線程處理，找一下run方法：
AllChannelHandler.run
這裏會判斷狀態，如果是received，則再次調用received，這裏的received實際上調用的是DecodeHandler.received
public void received(Channel channel, Object message) throws RemotingException {
if (message instanceof Decodeable) {
decode(message);
}
if (message instanceof Request) {
decode(((Request)message).getData());
}
if (message instanceof Response) {
decode( ((Response)message).getResult());
}
handler.received(channel, message);
}
這裏最後處理的received實際上調用的是
HeaderExchangeHandler.received
public void received(Channel channel, Object message) throws RemotingException {
channel.setAttribute(KEY_READ_TIMESTAMP, System.currentTimeMillis());
ExchangeChannel exchangeChannel = HeaderExchangeChannel.getOrAddChannel(channel);
try {
if (message instanceof Request) {
// handle request.
Request request = (Request) message;
if (request.isEvent()) {
handlerEvent(channel, request);
} else {
if (request.isTwoWay()) {
Response response = handleRequest(exchangeChannel, request);
channel.send(response);
} else {
handler.received(exchangeChannel, request.getData());
}
}
} else if (message instanceof Response) {
handleResponse(channel, (Response) message);
} else if (message instanceof String) {
if (isClientSide(channel)) {
Exception e = new Exception("Dubbo client can not supported string message: " + message + " in channel: " + channel + ", url: " + channel.getUrl());
logger.error(e.getMessage(), e);
} else {
String echo = handler.telnet(channel, (String) message);
if (echo != null && echo.length() > 0) {
channel.send(echo);
}
}
} else {
handler.received(exchangeChannel, message);
}
} finally {
HeaderExchangeChannel.removeChannelIfDisconnected(channel);
}
}
NettyServer
回到主線，在Nettyserver中，wrap了多個handler，再次調用
public NettyServer(URL url, ChannelHandler handler) throws RemotingException {
super(url, ChannelHandlers.wrap(handler, ExecutorUtil.setThreadName(url, SERVER_THREAD_POOL_NAME)));
}
protected ChannelHandler wrapInternal(ChannelHandler handler, URL url) {
return new MultiMessageHandler(new HeartbeatHandler(ExtensionLoader.getExtensionLoader(Dispatcher.class)
.getAdaptiveExtension().dispatch(handler, url)));
}
所以服務端的handler處理鏈爲
MultiMessageHandler(HeartbeatHandler(AllChannelHandler(DecodeHandler)))
MultiMessageHandler: 複合消息處理
HeartbeatHandler：心跳消息處理，接收心跳併發送心跳響應
AllChannelHandler：業務線程轉化處理器，把接收到的消息封裝成
ChannelEventRunnable可執行任務給線程池處理
DecodeHandler:業務解碼處理器
HeaderExchangeHandler.received
交互層請求響應處理，有三種處理方式
- handlerRequest，雙向請求
- handler.received 單向請求
- handleResponse 響應消息
public void received(Channel channel, Object message) throws RemotingException {
channel.setAttribute(KEY_READ_TIMESTAMP, System.currentTimeMillis());
ExchangeChannel exchangeChannel = HeaderExchangeChannel.getOrAddChannel(channel);
try {
if (message instanceof Request) {
// handle request.
Request request = (Request) message;
if (request.isEvent()) {
handlerEvent(channel, request);
} else {
if (request.isTwoWay()) {
Response response = handleRequest(exchangeChannel, request);
channel.send(response);
} else {
handler.received(exchangeChannel, request.getData());
}
}
} else if (message instanceof Response) {
handleResponse(channel, (Response) message);
} else if (message instanceof String) {
if (isClientSide(channel)) {
Exception e = new Exception("Dubbo client can not supported string message: " + message + " in channel: " + channel + ", url: " + channel.getUrl());
logger.error(e.getMessage(), e);
} else {
String echo = handler.telnet(channel, (String) message);
if (echo != null && echo.length() > 0) {
channel.send(echo);
}
}
} else {
handler.received(exchangeChannel, message);
}
} finally {
HeaderExchangeChannel.removeChannelIfDisconnected(channel);
}
}
handleRequest
處理請求並返回response
Response handleRequest(ExchangeChannel channel, Request req) throws RemotingException {
Response res = new Response(req.getId(), req.getVersion());
if (req.isBroken()) {
Object data = req.getData();
String msg;
if (data == null) msg = null;
else if (data instanceof Throwable) msg = StringUtils.toString((Throwable) data);
else msg = data.toString();
res.setErrorMessage("Fail to decode request due to: " + msg);
res.setStatus(Response.BAD_REQUEST);
return res;
}
// find handler by message class.
Object msg = req.getData();
try {
// handle data.
Object result = handler.reply(channel, msg);
res.setStatus(Response.OK);
res.setResult(result);
} catch (Throwable e) {
res.setStatus(Response.SERVICE_ERROR);
res.setErrorMessage(StringUtils.toString(e));
}
return res;
}
ExchangeHandlerAdaptive.replay(DubboProtocol)
調用DubboProtocol中定義的ExchangeHandlerAdaptive.replay方法處理消息
private ExchangeHandler requestHandler = new ExchangeHandlerAdapter() {
public Object reply(ExchangeChannel channel, Object message) throws RemotingException {
invoker.invoke(inv);
}
那接下來invoker.invoke會調用哪個類中的方法呢？還記得在RegistryDirectory中發佈本地方法的時候，對invoker做的包裝嗎？通過InvokerDelegete對原本的invoker做了一層包裝，而原本的invoker是什麼呢？是一個JavassistProxyFactory生成的動態代理吧。所以此處的invoker應該是
Filter(Listener(InvokerDelegete(AbstractProxyInvoker (Wrapper.invokeMethod)))
RegistryDirectory生成invoker的代碼如下
private <T> ExporterChangeableWrapper<T> doLocalExport(final Invoker<T> originInvoker){
String key = getCacheKey(originInvoker);
ExporterChangeableWrapper<T> exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
if (exporter == null) {
synchronized (bounds) {
exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
if (exporter == null) {
final Invoker<?> invokerDelegete = new InvokerDelegete<T>(originInvoker, getProviderUrl(originInvoker));
exporter = new ExporterChangeableWrapper<T>((Exporter<T>)protocol.export(invokerDelegete), originInvoker);
bounds.put(key, exporter);
}
}
}
return (ExporterChangeableWrapper<T>) exporter;
}
Directory
集羣目錄服務Directory， 代表多個Invoker, 可以看成List<Invoker>,它的值可能是動態變化的比如註冊中心推送變更。集羣選擇調用服務時通過目錄服務找到所有服務
StaticDirectory: 靜態目錄服務， 它的所有Invoker通過構造函數傳入， 服務消費方引用服務的時候， 服務對多註冊中心的引用，將Invokers集合直接傳入 StaticDirectory構造器，再由Cluster僞裝成一個Invoker；StaticDirectory的list方法直接返回所有invoker集合；
RegistryDirectory: 註冊目錄服務， 它的Invoker集合是從註冊中心獲取的， 它實現了NotifyListener接口實現了回調接口notify(List<Url>)
Directory目錄服務的更新過程
RegistryProtocol.doRefer方法，也就是消費端在初始化的時候，這裏涉及到了RegistryDirectory這個類。然後執行cluster.join(directory)方法。這些代碼在上節課有分析過。
cluster.join其實就是將Directory中的多個Invoker僞裝成一個Invoker, 對上層透明，包含集羣的容錯機制
private <T> Invoker<T> doRefer(Cluster cluster, Registry registry, Class<T> type, URL url) {
RegistryDirectory<T> directory = new RegistryDirectory<T>(type, url);//對多個invoker進行組裝
directory.setRegistry(registry); //ZookeeperRegistry
directory.setProtocol(protocol); //protocol=Protocol$Adaptive
//url=consumer://192.168.111....
URL subscribeUrl = new URL(Constants.CONSUMER_PROTOCOL, NetUtils.getLocalHost(), 0, type.getName(), directory.getUrl().getParameters());
//會把consumer://192... 註冊到註冊中心
if (! Constants.ANY_VALUE.equals(url.getServiceInterface())
&& url.getParameter(Constants.REGISTER_KEY, true)) {
//zkClient.create()
registry.register(subscribeUrl.addParameters(Constants.CATEGORY_KEY, Constants.CONSUMERS_CATEGORY,
Constants.CHECK_KEY, String.valueOf(false)));
}
directory.subscribe(subscribeUrl.addParameter(Constants.CATEGORY_KEY,
Constants.PROVIDERS_CATEGORY
+ "," + Constants.CONFIGURATORS_CATEGORY
+ "," + Constants.ROUTERS_CATEGORY));
//Cluster$Adaptive
return cluster.join(directory);
}
directory.subscribe
訂閱節點的變化
- 當zookeeper上指定節點發生變化以後，會通知到RegistryDirectory的notify方法
- 將url轉化爲invoker對象
調用過程中invokers的使用
再調用過程中，AbstractClusterInvoker.invoke方法中，
public Result invoke(final Invocation invocation) throws RpcException {
checkWhetherDestroyed();
LoadBalance loadbalance;
List<Invoker<T>> invokers = list(invocation);
if (invokers != null && invokers.size() > 0) {
loadbalance = ExtensionLoader.getExtensionLoader(LoadBalance.class).getExtension(invokers.get(0).getUrl()
.getMethodParameter(invocation.getMethodName(),Constants.LOADBALANCE_KEY, Constants.DEFAULT_LOADBALANCE));
} else {
loadbalance = ExtensionLoader.getExtensionLoader(LoadBalance.class).getExtension(Constants.DEFAULT_LOADBALANCE);
}
RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);
return doInvoke(invocation, invokers, loadbalance);
}
list方法
從directory中獲得invokers的集合，這裏獲取的方式基於netty，這一點我們簡單的在上一節有所分析
protected List<Invoker<T>> list(Invocation invocation) throws RpcException {
List<Invoker<T>> invokers = directory.list(invocation);
return invokers;
}
什麼是Cluster?
關於cluster，我們之前已經多次提到，這裏就不必重複造輪子了，引用網友的一篇文章：
Dubbo—Cluster剖析
LoadBalance
LoadBalance負載均衡， 負責從多個 Invokers中選出具體的一個Invoker用於本次調用，調用過程中包含了負載均衡的算法。
負載均衡代碼訪問入口
在AbstractClusterInvoker.invoke中代碼如下，通過名稱獲得指定的擴展點。RandomLoadBalance
public Result invoke(final Invocation invocation) throws RpcException {
checkWhetherDestroyed();
LoadBalance loadbalance;
List<Invoker<T>> invokers = list(invocation);
if (invokers != null && invokers.size() > 0) {
loadbalance = ExtensionLoader.getExtensionLoader(LoadBalance.class).getExtension(invokers.get(0).getUrl()
.getMethodParameter(invocation.getMethodName(),Constants.LOADBALANCE_KEY, Constants.DEFAULT_LOADBALANCE));
} else {
loadbalance = ExtensionLoader.getExtensionLoader(LoadBalance.class).getExtension(Constants.DEFAULT_LOADBALANCE);
}
RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);
return doInvoke(invocation, invokers, loadbalance);
}
AbstractClusterInvoker.doselect
調用LoadBalance.select方法，講invokers按照指定算法進行負載
private Invoker<T> doselect(LoadBalance loadbalance, Invocation invocation, List<Invoker<T>> invokers, List<Invoker<T>> selected) throws RpcException {
if (invokers == null || invokers.size() == 0)
return null;
if (invokers.size() == 1)
return invokers.get(0);
// 如果只有兩個invoker，退化成輪循
if (invokers.size() == 2 && selected != null && selected.size() > 0) {
return selected.get(0) == invokers.get(0) ? invokers.get(1) : invokers.get(0);
}
Invoker<T> invoker = loadbalance.select(invokers, getUrl(), invocation);
//如果 selected中包含（優先判斷） 或者 不可用&&availablecheck=true 則重試.
if( (selected != null && selected.contains(invoker))
||(!invoker.isAvailable() && getUrl()!=null && availablecheck)){
try{
Invoker<T> rinvoker = reselect(loadbalance, invocation, invokers, selected, availablecheck);
if(rinvoker != null){
invoker = rinvoker;
}else{
//看下第一次選的位置，如果不是最後，選+1位置.
int index = invokers.indexOf(invoker);
try{
//最後在避免碰撞
invoker = index <invokers.size()-1?invokers.get(index+1) :invoker;
}catch (Exception e) {
logger.warn(e.getMessage()+" may because invokers list dynamic change, ignore.",e);
}
}
}catch (Throwable t){
logger.error("clustor relselect fail reason is :"+t.getMessage() +" if can not slove ,you can set cluster.availablecheck=false in url",t);
}
}
return invoker;
}
默認情況下，LoadBalance使用的是Random算法，但是這個隨機和我們理解上的隨機還是不一樣的,因爲他還有個概念叫weight(權重)
RandomLoadBalance
假設有四個集羣節點A,B,C,D,對應的權重分別是1,2,3,4,那麼請求到A節點的概率就爲1/(1+2+3+4) = 10%.B,C,D節點依次類推爲20%,30%,40%.
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
int length = invokers.size(); // 總個數
int totalWeight = 0; // 總權重
boolean sameWeight = true; // 權重是否都一樣
for (int i = 0; i < length; i++) {
int weight = getWeight(invokers.get(i), invocation);
totalWeight += weight; // 累計總權重
if (sameWeight && i > 0
&& weight != getWeight(invokers.get(i - 1), invocation)) {
sameWeight = false; // 計算所有權重是否一樣
}
}
if (totalWeight > 0 && ! sameWeight) {
// 如果權重不相同且權重大於0則按總權重數隨機
int offset = random.nextInt(totalWeight);
// 並確定隨機值落在哪個片斷上
for (int i = 0; i < length; i++) {
offset -= getWeight(invokers.get(i), invocation);
if (offset < 0) {
return invokers.get(i);
}
}
}
// 如果權重相同或權重爲0則均等隨機
return invokers.get(random.nextInt(length));
}
後記
Dubbo源碼講到這裏就完結了，最後再給一張官方經典圖：
Dubbo中文註釋版：DubboV2.5.4下載地址
至此，關於Dubbo的介紹就告以段落了，下節預告：
分佈式專題-分佈式消息通信之ActiveMQ01-初識ActiveMQ
分佈式專題-分佈式消息通信之ActiveMQ02-ActiveMQ原理分析
更多架構知識，歡迎關注本套Java系列文章-地址導航：Java架構師成長之路