上一篇文章說到,現在這種每發起請求一次就新建一個client鏈接,但是如果在併發比較高的情況下,就會造成資源浪費,如果通過client和server進行長期鏈接,把需要處理的請求存到client裏面,並且通過異步的形式返回,便會減少資源浪費。
這裏有兩個主要的問題,1、如何實現異步返回?2、如何把client和server對應起來?
先看第一個問題,所謂的異步返回,可以以點奶茶爲例,當顧客(消費方)向奶茶店(服務端)點了一杯奶茶(發起請求),奶茶店給他一個號碼(憑證),然後客戶拿着號就去逛街幹其他事情,過了一段時間回來問奶茶店有沒有好了,此時奶茶店根據他的號碼來給他對應的奶茶。
這裏就是爲什麼上篇文章說的請求要帶上requestID,這就相當於憑證的標識,而客戶端鏈接可以保存這些請求並且返回一個憑證,當有返回時,把對應的結果塞到憑證裏面,客戶端可以做自己的事情或者一直等待憑證有結果,這時候結合線程池去做這個處理
第二個問題,需要一個鏈接池來管理鏈接和對應的遠端關係
看看編碼實現
第一個問題最重要的就是如何設計一個憑證,這時候需要用到Future接口來實現自己的憑證,通過一個內部同步器的管理,這個同步器主要是起到自旋改變狀態的作用,同步器因爲只有一個線程節點,會不斷自旋嘗試獲取鎖,獲取鎖的條件是遠端有返回並且改變了同步器的值。
/**
* @author: lele
* @date: 2019/11/21 上午10:54
* 實現異步返回
*/
public class RpcFuture implements Future<Object> {
private RpcResponse rpcResponse;
private RpcRequest rpcRequest;
/**
* 自定義同步器,這裏只是用來通過自選改變狀態
*/
private Sync sync;
public RpcFuture(RpcRequest rpcRequest) {
this.rpcRequest = rpcRequest;
this.sync = new Sync();
}
@Override
public boolean cancel(boolean mayInterruptIfRunning) {
throw new UnsupportedOperationException();
}
@Override
public boolean isCancelled() {
throw new UnsupportedOperationException();
}
/**
* 返回狀態是否改變了
* @return
*/
@Override
public boolean isDone() {
return sync.isDone();
}
/**
* 賦值並設置同步器鎖狀態爲1
* @param response
*/
public void done(RpcResponse response) {
this.rpcResponse = response;
sync.tryRelease(1);
}
/**
* 自選等待結果,這裏一直執行acquire
* 直到tryacquire方法return true即state爲1
* @return
* @throws InterruptedException
* @throws ExecutionException
*/
@Override
public Object get() throws InterruptedException, ExecutionException {
sync.acquire(-1);
return this.rpcResponse;
}
/**
* 超時拋異常
* @param timeout
* @param unit
* @return
* @throws InterruptedException
* @throws ExecutionException
* @throws TimeoutException
*/
@Override
public Object get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException {
//超時獲取
boolean success = sync.tryAcquireNanos(-1, unit.toNanos(timeout));
return get(success);
}
private Object get(boolean success) {
if (success) {
return this.rpcResponse;
} else {
throw new RuntimeException("超時:requestID" + rpcRequest.getRequestId() +
" method:" + rpcRequest.getMethodName() + " interface:" + rpcRequest.getInterfaceName()
);
}
}
/**
* 繼承同步器,這裏只是用來自旋改變狀態,根據state來實現,state初始爲0
*/
static class Sync extends AbstractQueuedSynchronizer {
/**
* 嘗試獲取鎖,如果獲取不了,加入同步隊列,阻塞自己,只由同步隊列的頭自旋獲取鎖
* 當狀態爲1,即有結果返回時可以獲取鎖進行後續操作,設置result
*這裏只有一個節點,會不斷自選嘗試獲取鎖
* @param arg
* @return
*/
@Override
protected boolean tryAcquire(int arg) {
return getState() == 1;
}
/**
* 用於遠端有返回時,設置狀態變更
* 從頭喚醒同步隊列的隊頭下一個等待的節點,如果下一個節點爲空,則從隊尾喚醒
* @param arg
* @return
*/
@Override
protected boolean tryRelease(int arg) {
//把狀態設置爲1,給tryAcquire獲取鎖進行操作
return getState() == 0 ? compareAndSetState(0, 1) : true;
}
public boolean isDone() {
return getState() == 1;
}
}
}
憑證寫好了,接下來就寫處理憑證的handler類,這個類有一個發送請求的方法,主要是把憑證和對應的標識存儲起來,並在有結果返回時設置返回狀態
package com.gdut.rpcstudy.demo.framework.protocol.netty.asyn;
import com.gdut.rpcstudy.demo.framework.URL;
import com.gdut.rpcstudy.demo.framework.protocol.netty.asyn.RpcFuture;
import com.gdut.rpcstudy.demo.framework.serialize.tranobject.RpcRequest;
import com.gdut.rpcstudy.demo.framework.serialize.tranobject.RpcResponse;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import lombok.Data;
import lombok.EqualsAndHashCode;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
/**
* @author: lele
* @date: 2019/11/21 下午4:07
* 異步模式下的處理
*/
@Data
@EqualsAndHashCode(callSuper = false)
public class NettyAsynHandler extends SimpleChannelInboundHandler<RpcResponse> {
//key:requestId,value自定義future
private ConcurrentHashMap<String, RpcFuture> resultMap = new ConcurrentHashMap<>();
private volatile Channel channel;
//對應的遠端URL
private final URL url;
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
this.channel = ctx.channel();
}
public void close() {
this.channel.close();
}
@Override
protected void channelRead0(ChannelHandlerContext channelHandlerContext, RpcResponse s) throws Exception {
System.out.println("收到結果:" + s);
String requestId = s.getRequestId();
//設置完成並移除future
RpcFuture future = resultMap.get(requestId);
if (s != null) {
future.done(s);
resultMap.remove(requestId);
}
}
public RpcFuture sendRequest(RpcRequest rpcRequest) {
final CountDownLatch latch = new CountDownLatch(1);
RpcFuture future = new RpcFuture(rpcRequest);
//放到請求列表裏面
resultMap.put(rpcRequest.getRequestId(), future);
//發送請求
channel.writeAndFlush(rpcRequest).addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture channelFuture) throws Exception {
System.out.println("發送了消息" + rpcRequest.toString());
latch.countDown();
}
});
try {
//等待結果
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
return future;
}
}
第一個問題的兩個比較核心的類就編寫到這裏。
然後到第二個問題,如何實現一個鏈接管理池?要爲每個服務下的每個server建立一個客戶端,則需要先通過註冊中心獲取地址,再進行後續操作,這裏獲取每個服務下的server地址
public static Map<String,List<URL>> getAllURL(){
Map<String,List<URL>> mapList=null;
try {
List<String> servcieList=client.getChildren().forPath("/");
mapList=new HashMap<>(servcieList.size());
for (String s : servcieList) {
mapList.put(s,getService(s));
}
} catch (Exception e) {
e.printStackTrace();
}
return mapList;
}
獲取地址後,我們可以建立關係了,採用{serviceName:{serverAddress:clienthandler}}的形式進行存儲,從client handler存放的就是上面說的客戶端處理器,通過這裏我們可以通過不同的選擇方式挑一個出來發送請求,這裏採用的選擇方式是輪詢。
大概是這樣的形式去獲取憑證,管理池根據服務名輪詢地選取一個客戶端,並且把憑證讓他處理,然後返回憑證,可這裏有個隱患,客戶端連接到服務端是需要時間的,再沒有可用的客戶端情況下,會報空指針異常
public RpcFuture sendFuture(String serviceName, RpcRequest request) {
NettyAsynHandler handler=ConnectManager.getInstance().getConnectionWithPolling(serviceName);
RpcFuture future = handler.sendRequest(request);
return future;
}
這時候我們需要爲加入鎖和條件隊列(Condition),當滿足條件時,纔可以鏈接,不然就自旋等待直到條件滿足,當客戶端鏈接上u對應的server,我們纔可以獲取對應的客戶端處理器,但是服務有這麼多,用一個鎖是可以,但是調用不相干的服務也要被鎖嗎?這時候可以細粒度化我們的鎖,爲每個服務建立一個鎖及其相應的條件隊列。這是連接池需要注意的點,下面看看代碼
/**
* @author: lele
* @date: 2019/11/21 上午11:58
* 管理連接池
*/
public class ConnectManager {
private Boolean isShutDown = false;
/**
* 客戶端鏈接服務端超時時間
*/
private long connectTimeoutMillis = 5000;
/**
* 自定義6個線程組用於客戶端服務
*/
private EventLoopGroup eventLoopGroup = new NioEventLoopGroup(6);
/**
* 存放服務對應的訪問數,用於輪詢
*/
private Map<String, AtomicInteger> pollingMap = new ConcurrentHashMap<>();
/**
* 對於每個服務都有一個鎖,每個鎖都有個條件隊列,用於控制鏈接獲取
*/
private Map<String, Object[]> serviceCondition = new ConcurrentHashMap<>();
/**
* 存放服務端地址和handler的關係
*/
private Map<String, Map<URL, NettyAsynHandler>> serverClientMap = new ConcurrentHashMap<>();
/**
* 用來初始化客戶端
*/
private ThreadPoolExecutor clientBooter = new ThreadPoolExecutor(
16, 16, 600, TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(1024)
, new BooterThreadFactory(), new ThreadPoolExecutor.AbortPolicy());
private static class Holder {
private static final ConnectManager j = new ConnectManager();
}
private ConnectManager() {
//初始化時把所有的url加進去,這裏可能沒有可用鏈接,所以需要添加對節點的監聽
Map<String, List<URL>> allURL = ZkRegister.getAllURL();
for (String s : allURL.keySet()) {
//爲每個服務添加鎖和條件隊列,通過條件隊列控制客戶端鏈接獲取
ReentrantLock lock =new ReentrantLock();
Condition condition = lock.newCondition();
serviceCondition.put(s,new Object[]{lock,condition});
}
addServer(allURL);
}
public static ConnectManager getInstance() {
return Holder.j;
}
/**
* 添加該服務對應的鏈接和handler
* @param serviceName
* @param url
* @param handler
*/
public void addConnection(String serviceName, URL url, NettyAsynHandler handler) {
Map<URL, NettyAsynHandler> handlerMap;
if (!serverClientMap.containsKey(serviceName)) {
handlerMap = new HashMap<>();
} else {
handlerMap = serverClientMap.get(serviceName);
}
handlerMap.put(url, handler);
//添加服務名和對應的url:客戶端鏈接
serverClientMap.put(serviceName, handlerMap);
//喚醒等待客戶端鏈接的線程
signalAvailableHandler(serviceName);
}
/**
* 獲取對應服務下的handler,通過輪詢獲取
* @param servicName
* @return
*/
public NettyAsynHandler getConnectionWithPolling(String servicName) {
Map<URL, NettyAsynHandler> urlNettyAsynHandlerMap = serverClientMap.get(servicName);
int size = 0;
//先嚐試獲取
if (urlNettyAsynHandlerMap != null) {
size = urlNettyAsynHandlerMap.size();
}
//不行就自選等待
while (!isShutDown && size <= 0) {
try {
//自旋等待可用服務出現,因爲客戶端與服務鏈接需要一定的時間,如果直接返回會出現空指針異常
boolean available = waitingForHandler(servicName);
if (available) {
urlNettyAsynHandlerMap = serverClientMap.get(servicName);
size = urlNettyAsynHandlerMap.size();
}
} catch (InterruptedException e) {
throw new RuntimeException("出錯", e);
}
}
//獲取對應的訪問次數
AtomicInteger count = pollingMap.get(servicName);
int index = (count.getAndAdd(1) + size) % size;
Iterator<Map.Entry<URL, NettyAsynHandler>> iterator = urlNettyAsynHandlerMap.entrySet().iterator();
//取出相應的handler
NettyAsynHandler nettyAsynHandler = null;
for (int i = 0; i <= index; i++) {
nettyAsynHandler = iterator.next().getValue();
}
return nettyAsynHandler;
}
/**
* 等待一定時間,等handler和相應的server建立建立鏈接,用條件隊列控制
* @param serviceName
* @return
* @throws InterruptedException
*/
private boolean waitingForHandler(String serviceName) throws InterruptedException {
Object[] objects = serviceCondition.get(serviceName);
ReentrantLock lock = (ReentrantLock) objects[0];
lock.lock();
Condition condition= (Condition) objects[1];
try {
return condition.await(this.connectTimeoutMillis, TimeUnit.MILLISECONDS);
} finally {
lock.unlock();
}
}
public void removeURL(URL url) {
List<String> list = new ArrayList<>();
for (Map.Entry<String, Map<URL, NettyAsynHandler>> map : serverClientMap.entrySet()) {
for (Map.Entry<URL, NettyAsynHandler> urlNettyAsynHandlerEntry : map.getValue().entrySet()) {
if (urlNettyAsynHandlerEntry.getKey().equals(url)) {
urlNettyAsynHandlerEntry.getValue().close();
list.add(map.getKey() + "@" + urlNettyAsynHandlerEntry.getKey());
}
}
}
for (String s : list) {
String[] split = s.split("@");
serverClientMap.get(split[0]).remove(split[1]);
}
}
/**
* 釋放對應服務的條件隊列,代表有客戶端鏈接可用了
* @param serviceName
*/
private void signalAvailableHandler(String serviceName) {
Object[] objects = serviceCondition.get(serviceName);
ReentrantLock lock = (ReentrantLock) objects[0];
lock.lock();
Condition condition= (Condition) objects[1];
try {
condition.signalAll();
} finally {
lock.unlock();
}
}
/**
* 添加server,並啓動對應的服務器
* @param allURL
*/
public void addServer(Map<String, List<URL>> allURL) {
for (String s : allURL.keySet()) {
pollingMap.put(s, new AtomicInteger(0));
List<URL> urls = allURL.get(s);
for (URL url : urls) {
//提交創建任務
clientBooter.submit(new Runnable() {
@Override
public void run() {
createClient(s, eventLoopGroup, url);
}
});
}
}
System.out.println("初始化客戶端ing");
}
/**
* 創建客戶端,持久化鏈接
* @param serviceName
* @param eventLoopGroup
* @param url
*/
public void createClient(String serviceName, EventLoopGroup eventLoopGroup, URL url) {
Bootstrap b = new Bootstrap();
b.group(eventLoopGroup)
.channel(NioSocketChannel.class)
.handler((new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline()
//把request實體變爲字節
.addLast(new RpcEncoder(RpcRequest.class))
//把返回的response字節變爲對象
.addLast(new RpcDecoder(RpcResponse.class))
.addLast(new NettyAsynHandler(url));
}
}));
ChannelFuture channelFuture = b.connect(url.getHostname(), url.getPort());
channelFuture.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(final ChannelFuture channelFuture) throws Exception {
//鏈接成功後的操作,把相應的url地址和客戶端鏈接存入
if (channelFuture.isSuccess()) {
NettyAsynHandler handler = channelFuture.channel().pipeline().get(NettyAsynHandler.class);
addConnection(serviceName, url, handler);
}
}
});
}
/**
* 關閉方法,關閉每個客戶端鏈接,釋放所有鎖,關掉創建鏈接的線程池,和客戶端的處理器
*/
public void stop() {
isShutDown = true;
for (Map<URL, NettyAsynHandler> urlNettyAsynHandlerMap : serverClientMap.values()) {
urlNettyAsynHandlerMap.values().forEach(e -> e.close());
}
for(String s:serviceCondition.keySet()){
signalAvailableHandler(s);
}
clientBooter.shutdown();
eventLoopGroup.shutdownGracefully();
}
/**
* 啓動客戶端鏈接的自定義線程工廠
*/
static class BooterThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
BooterThreadFactory() {
group = new ThreadGroup("connectManger");
group.setDaemon(false);
group.setMaxPriority(5);
namePrefix = "clientBooter-" +
poolNumber.getAndIncrement() +
"-thread-";
}
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r,
namePrefix + threadNumber.getAndIncrement(),
0);
return t;
}
}
}
這兩個問題解決好後,接下來對整體框架做修改,首先是接口註解,增加了異步模式,對象工廠根據註解來選擇不同的代理方式,
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
//用於接口上,name爲服務名,zk則在註冊服務改爲 服務名/ip,服務端通過傳來的接口名通過反射獲取類,或者通過給spring託管獲取其class
public @interface RpcStudyClient {
String name();
//結果返回是異步還是同步模式
int mode() default sync;
int sync=0;
int asyn=1;
}
@Data
@EqualsAndHashCode(callSuper = false)
public class RpcStudyClientFactoryBean implements FactoryBean<Object> {
private Class<?> type;
@Override
public Object getObject() throws Exception {
//根據RpcStudeClient的mode字段選擇以哪種方式代理
RpcStudyClient annotation = type.getAnnotation(RpcStudyClient.class);
int mode = annotation.mode();
return mode==RpcStudyClient.asyn?ProxyFactory.getAsyncProxy(this.type):ProxyFactory.getProxy(this.type);
}
@Override
public Class<?> getObjectType() {
return this.type;
}
}
代理工廠的新增異步代理,這裏需要自旋獲取結果,不然會導致結果獲取不到。
public static <T> T getAsyncProxy(Class interfaceClass) {
return (T) Proxy.newProxyInstance(interfaceClass.getClassLoader(), new Class[]{interfaceClass}, new InvocationHandler() {
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
//指定所用協議
Protocol protocol = ProtocolFactory.netty();
RpcStudyClient annotation = (RpcStudyClient) interfaceClass.getAnnotation(RpcStudyClient.class);
String requestId = UUID.randomUUID().toString().replace("-", "");
//封裝方法參數
RpcRequest rpcRequest = new RpcRequest(requestId, interfaceClass.getName(), method.getName(), args, method.getParameterTypes(), annotation.mode());
Future<RpcFuture> res = futureTask.submit(new Callable<RpcFuture>() {
@Override
public RpcFuture call() throws Exception {
RpcFuture res = protocol.sendFuture(annotation.fetch(), annotation.name(), rpcRequest);
//先嚐試一次
if (res.isDone()) {
return res;
}
//不行就自旋等待
while (!res.isDone()) {
}
return res;
}
;
});
//發送請求
//這裏的管理連接池通過服務名去訪問zk,獲取可用的url
return returnResult(res.get());
}
});
}
/**
* 具體的處理異步返回的方法
* @param res
* @return
* @throws ExecutionException
* @throws InterruptedException
*/
public static Object returnResult(RpcFuture res) throws ExecutionException, InterruptedException {
RpcResponse response = (RpcResponse) res.get();
if (response.getError() != null) {
throw new RuntimeException(response.getError());
} else {
return response.getResult();
}
}
請求實體類也增加多一個屬性mode,標識是同步還是異步模式,給serverhandler做處理,如果是異步,則不關閉鏈接,不然會導致客戶端發的請求無法接受,同步則關閉。
@Data
@AllArgsConstructor
public class RpcRequest {
private String requestId;
private String interfaceName;
private String methodName;
private Object[] params;
//防止重載
private Class[] paramsTypes;
//是否異步
private int mode;
}
ctx.writeAndFlush(response).addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture channelFuture) throws Exception {
System.out.println("發送了結果"+response);
//如果是mode爲同步,就直接關閉連接
if(rpcRequest.getMode()==RpcStudyClient.sync){
ctx.channel().close();
}
// 當異步模式時,不關閉鏈接
}
});
這裏還可以爲異步請求添加回調處理,慢請求日誌記錄等等,都可以在自定義的憑證裏面下文章。異步請求和連接池大概就到這裏,具體代碼https://github.com/97lele/rpcstudy/tree/withconcurrent