堵塞與非堵塞原理
傳統硬件的堵塞(IO)如下,從內存中讀取數據,然後寫到磁盤,而CPU一直等到磁盤寫完成,磁盤的寫操作是慢的,這段時間CPU被堵塞不能發揮效率。
使用非堵塞(NIO)的DMA如下圖:CPU只是發出寫操作這樣的指令,做一些初始化工作,DMA具體執行,從內存中讀取數據,然後寫到磁盤,當完成寫後發出一箇中斷事件給CPU。這段時間CPU是空閒的,可以做別的事情。這個原理稱爲Zero.copy零拷貝。
Netty是一個高性能事件驅動的異步的非堵塞的IO(NIO)框架,用於建立TCP等底層的連接,基於Netty可以建立高性能的Http服務器。Netty的一個線程服務於很多請求,如下圖,當從Java NIO獲得一個Selector事件,將激活通道Channel。
WebClient
WebClient是spring的一個響應式客戶端,它提供了RestTemplate的替代方法。它公開了一個功能齊全、流暢的API,非阻塞的、被動的客戶端執行HTTP請求。WebClient使用與WebFlux服務器應用程序相同的編解碼器,並與服務器功能Web框架共享公共基本包。默認情況下,它使用Reactor Netty作爲HTTP客戶端庫,但其他人可以通過自定義ClientHttpConnector插入。
WebClient運行時核心類庫
WebClient請求就是這麼簡單
WebClient.builder()
.baseUrl("http://www.baidu.com")
.build()
.get()
.exchange()
.flatMap(response->response.bodyToMono(String.class))
.subscribe(s->{
System.out.println(s);
});
運行時核心代碼分析
- 訂閱http請求響應
//reactor.core.publisher.Mono
public final Disposable subscribe(
@Nullable Consumer<? super T> consumer,
@Nullable Consumer<? super Throwable> errorConsumer,
@Nullable Runnable completeConsumer,
@Nullable Consumer<? super Subscription> subscriptionConsumer) {
return subscribeWith(new LambdaMonoSubscriber<>(consumer, errorConsumer,
completeConsumer, subscriptionConsumer));
}
//reactor.ipc.netty.http.client.MonoHttpClientResponse
public void subscribe(final CoreSubscriber<? super HttpClientResponse> subscriber) {
ReconnectableBridge bridge = new ReconnectableBridge();
bridge.activeURI = startURI;
Mono.defer(() -> parent.client.newHandler(new HttpClientHandler(this, bridge),
parent.options.getRemoteAddress(bridge.activeURI),
HttpClientOptions.isSecure(bridge.activeURI),
bridge))
.retry(bridge)
.cast(HttpClientResponse.class)
.subscribe(subscriber);
}
- 發起http請求
//reactor.ipc.netty.http.client.MonoHttpClientResponse
public Publisher<Void> apply(NettyInbound in, NettyOutbound out) {
try {
URI uri = bridge.activeURI;
HttpClientOperations ch = (HttpClientOperations) in;
String host = uri.getHost();
int port = uri.getPort();
if (port != -1 && port != 80 && port != 443) {
host = host + ':' + port;
}
ch.getNettyRequest()
.setUri(uri.getRawPath() + (uri.getQuery() == null ? "" :
"?" + uri.getRawQuery()))
.setMethod(parent.method)
.setProtocolVersion(HttpVersion.HTTP_1_1)
.headers()
.add(HttpHeaderNames.HOST, host)
.add(HttpHeaderNames.ACCEPT, ALL);
if (Objects.equals(parent.method, HttpMethod.GET)
|| Objects.equals(parent.method, HttpMethod.HEAD)
|| Objects.equals(parent.method, HttpMethod.DELETE)) {
ch.chunkedTransfer(false);
}
if (parent.handler != null) {
return parent.handler.apply(ch);
}
else {
return ch.send();
}
}
catch (Throwable t) {
return Mono.error(t);
}
}
- 接收http數據
exchange()
//org.springframework.web.reactive.function.client.ExchangeFunctions
public Mono<ClientResponse> exchange(ClientRequest request) {
return this.connector
.connect(request.method(), request.url(),
clientHttpRequest -> request.writeTo(clientHttpRequest, this.strategies))
.doOnSubscribe(subscription -> logger.debug("Subscriber present"))
.doOnRequest(n -> logger.debug("Demand signaled"))
.doOnCancel(() -> logger.debug("Cancelling request"))
.map(response -> {
if (logger.isDebugEnabled()) {
int status = response.getRawStatusCode();
HttpStatus resolvedStatus = HttpStatus.resolve(status);
logger.debug("Response received, status: " + status +
(resolvedStatus != null ? " " + resolvedStatus.getReasonPhrase() : ""));
}
return new DefaultClientResponse(response, this.strategies);
});
}
- 合併數據
//org.springframework.core.io.buffer.DataBufferUtils
public static Mono<DataBuffer> join(Publisher<DataBuffer> dataBuffers) {
return Flux.from(dataBuffers)
.collectList()
.filter(list -> !list.isEmpty())
.map(list -> {
DataBufferFactory bufferFactory = list.get(0).factory();
return bufferFactory.join(list);
});
}
- 轉換數據
flatMap(response->response.bodyToMono(String.class));
public Mono<T> decodeToMono(Publisher<DataBuffer> inputStream, ResolvableType elementType,
@Nullable MimeType mimeType, @Nullable Map<String, Object> hints) {
return DataBufferUtils.join(inputStream)
.map(buffer -> decodeDataBuffer(buffer, elementType, mimeType, hints));
}
protected String decodeDataBuffer(DataBuffer dataBuffer, ResolvableType elementType,
@Nullable MimeType mimeType, @Nullable Map<String, Object> hints) {
Charset charset = getCharset(mimeType);
CharBuffer charBuffer = charset.decode(dataBuffer.asByteBuffer());
DataBufferUtils.release(dataBuffer);
return charBuffer.toString();
}
reactor core核心接口分析
構建
接口 | 說明 | 舉例 |
---|---|---|
from | 從其他響應式流中創建reactor core的流,適用於所有不同reactor stream的實現間的適配 | Mono.from(Publisher<? extends T> source) |
defer | 延遲建立新的流,defer一開始不會馬上創建Publisher,直到有訂閱者訂閱時纔會創建,且每次都創建全新的 Publisher。 | Mono.defer(Supplier<? extends Mono<? extends T>> supplier) |
just | 基於對象創建流,參數是流將要發送的元素 | Mono.just(T t) |
轉換
接口 | 說明 | 舉例 |
---|---|---|
map | 將流的元素按指定的方法變換 | map(o-> o.toString()) |
flatMap | 異步轉換當前流發出的項,返回另一個流發出的值(可能更改值類型) | flatMap(s->Mono.just(s)) |
flatMap適用兩種場景
a. 將當前流的元素轉換爲多個元素,並在新的流中逐一發出轉換的多個元素。這裏的flat平鋪就是這個含義。
b. 由當前流的元素事件觸發一個新的流。在http請求場景下,Mono<ClientResponse> clientResponse = request.exchange(); clientResponse這個流在http請求response header返回時得到了流元素ClientResponse,但此時response body還在管道中傳輸,ClientResponse實際沒有拿到完整的body內容,所以ClientResponse將body封裝到了一個新的流中,如果下游需要取到這個流,就必須通過flatMap獲取。
參考文檔
使用 Reactor 進行反應式編程
https://www.ibm.com/developerworks/cn/java/j-cn-with-reactor-response-encode/index.html?lnk=hmhm
Reactive響應式編程
https://www.jdon.com/reactive.html
那些年我們錯過的響應式編程
http://www.open-open.com/lib/view/open1429151543096.html
Reactor Reactive編程介紹
https://luyiisme.github.io/2017/02/11/spring-reactor-programing/
Reactor 入門與實踐
https://studygolang.com/articles/11341?fr=sidebar
Reactor 實例解析
https://www.infoq.cn/article/reactor-by-example
Netty原理和使用
https://www.jdon.com/concurrent/netty.html
使用 UT 玩轉 defer 和 retryWhen
https://www.jianshu.com/p/d42793eba591
照虎畫貓深入理解響應式流規範
http://blog.51cto.com/liukang/2090922
Spring 5 WebClient
https://www.baeldung.com/spring-5-webclient