netty中粘包與拆包的處理

TCP是個流協議，流是一串沒有界限的數據。TCP會根據TCP緩衝區的實際情況對包進行劃分。因此造成一個完整的業務包，會被TCP分成多個包、把多個包封裝成一個大的包進行發送。

粘包與拆包現象

服務端分兩次讀取到了兩個獨立的數據包，分別是D1和D2，沒有粘包和拆包；
服務端一次接收到了兩個數據包，D1和D2粘合在一起，被稱爲TCP粘包；
服務端分兩次讀取到了兩個數據包，第一次讀取到了完整的D1包和D2包的部分內容，第二次讀取到了D2包的剩餘內容，這被稱爲TCP拆包；
服務端分兩次讀取到了兩個數據包，第一次讀取到了D1包的部分內容D1_1，第二次讀取到了D1包的剩餘內容D1_2和D2包的整包。

產生原因

應用程序write寫入的字節大小/大於套接口發送緩衝區大小；
進行MSS大小的TCP分段；
以太網幀的payload大於MTU進行IP分片。

對於Linux，發送緩衝區的默認值爲：16384。可使用下面命令查看：

# 接收
cat /proc/sys/net/ipv4/tcp_rmem
# min   default max
# 4096	87380	6291456 (單位：byte)
# 4K		85K		6M
# 發送(單位：byte)
cat /proc/sys/net/ipv4/tcp_wmem
# min   default max
# 4096	16384	4194304  (單位：byte)
# 4K		16K		4M

數據來自百度雲的雲服務器：

對於MacOS，可參考：sysctl net.inet.tcp，但是好像沒找到與linux類似的參數。

如何解決

Netty如何解決

Netty中主要是在收到數據後，對數據進行處理解碼處理時，根據不同的策略，進行了拆包操作，然後將得到的完整的業務數據包傳遞給下個處理邏輯。分割前後的邏輯主要在ByteToMessageDecoder這個類中。它的繼承如下：

每次從TCP緩衝區讀到數據都會調用其channelRead()方法。這個函數的處理邏輯是：

用累加器cumulator將新讀入的數據(ByteBuf)存儲到cumulation中；
調用解碼器

累加器

存在兩個累加器，MERGE_CUMULATOR和COMPOSITE_CUMULATOR。默認的是前者，即:private Cumulator cumulator = MERGE_CUMULATOR;。

MERGE_CUMULATOR會先判斷是否需要擴容，然後再將收到的msg拷貝到cumulation中。

/**
    * Cumulate {@link ByteBuf}s by merge them into one {@link ByteBuf}'s, using memory copies.
    */
public static final Cumulator MERGE_CUMULATOR = new Cumulator() {
    @Override
    public ByteBuf cumulate(ByteBufAllocator alloc, ByteBuf cumulation, ByteBuf in) {
        try {
            final int required = in.readableBytes();
            if (required > cumulation.maxWritableBytes() ||
                    (required > cumulation.maxFastWritableBytes() && cumulation.refCnt() > 1) ||
                    cumulation.isReadOnly()) {
                // Expand cumulation (by replacing it) under the following conditions:
                // - cumulation cannot be resized to accommodate the additional data
                // - cumulation can be expanded with a reallocation operation to accommodate but the buffer is
                //   assumed to be shared (e.g. refCnt() > 1) and the reallocation may not be safe.
                return expandCumulation(alloc, cumulation, in);
            }
            return cumulation.writeBytes(in);
        } finally {
            // We must release in in all cases as otherwise it may produce a leak if writeBytes(...) throw
            // for whatever release (for example because of OutOfMemoryError)
            in.release();
        }
    }
};

擴容的過程是先得到一個能夠容納下原數據+當前數據的收集器，然後將原數據和當前數據依次拷貝進入收集器，最後釋放舊的收集器裏面的數據。

private static ByteBuf expandCumulation(ByteBufAllocator alloc, ByteBuf oldCumulation, ByteBuf in) {
    ByteBuf newCumulation = alloc.buffer(alloc.calculateNewCapacity(
            oldCumulation.readableBytes() + in.readableBytes(), MAX_VALUE));
    ByteBuf toRelease = newCumulation;
    try {
        newCumulation.writeBytes(oldCumulation);
        newCumulation.writeBytes(in);
        toRelease = oldCumulation;
        return newCumulation;
    } finally {
        toRelease.release();
    }
}

COMPOSITE_CUMULATOR是將每個新收到的消息，作爲一個Component存儲到收集器CompositeByteBuf中的components數組中。

/**
    * Cumulate {@link ByteBuf}s by add them to a {@link CompositeByteBuf} and so do no memory copy whenever possible.
    * Be aware that {@link CompositeByteBuf} use a more complex indexing implementation so depending on your use-case
    * and the decoder implementation this may be slower then just use the {@link #MERGE_CUMULATOR}.
    */
public static final Cumulator COMPOSITE_CUMULATOR = new Cumulator() {
    @Override
    public ByteBuf cumulate(ByteBufAllocator alloc, ByteBuf cumulation, ByteBuf in) {
        try {
            if (cumulation.refCnt() > 1) {
                // Expand cumulation (by replace it) when the refCnt is greater then 1 which may happen when the
                // user use slice().retain() or duplicate().retain().
                //
                // See:
                // - https://github.com/netty/netty/issues/2327
                // - https://github.com/netty/netty/issues/1764
                return expandCumulation(alloc, cumulation, in);
            }
            final CompositeByteBuf composite;
            if (cumulation instanceof CompositeByteBuf) {
                composite = (CompositeByteBuf) cumulation;
            } else {
                composite = alloc.compositeBuffer(MAX_VALUE);
                composite.addComponent(true, cumulation);
            }
            composite.addComponent(true, in);
            in = null;
            return composite;
        } finally {
            if (in != null) {
                // We must release if the ownership was not transferred as otherwise it may produce a leak if
                // writeBytes(...) throw for whatever release (for example because of OutOfMemoryError).
                in.release();
            }
        }
    }
};

拆包解碼流程

callDecode()方法中的decodeRemovalReentryProtection()將調用decode()方法，其中decode()是一個抽象方法，由子類去實現。主要的子類有：

`FixedLengthFrameDecoder`

裏面有一個屬性叫frameLength，用來表示消息的長度。

A decoder that splits the received ByteBufs by the fixed number of bytes. For example, if you received the following four fragmented packets:
   +---+----+------+----+
   | A | BC | DEFG | HI |
   +---+----+------+----+

A FixedLengthFrameDecoder(3) will decode them into the following three packets with the fixed length:
   +-----+-----+-----+
   | ABC | DEF | GHI |
   +-----+-----+-----+

流程也比較簡單，收集器裏面的數據長度夠frameLength，就從收集器中截取frameLengthbyte，然後返回一個新的ByteBuf。

@Override
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
    Object decoded = decode(ctx, in);
    if (decoded != null) {
        out.add(decoded);
    }
}

/**
 * Create a frame out of the {@link ByteBuf} and return it.
 *
 * @param   ctx             the {@link ChannelHandlerContext} which this {@link ByteToMessageDecoder} belongs to
 * @param   in              the {@link ByteBuf} from which to read data
 * @return  frame           the {@link ByteBuf} which represent the frame or {@code null} if no frame could
 *                          be created.
 */
protected Object decode(
        @SuppressWarnings("UnusedParameters") ChannelHandlerContext ctx, ByteBuf in) throws Exception {
    if (in.readableBytes() < frameLength) {
        return null;// 長度不夠，此次decode不產生消息
    } else {
        return in.readRetainedSlice(frameLength);
    }
}

有一個問題，如果一次收到的數據長度爲2 * frameLength，且這個數據是最後一個數據，那麼是否存在解碼出現異常的情況？

有一個循環
輸入結束的時候再次調用解碼

`LineBasedFrameDecoder`

流程是先找到當前消息中的換行符，存在且沒有超過最大長度，返回解釋到的數據。

`DelimiterBasedFrameDecoder`

根據特定的字符進行分割，其中如果分割符是行標誌，會調用LineBasedFrameDecoder進行分割解碼。

// decode()方法中
if (lineBasedDecoder != null) {
	return lineBasedDecoder.decode(ctx, buffer);
}

// lineBasedDecoder不爲空的情況是分割字符是行分割字符
// 構造方法中
if (isLineBased(delimiters) && !isSubclass()) {
	lineBasedDecoder = new LineBasedFrameDecoder(maxFrameLength, stripDelimiter, failFast);
	this.delimiters = null;
}

判斷分割符是否爲行分割符的代碼如下：

private static boolean isLineBased(final ByteBuf[] delimiters) {
	if (delimiters.length != 2) {
		return false;
	}
	ByteBuf a = delimiters[0];
	ByteBuf b = delimiters[1];
	if (a.capacity() < b.capacity()) {
		a = delimiters[1];
		b = delimiters[0];
	}
	return a.capacity() == 2 && b.capacity() == 1
		&& a.getByte(0) == '\r' && a.getByte(1) == '\n'
		&& b.getByte(0) == '\n';
}

因爲分割字符可能是多個，當數據中存在多個分割字符的情況下，會用分割後得到的數據最短的那個分割字符。如下：

// Try all delimiters and choose the delimiter which yields the shortest frame.
int minFrameLength = Integer.MAX_VALUE;
ByteBuf minDelim = null;
for (ByteBuf delim: delimiters) {
	int frameLength = indexOf(buffer, delim);
	if (frameLength >= 0 && frameLength < minFrameLength) {
		minFrameLength = frameLength;
		minDelim = delim;
	}
}

For example, if you have the following data in the buffer:
±-------------+
| ABC\nDEF\r\n |
±-------------+

a DelimiterBasedFrameDecoder(Delimiters.lineDelimiter()) will choose ‘\n’ as the first delimiter and produce two frames:
±----±----+
| ABC | DEF |
±----±----+

rather than incorrectly choosing ‘\r\n’ as the first delimiter:
±---------+
| ABC\nDEF |
±---------+