爲何要封包拆包
TCP是個"流"協議,所謂流,就是沒有界限的一串數據.大家可以想想河裏的流水,是連成一片的,其間是沒有分界線的.但一般通訊程序開發是需要定義一個個相互獨立的數據包的,比如用於登陸的數據包,用於註銷的數據包.由於TCP"流"的特性以及網絡狀況,在進行數據傳輸時會出現以下幾種情況.
假設我們連續調用兩次send分別發送兩段數據data1和data2,在接收端有以下幾種接收情況(當然不止這幾種情況,這裏只列出了有代表性的情況).A.先接收到data1,然後接收到data2.
B.先接收到data1的部分數據,然後接收到data1餘下的部分以及data2的全部.
C.先接收到了data1的全部數據和data1的部分數據,然後接收到了data2的餘下的數據.
D.一次性接收到了data1和data2的全部數據.
如何封包
最初遇到"粘包"的問題時,我是通過在兩次send之間調用sleep來休眠一小段時間來解決.這個解決方法的缺點是顯而易見的,使傳輸效率大大降低,而且也並不可靠.後來就是通過應答的方式來解決,儘管在大多數時候是可行的,但是不能解決象B的那種情況,而且採用應答方式增加了通訊量,加重了網絡負荷.再後來就是對數據包進行封包和拆包的操作.
封包:封包就是給一段數據加上包頭,這樣一來數據包就分爲包頭和包體兩部分內容了(以後講過濾非法包時封包會加入"包尾"內容).包頭其實上是個大小固定的結構體,其中有個結構體成員變量表示包體的長度,這是個很重要的變量,其他的結構體成員可根據需要自己定義.根據包頭長度固定以及包頭中含有包體長度的變量就能正確的拆分出一個完整的數據包.
對於拆包目前我最常用的是以下兩種方式.
1.動態緩衝區暫存方式.之所以說緩衝區是動態的是因爲當需要緩衝的數據長度超出緩衝區的長度時會增大緩衝區長度.
大概過程描述如下:
A,爲每一個連接動態分配一個緩衝區,同時把此緩衝區和SOCKET關聯,常用的是通過結構體關聯.
B,當接收到數據時首先把此段數據存放在緩衝區中.
C,判斷緩存區中的數據長度是否夠一個包頭的長度,如不夠,則不進行拆包操作.
D,根據包頭數據解析出裏面代表包體長度的變量.
E,判斷緩存區中除包頭外的數據長度是否夠一個包體的長度,如不夠,則不進行拆包操作.
F,取出整個數據包.這裏的"取"的意思是不光從緩衝區中拷貝出數據包,而且要把此數據包從緩存區中刪除掉.刪除的辦法就是把此包後面的數據移動到緩衝區的起始地址.
開始
定義協議
包頭:爲何是4個Byte呢?因給我給包頭的定義是一個長度代表數據區的長度,長度使用int記錄的,而int類型佔有4個字節,一個簡單的協議就定義好了封包過程
下面是一個心跳的封包,還有其他的都是類似的例如TokenData丶AcceptSuccessData等
public class PulseData implements IPulseSendable { private String str; public PulseData() { JSONObject jsonObject = new JSONObject(); try { jsonObject.put("token", DeviceUtils.getTokennId()); jsonObject.put("type", 0); str = jsonObject.toString(); } catch (JSONException e) { e.printStackTrace(); } } @Override public byte[] parse() { byte[] body = str.getBytes(Charset.defaultCharset()); ByteBuffer bb = ByteBuffer.allocate(body.length + 8); bb.order(ByteOrder.LITTLE_ENDIAN); bb.putInt(1000);// 這個type在服務端用的 現在沒啥用處 bb.putInt(body.length); bb.put(body); byte[] array = bb.array(); return array; } }拆包
根據上面約定的協議,我要將服務端推送的封包數據,進行拆包,分割成1條1條的數據
ReaderImpl的Read方法
原理就是originalData就是一條數據,他會先讀取4個字節,然後將4個自己轉換成int類型,這樣就知道數據的長度,然後讀取int常讀數據,具體看方法readBodyFromChannel讀取數據體的方法public void read() throws RuntimeException { OriginalData originalData = new OriginalData(); IHeaderProtocol headerProtocol = mOkOptions.getHeaderProtocol(); ByteBuffer headBuf = ByteBuffer.allocate(headerProtocol.getHeaderLength()); headBuf.order(mOkOptions.getReadByteOrder()); try { if (mRemainingBuf != null) { mRemainingBuf.flip(); int length = Math.min(mRemainingBuf.remaining(), headerProtocol.getHeaderLength()); headBuf.put(mRemainingBuf.array(), 0, length); if (length < headerProtocol.getHeaderLength()) { //there are no data left mRemainingBuf = null; readHeaderFromChannel(headBuf, headerProtocol.getHeaderLength() - length); } else { mRemainingBuf.position(headerProtocol.getHeaderLength()); } } else { readHeaderFromChannel(headBuf, headBuf.capacity()); } originalData.setHeadBytes(headBuf.array()); if (OkSocketOptions.isDebug()) { SL.i("read head: " + BytesUtils.toHexStringForLog(headBuf.array())); } int bodyLength = headerProtocol.getBodyLength(originalData.getHeadBytes(), mOkOptions.getReadByteOrder()); if (OkSocketOptions.isDebug()) { SL.i("need read body length: " + bodyLength); } if (bodyLength > 0) { if (bodyLength > mOkOptions.getMaxReadDataMB() * 1024 * 1024) { throw new ReadException("Need to follow the transmission protocol.\r\n" + "Please check the client/server code.\r\n" + "According to the packet header data in the transport protocol, the package length is " + bodyLength + " Bytes."); } ByteBuffer byteBuffer = ByteBuffer.allocate(bodyLength); byteBuffer.order(mOkOptions.getReadByteOrder()); if (mRemainingBuf != null) { int bodyStartPosition = mRemainingBuf.position(); int length = Math.min(mRemainingBuf.remaining(), bodyLength); byteBuffer.put(mRemainingBuf.array(), bodyStartPosition, length); mRemainingBuf.position(bodyStartPosition + length); if (length == bodyLength) { if (mRemainingBuf.remaining() > 0) {//there are data left ByteBuffer temp = ByteBuffer.allocate(mRemainingBuf.remaining()); temp.order(mOkOptions.getReadByteOrder()); temp.put(mRemainingBuf.array(), mRemainingBuf.position(), mRemainingBuf.remaining()); mRemainingBuf = temp; } else {//there are no data left mRemainingBuf = null; } //cause this time data from remaining buffer not from channel. originalData.setBodyBytes(byteBuffer.array()); mStateSender.sendBroadcast(IAction.ACTION_READ_COMPLETE, originalData); return; } else {//there are no data left in buffer and some data pieces in channel mRemainingBuf = null; } } readBodyFromChannel(byteBuffer); originalData.setBodyBytes(byteBuffer.array()); } else if (bodyLength == 0) { originalData.setBodyBytes(new byte[0]); if (mRemainingBuf != null) { //the body is empty so header remaining buf need set null if (mRemainingBuf.hasRemaining()) { ByteBuffer temp = ByteBuffer.allocate(mRemainingBuf.remaining()); temp.order(mOkOptions.getReadByteOrder()); temp.put(mRemainingBuf.array(), mRemainingBuf.position(), mRemainingBuf.remaining()); mRemainingBuf = temp; } else { mRemainingBuf = null; } } } else if (bodyLength < 0) { throw new ReadException( "this socket input stream is end of file read " + bodyLength + " ,we'll disconnect"); } mStateSender.sendBroadcast(IAction.ACTION_READ_COMPLETE, originalData); } catch (Exception e) { ReadException readException = new ReadException(e); throw readException; } }private void readBodyFromChannel(ByteBuffer byteBuffer) throws IOException { while (byteBuffer.hasRemaining()) { try { int remaining = byteBuffer.remaining(); // 判斷下需要讀取到長度,不需要每次都申請緩衝大小的byte 內存抖動會小一些 Write也最好改下 int length = mOkOptions.getReadPackageBytes(); if (remaining <= mOkOptions.getReadPackageBytes()) { length = remaining; } byte[] bufArray = new byte[length]; int len = mInputStream.read(bufArray); if (len == -1) { break; } if (len > remaining) { byteBuffer.put(bufArray, 0, remaining); mRemainingBuf = ByteBuffer.allocate(len - remaining); mRemainingBuf.order(mOkOptions.getReadByteOrder()); mRemainingBuf.put(bufArray, remaining, len - remaining); } else { byteBuffer.put(bufArray, 0, len); } } catch (Exception e) { throw e; } } if (OkSocketOptions.isDebug()) { SL.i("read total bytes: " + BytesUtils.toHexStringForLog(byteBuffer.array())); SL.i("read total length:" + (byteBuffer.capacity() - byteBuffer.remaining())); } }連接成功的業務邏輯
當連接服務成功,會立刻將設備Token發送到服務端,服務端會讀取當前設備還有那些歷史推送沒有推送過,會再次推送給設備,並且開啓心跳,5秒發送一次心跳數據
@Override public void onSocketConnectionSuccess(Context context, ConnectionInfo info, String action) { SL.e("連接成功"); if (listener != null) { listener.onPushStart(); } IConnectionManager connectionManager = OkSocket.open(info); if (connectionManager != null) { //發送token設備數據 connectionManager.send(new TokenData()); OkSocketOptions option = connectionManager.getOption(); boolean pulse = option.isPulse(); if (pulse) { //啓動心跳 PulseData pulseData = new PulseData(); PulseManager pulseManager = connectionManager.getPulseManager(); pulseManager.setPulseSendable(pulseData); pulseManager.pulse(); } } }收到推送消息
- 解析數據,獲取到推送的ID
- 判斷消息類型,如果是心跳類型,進行喂狗操作(就是就是將心跳的一個標記置換成0)
- 讀取數據庫判斷是否以前收到的這條推送,如果收到的,就不回調 listener.receivePush(pushEntity),
- 推送消息入庫
- RxBus.getInstance().post(new AcceptSuccessData(pushEntity.id))告訴服務端我收到了這條ID的推送消息推送完成
@Override
public void onSocketReadResponse(Context context, ConnectionInfo info, String action, OriginalData data) {
SL.e("收到數據");
byte[] bodyBytes = data.getBodyBytes();
byte[] headBytes = data.getHeadBytes();
String dataJson = new String(bodyBytes);
PushEntity pushEntity = new GsonBuilder().create().fromJson(dataJson, PushEntity.class);
if (pushEntity.type == PushEntity.PULSE) {
//代表的是心跳數據
IConnectionManager connectionManager = OkSocket.open(info);
if (connectionManager != null) {
// 喂狗心跳置0
SL.e("心跳喂狗完成");
connectionManager.getPulseManager().feed();
}
} else {
// 判斷是否收到過這條推送消息
PushEntity unique = PushApp.getDb().getPushEntityDao()
.queryBuilder()
.where(PushEntityDao.Properties.Id.eq(pushEntity.id))
.unique();
if (unique == null) {
if (listener != null) {
listener.receivePush(pushEntity);
}
}
// 收到消息通知服務的 收推送消息完成
RxBus.getInstance().post(new AcceptSuccessData(pushEntity.id));
// 推送消息入庫
PushApp.getDb().getPushEntityDao().insertOrReplace(pushEntity);
}
}使用技術
- RxBus
- OkScket
- GreenDao
效果圖
正常推送,客戶端連接服務端正常,推送Toast丶通知丶圖片
