從janus中學習webrtc的ice簡單交換過程

1. 簡介：

本文通過web和janus進行實時音視頻通信的Demo，結合rfc-5245來學習ice交換的過程。

2. 測試模型

本文測試模型爲一個NAT內的web的客戶端，向一個NAT內的janus服務器發起請求。

3. rfc 5245

本文只撿基礎的一個流程進行分析，5245協議本身囊括的內容較多。(這裏主要圍繞2, 2.1, 2.2三段)。

ICE的過程主要分爲三個過程：

1) 各個peer首先向各自的stun server請求，獲取各自的對外IP。(非必須)

2) 各個peer收集自己的ice candidate，這裏包括了本機的local地址，及打洞後獲取的對外IP，雙方交換ice candidate。

3) 各個peer收到對應的ice candidate後，排序後向各個candidate發起stun請求，用stun協議進行連通性測試，通過後擇優。

4. 過程分析

本次例子客戶端發往打洞服務器被牆……，從抓包上來看，客戶端打洞請求發出，但是沒有收到對端服務器的響應，而服務端響應正常。

下圖爲客戶端打洞請求，但是並沒有從相應的服務器得到相應的回覆。

下圖爲服務端的打洞請求，服務端收到了打洞服務器的響應，並得到了自己的對外IP和端口。

接着我們繼續查看客戶端發給服務端的請求，這裏還是查找POST請求，我們會發現，這裏發送給服務端的ice candidate中都是NAT內部的IP。

服務端日誌上正是收到如上部分的candidate。

[5261413890922073] Trickle candidate (sdparta_0): candidate:0 1 UDP 2122252543 192.168.0.102 55349 typ host
[5261413890922073] Trickle candidate (sdparta_0): candidate:0 2 UDP 2122252542 192.168.0.102 59517 typ host
[5261413890922073] Trickle candidate (sdparta_1): candidate:0 2 UDP 2122252542 192.168.0.102 50268 typ host
[5261413890922073] Trickle candidate (sdparta_1): candidate:0 1 UDP 2122252543 192.168.0.102 50842 typ host
[5261413890922073] Trickle candidate (sdparta_2): candidate:0 1 UDP 2122252543 192.168.0.102 55799 typ host

服務端收到所有ice後，回覆的sdp內容如下，服務端的candidate包含了本機打洞後的IP。

"v=0\r\no=mozilla...THIS_IS_SDPARTA-50.1.0 1487429580697208 1 IN IP4 47.89.12.119\r\ns=-\r\nt=0 0\r\na=group:BUNDLE sdparta_0 sdparta_1 sdparta_2\r\na=msid-semantic: WMS janus\r\nm=audio 9 UDP/TLS/RTP/SAVPF 109 9 0 8\r\nc=IN IP4
47.89.12.119\r\na=sendrecv\r\na=mid:sdparta_0\r\na=rtcp-mux\r\na=ice-ufrag:Aa6k\r\na=ice-pwd:Iyz7Wyfm+iBlPYo+OTJiMV\r\na=ice-options:trickle\r\na=fingerprint:sha-256
9B:0E:EE:53:74:3F:C1:A3:0F:95:F5:15:84:38:8F:3D:8D:8C:00:DB:21:68:15:5E:6D:CA:F6:21:EE:D9:97:34\r\na=setup:active\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=fmtp:109
maxplaybackrate=48000;stereo=1;useinbandfec=1\r\na=rtpmap:109 opus/48000/2\r\na=rtpmap:9 G722/8000/1\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=ssrc:2925740869 cname:janusaudio\r\na=ssrc:2925740869 msid:janus
janusa0\r\na=ssrc:2925740869 mslabel:janus\r\na=ssrc:2925740869 label:janusa0\r\na=candidate:1 1 udp 2013266431 172.31.40.96 49432 typ host\r\na=candidate:2 1 udp 1677721855 47.89.12.119 49432 typ srflx raddr 172.31.40.96 rport
49432\r\nm=video 9 UDP/TLS/RTP/SAVPF 120 126 97\r\nc=IN IP4 47.89.12.119\r\na=sendrecv\r\na=mid:sdparta_1\r\na=rtcp-mux\r\na=ice-ufrag:Aa6k\r\na=ice-pwd:Iyz7Wyfm+iBlPYo+OTJiMV\r\na=ice-options:trickle\r\na=fingerprint:sha-256
9B:0E:EE:53:74:3F:C1:A3:0F:95:F5:15:84:38:8F:3D:8D:8C:00:DB:21:68:15:5E:6D:CA:F6:21:EE:D9:97:34\r\na=setup:active\r\na=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1\r\na=fmtp:97
profile-level-id=42e01f;level-asymmetry-allowed=1\r\na=fmtp:120 max-fs=12288;max-fr=60\r\na=rtcp-fb:120 nack\r\na=rtcp-fb:120 nack pli\r\na=rtcp-fb:120 ccm fir\r\na=rtcp-fb:120 goog-remb\r\na=rtcp-fb:126 nack\r\na=rtcp-fb:126
nack pli\r\na=rtcp-fb:126 ccm fir\r\na=rtcp-fb:126 goog-remb\r\na=rtcp-fb:97 nack\r\na=rtcp-fb:97 nack pli\r\na=rtcp-fb:97 ccm fir\r\na=rtcp-fb:97 goog-remb\r\na=rtpmap:120 VP8/90000\r\na=rtpmap:126 H264/90000\r\na=rtpmap:97
H264/90000\r\na=ssrc:3393709355 cname:janusvideo\r\na=ssrc:3393709355 msid:janus janusv0\r\na=ssrc:3393709355 mslabel:janus\r\na=ssrc:3393709355 label:janusv0\r\na=candidate:1 1 udp 2013266431 172.31.40.96 49432 typ
host\r\na=candidate:2 1 udp 1677721855 47.89.12.119 49432 typ srflx raddr 172.31.40.96 rport 49432\r\nm=application 9 DTLS/SCTP 5000\r\nc=IN IP4 47.89.12.119\r\na=sendrecv\r\na=sctpmap:5000 webrtc-datachannel
16\r\na=mid:sdparta_2\r\na=ice-ufrag:Aa6k\r\na=ice-pwd:Iyz7Wyfm+iBlPYo+OTJiMV\r\na=ice-options:trickle\r\na=fingerprint:sha-256
9B:0E:EE:53:74:3F:C1:A3:0F:95:F5:15:84:38:8F:3D:8D:8C:00:DB:21:68:15:5E:6D:CA:F6:21:EE:D9:97:34\r\na=setup:active\r\na=candidate:1 1 udp 2013266431 172.31.40.96 49432 typ host\r\na=candidate:2 1 udp 1677721855 47.89.12.119 49432
typ srflx raddr 172.31.40.96 rport 49432\r\n"

接着雙方進行連通性測試，這裏我們只看服務端的抓包信息對應的stun協議包。

從下面抓包信息可以看出，服務端收到的candidate只有一個IP，192.168.0.102，服務端在對192.168.0.102:55349進行連通性測試時，並沒有收到對應的消息回覆，且持續發送(中間已經開始推流了)，這是服務端的行爲。

除了服務端發起的連通性測試外，客戶端也發起了連通性測試，但是此時的連通性測試發起的IP是一個客戶端自己沒有記錄的IP，服務端收到這個請求後，會認爲發現了一個新的candidate，這時候服務端正常回復stun消息後，也再次向該地址:端口發起連通性測試。

5. 部分參數

在ice的連通性檢查時，需要指明這次連通性測試的用戶名和密碼，如上圖服務端的抓包信息，在Binding Request user中指明瞭【對端用戶名:本地用戶名】，這兩個用戶名是在sdp信息中的ice-ufrag字段指明，可以從上文中的sdp中找到相應的用戶名Aa6k，同時在字段ice-pwd中指明連通性測試時的密碼。此外，還有一個ice-options指明瞭一些ice使用的參數，這裏主要指明瞭ice的類型是trickle。

6. ICE確認連接

這一部分需要解讀rfc。

6.1 兩種提名(regular nomination & aggressive nomination)

分爲常規的提名和激進的提名。

常規的提名在協商中controlling一端在發送binding request時候不會攜帶確認標誌，當所有協商完成後對所有的candidate進行評估，最後會再次發送一個帶有標誌位的請求來表示確認。

而激進提名方式controlling在發送binding request的時候就會攜帶對應的標誌位，當該次連通性測試完成時，就選定該連接，這種時候不會發送第二次的binding request。

With regular nomination, the controlling agent lets the checks
continue until at least one valid candidate pair for each media
stream is found.  Then, it picks amongst those that are valid, and
sends a second STUN request on its NOMINATED candidate pair, but this
time with a flag set to tell the peer that this pair has been
nominated for use.  This is shown in Figure 4.

L                        R
-                        -
STUN request ->             \  L's
          <- STUN response  /  check

           <- STUN request  \  R's
STUN response ->            /  check

STUN request + flag ->      \  L's
          <- STUN response  /  check

                    Figure 4: Regular Nomination

Once the STUN transaction with the flag completes, both sides cancel
any future checks for that media stream.  ICE will now send media
using this pair.  The pair an ICE agent is using for media is called
the SELECTED PAIR.

In aggressive nomination, the controlling agent puts the flag in
every STUN request it sends.  This way, once the first check
succeeds, ICE processing is complete for that media stream and the
controlling agent doesn't have to send a second STUN request.  The
selected pair will be the highest-priority valid pair whose check
succeeded.  Aggressive nomination is faster than regular nomination,
but gives less flexibility.  Aggressive nomination is shown in
Figure 5.

L                        R
-                        -
STUN request + flag ->      \  L's
          <- STUN response  /  check

           <- STUN request  \  R's
STUN response ->            /  check

                   Figure 5: Aggressive Nomination

Once all of the media streams are completed, the controlling endpoint
sends an updated offer if the candidates in the m and c lines for the
media stream (called the DEFAULT CANDIDATES) don't match ICE's
SELECTED CANDIDATES.

6.2 標誌位

ICE擴展了以下幾個stun的attribute，其中6.1中的標誌位是USE_CANDIDATE，controlling在選擇candidate的時候會在binding request中攜帶該標誌位，而controlled在binding response和binding request都不會攜帶該標誌位。

This specification defines four new attributes, PRIORITY, USE-
CANDIDATE, ICE-CONTROLLED, and ICE-CONTROLLING.

The PRIORITY attribute indicates the priority that is to be
associated with a peer reflexive candidate, should one be discovered
by this check.  It is a 32-bit unsigned integer, and has an attribute
value of 0x0024.

The USE-CANDIDATE attribute indicates that the candidate pair
resulting from this check should be used for transmission of media.
The attribute has no content (the Length field of the attribute is
zero); it serves as a flag.  It has an attribute value of 0x0025.

The ICE-CONTROLLED attribute is present in a Binding request and
indicates that the client believes it is currently in the controlled
role.  The content of the attribute is a 64-bit unsigned integer in
network byte order, which contains a random number used for tie-
breaking of role conflicts.

The ICE-CONTROLLING attribute is present in a Binding request and
indicates that the client believes it is currently in the controlling
role.  The content of the attribute is a 64-bit unsigned integer in
network byte order, which contains a random number used for tie-
breaking of role conflicts.

6.3 Janus服務器中的controlling判斷

從日誌中可以看出Janus的服務器屬於controlled，因此連接的最終選擇權在客戶端。在日誌中我們可以看到如下log信息，並搜索對應代碼，最終在janus_ice_set_local函數中。

[3814770425474886] Setting ICE locally: got OFFER (1 audios, 1 videos)
[3814770425474886] Creating ICE agent (ICE Full mode, controlled)

我們可以看出此時服務端收到的是客戶端的offer，並找到對應分支，controlling的判斷首先基於ice的設置，看看服務器是處於lite模式或者是full模式，lite模式時服務端總是controlled，而在full模式時，則根據收到offer或者是發起offer來判斷，先發起offer的是controlling。

handle->controlling = janus_ice_lite_enabled ? FALSE : !offer;
JANUS_LOG(LOG_INFO, "[%"SCNu64"] Creating ICE agent (ICE %s mode, %s)\n", handle->handle_id,
janus_ice_lite_enabled ? "Lite" : "Full", handle->controlling ? "controlling" : "controlled");

結合rfc的2.7節：總是處於公網IP下的服務端可以使用lite實現，這種工作模式下的服務端不用收集ice candidate信息，也不用主動發起連通性測試，只需要進行被動響應即可。

In order for ICE to be used in a call, both agents need to support
it.  However, certain agents will always be connected to the public
Internet and have a public IP address at which it can receive packets
from any correspondent.  To make it easier for these devices to
support ICE, ICE defines a special type of implementation called LITE
(in contrast to the normal FULL implementation).  A lite
implementation doesn't gather candidates; it includes only host
candidates for any media stream.  Lite agents do not generate
connectivity checks or run the state machines, though they need to be
able to respond to connectivity checks.  When a lite implementation
connects with a full implementation, the full agent takes the role of
the controlling agent, and the lite agent takes on the controlled
role.  When two lite implementations connect, no checks are sent.

For guidance on when a lite implementation is appropriate, see the
discussion in Appendix A.

It is important to note that the lite implementation was added to
this specification to provide a stepping stone to full
implementation.  Even for devices that are always connected to the
public Internet, a full implementation is preferable if achievable.

從janus中學習webrtc的ice簡單交換過程

用ffmpeg將mp3轉爲wav

centos搭建janus服務器

ice交互的深度好文

webrtc服務器janus android客戶端編譯學習三

聊一聊分佈式鎖的設計（轉）

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結