火山引擎 RTC 在互娛場景下的最佳實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"火山引擎 RTC 脫胎於字節跳動自研的 RTC 技術中臺。目前，字節跳動旗下約 40+ 業務產品都由此技術中臺提供底層 RTC 服務，其中不乏抖音這樣億級 DAU 的國民應用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了互娛場景之外，火山引擎 RTC 也在在線教育、遊戲語音、企業通信等領域拓展服務場景。目前，火山引擎 RTC 的月用量已經達到了百億分鐘級別，並仍在快速增長。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文將分享火山引擎 RTC 在互娛場景下的最佳實踐，主要包括千人聊天、直播連麥和雲渲染這三個具體場景。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"千人聊天的技術挑戰"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"千人聊天場景來源於 2021 年初非常火爆的語音沙龍場景。千人聊天場景下，一個頻道需要支持多至上千個參與者，頻道內所有有麥的用戶（主播）都可以就頻道話題進行發言。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從技術角度來講，主播數量的每一點增大都會額外帶來巨大的計算量。假設主播數量是 n，且每個主播都開麥，那麼，每個主播都要訂閱除自己以外所有主播的音頻，也就是 (n-1) 路音頻。此時，訂閱的量級就是 n*（n-1）。當 n=1000 時，運算量就達到了 100 萬。並且，頻道內的每個用戶還會進行各種高頻動作，比如進出頻道，或者主播上下麥。每個簡單的操作，在頻道內有極大的人數時，都會觸發大量的運算，很容易造成 RTC 服務端的消息風暴。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"頻道內巨大的人數也會對應用客戶端造成壓力。應用客戶端需要維護 n 份 ICE 連接，對下行帶寬和處理內存有非常高的要求。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/3a\/3a8b20870eba32ba0a47a326a89bdbac.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對這個場景，我們看到市場上已有一些方案。這些方案通常會區分參與者的角色（主播\/觀衆），限定只有主播纔可以發言，並限定主播人數不能超過 50 人，而觀衆角色的參與者只能收聽或收看。構建千人聊天場景的業務方很容易就發現這樣的方案會對業務形態造成限制。同時，業務方在進行場景構建時，還需關注人數上限，添加兜底邏輯。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"火山引擎 RTC 認爲，這樣的方案是不合理的。我們希望這個場景下人人都可以是主播，且人數的上限達到或超過 1000，甚至沒有限制。這樣才能幫助我們的業務方無拘無束的構建理想中的場景。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"千人音頻聊天解決方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"經過反覆的思考和積極的嘗試之後，我們認爲能通過優化對音頻訂閱邏輯的處理，實現我們理想的方案。解決思路可以概括爲服務端選流。思路如圖："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/0f\/0f8cfdddbb06699895c46a7bbf97bf3b.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"混音和選流"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在瞭解這個思路之前，需要先了解一下混音策略。客戶端播放來自遠端的音頻信號之前，需要先把多路音頻混成一路。混音會消耗算力，當進行混音的音頻流數量（n) 特別多，比如達到上百的量級時，客戶端的混音會需要很多時間。因此，大多數 RTC 方案中，在客戶端混音時，添加選流策略，同一時刻只對於音量最大的 n 路音頻流（n 通常爲 3）進行混音，拋棄其他的音頻流。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個合理的懷疑是，選流策略會導致一些有用的音頻流被拋棄。其實絕大部分場景下是不會的。因爲現實中，多人同時說話時信息的傳遞準確率是很低的，如果一個頻道里有大於兩個人同時在說話，其他人就基本聽不清說話內容了。對於搶答、齊讀等特殊場景，也只要把 n 調整爲 10，就基本可以解決問題了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"選流策略是多人音頻場景下一個普遍的策略。但仔細思考就會發現，如果這個策略在應用客戶端實現，就意味着如果頻道內有 n 路音頻，客戶端仍然需要訂閱 n-1 路音頻流，再進行選流和混音。當 n 特別大時，下行帶寬和計算資源的浪費就很明顯。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"服務端選流"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"發現以上的弊端後，我們就考慮是不是把選流策略放在服務端進行；完成選流後，只向客戶端發送有效的音頻流就好了？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設，最終我們從 n 路音頻流中選取 m 路音頻流（3≤m≤10)。那麼選取完成後，服務端只需要向單個客戶端建立 m 個通道，進行音頻流的傳輸。這樣一來，服務端實際處理的消息數量就變成了 m"},{"type":"text","marks":[{"type":"italic"}],"text":"n（而非 n"},{"type":"text","text":"n），計算複雜度就從 O(n^2) 降到了 O(n)。無論是服務端的消息壓力，還是客戶端的內存和帶寬壓力，都能夠大大減小。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實際上，我們也正是這樣去做的，這一方案在千人場景測試中表現非常好。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"千人視頻聊天解決方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"千人場景下的音頻我們解決了，如果還要加上視頻，該怎麼處理？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"和一般人想的不同，視頻的處理其實比音頻的處理要簡單得多。業務方對需要訂閱哪幾路視頻流有很明確的想法。作爲 RTC 技術提供方，我們只需要將音頻和視頻訂閱邏輯分離，幫助業務方根據自己的想法添加視頻訂閱邏輯，由我們來幫助管理音頻訂閱邏輯。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，我們也可以支持根據音頻選流的思路進行視頻選流：顯示頻道內說話聲音最大的用戶的視頻。如果有多路視頻，則按照音量降序，顯示多路視頻。這個思路在很多視頻會議應用中經常使用。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"直播連麥的極致體驗"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"直播連麥是指主播之間或主播和觀衆之間，通過實時音視頻進行互動，並把互動的音視頻合成一路進行直播的場景。常見的直播連麥細分場景包括主播 PK 、主播和主播連麥以及主播和觀衆連麥等。各種玩法非常豐富，這裏就不一一贅述了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這一場景下，使用 RTC 產品傳輸的音視頻流，都需要合流，轉碼成 RTMP 流並推送到 CDN 處。業內主流方案在實現這一邏輯時，這些合流轉碼推送的步驟都是在服務端實現的，也就是大家熟悉的旁路轉推的方案。如下："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/55\/5568702a1845ccbf076b8f099cd3a549.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在字節跳動內部測試中，我們發現這個方案會帶來不可避免的體驗瑕疵："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"黑屏和卡頓：主播的上下麥時，需要從直播流轉成 RTC 流，或從 RTC 流轉成直播流。線路的切換經常會造成黑屏和卡頓。在弱網情況下，這一現象尤其明顯。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"清晰度下降：主播採集的視頻經過一次編碼推到 RTC 服務端；RTC 服務端收到後，對這一視頻流進行解碼，和其他視頻流合碼，進行第二次編碼再推送到 CDN 處。其中的多次編碼解碼會造成明顯的清晰度下降。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"智能合流"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"經過仔細的思考後，我們覺得把合流、轉碼推流這一系列過程放到主播的客戶端，似乎有助於解決上面的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在客戶端合流轉碼時，可以減少一次編解碼，減小清晰度的損失；其次不管是主播的單路 RTMP 流還是合流後 RTC 流轉成的 RTMP 流，都是在主播的客戶端實現推流的，兩者切換過程中，就可以做到沒有黑幀。以下是示意圖，和服務端合流轉推相比，少了轉碼服務器的參與："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/40\/40395b8c0b87d5b6ce7c2cad67eb1ca3.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們當然也意識到了這個方案的侷限，那就是對主播的設備的性能和網絡帶寬要求非常高。把合流、轉碼、推流這些步驟從服務端上轉移到客戶端上，會對客戶端造成非常大的壓力。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最終，我們決定將客戶端合流和服務端合流融爲一體，並根據主播的設備和網絡質量情況，判斷適用的方案。這一判斷邏輯是比較複雜的，我們將真實場景下測試得到的最佳實踐封裝在產品中，供業務方直接使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後的數據結果表明：在客戶端上進行轉碼推流，清晰度提升的收益大概在 20%-40%。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"解鎖更多玩法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在客戶端上進行合流、轉碼還可以解鎖更多玩法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比如，有些場景需要在連麥直播過程中加上 CV 等效果，並希望渲染出來的畫面能夠和 RTC 連麥的畫面高度一致。這時候採用服務端合流的方案造成的延時將遠高於客戶端合流的方案。如果業務方對實時性有要求，就只能使用客戶端合流了。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"雲渲染方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一些複雜的渲染計算，比如高畫質的遊戲或者精美的 3D 模型，會消耗非常多的計算資源。要達到最佳的顯示效果，對終端的圖形計算性能就有很高的要求。隨着 RTC 技術的不斷髮展，我們可以在雲端進行渲染，把渲染得到的音視頻畫面，用 RTC 傳輸到本機。終端用戶只需要滿足網絡開銷和播放音視頻的性能開銷，就能獲得極致的視聽體驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常見的雲渲染應用場景包括：雲遊戲，雲手機、雲電腦等。我們和火山引擎的多媒體中臺、AI 中臺一起共建的雲特效方案亦屬此列。雲特效能夠把主播拍攝的畫面先通過上傳到雲端進行特效渲染，然後再把渲染後的視頻推回到主播的客戶端。主播在客戶端上看到的加了特效的視頻，在非常短的時間內其實經過了上傳、特效渲染和下載的過程。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"雲渲染面臨的挑戰"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然雲渲染的場景也面臨一些挑戰："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"超大的計算：雲渲染會消耗大量的 GPU 計算資源。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"超大碼率：要有更精美的畫質，移動端一般要求 720P 30 幀；PC 端則是 1080P 60 幀。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"超低延時：雲渲染對響應延時的要求一般在 100 毫秒以內，遠低於傳統 RTC 應用的延時要求（ 400 毫秒以內）。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"超高可靠：應用在雲遊戲場景中時，會有超高的可靠性要求。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"總結成一句話，就是既要更大的碼率，又要更低的延時，還要更高的可靠性。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"傳統雲渲染的傳輸方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"渲染質量等挑戰不是本文關注的內容。以下我們討論如何應對雲渲染場景對低傳輸時延的要求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"傳統的雲渲染有兩種傳輸方案："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雲渲染的主機和終端採用 P2P 方案進行傳輸。這一方案時延較低，但可靠性較難保障。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雲渲染的主機和終端採用 RTC 方案進行傳輸。這一方案可靠性較高，但時延一般高於 P2P。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"改進的傳輸方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們認爲 RTC 方案能夠比 P2P 方案實現更高的可靠性，我們只需降低 RTC 方案的延遲即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們發現，RTC 傳輸的時延很大一部分是由於 RTC 和雲渲染服務器之間的傳輸延時造成的。於是我們和火山引擎雲渲染團隊一起對傳輸網絡進行了改造升級：將 RTC 的邊緣節點和雲渲染服務端進行一體化的部署，消除了從 RTC 服務端到雲渲染服務端之間的傳輸延時。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，對於火山引擎以外的雲渲染服務商，RTC 的邊緣節點架構也支持同樣的合作優化。根據需要，我們可以提供兩種優化方式："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把火山引擎 RTC 的邊緣節點部署到雲渲染服務方的機房裏。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把雲渲染服務方的實例部署到火山引擎 RTC 的邊緣節點裏。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"優化結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"改造後的相應延時數據如下："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雲遊戲和雲手機的響應延時在 80 毫秒左右；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雲特效的響應延時在 135 毫秒左右。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏響應延時的定義是："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於雲遊戲和雲手機，從用戶的終端觸控開始，到用戶最終看到畫面的時間。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於雲特效，從用戶拍攝自己的畫面開始，到在終端上看到這個畫面的時間。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文主要介紹了火山引擎 RTC 在千人聊天、直播連麥、雲渲染三個場景下的最佳實踐。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在千人聊天場景下，音頻訂閱採用了服務端選流的策略，視頻訂閱則採用音視頻訂閱分離以及按音量顯示視頻的策略，保證視聽效果的前提下，大大降低了客戶端性能開銷。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在直播連麥場景下，採用智能合流的方式，根據具體情況應用客戶端和服務端合流轉推，減少了黑屏卡頓，並提升了清晰度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在雲渲染場景下，將 RTC 邊緣節點和雲渲染主機進行一體化部署，並自研了雲渲染主機實例底層架構，降低了雲渲染場景下的響應延時。同時，這一架構還支持第三方雲渲染主機接入。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Q&A"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Q：千人聊天這個方案可以用於視頻會議場景嗎？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"A："},{"type":"text","text":"可以。事實上很多 RTC 場景就是從視頻會議場景演化而來的，千人聊天甚至萬人聊天的場景就是其中之一。但是有別於傳統視頻會議的 MCU 架構，我們這個架構還是基於 SFU 的，它只是在服務端做選流，並不做合流（MCU 的方案）。因爲合流會造成一定的成本消耗和延時增加，並且多方音頻的合流方案並不經濟，因爲不能把每個人自己的聲音合進去。不過我們也有 MCU 的融合方案，可以融合到現有方案中，針對部分低端設備提供合成後的音視頻流，這就是另一個話題了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Q：直播連麥時，合流特效是直接在主播端合成到了 RTMP 流裏面嗎？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"A："},{"type":"text","text":"一些美顏特效在編碼傳輸前添加，方便主播在本地預覽；單人直播時，會合到 RTMP 流裏面；連麥直播時，就合到 RTC 流裏面。對於一些需要後期疊加的渲染，比如小遊戲的場景，就不會合成到視頻流裏面，而是在接收端渲染。渲染時，可以利用 SEI 來控制同步。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Q：雲渲染的延時還可以怎麼控制？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"A："},{"type":"text","text":"除了文中提到的，把 RTC 的服務器和雲渲染服務器一體化部署，減少傳輸延時的方案以外，我們還有很多其他策略：比如使用感興趣區域編碼策略（ROI），對感興趣的區域應用較高的編碼質量，不感興趣的區域應用較低的編碼質量。用這樣的方法可以降低整體碼率和編碼時間。另外，對於有很多靜止畫面的情況，提高編碼壓縮率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：字節跳動技術團隊(ID：toutiaotechblog）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/rZ1GKVvSP-V1Kq6SzosH7Q","title":"xxx","type":null},"content":[{"type":"text","text":"火山引擎 RTC 在互娛場景下的最佳實踐"}]}]}]}