{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"7月29日-7月30日，由青雲科技舉辦的 CIC2021 雲計算峯會在北京成功舉辦，拍樂雲服務端專家沈偉鋒受邀出席峯會，並在音視頻技術論壇上以《大規模實時音視頻技術架構的實踐和演進》爲演講主題，分享了實時音視頻通訊的幾種常見架構和網絡拓撲，構建實時音視頻實際場景的複雜性和多樣性，以及拍樂雲在超大規模實時音視頻系統的一些實踐。全球疫情持續反覆，線上互動依然是後疫情時代人們工作、生活、娛樂的常態，實時音視頻的需求還在增加。經過不斷的演進，拍樂雲可以支持","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"單房間萬人在線、千人視頻連麥、萬人音頻連麥","attrs":{}},{"type":"text","text":"，並做到","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"99.95%","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"的高可用","attrs":{}},{"type":"text","text":"，服務於全球用戶。本篇演講實錄將由淺入深，一步一步帶大家瞭解時音視頻通訊系統背後的技術細節。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"實時音視頻常見架構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"直連","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"直連，也叫對等網絡(P2P)。這種結構中，每個客戶端在啓動的時候，都需要註冊到註冊服務器，便於其他人能找到自己，正常情況下，在建立音視頻通訊的時候不需要媒體服務器的介入，每個客戶端相互連接，直接進行音視頻通訊。但當一個或多個客戶端在 NAT 之後（甚至多層NAT之後）時，直連會變得比較困難，有時候甚至無法聯通。這個時候需要通過 STUN 進行“打洞”來穿越 NAT 進行通訊，如果“打洞”失敗，還需要引入服務器中繼才能通訊。具體的NAT打洞，或服務中繼的細節可以參考拍樂雲的系列文章","attrs":{}},{"type":"link","attrs":{"href":"https://link.juejin.cn/?target=http%3A%2F%2Fmp.weixin.qq.com%2Fs%3F__biz%3DMzU1MjAxNjI0Ng%3D%3D%26mid%3D2247484850%26idx%3D1%26sn%3D0c4d10c987647803266ceaa0e4a4158a%26chksm%3Dfb89c5abccfe4cbdb3af79851e29fea440bb636bdb954dc9217943de959a69538acccf563235%26scene%3D21%23wechat_redirect","title":"http://mp.weixin.qq.com/s?__biz=MzU1MjAxNjI0Ng==&mid=2247484850&idx=1&sn=0c4d10c987647803266ceaa0e4a4158a&chksm=fb89c5abccfe4cbdb3af79851e29fea440bb636bdb954dc9217943de959a69538acccf563235&scene=21#wechat_redirect","type":null},"content":[{"type":"text","text":"《穿越防火牆的奧祕：ICE協議詳解》","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a1/a137e44b9cd7c009c72f90e214f24216.png","alt":"222221.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"MCU","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MCU(Multipoint Conferencing Unit)方案出現得比較早，相應的技術也非常成熟，該方案由一個服務器和多個客戶端組成一個星形結構，各個端都將音視頻數據發送給服務端，服務端會把所有客戶端的音視頻數據經過解碼，同步，重採樣，佈局，混流，編碼等，最後把媒體數據推送給所有的客戶端。實際上服務器端就是一個音視頻混合器，這種方案服務器的計算壓力會非常大。一般情況下，在音頻數據混流之前，服務端會把目標用戶自己的音頻數據移除，避免客戶端聽到自己的回聲。在視頻數據混流前，服務端可能檢測每個目標用戶是否有自定義的佈局，否則就按系統默認的佈局混流編碼。在一些網絡比較複雜的環境下，MCU 也可以按目標用戶的帶寬對視頻數據的編碼碼率做一些自適應的調整。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/12/12bfc0d8652a354dadc84c3e39713a96.png","alt":"222222.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"SFU","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SFU(Selective Forwarding Unit)是最近幾年流行的新架構，SFU 的方案跟 MCU 類似，每個客戶端都把音視頻數據發給服務端，然後由服務端轉發給不同的客戶端。跟 MCU 不同的地方，SFU 不對音視頻進行混流，收到某個客戶端的音視頻數據後，按需(目標客戶端是否訂閱)將音視頻數據原封不動的轉發給目標客戶端。它實際上就是一個音視頻路由轉發器。在這種方案裏，所有的混流都是在客戶端做的，對服務端的計算要求大大降低。在一些複雜的網絡環境，視頻的數據源端會使用 Simulcast 或 SVC 發送多層不同分辨率的視頻流數據，服務端根據目標客戶端的不同網絡帶寬和網絡狀況轉發最合適的分辨率給目標客戶端，使每個客戶端的體驗達到最佳。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d8/d874fcca335b3f42d7dc0d89c8a62fcc.png","alt":"2222223.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"對比和總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e5/e5ee15e1783b417e8a9c95fff91f2d33.png","alt":"QQ圖片20210803115314.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從上面的對比，我們可以看到，直連方案基本不大適合大會場景，而且無法對網絡內容進行審覈，直連方案目前市場上基本只有在免費場景中看到。而隨着計算成本和帶寬成本的大幅度下降，及超大併發的需求，SFU方案的優勢變得非常明顯，而MCU方案在一些企業內基於音視頻終端的通訊等傳統應用場景目前還是比較常見的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面的音視頻通訊架構，是最基礎的音視頻服務架構，還沒有辦法做到高可用和高併發，假設服務器宕機，服務就變的不可用了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"構建實時通信的網絡拓撲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"環狀網絡結構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"環型結構由網絡中若干節點通過點到點的鏈路首尾相連形成一個閉合的環，這種結構通過公共傳輸鏈路組成環型連接，數據在環路中沿着一個方向在各個節點間傳輸，信息從一個節點傳到另一個節點。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這種網絡實現，構建和路由選路都很簡單，沒有中心依賴。但環狀網絡結構增加刪除節點比較困難，環中任何一個節點失效，環路傳輸就會中斷，導致整個網絡癱瘓。環狀網絡結構開始主要用於令牌網，目前已經很少被採用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/66/6605202fd3d7cf97fbe7ec9af4adf183.png","alt":"222224.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"星型/樹型網絡結構","attrs":{}},{"type":"text","text":"星型拓撲結構是一箇中心，多個分節點。它結構簡單，連接方便，管理和維護都相對容易，而且擴展性強，網絡延遲小。中心節點是瓶頸，一旦失效，整個網絡就癱瘓，單葉子結點互相獨立，互不影響。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樹形拓撲結構形狀像一棵倒置的樹,頂端是樹根，樹根向下分支，每個分支還可再向下分支，樹根接收各站點發送的數據，然後再廣播發送到全網。好擴展，容易診斷錯誤，根節點是瓶頸。在音視頻服務架構中，當需要大規模擴展併發用戶的時候，一般都會採取部署多個邊緣計算控制節點，並通過樹型方式連接到中心DC。通過這樣方式接入的用戶，延遲會略有增加。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/17/17136b39831c81c65c76a6562cb49ba4.png","alt":"222225.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"網狀網絡結構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在網狀拓撲結構中，網絡的每臺設備之間均有點到點的鏈路連接，網狀拓撲結構是應用最廣泛的。它的優點是沒有中心節點，可靠性高，容錯能力強，延遲低，但結構複雜，因有多條傳輸路徑，選路和流量控制比較複雜，消息的時序一致性無法保證。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在音視頻服務架構中，我們的設計目標是：極低的延遲，傳輸高效，吞吐量大，但對不同用戶來的媒體數據的時序性一致性並沒有要求。因此網狀結構一般是音視頻服務架構中主DC服務控制節點間組網的首選拓撲結構。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9f/9f895baacddd9193ff6f978b403cb352.png","alt":"222226.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"實際場景的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"網絡接入的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"移動網絡：","attrs":{}},{"type":"text","text":" 3G/4G/5G的接入帶寬各不相同(3G:<2Mbps；4G:10~ 100Mbps；5G:~10Gbps)，並且信號強弱，接入基站隨移動變化。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"有線寬帶","attrs":{}},{"type":"text","text":"：LAN接入，共享出口帶寬，容易出現用戶間帶寬競爭；ADSL接入，上下行帶寬不對稱，上行帶寬低；PON & FTTH 直接光纖接入 ，帶寬穩定。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"無線接入","attrs":{}},{"type":"text","text":"：主流路由器 < 150Mbps (2.4G頻段最大300Mbps，5G頻段最大867Mbps)。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"傳輸路徑上設備的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"傳輸路徑上路由器、交換機吞吐量各不相同，出現性能瓶頸時，丟包策略和資源預留策略會有差異。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"終端設備的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們需要面對的設備有桌面設備、移動設備、穿戴設備、物聯網設備等，這些設備上的關鍵模塊的質量，性能參差不齊。這些關鍵模塊包括：網絡模塊、媒體採集模塊(Camera/Mic)、計算模塊(CPU)、渲染模塊(GPU)等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務端接入的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BGP(Border Gateway Protocol)機房實現單IP多線接入，具備智能路由選擇，線路備份，故障後自動切換到可用線路等。多運營商專線接入，需要從應用層處理選路，故障時線路切換等。單運營商專線接入，無法解決不同運營商之間的互聯互通的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正是由於實際場景的複雜性，多樣性導致了：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"網絡的動態變化","attrs":{}},{"type":"text","text":"：帶寬，丟包，抖動，延遲等；","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"採集數據的動態變化","attrs":{}},{"type":"text","text":"：噪音(噪點)，畸變，輸出數據抖動等。\\","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"架構演進和拍樂雲實踐","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務的高併發，高可用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要做到服務高併發，高可用，主要涉及到下面幾項技術：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"高併發服務集羣","attrs":{}},{"type":"text","text":"我們在音視頻服務基礎架構中講到，單機服務併發量受限於服務器的計算資源，當需要非常高併發的時候，我們必須擴展服務端計算資源，組成服務集羣，並通過前面的負載均衡使客戶端來服務請求被均衡的分配到集羣中的每個計算資源。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5a/5a0e3e315140f5724da835820beb384f.png","alt":"222227.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務故障的自動恢復和降級","attrs":{}},{"type":"text","text":"我們知道任何代碼都或多或少會存在邏輯缺陷，即使是世界上最厲害的大牛也無法避免，因此，我們需要一個機制來保障，當故障發生時，系統能自動捕獲錯誤，並恢復服務的可用性。當一些物理上的瓶頸出現的時候，比如服務計算資源，網絡帶寬出現瓶頸的時候，繼續按正常方式提供服務，可能會產生雪崩效應，導致整個服務不可用，在這個時候，我們需要通過服務降級的方式來保障最主要的功能是可用的。比如關閉視頻並不會對溝通造成嚴重的影響，但關閉音頻可能使溝通無法繼續，那就保留音頻，關閉視頻，來保證整個服務的可用性。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務資源彈性伸縮","attrs":{}},{"type":"text","text":"在虛擬化，SDN等技術的加持下，使得計算資源，網絡資源的動態分配成爲可能，因此在服務資源成爲瓶頸時，動態伸縮服務資源在技術上是可行的。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"異地容災備份","attrs":{}},{"type":"text","text":"在一些極端情況下，比如我們部署服務集羣的機房發生火災，如何來保證我們的服務可用？這個時候，我們會在不同地理位置部署多個服務集羣，在正常情況下，不同地理位置的服務集羣都會同時提供服務，當某個集羣發生不可用的時候，我們的服務監控會檢測到相應的事件，並及時調整路由，把新的服務請求調度到可用的其他服務集羣。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/4f/4f5da1e295ca948301ff52f7e8d32651.png","alt":"222228.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在以上幾個主要技術的加持下，拍樂雲目前已經可以做到99.95%的高可用，並服務於全球用戶。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務的高質量","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"面對複雜多變的網絡環境，如何保證提供高質量的音視頻服務是音視頻服務系統非常重要的任務，在這方面，拍樂雲主要使用了這些技術來保證服務的高質量：帶寬評估、擁塞控制和平滑發送、丟包重傳和前向冗餘糾錯、錯誤隱藏和恢復、基於時域和空域的多層分發(SVC & Simulcast AVC)、基於圖像和語音的去噪和增強、迴音消除、音量自適應調整、網絡資源預留。通過這些技術的應用，拍樂雲即使在70%的丟包率下，依然能提供非常高質量的音視頻服務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"超大規模，超高併發","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"SFU","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"架構中的有選擇的數據轉發","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大家知道，在SFU方案中，音視頻數據是全量轉發的，也就是說在10個人的會議中，服務端要把每個人的數據轉發給其他9個用戶(10","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"9),在用戶數小的時候，問題不會太嚴重，但隨着用戶量的增加，問題會變的越來越嚴重。假設會中有100個人，每個人的視頻數據是1Mbps，實際服務端需要轉發的是100","attrs":{}},{"type":"text","text":"99 = 9.9Gbps，在現實情況下，這幾乎是不可能的。在實際情況中，受屏幕大小的限制，每個人不可能同時去看另外99個用戶的視頻，最常見的情況是1大+6小，或2","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"2、3","attrs":{}},{"type":"text","text":"3、4","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"4、5","attrs":{}},{"type":"text","text":"5等幾種模式，這樣，通過按需轉發的方式，數據量可以大幅度的減少。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"邊緣計算與加速節點","attrs":{}},{"type":"text","text":"在上面實時通訊系統的網絡拓撲結構中我們講到，通過部署多個邊緣計算節點，按樹型結構鏈接到主DC，可以大幅度擴大會議的規模。邊緣計算節點也可以通過就近接入來解決最後一公里的接入問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在單向直播的場景中，我們也可以通過第三方CDN網絡來擴展會議規模，但這種方案的延遲會比較大，會達到3～10秒的延遲，基本無法互動溝通，只能單向直播，當需要互動溝通的時候，必須切換接入方式到邊緣計算節點或中心DC。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"拍樂雲音視頻系統技術架構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/0c/0c2682fa3d907bd39c0842866121d609.png","alt":"222229.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"架構圖的左邊主要是服務的註冊、認證、配置、發現、調度。右邊主要是大數據分析平臺，服務健康狀況監控報警，服務資源彈性伸縮。中間是拍樂雲的產品服務：語音通話、視頻通話、互動白板、互動直播，雲端錄製等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#05","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"業界動向與最新技術","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近年來，音視頻通訊領域的發展非常快，出現了各種前沿新興的技術，有的已經落地，有的還在深入的研究之中，很多技術的應用前景都非常看好。在這裏我們舉幾個例子：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"WebRTC","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2010年5月，Google收購Global IP Solutions的GIPS引擎，將其開源並改名爲WebRTC。2014年7月，WebRTC成爲W3C標準，併發布瀏覽器標準API1.0。自此以後，實時音視頻通訊服務的門檻大幅度降低，很多基於WebRTC的實時音視頻服務如雨後春筍般蓬勃發展起來。可以說WebRTC的出現改變了音視頻通訊領域的市場格局。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"WebRTC目前還在不停的演進之中，有興趣的可以從下面的鏈接獲取最新的信息：","attrs":{}},{"type":"link","attrs":{"href":"https://link.juejin.cn/?target=https%3A%2F%2Fwebrtc.org%2Fhttps%3A%2F%2Fgroups.google.com%2Fforum%2F%23!forum%2Fdiscuss-webrtchttps%3A%2F%2Ftwitter.com%2Fwebrtc","title":"https://webrtc.org/https://groups.google.com/forum/#!forum/discuss-webrtchttps://twitter.com/webrtc","type":null},"content":[{"type":"text","text":"webrtc.org/https://gro…","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://link.juejin.cn/?target=https%3A%2F%2Fwebrtcweekly.com%2F","title":"https://webrtcweekly.com/","type":null},"content":[{"type":"text","text":"webrtcweekly.com/","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"SDN","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Software Defined Network即軟件定義網絡，最初是由美國斯坦福大學CLean State研究組提出的一種新型網絡創新架構，可通過軟件編程的形式定義和控制網絡，其控制平面和轉發平面分離及開放性可編程的特點，被認爲是網絡領域的一場革命，爲新型互聯網體系結構研究提供了新的實驗途徑，也極大地推動了下一代互聯網的發展。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其核心概念是：控制與轉發分離,管理與控制分離。其可編程和虛擬化特點可以幫助快速定義網絡，並實現自動化部署和運維。控制和管理的集中，使得網絡路徑最優化變得更加容易實現。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"基於機器學習的新算法","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"機器學習在很多領域都得到了廣泛的應用，在實時音視頻通訊領域上也出現了很多應用方向，比如：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網絡傳輸相關：智能擁塞算法，智能帶寬評估算法，智能路由等；視頻圖像相關：虛擬背景，超分辯率，視頻融合，deepfake等；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"語音相關的：語音識別，語音增強等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"虛擬現實、增強現實和","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"3D","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"技術","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"虛擬現實(Virtual Reality)，就是虛擬和現實相互結合，是一種可以創建和體驗虛擬世界的計算機仿真系統，它利用計算機生成一種模擬環境，使用戶沉浸到該環境中。增強現實(Augmented Reality)是一種將虛擬信息與真實世界巧妙融合的技術，真實環境和虛擬物體之間重疊之後，能夠在同一個畫面以及空間中同時存在。通過VR，AR，3D技術的結合，相信不久的將來，實時音視頻通訊可以實現類似於現實世界中會議室一起開會的效果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：拍樂雲Pano","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"鏈接：https://juejin.cn/post/6992125317858983966/","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
萬人連麥的幕後技術詳解
{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"7月29日-7月30日，由青雲科技舉辦的 CIC2021 雲計算峯會在北京成功舉辦，拍樂雲服務端專家沈偉鋒受邀出席峯會，並在音視頻技術論壇上以《大規模實時音視頻技術架構的實踐和演進》爲演講主題，分享了實時音視頻通訊的幾種常見架構和網絡拓撲，構建實時音視頻實際場景的複雜性和多樣性，以及拍樂雲在超大規模實時音視頻系統的一些實踐。全球疫情持續反覆，線上互動依然是後疫情時代人們工作、生活、娛樂的常態，實時音視頻的需求還在增加。經過不斷的演進，拍樂雲可以支持","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"單房間萬人在線、千人視頻連麥、萬人音頻連麥","attrs":{}},{"type":"text","text":"，並做到","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"99.95%","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"的高可用","attrs":{}},{"type":"text","text":"，服務於全球用戶。本篇演講實錄將由淺入深，一步一步帶大家瞭解時音視頻通訊系統背後的技術細節。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"實時音視頻常見架構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"直連","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"直連，也叫對等網絡(P2P)。這種結構中，每個客戶端在啓動的時候，都需要註冊到註冊服務器，便於其他人能找到自己，正常情況下，在建立音視頻通訊的時候不需要媒體服務器的介入，每個客戶端相互連接，直接進行音視頻通訊。但當一個或多個客戶端在 NAT 之後（甚至多層NAT之後）時，直連會變得比較困難，有時候甚至無法聯通。這個時候需要通過 STUN 進行“打洞”來穿越 NAT 進行通訊，如果“打洞”失敗，還需要引入服務器中繼才能通訊。具體的NAT打洞，或服務中繼的細節可以參考拍樂雲的系列文章","attrs":{}},{"type":"link","attrs":{"href":"https://link.juejin.cn/?target=http%3A%2F%2Fmp.weixin.qq.com%2Fs%3F__biz%3DMzU1MjAxNjI0Ng%3D%3D%26mid%3D2247484850%26idx%3D1%26sn%3D0c4d10c987647803266ceaa0e4a4158a%26chksm%3Dfb89c5abccfe4cbdb3af79851e29fea440bb636bdb954dc9217943de959a69538acccf563235%26scene%3D21%23wechat_redirect","title":"http://mp.weixin.qq.com/s?__biz=MzU1MjAxNjI0Ng==&mid=2247484850&idx=1&sn=0c4d10c987647803266ceaa0e4a4158a&chksm=fb89c5abccfe4cbdb3af79851e29fea440bb636bdb954dc9217943de959a69538acccf563235&scene=21#wechat_redirect","type":null},"content":[{"type":"text","text":"《穿越防火牆的奧祕：ICE協議詳解》","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a1/a137e44b9cd7c009c72f90e214f24216.png","alt":"222221.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"MCU","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MCU(Multipoint Conferencing Unit)方案出現得比較早，相應的技術也非常成熟，該方案由一個服務器和多個客戶端組成一個星形結構，各個端都將音視頻數據發送給服務端，服務端會把所有客戶端的音視頻數據經過解碼，同步，重採樣，佈局，混流，編碼等，最後把媒體數據推送給所有的客戶端。實際上服務器端就是一個音視頻混合器，這種方案服務器的計算壓力會非常大。一般情況下，在音頻數據混流之前，服務端會把目標用戶自己的音頻數據移除，避免客戶端聽到自己的回聲。在視頻數據混流前，服務端可能檢測每個目標用戶是否有自定義的佈局，否則就按系統默認的佈局混流編碼。在一些網絡比較複雜的環境下，MCU 也可以按目標用戶的帶寬對視頻數據的編碼碼率做一些自適應的調整。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/12/12bfc0d8652a354dadc84c3e39713a96.png","alt":"222222.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"SFU","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SFU(Selective Forwarding Unit)是最近幾年流行的新架構，SFU 的方案跟 MCU 類似，每個客戶端都把音視頻數據發給服務端，然後由服務端轉發給不同的客戶端。跟 MCU 不同的地方，SFU 不對音視頻進行混流，收到某個客戶端的音視頻數據後，按需(目標客戶端是否訂閱)將音視頻數據原封不動的轉發給目標客戶端。它實際上就是一個音視頻路由轉發器。在這種方案裏，所有的混流都是在客戶端做的，對服務端的計算要求大大降低。在一些複雜的網絡環境，視頻的數據源端會使用 Simulcast 或 SVC 發送多層不同分辨率的視頻流數據，服務端根據目標客戶端的不同網絡帶寬和網絡狀況轉發最合適的分辨率給目標客戶端，使每個客戶端的體驗達到最佳。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d8/d874fcca335b3f42d7dc0d89c8a62fcc.png","alt":"2222223.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"對比和總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e5/e5ee15e1783b417e8a9c95fff91f2d33.png","alt":"QQ圖片20210803115314.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從上面的對比，我們可以看到，直連方案基本不大適合大會場景，而且無法對網絡內容進行審覈，直連方案目前市場上基本只有在免費場景中看到。而隨着計算成本和帶寬成本的大幅度下降，及超大併發的需求，SFU方案的優勢變得非常明顯，而MCU方案在一些企業內基於音視頻終端的通訊等傳統應用場景目前還是比較常見的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面的音視頻通訊架構，是最基礎的音視頻服務架構，還沒有辦法做到高可用和高併發，假設服務器宕機，服務就變的不可用了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"構建實時通信的網絡拓撲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"環狀網絡結構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"環型結構由網絡中若干節點通過點到點的鏈路首尾相連形成一個閉合的環，這種結構通過公共傳輸鏈路組成環型連接，數據在環路中沿着一個方向在各個節點間傳輸，信息從一個節點傳到另一個節點。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這種網絡實現，構建和路由選路都很簡單，沒有中心依賴。但環狀網絡結構增加刪除節點比較困難，環中任何一個節點失效，環路傳輸就會中斷，導致整個網絡癱瘓。環狀網絡結構開始主要用於令牌網，目前已經很少被採用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/66/6605202fd3d7cf97fbe7ec9af4adf183.png","alt":"222224.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"星型/樹型網絡結構","attrs":{}},{"type":"text","text":"星型拓撲結構是一箇中心，多個分節點。它結構簡單，連接方便，管理和維護都相對容易，而且擴展性強，網絡延遲小。中心節點是瓶頸，一旦失效，整個網絡就癱瘓，單葉子結點互相獨立，互不影響。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樹形拓撲結構形狀像一棵倒置的樹,頂端是樹根，樹根向下分支，每個分支還可再向下分支，樹根接收各站點發送的數據，然後再廣播發送到全網。好擴展，容易診斷錯誤，根節點是瓶頸。在音視頻服務架構中，當需要大規模擴展併發用戶的時候，一般都會採取部署多個邊緣計算控制節點，並通過樹型方式連接到中心DC。通過這樣方式接入的用戶，延遲會略有增加。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/17/17136b39831c81c65c76a6562cb49ba4.png","alt":"222225.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"網狀網絡結構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在網狀拓撲結構中，網絡的每臺設備之間均有點到點的鏈路連接，網狀拓撲結構是應用最廣泛的。它的優點是沒有中心節點，可靠性高，容錯能力強，延遲低，但結構複雜，因有多條傳輸路徑，選路和流量控制比較複雜，消息的時序一致性無法保證。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在音視頻服務架構中，我們的設計目標是：極低的延遲，傳輸高效，吞吐量大，但對不同用戶來的媒體數據的時序性一致性並沒有要求。因此網狀結構一般是音視頻服務架構中主DC服務控制節點間組網的首選拓撲結構。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9f/9f895baacddd9193ff6f978b403cb352.png","alt":"222226.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"實際場景的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"網絡接入的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"移動網絡：","attrs":{}},{"type":"text","text":" 3G/4G/5G的接入帶寬各不相同(3G:<2Mbps；4G:10~ 100Mbps；5G:~10Gbps)，並且信號強弱，接入基站隨移動變化。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"有線寬帶","attrs":{}},{"type":"text","text":"：LAN接入，共享出口帶寬，容易出現用戶間帶寬競爭；ADSL接入，上下行帶寬不對稱，上行帶寬低；PON & FTTH 直接光纖接入 ，帶寬穩定。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"無線接入","attrs":{}},{"type":"text","text":"：主流路由器 < 150Mbps (2.4G頻段最大300Mbps，5G頻段最大867Mbps)。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"傳輸路徑上設備的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"傳輸路徑上路由器、交換機吞吐量各不相同，出現性能瓶頸時，丟包策略和資源預留策略會有差異。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"終端設備的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們需要面對的設備有桌面設備、移動設備、穿戴設備、物聯網設備等，這些設備上的關鍵模塊的質量，性能參差不齊。這些關鍵模塊包括：網絡模塊、媒體採集模塊(Camera/Mic)、計算模塊(CPU)、渲染模塊(GPU)等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務端接入的多樣性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BGP(Border Gateway Protocol)機房實現單IP多線接入，具備智能路由選擇，線路備份，故障後自動切換到可用線路等。多運營商專線接入，需要從應用層處理選路，故障時線路切換等。單運營商專線接入，無法解決不同運營商之間的互聯互通的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正是由於實際場景的複雜性，多樣性導致了：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"網絡的動態變化","attrs":{}},{"type":"text","text":"：帶寬，丟包，抖動，延遲等；","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"採集數據的動態變化","attrs":{}},{"type":"text","text":"：噪音(噪點)，畸變，輸出數據抖動等。\\","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"架構演進和拍樂雲實踐","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務的高併發，高可用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要做到服務高併發，高可用，主要涉及到下面幾項技術：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"高併發服務集羣","attrs":{}},{"type":"text","text":"我們在音視頻服務基礎架構中講到，單機服務併發量受限於服務器的計算資源，當需要非常高併發的時候，我們必須擴展服務端計算資源，組成服務集羣，並通過前面的負載均衡使客戶端來服務請求被均衡的分配到集羣中的每個計算資源。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5a/5a0e3e315140f5724da835820beb384f.png","alt":"222227.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務故障的自動恢復和降級","attrs":{}},{"type":"text","text":"我們知道任何代碼都或多或少會存在邏輯缺陷，即使是世界上最厲害的大牛也無法避免，因此，我們需要一個機制來保障，當故障發生時，系統能自動捕獲錯誤，並恢復服務的可用性。當一些物理上的瓶頸出現的時候，比如服務計算資源，網絡帶寬出現瓶頸的時候，繼續按正常方式提供服務，可能會產生雪崩效應，導致整個服務不可用，在這個時候，我們需要通過服務降級的方式來保障最主要的功能是可用的。比如關閉視頻並不會對溝通造成嚴重的影響，但關閉音頻可能使溝通無法繼續，那就保留音頻，關閉視頻，來保證整個服務的可用性。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務資源彈性伸縮","attrs":{}},{"type":"text","text":"在虛擬化，SDN等技術的加持下，使得計算資源，網絡資源的動態分配成爲可能，因此在服務資源成爲瓶頸時，動態伸縮服務資源在技術上是可行的。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"異地容災備份","attrs":{}},{"type":"text","text":"在一些極端情況下，比如我們部署服務集羣的機房發生火災，如何來保證我們的服務可用？這個時候，我們會在不同地理位置部署多個服務集羣，在正常情況下，不同地理位置的服務集羣都會同時提供服務，當某個集羣發生不可用的時候，我們的服務監控會檢測到相應的事件，並及時調整路由，把新的服務請求調度到可用的其他服務集羣。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/4f/4f5da1e295ca948301ff52f7e8d32651.png","alt":"222228.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在以上幾個主要技術的加持下，拍樂雲目前已經可以做到99.95%的高可用，並服務於全球用戶。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務的高質量","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"面對複雜多變的網絡環境，如何保證提供高質量的音視頻服務是音視頻服務系統非常重要的任務，在這方面，拍樂雲主要使用了這些技術來保證服務的高質量：帶寬評估、擁塞控制和平滑發送、丟包重傳和前向冗餘糾錯、錯誤隱藏和恢復、基於時域和空域的多層分發(SVC & Simulcast AVC)、基於圖像和語音的去噪和增強、迴音消除、音量自適應調整、網絡資源預留。通過這些技術的應用，拍樂雲即使在70%的丟包率下，依然能提供非常高質量的音視頻服務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"超大規模，超高併發","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"SFU","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"架構中的有選擇的數據轉發","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大家知道，在SFU方案中，音視頻數據是全量轉發的，也就是說在10個人的會議中，服務端要把每個人的數據轉發給其他9個用戶(10","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"9),在用戶數小的時候，問題不會太嚴重，但隨着用戶量的增加，問題會變的越來越嚴重。假設會中有100個人，每個人的視頻數據是1Mbps，實際服務端需要轉發的是100","attrs":{}},{"type":"text","text":"99 = 9.9Gbps，在現實情況下，這幾乎是不可能的。在實際情況中，受屏幕大小的限制，每個人不可能同時去看另外99個用戶的視頻，最常見的情況是1大+6小，或2","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"2、3","attrs":{}},{"type":"text","text":"3、4","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"4、5","attrs":{}},{"type":"text","text":"5等幾種模式，這樣，通過按需轉發的方式，數據量可以大幅度的減少。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"邊緣計算與加速節點","attrs":{}},{"type":"text","text":"在上面實時通訊系統的網絡拓撲結構中我們講到，通過部署多個邊緣計算節點，按樹型結構鏈接到主DC，可以大幅度擴大會議的規模。邊緣計算節點也可以通過就近接入來解決最後一公里的接入問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在單向直播的場景中，我們也可以通過第三方CDN網絡來擴展會議規模，但這種方案的延遲會比較大，會達到3～10秒的延遲，基本無法互動溝通，只能單向直播，當需要互動溝通的時候，必須切換接入方式到邊緣計算節點或中心DC。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"拍樂雲音視頻系統技術架構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/0c/0c2682fa3d907bd39c0842866121d609.png","alt":"222229.png","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"架構圖的左邊主要是服務的註冊、認證、配置、發現、調度。右邊主要是大數據分析平臺，服務健康狀況監控報警，服務資源彈性伸縮。中間是拍樂雲的產品服務：語音通話、視頻通話、互動白板、互動直播，雲端錄製等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"#05","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"業界動向與最新技術","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近年來，音視頻通訊領域的發展非常快，出現了各種前沿新興的技術，有的已經落地，有的還在深入的研究之中，很多技術的應用前景都非常看好。在這裏我們舉幾個例子：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"WebRTC","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2010年5月，Google收購Global IP Solutions的GIPS引擎，將其開源並改名爲WebRTC。2014年7月，WebRTC成爲W3C標準，併發布瀏覽器標準API1.0。自此以後，實時音視頻通訊服務的門檻大幅度降低，很多基於WebRTC的實時音視頻服務如雨後春筍般蓬勃發展起來。可以說WebRTC的出現改變了音視頻通訊領域的市場格局。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"WebRTC目前還在不停的演進之中，有興趣的可以從下面的鏈接獲取最新的信息：","attrs":{}},{"type":"link","attrs":{"href":"https://link.juejin.cn/?target=https%3A%2F%2Fwebrtc.org%2Fhttps%3A%2F%2Fgroups.google.com%2Fforum%2F%23!forum%2Fdiscuss-webrtchttps%3A%2F%2Ftwitter.com%2Fwebrtc","title":"https://webrtc.org/https://groups.google.com/forum/#!forum/discuss-webrtchttps://twitter.com/webrtc","type":null},"content":[{"type":"text","text":"webrtc.org/https://gro…","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://link.juejin.cn/?target=https%3A%2F%2Fwebrtcweekly.com%2F","title":"https://webrtcweekly.com/","type":null},"content":[{"type":"text","text":"webrtcweekly.com/","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"SDN","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Software Defined Network即軟件定義網絡，最初是由美國斯坦福大學CLean State研究組提出的一種新型網絡創新架構，可通過軟件編程的形式定義和控制網絡，其控制平面和轉發平面分離及開放性可編程的特點，被認爲是網絡領域的一場革命，爲新型互聯網體系結構研究提供了新的實驗途徑，也極大地推動了下一代互聯網的發展。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其核心概念是：控制與轉發分離,管理與控制分離。其可編程和虛擬化特點可以幫助快速定義網絡，並實現自動化部署和運維。控制和管理的集中，使得網絡路徑最優化變得更加容易實現。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"基於機器學習的新算法","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"機器學習在很多領域都得到了廣泛的應用，在實時音視頻通訊領域上也出現了很多應用方向，比如：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網絡傳輸相關：智能擁塞算法，智能帶寬評估算法，智能路由等；視頻圖像相關：虛擬背景，超分辯率，視頻融合，deepfake等；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"語音相關的：語音識別，語音增強等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"04","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"虛擬現實、增強現實和","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"3D","attrs":{}},{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"技術","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"虛擬現實(Virtual Reality)，就是虛擬和現實相互結合，是一種可以創建和體驗虛擬世界的計算機仿真系統，它利用計算機生成一種模擬環境，使用戶沉浸到該環境中。增強現實(Augmented Reality)是一種將虛擬信息與真實世界巧妙融合的技術，真實環境和虛擬物體之間重疊之後，能夠在同一個畫面以及空間中同時存在。通過VR，AR，3D技術的結合，相信不久的將來，實時音視頻通訊可以實現類似於現實世界中會議室一起開會的效果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：拍樂雲Pano","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"鏈接：https://juejin.cn/post/6992125317858983966/","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章