Netflix如何利用開源軟件打包視頻流

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong"}],"text":"本文最初發佈於Netflix技術博客，經授權由InfoQ中文站翻譯並分享。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在之前的博文中，我們在Netflix的同事解釋了"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/optimized-shot-based-encodes-for-4k-now-streaming-47b516b10bbb","title":null,"type":null},"content":[{"type":"text","text":"如何優化4K視頻流"}]},{"type":"text","text":"，"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/improving-our-video-encodes-for-legacy-devices-2b6b56eec5c9","title":null,"type":null},"content":[{"type":"text","text":"如何改進傳統視頻流"}]},{"type":"text","text":"，以及最近的，"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/optimizing-the-aural-experience-on-android-devices-with-xhe-aac-c27714292a33","title":null,"type":null},"content":[{"type":"text","text":"如何使用新的音頻編解碼器爲我們的會員提供更好的聽覺體驗"}]},{"type":"text","text":"。在所有這些情況下，在通過內容分發網絡Open Connect傳送之前，我們需要對那些屢獲殊榮的電視節目、電影和紀錄片（如《王冠》）進行打包，以便可以爲我們的會員提供一些關鍵特性。在這篇文章中，我們將解釋這些特性以及我們如何依靠獲獎的標準格式和開源軟件來實現它們。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"關鍵打包特性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在典型的流管道中，打包是在編碼之後進行的步驟，如下圖所示。編碼器的輸出是一個字節序列，稱爲基本流，要對它進行解析需要了解基本的流語法。例如，檢測AV1視頻流中的幀邊界需要能夠解析所謂的開放比特流單元（OBU）並識別時間分隔符OBU。但是，在客戶端設備上執行的高級操作，例如搜索，則不需要了解基本語法，並且可以從編解碼器無關的格式中受益。打包步驟旨在產生這樣一種與編解碼器無關的字節序列，稱爲打包格式或容器格式，在某種程度上，無需深入瞭解編碼格式就可以對其進行操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ac\/e0\/ac1260b5017e4542dd64c92ff0270ee0.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"圖1：流預備管道的簡化架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在播放音頻、視頻和定時文本時，我們應該爲會員提供的一個關鍵功能是同步。在Netflix，我們努力提供一種體驗，讓你永遠不會在聽到《王冠》中相應的對話之前，看到英國女王的嘴脣在動。同步是通過信號的基本元素來實現的，比如在打包內容中提供的時鐘或時間線、時間戳和時間刻度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"會員不是簡單地從頭到尾觀看節目。當繼續觀看《布里奇頓》時，他們會搜索。他們在觀看《"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"女王的棋局》時會"},{"type":"text","text":"重放他們最喜歡的棋局。當他們迫不及待地觀看"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"《亞森·羅賓》"},{"type":"text","text":"時，他們會跳過介紹和回顧。他們會在觀看互動節目《你與荒野》時回放。由於音頻或視頻壓縮技術的性質，播放器不可能從會員想要的地方開始解碼流。其底層機制是，播放器必須在流中找到可以開始解碼的點，並儘可能快地解碼，找到用戶的搜索點之開始播放。這是打包的另一個基本特性：信號幀類型，特別是隨機接入點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當會員的孩子在父母的汽車後座上觀看《神偷卡門》時，或者更一般的來說，當網絡吞吐量變化時，我們運用了自適應流媒體技術來提供這種網絡條件下最佳的觀看體驗。"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/engineering-a-studio-quality-experience-with-high-quality-audio-at-netflix-eaa0b6145f32","title":null,"type":null},"content":[{"type":"text","text":"自適應流媒體技術"}]},{"type":"text","text":"要求在對各種質量的流進行編碼時遵循共同的約束條件，但它們也依賴於打包的另一個關鍵特性“索引”來提供無縫的質量切換。索引讓播放器可以只獲取新流的相應片段。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們在打包的內容中提供了許多其他的信號元素，以期讓會員可以儘快開始觀看。解密模塊需要用合適的方案和初始化向量進行初始化。硬件視頻解碼器需要提前知道視頻流的分辨率和位深，以便分配解碼緩衝區。渲染管道需要提前知道音頻流的揚聲器配置，或者視頻流是HDR還是SDR。能夠標記所有這些元素也是現代打包格式的一個關鍵特性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"標準和開源軟件的作用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們的2億多會員在各種各樣的設備上觀看Netflix，從智能手機到筆記本電腦，再到電視和由大量合作伙伴開發的許多其他設備。我們要減少在新設備上播放的阻力，並確保我們的內容在很長一段時間內可以在舊設備上播放，這非常重要。標準在這裏發揮了關鍵性的作用。ISO基本媒體文件格式（ISOBMFF）是娛樂行業的關鍵打包標準，最近獲得了美國國家電視藝術與科學學院（NATAS）頒發的"},{"type":"link","attrs":{"href":"https:\/\/theemmys.tv\/tech-72nd-award-recipients\/","title":null,"type":null},"content":[{"type":"text","text":"技術與工程艾美獎®"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ISOBMFF提供了上面提到的所有關鍵打包特性，而且歷史已經證明，它是通用且可擴展的，它具備添加新信號特性和編解碼器支持的能力。使用成熟編解碼器（如AVC和AAC）編碼的流可以通過ISOBMFF文件傳輸，但該規範也會定期擴展，以支持最新的編解碼器。Netflix的媒體系統團隊積極致力於ISOBMFF的開發、維護和採用。例如，Netflix主導了"},{"type":"link","attrs":{"href":"https:\/\/aomediacodec.github.io\/av1-isobmff\/","title":null,"type":null},"content":[{"type":"text","text":"ISOBMFF中AOM AV1視頻流的傳輸規範"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"經過20多年的發展，ISOBMFF爲各種用例積累了大量的技術工具。圖2通過“brand”的概念說明了現如今ISOBMFF的複雜性，這個概念類似於音頻或視頻標準中的配置文件。最初，該標準有限且可以良好地嵌套，但現在已經非常廣泛，並向各個方向發展。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/59\/37\/59c9585ac3ee2e201440f022e2197637.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"圖2：說明ISOBMFF第6版的複雜性。每個矩形代表一個“brand”(由粗體的四字符代碼表示）和它所需的工具集（由“+”號行表示）。Brand是嵌套的。所有內層brand的工具都是外層brand所需要的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於Netflix的流媒體服務，我們依賴於由公共媒體應用程序格式（CMAF）標準確定的這些工具的一個子集，以及公共加密（CENC）標準中定義的內容保護工具。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"像ISOBMFF、CMAF和CENC這樣的多媒體標準都有相應的開源軟件實現。開源軟件可以示範該標準的特性，使行業瞭解其好處並擴大其採用範圍。開源軟件還可以通過中立的參考實現來突出可能存在的歧義條款，從而幫助提高標準的質量。Netflix的媒體系統團隊爲SMPTE IMF標準維護了一個名爲"},{"type":"link","attrs":{"href":"https:\/\/github.com\/Netflix\/photon","title":null,"type":null},"content":[{"type":"text","text":"Photon"}]},{"type":"text","text":"的開源參考實現。對於ISOBMFF, Netflix使用了來自"},{"type":"link","attrs":{"href":"http:\/\/gpac.io\/","title":null,"type":null},"content":[{"type":"text","text":"GPAC團隊"}]},{"type":"text","text":"的開源參考實現"},{"type":"link","attrs":{"href":"https:\/\/github.com\/gpac\/gpac\/wiki\/MP4Box","title":null,"type":null},"content":[{"type":"text","text":"MP4Box"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個由標準和開源軟件組成的打包生態系統中，媒體系統團隊的工作包括從現有標準中找出工具來解決新的流媒體用例。當這樣的工具不存在時，我們就定義新的標準或擴展現有的標準，包括ISOBMFF和CMAF，並支持開源軟件來匹配這些標準。例如，當我們的視頻編碼同事設計了動態優化的編碼方案，用於製作具有可變時長的流片段，我們修改了工作流程，以確保具有不同比特率的視頻流段在時間上保持一致。類似地，當我們的音頻編碼同事引入xHE-AAC時（這就打破了以前每個音頻幀都可解碼的假設），也就保證了音頻\/視頻片段的一致性。最後，當我們希望幫助業界匯聚到一種用於新視頻編解碼器（如AV1）的通用加密方案時，我們就協調討論選出方案，這裏是基於模式的子樣本加密（簡稱“cbcs”），並通過提供參考位流作出示範。當然，我們的工作也包括在在沒有適當標準支持的領域處理許多類型的設備。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"小結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們希望這篇文章能讓你更好地瞭解Netflix媒體系統團隊的部分工作，希望下次你觀看我們的獲獎節目時，會發現ISOBMFF所發揮的作用，這是一項關鍵的獲獎技術。感興趣的話，你可以瞭解下這個團隊其他方面的工作，看看其他獲獎的技術，比如"},{"type":"link","attrs":{"href":"https:\/\/theemmys.tv\/tech-67th-award-recipients\/","title":null,"type":null},"content":[{"type":"text","text":"TTML"}]},{"type":"text","text":"，"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/implementing-japanese-subtitles-on-netflix-c165fbe61989","title":null,"type":null},"content":[{"type":"text","text":"我們用它來製作日語字幕"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查看英文原文："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/packaging-award-winning-shows-with-award-winning-technology-c1010594ba39","title":null,"type":null},"content":[{"type":"text","text":"Packaging award-winning shows with award-winning technology"}]}]}]}