微幀Film Grain編碼技術，致敬電影膠片顆粒的獨特魅力

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“膠片成像的顆粒感，是數字影像無法複製的獨特魅力。”著名導演克里斯托弗·諾蘭在《信條》上映時表示。作爲膠片電影的忠實追隨者，自處女作起，諾蘭每一部影片都採用膠片進行拍攝，其中包括《星際穿越》《盜夢空間》等風靡全球的著作。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"諾蘭導演口中的“膠片成像的顆粒感”，正是本文要討論的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Film Grain（膠片顆粒）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲何Film Grain作爲一種噪聲，卻能得到大量導演和觀衆的喜愛？其實是因爲在某些特定的非線性系統中，噪聲的存在能夠增強微弱信號的檢測能力，這種現象就稱爲隨機共振。而在圖像系統中，在一定噪聲（如Film Grain）強度條件下，圖像視覺質量可以達到更優的效果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此，Film Grain作爲影視創作意圖的一部分，在對視頻進行編碼時，有效保留Film Grain是非常重要的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但Film Grain作爲圖像噪聲，分佈和大小都不規則，具有隨機性。Film Grain的隨機性使得傳統編碼技術難以有效壓縮，同時也使得預測變得非常困難，運動估計的精度也會降低。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就此，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"微幀科技在微幀Aurora AV1基礎上研發了Film Grain編碼技術，","attrs":{}},{"type":"text","text":"此技術已成功應用於某知名PGC視頻平臺，並獲得了客戶及平臺用戶的認可。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用AV1能更高效地保留與合成Film Grain。因爲傳統視頻標準在制定時，並沒有將Film Grain加進標準，運用傳統編碼方式對視頻編碼，容易造成grain分佈不均勻、保留不完整等問題。而AV1在制定時就把Film Grain處理工具加進了標準，對於Film Grain編碼和解碼的支持度更高。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"與開源libaom相比，微幀Aurora-AV1的優勢在於編碼速度更快、效率更高，其次是能支持10bit Film Grain的去噪處理；並且，在開源libaom編碼器上保留grain容易出現明顯的banding，微幀Aurora-AV1的自適應量化模式也能更好的避免這種情況。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fb/fb911f3689ed7fa0c7b969083445c6b6.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"banding示意","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對不同類型的grain，微幀科技採用Aurora-AV1視頻編碼器進行了3個方向的嘗試：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"去噪+合成（Grain Synthesis）、直接保留（Grain Preservation）、保留+合成（Preservation+Synthesis）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"01、去噪+合成（Grain Synthesis）","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Grain Synthesis是通過去噪處理去除視頻中的grain，並估計出grain參數，這些參數將隨壓縮視頻流一起發送至解碼器，解碼後，再將grain合成並添加到重建的視頻幀中。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b0/b0dccc761e0093b1c69a896eb1085c23.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此方法有助於保存已編碼視頻的grain外觀，與直接對grain進行編碼比，能保持更低的碼率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但Grain Synthesis的缺點在於：合成過程中的第一步需要生成一個64x64的模板，之後隨機從模板中選取一個32x32的塊。在選取的時候，x,、y的座標需要小於32，導致相鄰塊選取的32x32塊會有重疊，且朝向一致，在grain強的序列中的平坦區域非常容易產生pattern，以及明顯的artifacts。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就此問題，微幀的解決方法是：選取grain的種子，只採用pattern沒那麼明顯的種子進行合成。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ae/aea38bf50849318db235c8ace20f123b.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"左：源視頻","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"右：通過微幀Aurora-AV1編碼合成的grain","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"兩者幾乎無差","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"02、直接保留（Grain Preservation）","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若源視頻中的Film Grain是未經壓縮的，可以用AV1 Film Grain工具進行重建；但對於已被轉碼壓縮的視頻，則很難重建。因爲grain經過壓縮後，會出現低頻分量，產生一定的pattern，這些很難用AR model完全表示出來。因此，對於明顯有低頻分量的grain pattern，需要通過直接編碼的方式進行保留。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除此之外，對於grain比較細的序列，採用直接保留grain的方式，能讓grain保留的更均勻，也能保證更好的清晰度，使得在無grain artifact的同時生成較好的grain視頻。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得一提的是，採用微幀Aurora-AV1直接編碼保留grain，能夠抑制grain不一致和不同幀清晰度不一致的現象，這是普通AV1編碼器目前所達不到的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"03、保留+合成（Preservation+Synthesis）","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"編碼過程中還會遇到同一幀grain分佈非常不均勻的情況，比如上半部分有grain，下半部分沒有grain，這種情況不能依賴Grain Synthesis的方法，因爲標準全圖都爲同一種grain model。直接編碼的方式則可能會出現grain保留不完整、不一致等情況。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於類似這種複雜的grain，微幀採用的方法是：將grain分解，針對難以用Grain Synthesis標準生成的grain，直接依賴編碼器進行編碼；針對Grain Synthesis標準比較好生成的grain，採用標準生成grain，如高頻的高斯噪聲部分則可以採用Grain Synthesis來合成。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Preservation+Synthesis的方式能解決Grain Synthesis的侷限性，重現比較複雜pattern的grain。同時，增加了grain合成的方式會比直接編碼的方法，能保留更多的細小的grain。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/13/13eb70fc71d0d56b54ed279b7b2bb323.jpeg","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"右：微幀微幀Aurora-AV1編碼合成的grain","attrs":{}}]}]}