微帧Film Grain编码技术，致敬电影胶片颗粒的独特魅力

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“胶片成像的颗粒感，是数字影像无法复制的独特魅力。”著名导演克里斯托弗·诺兰在《信条》上映时表示。作为胶片电影的忠实追随者，自处女作起，诺兰每一部影片都采用胶片进行拍摄，其中包括《星际穿越》《盗梦空间》等风靡全球的著作。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"诺兰导演口中的“胶片成像的颗粒感”，正是本文要讨论的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Film Grain（胶片颗粒）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为何Film Grain作为一种噪声，却能得到大量导演和观众的喜爱？其实是因为在某些特定的非线性系统中，噪声的存在能够增强微弱信号的检测能力，这种现象就称为随机共振。而在图像系统中，在一定噪声（如Film Grain）强度条件下，图像视觉质量可以达到更优的效果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此，Film Grain作为影视创作意图的一部分，在对视频进行编码时，有效保留Film Grain是非常重要的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但Film Grain作为图像噪声，分布和大小都不规则，具有随机性。Film Grain的随机性使得传统编码技术难以有效压缩，同时也使得预测变得非常困难，运动估计的精度也会降低。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就此，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"微帧科技在微帧Aurora AV1基础上研发了Film Grain编码技术，","attrs":{}},{"type":"text","text":"此技术已成功应用于某知名PGC视频平台，并获得了客户及平台用户的认可。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用AV1能更高效地保留与合成Film Grain。因为传统视频标准在制定时，并没有将Film Grain加进标准，运用传统编码方式对视频编码，容易造成grain分布不均匀、保留不完整等问题。而AV1在制定时就把Film Grain处理工具加进了标准，对于Film Grain编码和解码的支持度更高。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"与开源libaom相比，微帧Aurora-AV1的优势在于编码速度更快、效率更高，其次是能支持10bit Film Grain的去噪处理；并且，在开源libaom编码器上保留grain容易出现明显的banding，微帧Aurora-AV1的自适应量化模式也能更好的避免这种情况。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fb/fb911f3689ed7fa0c7b969083445c6b6.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"banding示意","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"针对不同类型的grain，微帧科技采用Aurora-AV1视频编码器进行了3个方向的尝试：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"去噪+合成（Grain Synthesis）、直接保留（Grain Preservation）、保留+合成（Preservation+Synthesis）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"01、去噪+合成（Grain Synthesis）","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Grain Synthesis是通过去噪处理去除视频中的grain，并估计出grain参数，这些参数将随压缩视频流一起发送至解码器，解码后，再将grain合成并添加到重建的视频帧中。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b0/b0dccc761e0093b1c69a896eb1085c23.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此方法有助于保存已编码视频的grain外观，与直接对grain进行编码比，能保持更低的码率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但Grain Synthesis的缺点在于：合成过程中的第一步需要生成一个64x64的模板，之后随机从模板中选取一个32x32的块。在选取的时候，x,、y的座标需要小于32，导致相邻块选取的32x32块会有重叠，且朝向一致，在grain强的序列中的平坦区域非常容易产生pattern，以及明显的artifacts。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就此问题，微帧的解决方法是：选取grain的种子，只采用pattern没那么明显的种子进行合成。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ae/aea38bf50849318db235c8ace20f123b.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"左：源视频","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"右：通过微帧Aurora-AV1编码合成的grain","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"两者几乎无差","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"02、直接保留（Grain Preservation）","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若源视频中的Film Grain是未经压缩的，可以用AV1 Film Grain工具进行重建；但对于已被转码压缩的视频，则很难重建。因为grain经过压缩后，会出现低频分量，产生一定的pattern，这些很难用AR model完全表示出来。因此，对于明显有低频分量的grain pattern，需要通过直接编码的方式进行保留。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除此之外，对于grain比较细的序列，采用直接保留grain的方式，能让grain保留的更均匀，也能保证更好的清晰度，使得在无grain artifact的同时生成较好的grain视频。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得一提的是，采用微帧Aurora-AV1直接编码保留grain，能够抑制grain不一致和不同帧清晰度不一致的现象，这是普通AV1编码器目前所达不到的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"03、保留+合成（Preservation+Synthesis）","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"编码过程中还会遇到同一帧grain分布非常不均匀的情况，比如上半部分有grain，下半部分没有grain，这种情况不能依赖Grain Synthesis的方法，因为标准全图都为同一种grain model。直接编码的方式则可能会出现grain保留不完整、不一致等情况。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于类似这种复杂的grain，微帧采用的方法是：将grain分解，针对难以用Grain Synthesis标准生成的grain，直接依赖编码器进行编码；针对Grain Synthesis标准比较好生成的grain，采用标准生成grain，如高频的高斯噪声部分则可以采用Grain Synthesis来合成。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Preservation+Synthesis的方式能解决Grain Synthesis的局限性，重现比较复杂pattern的grain。同时，增加了grain合成的方式会比直接编码的方法，能保留更多的细小的grain。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/13/13eb70fc71d0d56b54ed279b7b2bb323.jpeg","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"right","origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"size","attrs":{"size":9}}],"text":"右：微帧微帧Aurora-AV1编码合成的grain","attrs":{}}]}]}