一种基于视频帧差异视频卡顿检测方案

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"奇技 · 指南"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在视频质量检测中，检测视频是否卡顿也属于视频质量检测的标准之一，在构建视频检测平台中,这一步至关重要。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文要说明的是把视频转换为帧序列,根据计算帧之间的差值,寻找帧序列中是否有断层,来判断当前视频是否存在卡顿的现象。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"结果为一个数据, 0 代表无卡顿现象, 1代表存在卡顿现象"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技术与架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用户上传视频文件后,使用ffmpeg 转换为图片序列,抽取图片信息,计算所有序列帧的图片运动像素,计算所有序列图片的平均运动水平,动态计算动态因子,输出判断结果, 0表示当前不存在卡顿点, 1表示当前存在卡顿点。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/54\/547058587cdf06a870819d1004a259b1.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"整体方案主要分为六个部分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1. 图片处理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2. 图像相邻帧像素计算"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3. 计算所有图片运动量,组合为运动集合"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4. 消除视频图片场景剪辑比例,计算平均运动量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"5. 计算动态因子"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"6. 返回结果"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技术优势"}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不需要准备大量的数据集来训练模型，只针对当前要处理的视频进行计算；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不会因为大量的动态场景和静态场景影响卡顿检测的结果；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"精准高效,计算量相对较低"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技术实现"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"图片处理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这里采用灰度图片来作为视频卡顿检测的输入序列图片数据,重新设置当前图片的大小为 360*640,当前区域为我们后面计算的兴趣区域.设定兴趣区域,可以有效的避免一些像素点计算的噪声影响。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"图像相邻帧计算"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"A.遍历当前图像集,使用t+1(下一时刻帧) 的像素减去t(当前时刻帧)的像素值,计算出两帧之间的差异信息。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"B.设定阈值,这里的阈值是一个常量值 = 30,当两帧之间的差异值> 30的时候,就任务图片存在运动像素,否则,没有存在运动,值为 0,此步骤消除了低运动噪声,或感知能力下的运动像素。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"计算所有图片的运动量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"将步骤2中的值进行平方,将幅度转换为能量,并计算每个视频帧的平均值.该平均值就是当前帧的能量值,所有帧的动量值记为TI2。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"消除视频图片场景切换比例,计算平均运动水量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"A.在计算平均值之前要消除场景剪辑比例,这里使用的常量值为 0.02,就是说我们有100个帧要消除2个场景的剪辑。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"B.对 TI2 从小到大排序,在这个序列中,根据B中的比例值,消除最高和最低的两个噪声值,循环遍历TI2,计算t时刻帧之前所有帧的平均值,并把这个平均值进行累加.当场景切换的时候,TI2 序列的低点和高点均被消除,平均TI2值(TI2_AVG)不会收到影响。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"计算动态因子"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在近乎静态的场景和动态的场景中,由于像素的变动很小,或者像素变动很大,卡顿的帧\/丢失帧会存在少量\/大量的运动信息,在确定运动水平的时候需要涉及主观直觉的要素,所以需要利用动态阈值来确定卡顿的视频帧。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在视频中,存在的动态场景较多,该阈值增加,静态场景,阈值减少。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dfact = a + b * log(TI2_AVG)  a ,b, c 都为常量,分别为 2.5, 1.25, 0.1,c为限制Dfact 较小的一个值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当 Dfact < c = Dfact else 等于 c, Dfact 取值范围是[0, 0.1]之前的一个值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"帧的丢弃和运动量是线性依赖于平均运动能量的对数。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"返回结果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"循环遍历视频帧,获取每一帧的TI2值,如果当前的TI2值<= Dfact * Mdrop,认为当前的帧是卡顿的,也就是值为1,如果当前的TI2值> Dfact * Mdrop, 任务当前帧不存在卡顿,把视频所有的帧按找时间顺序排序后,就是我们当前视频卡顿检测的列表值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dfact 为上述计算的动态因子,Mdrop为固定运动能量阈值来确定帧的卡顿.Mdrop 为常量值 0.015。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"效果展示"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/98\/9800ccd3354134d713dbbd68a4540984.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"选择9张连续的视频帧图片"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/72\/72ecc75d33471dce8776b0862c126d04.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"返回检测结果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文转载自：360技术（ID：qihoo_tech）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/jvVhENz1Stmyld2e1Oeo1g","title":"xxx","type":null},"content":[{"type":"text","text":"一种基于视频帧差异视频卡顿检测方案"}]}]}]}