一種基於視頻幀差異視頻卡頓檢測方案

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"奇技 · 指南"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在視頻質量檢測中，檢測視頻是否卡頓也屬於視頻質量檢測的標準之一，在構建視頻檢測平臺中,這一步至關重要。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文要說明的是把視頻轉換爲幀序列,根據計算幀之間的差值,尋找幀序列中是否有斷層,來判斷當前視頻是否存在卡頓的現象。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"結果爲一個數據, 0 代表無卡頓現象, 1代表存在卡頓現象"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技術與架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用戶上傳視頻文件後,使用ffmpeg 轉換爲圖片序列,抽取圖片信息,計算所有序列幀的圖片運動像素,計算所有序列圖片的平均運動水平,動態計算動態因子,輸出判斷結果, 0表示當前不存在卡頓點, 1表示當前存在卡頓點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/54\/547058587cdf06a870819d1004a259b1.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"整體方案主要分爲六個部分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1. 圖片處理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2. 圖像相鄰幀像素計算"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3. 計算所有圖片運動量,組合爲運動集合"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4. 消除視頻圖片場景剪輯比例,計算平均運動量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"5. 計算動態因子"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"6. 返回結果"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技術優勢"}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不需要準備大量的數據集來訓練模型，只針對當前要處理的視頻進行計算；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不會因爲大量的動態場景和靜態場景影響卡頓檢測的結果；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"精準高效,計算量相對較低"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技術實現"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"圖片處理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏採用灰度圖片來作爲視頻卡頓檢測的輸入序列圖片數據,重新設置當前圖片的大小爲 360*640,當前區域爲我們後面計算的興趣區域.設定興趣區域,可以有效的避免一些像素點計算的噪聲影響。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"圖像相鄰幀計算"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"A.遍歷當前圖像集,使用t+1(下一時刻幀) 的像素減去t(當前時刻幀)的像素值,計算出兩幀之間的差異信息。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"B.設定閾值,這裏的閾值是一個常量值 = 30,當兩幀之間的差異值> 30的時候,就任務圖片存在運動像素,否則,沒有存在運動,值爲 0,此步驟消除了低運動噪聲,或感知能力下的運動像素。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"計算所有圖片的運動量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將步驟2中的值進行平方,將幅度轉換爲能量,並計算每個視頻幀的平均值.該平均值就是當前幀的能量值,所有幀的動量值記爲TI2。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"消除視頻圖片場景切換比例,計算平均運動水量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"A.在計算平均值之前要消除場景剪輯比例,這裏使用的常量值爲 0.02,就是說我們有100個幀要消除2個場景的剪輯。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"B.對 TI2 從小到大排序,在這個序列中,根據B中的比例值,消除最高和最低的兩個噪聲值,循環遍歷TI2,計算t時刻幀之前所有幀的平均值,並把這個平均值進行累加.當場景切換的時候,TI2 序列的低點和高點均被消除,平均TI2值(TI2_AVG)不會收到影響。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"計算動態因子"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在近乎靜態的場景和動態的場景中,由於像素的變動很小,或者像素變動很大,卡頓的幀\/丟失幀會存在少量\/大量的運動信息,在確定運動水平的時候需要涉及主觀直覺的要素,所以需要利用動態閾值來確定卡頓的視頻幀。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在視頻中,存在的動態場景較多,該閾值增加,靜態場景,閾值減少。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dfact = a + b * log(TI2_AVG)  a ,b, c 都爲常量,分別爲 2.5, 1.25, 0.1,c爲限制Dfact 較小的一個值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當 Dfact < c = Dfact else 等於 c, Dfact 取值範圍是[0, 0.1]之前的一個值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"幀的丟棄和運動量是線性依賴於平均運動能量的對數。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"返回結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"循環遍歷視頻幀,獲取每一幀的TI2值,如果當前的TI2值<= Dfact * Mdrop,認爲當前的幀是卡頓的,也就是值爲1,如果當前的TI2值> Dfact * Mdrop, 任務當前幀不存在卡頓,把視頻所有的幀按找時間順序排序後,就是我們當前視頻卡頓檢測的列表值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dfact 爲上述計算的動態因子,Mdrop爲固定運動能量閾值來確定幀的卡頓.Mdrop 爲常量值 0.015。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"效果展示"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/98\/9800ccd3354134d713dbbd68a4540984.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"選擇9張連續的視頻幀圖片"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/72\/72ecc75d33471dce8776b0862c126d04.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"返回檢測結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：360技術（ID：qihoo_tech）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/jvVhENz1Stmyld2e1Oeo1g","title":"xxx","type":null},"content":[{"type":"text","text":"一種基於視頻幀差異視頻卡頓檢測方案"}]}]}]}