RTSP媒體協議流的錄製方案及其覆蓋策略詳解

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"前言","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在安防和監控領域，RTSP媒體協議流有很廣泛的使用。本文將介紹一種針對RTSP媒體流的錄製方案及其相應的覆蓋策略。據我所知，聲網的實時錄製功能支持三種模式，分別是雲端錄製、本地服務端錄製和頁面錄製，今天我們介紹的錄製方案和聲網的雲端錄製類似。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"正文","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文將從錄製視頻格式的調研、錄製方案的選擇、異常狀況的處理、覆蓋策略的執行四個大方面進行介紹。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1. 錄製視頻格式調研","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果想要實現RTSP媒體流的錄製功能，就需要考慮錄製目標文件的格式，也就是把媒體流錄製成哪種格式的視頻文件。起初我們預設了三種方案，經過一系列調研後，最終選擇了m3u8。接下來，我們簡單介紹一下這個選擇過程。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.1 爲什麼不用mp4格式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"mp4是點播視頻中最爲常見的視頻格式，綜合分析下來並不符合我們的使用場景。一般情況下，一個電影視頻的最大時長也就兩到三個小時左右，保存成一個mp4文件就夠用了，但是在安防和監控場景下，一個攝像頭對應的錄製視頻文件的長度可能是十幾個小時，甚至是十幾天。所以，對比下來，mp4格式更適用於電影網站。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這就引出了mp4格式的一個缺點，如果錄製存儲爲一個mp4格式，那文件體積可能會非常大。那麼，存儲的時候就會面臨一系列問題，比如磁盤空間不足、大文件分片等狀況的處理，特別是錄製過程中數據流異常中斷可能會導致已經錄製的mp4文件不可用，這是其一。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ee/ee86ba4d6f8f0da3c6d3f0ba61f08500.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們知道mp4文件是由許多Box和FullBox組成的，可以參考上圖的Box樹形圖，其中，FullBox是Box的擴展，每個Box又包含Header和Data兩部分，moov Box記錄了整個mp4文件的音視頻媒體信息。而moov Box一般是在mp4文件寫完時纔在文件尾部添加。因此，又引出了另外一個缺點，如果mp4文件特別大，那麼在播放的時候，播放器需要加載全部的視頻文件到內存中，如果視頻文件特別大，這幾乎是不現實的。因此，我們在錄製結束保存mp4的時候，需要把moov Box調整到文件頭部來避免這個問題。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.2 爲什麼不用mpd格式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"mpd格式類似於m3u8格式，但是它採用的是XML的組織形式。我們不選擇它的原因也有兩個，其一，mpd格式在現有產品線上沒有類似使用場景，我們使用更多的是m3u8，換句話說就是技術儲備不足。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其二，播放器方案的通用性上存在問題，如果使用mpd格式，那麼我們的播放器方案需要調整，能夠支持mpd格式媒體的播放，這樣一來會給播放器帶來一定的工作量和隱含的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，給出一個mpd的文件示例，讓大家對其有一個更加直觀的瞭解。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"html"},"content":[{"type":"text","text":"\n\n\n \n \n \n \n tears_audio_eng.mp4\n \n \n \n \n \n \n \n tears_h264_baseline_240p_800.mp4\n \n \n \n \n \n tears_h264_main_480p_2000.mp4\n \n \n \n \n \n tears_h264_main_720p_8000.mp4\n \n \n \n \n \n tears_h264_high_1080p_20000.mp4\n \n \n \n \n \n \n\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過上述文件，我們可以知道這個mpd文件包含了一路音頻流，同時支持三種不同分辨率和碼率的視頻流。不同的媒體類型是用AdaptationSet標籤表示的，內部還可以使用Representation標籤標記不同分辨率和碼率的媒體流。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.3 爲什麼最終選擇m3u8格式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"選擇m3u8的話，優勢就會更加明顯，除了規避上述方案的問題外，還有一些自身的優勢，具體表現如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1）本身就是ts分片存儲形式，不需要再單獨考慮大文件的切片問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2）現有播放器方案支持m3u8格式，不需要再單獨進行適配。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3）具有一定的技術儲備，開發上手快，開發週期可控。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4）相應的覆蓋策略執行起來會更加方便。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，給出一個m3u8的文件示例，讓大家對其有一個更加直觀的瞭解。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"html"},"content":[{"type":"text","text":"#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-TARGETDURATION:17\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:11.933333,\nindex_0000.ts\n#EXTINF:3.866667,\nindex_0001.ts\n#EXTINF:7.333333,\nindex_0002.ts\n#EXTINF:16.666667,\nindex_0003.ts\n#EXTINF:4.133333,\nindex_0004.ts\n#EXT-X-ENDLIST","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過上述文件，我們可以知道這個m3u8文件包含了5個ts分片，以及它們各自的時長信息。文件以#EXTM3U標籤開始，並以#EXT-X-ENDLIST標籤結束。這裏有一點需要注意，如果是直播使用的m3u8文件，它是沒有#EXT-X-ENDLIST標籤的。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2. 錄製方案選擇","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"既然已經確定了目標文件的格式，那麼我們就要考慮怎麼實現了。目前有兩個方案可以考慮，一個是Golang純原生方案，另一個是利用ffmpeg實現，接下來分別介紹。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.1 Go原生","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"利用純原生的Golang實現，其實，Golang處理音視頻數據還是有一定優勢的，通過解封裝RTSP媒體流，得到音頻數據和視頻數據，然後創建對應的解碼器，得到對應的原始音頻PCM數據和原始視頻YUV數據，再分別編碼成AAC的音頻和H264的視頻，最後保存成m3u8格式的錄製文件。整個過程可以參考下圖：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f1/f154527c5e9675a9d8fd0df2bac3854d.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這種方案，編碼的工作量會稍微大一些，同時有很多音視頻數據處理的細節問題，負載度和難易程度上不如ffmpeg方案。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.2 ffmpeg","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"利用ffmpeg工具庫，通過啓用ffmpeg進程來完成對應的RTSP流數據接收和m3u8文件錄製保存工作，這樣會更加簡單，我們只需要管理好進程的創建、釋放和異常處理工作。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3. 異常處理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"錄製過程中會遇到各種各樣的問題，接下會分別介紹。有一點是相同的，所有的異常狀況都會通知到錄製調度服務，由調度服務進行統一分析和管理，同時支持熱備機制，我們通過Nacos的服務發現機制監測錄製調度服務的運行狀態，具體關係可以參考下圖：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/07/076db91b0214cc93938bde0f5ec9c941.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 CPU、磁盤","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU負載過高和磁盤空間不足是最爲常見的兩種錄製時的異常狀況，大致的處理邏輯也是較爲相似的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU過高的處理邏輯，可以參考下圖：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/33/33825244bb3aaf0ef964f944194099f1.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前機器接收到任務後，進行自檢操作，發現CPU負載過高會停止當前錄製任務的執行，同時上報調度服務，重新分配別的機器執行該錄製任務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前機器正在執行錄製任務，突然發現CPU負載超過閾值，會持續觀察一段時間，假定觀察週期爲10秒，如果CPU負載連續10秒鐘超高，那麼會停止當前錄製任務，同時上報調度服務，請求別的機器繼續執行該錄製任務，最後將兩臺機器上的錄製文件進行邏輯關聯保存到數據庫中。如果CPU負載在10秒內恢復到正常值，我們將繼續執行當前錄製任務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"磁盤空間不足的處理邏輯和CPU負載過高有類似的處理邏輯，具體可以參考下圖：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d5/d5849825aec4fc2d6c825e4c8ed1313a.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過流程圖，我們也可以知道磁盤空間不足的處理邏輯和CPU負載過高時類似，上圖已經展示的非常明確了，這裏就不過多贅述了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 異常處理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一些其他的異常處理情況，比如崩潰，整體流程可以參考下圖：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1b/1b3cfcf9dad004469c1be8ce4ddbe55c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"異常發生時，如果是一般異常，我們只需要將狀態通知調度服務即可，調度服務記錄相關日誌，綜合分析整個錄製服務的狀態。如果60%的錄製機器觸發了相同的異常，調度服務就要採取相應的策略。如果是崩潰等重大異常，就需要重啓機器或者調度新的機器繼續執行錄製任務。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 錄製超時","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果發生了錄製超時，比如我們想錄制24個小時的視頻，現在時長已經錄夠了，接下來應該怎麼做呢？一般有兩種處理方法，第一種是直接停止當前錄製，上報通知調度服務即可，這種處理方式比較簡單粗暴，但是在安防和監控領域是不合適的。第二種是執行特定規則的覆蓋策略，實現循環覆蓋，始終保留最近24小時之內的視頻畫面內容。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c4/c4902cfc0565dc56c89009aff14061e7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對比上述兩種處理方式，當發生錄製超時時，第二種方式是最符合安防和監控領域的通用做法。那麼覆蓋策略又是怎麼實現的呢，這就引出了下面的內容——覆蓋策略。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4. 覆蓋策略","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"覆蓋策略在原理上理解起來很簡單，但是具體執行時，就不那麼簡單了。首先，我們也先通過一個流程圖對覆蓋策略的處理邏輯有一個整體上的認識。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/da/da2f7a3ac34b46e9e1d1892ff486ad18.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.1 一級定時器","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當錄製任務啓動時，我們同時啓動一個定時器（一級定時器），定時器的時長就是錄製任務的目標時長，這個非常好理解。但是，這個定時器只生效一次或者一次都不生效。只有一級定時器生效後，纔會啓動二級定時器。如果一級定時器沒有啓動，那麼二級定時器也不會啓動。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們可以這樣理解，只有一級定時器觸發，錄製服務纔會執行對應的覆蓋策略。當覆蓋策略啓動後，一級定時器銷燬，二級定時器生效。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.2 二級定時器","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當文件時長達到了預設的最大時長時，我們將啓動二級定時器。其實，二級定時器控制的是覆蓋策略的刪除頻率，每次時間到了，就刪除早些時候到錄製文件分片。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.3 執行覆蓋","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"具體覆蓋的執行邏輯是，根據ts分片的時長和二級定時器的時間週期，計算需要刪除的ts分片個數，同時更新m3u8中的索引列表，然後循環執行該策略，最終實現動態循環的錄製覆蓋策略。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/78/78745cd656da6c537fb81d4f8b6da66b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"覆蓋策略的執行過程如上圖所示，相信通過上文的解釋，大家理解起來還是非常容易的。需要特別說明的是，由於二級定時器執行週期 t 的限制，錄製文件的實際時長在最大錄製時長 T 和（T+t）之間。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"結尾","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"好了，現在關於RTSP媒體流的錄製方案和覆蓋策略就介紹完了，相信大家對雲端錄製方案也有了一定認識，有自己想法和感興趣的小夥伴，歡迎評論留言。關注我，分享更多音視頻和流媒體服務器內容。","attrs":{}}]}]}