RTSP媒体协议流的录制方案及其覆盖策略详解

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"前言","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在安防和监控领域，RTSP媒体协议流有很广泛的使用。本文将介绍一种针对RTSP媒体流的录制方案及其相应的覆盖策略。据我所知，声网的实时录制功能支持三种模式，分别是云端录制、本地服务端录制和页面录制，今天我们介绍的录制方案和声网的云端录制类似。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"正文","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文将从录制视频格式的调研、录制方案的选择、异常状况的处理、覆盖策略的执行四个大方面进行介绍。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1. 录制视频格式调研","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果想要实现RTSP媒体流的录制功能，就需要考虑录制目标文件的格式，也就是把媒体流录制成哪种格式的视频文件。起初我们预设了三种方案，经过一系列调研后，最终选择了m3u8。接下来，我们简单介绍一下这个选择过程。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.1 为什么不用mp4格式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"mp4是点播视频中最为常见的视频格式，综合分析下来并不符合我们的使用场景。一般情况下，一个电影视频的最大时长也就两到三个小时左右，保存成一个mp4文件就够用了，但是在安防和监控场景下，一个摄像头对应的录制视频文件的长度可能是十几个小时，甚至是十几天。所以，对比下来，mp4格式更适用于电影网站。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这就引出了mp4格式的一个缺点，如果录制存储为一个mp4格式，那文件体积可能会非常大。那么，存储的时候就会面临一系列问题，比如磁盘空间不足、大文件分片等状况的处理，特别是录制过程中数据流异常中断可能会导致已经录制的mp4文件不可用，这是其一。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ee/ee86ba4d6f8f0da3c6d3f0ba61f08500.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们知道mp4文件是由许多Box和FullBox组成的，可以参考上图的Box树形图，其中，FullBox是Box的扩展，每个Box又包含Header和Data两部分，moov Box记录了整个mp4文件的音视频媒体信息。而moov Box一般是在mp4文件写完时才在文件尾部添加。因此，又引出了另外一个缺点，如果mp4文件特别大，那么在播放的时候，播放器需要加载全部的视频文件到内存中，如果视频文件特别大，这几乎是不现实的。因此，我们在录制结束保存mp4的时候，需要把moov Box调整到文件头部来避免这个问题。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.2 为什么不用mpd格式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"mpd格式类似于m3u8格式，但是它采用的是XML的组织形式。我们不选择它的原因也有两个，其一，mpd格式在现有产品线上没有类似使用场景，我们使用更多的是m3u8，换句话说就是技术储备不足。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其二，播放器方案的通用性上存在问题，如果使用mpd格式，那么我们的播放器方案需要调整，能够支持mpd格式媒体的播放，这样一来会给播放器带来一定的工作量和隐含的问题。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后，给出一个mpd的文件示例，让大家对其有一个更加直观的了解。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"html"},"content":[{"type":"text","text":"\n\n\n \n \n \n \n tears_audio_eng.mp4\n \n \n \n \n \n \n \n tears_h264_baseline_240p_800.mp4\n \n \n \n \n \n tears_h264_main_480p_2000.mp4\n \n \n \n \n \n tears_h264_main_720p_8000.mp4\n \n \n \n \n \n tears_h264_high_1080p_20000.mp4\n \n \n \n \n \n \n\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过上述文件，我们可以知道这个mpd文件包含了一路音频流，同时支持三种不同分辨率和码率的视频流。不同的媒体类型是用AdaptationSet标签表示的，内部还可以使用Representation标签标记不同分辨率和码率的媒体流。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.3 为什么最终选择m3u8格式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"选择m3u8的话，优势就会更加明显，除了规避上述方案的问题外，还有一些自身的优势，具体表现如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1）本身就是ts分片存储形式，不需要再单独考虑大文件的切片问题。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2）现有播放器方案支持m3u8格式，不需要再单独进行适配。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3）具有一定的技术储备，开发上手快，开发周期可控。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4）相应的覆盖策略执行起来会更加方便。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后，给出一个m3u8的文件示例，让大家对其有一个更加直观的了解。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"html"},"content":[{"type":"text","text":"#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-TARGETDURATION:17\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:11.933333,\nindex_0000.ts\n#EXTINF:3.866667,\nindex_0001.ts\n#EXTINF:7.333333,\nindex_0002.ts\n#EXTINF:16.666667,\nindex_0003.ts\n#EXTINF:4.133333,\nindex_0004.ts\n#EXT-X-ENDLIST","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过上述文件，我们可以知道这个m3u8文件包含了5个ts分片，以及它们各自的时长信息。文件以#EXTM3U标签开始，并以#EXT-X-ENDLIST标签结束。这里有一点需要注意，如果是直播使用的m3u8文件，它是没有#EXT-X-ENDLIST标签的。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2. 录制方案选择","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"既然已经确定了目标文件的格式，那么我们就要考虑怎么实现了。目前有两个方案可以考虑，一个是Golang纯原生方案，另一个是利用ffmpeg实现，接下来分别介绍。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.1 Go原生","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"利用纯原生的Golang实现，其实，Golang处理音视频数据还是有一定优势的，通过解封装RTSP媒体流，得到音频数据和视频数据，然后创建对应的解码器，得到对应的原始音频PCM数据和原始视频YUV数据，再分别编码成AAC的音频和H264的视频，最后保存成m3u8格式的录制文件。整个过程可以参考下图：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f1/f154527c5e9675a9d8fd0df2bac3854d.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这种方案，编码的工作量会稍微大一些，同时有很多音视频数据处理的细节问题，负载度和难易程度上不如ffmpeg方案。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.2 ffmpeg","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"利用ffmpeg工具库，通过启用ffmpeg进程来完成对应的RTSP流数据接收和m3u8文件录制保存工作，这样会更加简单，我们只需要管理好进程的创建、释放和异常处理工作。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3. 异常处理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"录制过程中会遇到各种各样的问题，接下会分别介绍。有一点是相同的，所有的异常状况都会通知到录制调度服务，由调度服务进行统一分析和管理，同时支持热备机制，我们通过Nacos的服务发现机制监测录制调度服务的运行状态，具体关系可以参考下图：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/07/076db91b0214cc93938bde0f5ec9c941.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 CPU、磁盘","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU负载过高和磁盘空间不足是最为常见的两种录制时的异常状况，大致的处理逻辑也是较为相似的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU过高的处理逻辑，可以参考下图：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/33/33825244bb3aaf0ef964f944194099f1.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当前机器接收到任务后，进行自检操作，发现CPU负载过高会停止当前录制任务的执行，同时上报调度服务，重新分配别的机器执行该录制任务。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当前机器正在执行录制任务，突然发现CPU负载超过阈值，会持续观察一段时间，假定观察周期为10秒，如果CPU负载连续10秒钟超高，那么会停止当前录制任务，同时上报调度服务，请求别的机器继续执行该录制任务，最后将两台机器上的录制文件进行逻辑关联保存到数据库中。如果CPU负载在10秒内恢复到正常值，我们将继续执行当前录制任务。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"磁盘空间不足的处理逻辑和CPU负载过高有类似的处理逻辑，具体可以参考下图：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d5/d5849825aec4fc2d6c825e4c8ed1313a.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过流程图，我们也可以知道磁盘空间不足的处理逻辑和CPU负载过高时类似，上图已经展示的非常明确了，这里就不过多赘述了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 异常处理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一些其他的异常处理情况，比如崩溃，整体流程可以参考下图：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1b/1b3cfcf9dad004469c1be8ce4ddbe55c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"异常发生时，如果是一般异常，我们只需要将状态通知调度服务即可，调度服务记录相关日志，综合分析整个录制服务的状态。如果60%的录制机器触发了相同的异常，调度服务就要采取相应的策略。如果是崩溃等重大异常，就需要重启机器或者调度新的机器继续执行录制任务。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 录制超时","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果发生了录制超时，比如我们想录制24个小时的视频，现在时长已经录够了，接下来应该怎么做呢？一般有两种处理方法，第一种是直接停止当前录制，上报通知调度服务即可，这种处理方式比较简单粗暴，但是在安防和监控领域是不合适的。第二种是执行特定规则的覆盖策略，实现循环覆盖，始终保留最近24小时之内的视频画面内容。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c4/c4902cfc0565dc56c89009aff14061e7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对比上述两种处理方式，当发生录制超时时，第二种方式是最符合安防和监控领域的通用做法。那么覆盖策略又是怎么实现的呢，这就引出了下面的内容——覆盖策略。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4. 覆盖策略","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"覆盖策略在原理上理解起来很简单，但是具体执行时，就不那么简单了。首先，我们也先通过一个流程图对覆盖策略的处理逻辑有一个整体上的认识。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/da/da2f7a3ac34b46e9e1d1892ff486ad18.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.1 一级定时器","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当录制任务启动时，我们同时启动一个定时器（一级定时器），定时器的时长就是录制任务的目标时长，这个非常好理解。但是，这个定时器只生效一次或者一次都不生效。只有一级定时器生效后，才会启动二级定时器。如果一级定时器没有启动，那么二级定时器也不会启动。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们可以这样理解，只有一级定时器触发，录制服务才会执行对应的覆盖策略。当覆盖策略启动后，一级定时器销毁，二级定时器生效。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.2 二级定时器","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当文件时长达到了预设的最大时长时，我们将启动二级定时器。其实，二级定时器控制的是覆盖策略的删除频率，每次时间到了，就删除早些时候到录制文件分片。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.3 执行覆盖","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"具体覆盖的执行逻辑是，根据ts分片的时长和二级定时器的时间周期，计算需要删除的ts分片个数，同时更新m3u8中的索引列表，然后循环执行该策略，最终实现动态循环的录制覆盖策略。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/78/78745cd656da6c537fb81d4f8b6da66b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"覆盖策略的执行过程如上图所示，相信通过上文的解释，大家理解起来还是非常容易的。需要特别说明的是，由于二级定时器执行周期 t 的限制，录制文件的实际时长在最大录制时长 T 和（T+t）之间。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"结尾","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"好了，现在关于RTSP媒体流的录制方案和覆盖策略就介绍完了，相信大家对云端录制方案也有了一定认识，有自己想法和感兴趣的小伙伴，欢迎评论留言。关注我，分享更多音视频和流媒体服务器内容。","attrs":{}}]}]}