十二 h264 rtp包的時間戳
這次我們一起來分析一下live555中是怎樣爲rtp包打時間戳的．就以h264爲例吧．
void H264VideoRTPSink::doSpecialFrameHandling(unsigned /*fragmentationOffset*/,
unsigned char* /*frameStart*/,
unsigned /*numBytesInFrame*/,
struct timeval framePresentationTime,
unsigned /*numRemainingBytes*/)
{
// Set the RTP 'M' (marker) bit iff
// 1/ The most recently delivered fragment was the end of (or the only fragment of) an NAL unit, and
// 2/ This NAL unit was the last NAL unit of an 'access unit' (i.e. video frame).
if (fOurFragmenter != NULL) {
H264VideoStreamFramer* framerSource = (H264VideoStreamFramer*) (fOurFragmenter->inputSource());
// This relies on our fragmenter's source being a "H264VideoStreamFramer".
if (fOurFragmenter->lastFragmentCompletedNALUnit()
&& framerSource != NULL && framerSource->pictureEndMarker()) {
setMarkerBit();
framerSource->pictureEndMarker() = False;
}
}
setTimestamp(framePresentationTime);
}
函數中先檢測是否是一個幀的最後一個包，如果是，打上'M'標記．然後就設置時間戳．這個間戳是哪來的呢？需看函數doSpecialFrameHandling()是被誰調用的，經查找，是被MultiFramedRTPSink::afterGettingFrame1()調用的．MultiFramedRTPSink::afterGettingFrame1()的參數presentationTime傳給了doSpecialFrameHandling()．MultiFramedRTPSink::afterGettingFrame1()是在調用source的getNextFrame()時傳給了source．傳給哪個source呢？傳給了H264FUAFragmenter,還記得暗渡陳倉那件事嗎？所以H264FUAFragmenter在獲取一個nal
unit後調用了MultiFramedRTPSink::afterGettingFrame1()．也就是H264FUAFragmenter::afterGettingFrame1()調用了MultiFramedRTPSink::afterGettingFrame1()．
H264FUAFragmenter::afterGettingFrame1()是被它自己的source的afterGettingFrame1()調用的．H264FUAFragmenter的source是誰呢？是H264VideoStreamFramer，是在暗渡陳倉時傳給H264FUAFragmenter的構造函數的．
H264VideoStreamFramer的afterGettingFrame1()是沒有的，代替之的是MPEGVideoStreamFramer::continueReadProcessin()．它被MPEGVideoStreamParser暗中傳給了StreamParser的構造函數．所以StreamParser在分析完一幀（或nal unit)之後，調用的就是MPEGVideoStreamFramer::continueReadProcessin()．以下即是證明：(補充：以下函數並不是在parser分析完一幀（或nal
unit）之後調用，而是parser利用ByteStreamFileSuorce獲取到原始數據後調用，然後MPEGVideoStreamFramer再調用Parser的parser()函數分析原始數據)
void StreamParser::afterGettingBytes(void* clientData,
unsigned numBytesRead,
unsigned /*numTruncatedBytes*/,
struct timeval presentationTime,
unsigned /*durationInMicroseconds*/)
{
StreamParser* parser = (StreamParser*) clientData;
if (parser != NULL)
parser->afterGettingBytes1(numBytesRead, presentationTime);
}
void StreamParser::afterGettingBytes1(unsigned numBytesRead,
struct timeval presentationTime)
{
// Sanity check: Make sure we didn't get too many bytes for our bank:
if (fTotNumValidBytes + numBytesRead > BANK_SIZE) {
fInputSource->envir()
<< "StreamParser::afterGettingBytes() warning: read "
<< numBytesRead << " bytes; expected no more than "
<< BANK_SIZE - fTotNumValidBytes << "\n";
}
fLastSeenPresentationTime = presentationTime;
unsigned char* ptr = &curBank()[fTotNumValidBytes];
fTotNumValidBytes += numBytesRead;
// Continue our original calling source where it left off:
restoreSavedParserState();
// Sigh... this is a crock; things would have been a lot simpler
// here if we were using threads, with synchronous I/O...
fClientContinueFunc(fClientContinueClientData, ptr, numBytesRead,
presentationTime);
}
fClientContinueFunc就是MPEGVideoStreamFramer::continueReadProcessin()，而且我們看到時間戳被傳入fClientContinueFunc．
然而，MPEGVideoStreamFramer::continueReadProcessin()中跟本就不理這個時間戳，因爲這個時間戳是ByteStreamFileSource計算出來的，它跟本就不可能正確．
void MPEGVideoStreamFramer::continueReadProcessing(void* clientData,
unsigned char* /*ptr*/,
unsigned /*size*/,
struct timeval /*presentationTime*/)
{
MPEGVideoStreamFramer* framer = (MPEGVideoStreamFramer*) clientData;
framer->continueReadProcessing();
}
看來真正的時間戳是在MPEGVideoStreamFramer中計算的，但是H264VideoStreamFramer並沒有用到MPEGVideoStreamFramer中那些計算時間戳的函數，而是另外計算，其實H264VideoStreamFramer也沒有自己去計算，而是利用H264VideoStreamParser計算的．是在哪個函數中呢？在parser()中！
unsigned H264VideoStreamParser::parse()
{
try {
// The stream must start with a 0x00000001:
if (!fHaveSeenFirstStartCode) {
// Skip over any input bytes that precede the first 0x00000001:
u_int32_t first4Bytes;
while ((first4Bytes = test4Bytes()) != 0x00000001) {
get1Byte();
setParseState(); // ensures that we progress over bad data
}
skipBytes(4); // skip this initial code
setParseState();
fHaveSeenFirstStartCode = True; // from now on
}
if (fOutputStartCodeSize > 0) {
// Include a start code in the output:
save4Bytes(0x00000001);
}
// Then save everything up until the next 0x00000001 (4 bytes) or 0x000001 (3 bytes), or we hit EOF.
// Also make note of the first byte, because it contains the "nal_unit_type":
u_int8_t firstByte;
if (haveSeenEOF()) {
// We hit EOF the last time that we tried to parse this data,
// so we know that the remaining unparsed data forms a complete NAL unit:
unsigned remainingDataSize = totNumValidBytes() - curOffset();
if (remainingDataSize == 0)
(void) get1Byte(); // forces another read, which will cause EOF to get handled for real this time
if (remainingDataSize == 0)
return 0;
firstByte = get1Byte();
saveByte(firstByte);
while (--remainingDataSize > 0) {
saveByte(get1Byte());
}
} else {
u_int32_t next4Bytes = test4Bytes();
firstByte = next4Bytes >> 24;
while (next4Bytes != 0x00000001
&& (next4Bytes & 0xFFFFFF00) != 0x00000100) {
// We save at least some of "next4Bytes".
if ((unsigned) (next4Bytes & 0xFF) > 1) {
// Common case: 0x00000001 or 0x000001 definitely doesn't begin anywhere in "next4Bytes", so we save all of it:
save4Bytes(next4Bytes);
skipBytes(4);
} else {
// Save the first byte, and continue testing the rest:
saveByte(next4Bytes >> 24);
skipBytes(1);
}
next4Bytes = test4Bytes();
}
// Assert: next4Bytes starts with 0x00000001 or 0x000001, and we've saved all previous bytes (forming a complete NAL unit).
// Skip over these remaining bytes, up until the start of the next NAL unit:
if (next4Bytes == 0x00000001) {
skipBytes(4);
} else {
skipBytes(3);
}
}
u_int8_t nal_ref_idc = (firstByte & 0x60) >> 5;
u_int8_t nal_unit_type = firstByte & 0x1F;
switch (nal_unit_type) {
case 6: { // Supplemental enhancement information (SEI)
analyze_sei_data();
// Later, perhaps adjust "fPresentationTime" if we saw a "pic_timing" SEI payload??? #####
break;
}
case 7: { // Sequence parameter set
// First, save a copy of this NAL unit, in case the downstream object wants to see it:
usingSource()->saveCopyOfSPS(fStartOfFrame + fOutputStartCodeSize,
fTo - fStartOfFrame - fOutputStartCodeSize);
// Parse this NAL unit to check whether frame rate information is present:
unsigned num_units_in_tick, time_scale, fixed_frame_rate_flag;
analyze_seq_parameter_set_data(num_units_in_tick, time_scale,
fixed_frame_rate_flag);
if (time_scale > 0 && num_units_in_tick > 0) {
usingSource()->fFrameRate = time_scale
/ (2.0 * num_units_in_tick);
} else {
}
break;
}
case 8: { // Picture parameter set
// Save a copy of this NAL unit, in case the downstream object wants to see it:
usingSource()->saveCopyOfPPS(fStartOfFrame + fOutputStartCodeSize,
fTo - fStartOfFrame - fOutputStartCodeSize);
}
}
//更新時間戳變量
usingSource()->setPresentationTime();
// If this NAL unit is a VCL NAL unit, we also scan the start of the next NAL unit, to determine whether this NAL unit
// ends the current 'access unit'. We need this information to figure out when to increment "fPresentationTime".
// (RTP streamers also need to know this in order to figure out whether or not to set the "M" bit.)
Boolean thisNALUnitEndsAccessUnit = False; // until we learn otherwise
if (haveSeenEOF()) {
// There is no next NAL unit, so we assume that this one ends the current 'access unit':
thisNALUnitEndsAccessUnit = True;
} else {
Boolean const isVCL = nal_unit_type <= 5 && nal_unit_type > 0; // Would need to include type 20 for SVC and MVC #####
if (isVCL) {
u_int32_t first4BytesOfNextNALUnit = test4Bytes();
u_int8_t firstByteOfNextNALUnit = first4BytesOfNextNALUnit
>> 24;
u_int8_t next_nal_ref_idc = (firstByteOfNextNALUnit & 0x60)
>> 5;
u_int8_t next_nal_unit_type = firstByteOfNextNALUnit & 0x1F;
if (next_nal_unit_type >= 6) {
// The next NAL unit is not a VCL; therefore, we assume that this NAL unit ends the current 'access unit':
thisNALUnitEndsAccessUnit = True;
} else {
// The next NAL unit is also a VLC. We need to examine it a little to figure out if it's a different 'access unit'.
// (We use many of the criteria described in section 7.4.1.2.4 of the H.264 specification.)
Boolean IdrPicFlag = nal_unit_type == 5;
Boolean next_IdrPicFlag = next_nal_unit_type == 5;
if (next_IdrPicFlag != IdrPicFlag) {
// IdrPicFlag differs in value
thisNALUnitEndsAccessUnit = True;
} else if (next_nal_ref_idc != nal_ref_idc
&& next_nal_ref_idc * nal_ref_idc == 0) {
// nal_ref_idc differs in value with one of the nal_ref_idc values being equal to 0
thisNALUnitEndsAccessUnit = True;
} else if ((nal_unit_type == 1 || nal_unit_type == 2
|| nal_unit_type == 5)
&& (next_nal_unit_type == 1
|| next_nal_unit_type == 2
|| next_nal_unit_type == 5)) {
// Both this and the next NAL units begin with a "slice_header".
// Parse this (for each), to get parameters that we can compare:
// Current NAL unit's "slice_header":
unsigned frame_num, pic_parameter_set_id, idr_pic_id;
Boolean field_pic_flag, bottom_field_flag;
analyze_slice_header(
fStartOfFrame + fOutputStartCodeSize, fTo,
nal_unit_type, frame_num, pic_parameter_set_id,
idr_pic_id, field_pic_flag, bottom_field_flag);
// Next NAL unit's "slice_header":
u_int8_t next_slice_header[NUM_NEXT_SLICE_HEADER_BYTES_TO_ANALYZE];
testBytes(next_slice_header, sizeof next_slice_header);
unsigned next_frame_num, next_pic_parameter_set_id,
next_idr_pic_id;
Boolean next_field_pic_flag, next_bottom_field_flag;
analyze_slice_header(next_slice_header,
&next_slice_header[sizeof next_slice_header],
next_nal_unit_type, next_frame_num,
next_pic_parameter_set_id, next_idr_pic_id,
next_field_pic_flag, next_bottom_field_flag);
if (next_frame_num != frame_num) {
// frame_num differs in value
thisNALUnitEndsAccessUnit = True;
} else if (next_pic_parameter_set_id
!= pic_parameter_set_id) {
// pic_parameter_set_id differs in value
thisNALUnitEndsAccessUnit = True;
} else if (next_field_pic_flag != field_pic_flag) {
// field_pic_flag differs in value
thisNALUnitEndsAccessUnit = True;
} else if (next_bottom_field_flag
!= bottom_field_flag) {
// bottom_field_flag differs in value
thisNALUnitEndsAccessUnit = True;
} else if (next_IdrPicFlag == 1
&& next_idr_pic_id != idr_pic_id) {
// IdrPicFlag is equal to 1 for both and idr_pic_id differs in value
// Note: We already know that IdrPicFlag is the same for both.
thisNALUnitEndsAccessUnit = True;
}
}
}
}
}
//注意！注意！注意！此處計算時間戳！！
if (thisNALUnitEndsAccessUnit) {
usingSource()->fPictureEndMarker = True;
++usingSource()->fPictureCount;
// Note that the presentation time for the next NAL unit will be different:
struct timeval& nextPT = usingSource()->fNextPresentationTime; // alias
nextPT = usingSource()->fPresentationTime;
double nextFraction = nextPT.tv_usec / 1000000.0
+ 1 / usingSource()->fFrameRate;
unsigned nextSecsIncrement = (long) nextFraction;
nextPT.tv_sec += (long) nextSecsIncrement;
nextPT.tv_usec = (long) ((nextFraction - nextSecsIncrement)
* 1000000);
}
setParseState();
return curFrameSize();
} catch (int /*e*/) {
return 0; // the parsing got interrupted
}
}
每當開始一個新幀時，計算新的時間戳．時間戳保存在fNextPresentationTime中，在usingSource()->setPresentationTime()中傳給fPresentationTime.
哇，我們看到live555的類之間調用關係曲折複雜，的確有點不易維護啊！同時我寫的也不夠清析，自己看着都暈，如果把你搞暈了，這很正常哦！
fPresentationTime是64位的時間，經convertToRTPTimestamp轉換爲32的rtp時間戳，見函數：
u_int32_t RTPSink::convertToRTPTimestamp(struct timeval tv)
{
// Begin by converting from "struct timeval" units to RTP timestamp units:
u_int32_t timestampIncrement = (fTimestampFrequency * tv.tv_sec);
timestampIncrement += (u_int32_t)(
(2.0 * fTimestampFrequency * tv.tv_usec + 1000000.0) / 2000000);
// note: rounding
// Then add this to our 'timestamp base':
if (fNextTimestampHasBeenPreset) {
// Make the returned timestamp the same as the current "fTimestampBase",
// so that timestamps begin with the value that was previously preset:
fTimestampBase -= timestampIncrement;
fNextTimestampHasBeenPreset = False;
}
u_int32_t const rtpTimestamp = fTimestampBase + timestampIncrement;
return rtpTimestamp;
}
其實時間戳的轉換主要就是把以秒爲單位的時間，提升成按頻率爲單位的時間．也就是提升後，時間間隔不是以秒爲單位，而是以1/fTimestampFrequency爲單位，也就是1/9000秒。然後再強轉爲32。
