今天對VTM5.0中的亮度幀內預測的入口函數estIntraPredLumaQT()進行詳細的學習
estIntraPredLumaQT中完成了亮度分量的幀內預測，其主要流程如下：
一、初始化各種參數。
二、爲了減少最終RDcost的次數，降低編碼端的複雜度，VVC的VTM5.0版本默認使用幀內快速搜索算法，主要經過兩輪的SATD的粗選階段和一輪RDcost的細選階段
1、第一輪的SATD粗選，首先對35種非擴展的傳統的亮度幀內模式進行第一輪SATD循環粗選，這裏粗選的時候使用predIntraAng函數計算預測值，然後計算比較哈達瑪失真，取SAD和SATD（HAD）之中的最小值作爲最小的代價，然後調用updateCandList函數構建全率失真優化候選列表。全率失真優化候選列表長度numModesForFullRD由塊寬度決定。
2、第二輪的SATD粗選，如今的VTM5.0因爲引入了很多的新模式，因此整個第二輪的流程複雜了很多，如下：
①對第一輪後剩餘的擴展角度模式的SATD粗選，這裏粗選的時候仍然用使用predIntraAng函數計算預測值，然後計算比較哈達瑪失真，取SAD和SATD（HAD）之中的最小值作爲最小的代價，然後調用updateCandList函數構建全率失真優化候選列表。全率失真優化候選列表長度numModesForFullRD由塊寬度決定。
②對傳統的MPM列表進行SATD的粗選，選出最有可能的幾種模式加入到RD候選列表中去
③在兩輪傳統角度模式的SATD以及MPM的SATD結束以後，對使用哈達嗎的MIp模式單獨進
行候選模式的推導，關於Mip模式的技術細節我之前的博客已經講過啦，這裏就不做過多闡釋，直接給出鏈接H.266/VVC代碼學習筆記4：帶你深入解析VTM5.0中的最新技術ALWIP，在Mip的SATD之後這裏對RD候選列表CandHadList進行第一次縮減reduceHadCandList，也是爲了再次降低編碼端的複雜度。
④FastUDI最可能的MPM列表模式，這裏設置了一個numCand，定義MPM列表中最可能的模式數量,然後只對MPM中最可能的那幾個模式循環進行SATD粗選，其餘模式不進行，在一定程度上也節省了幾次SATD的比較和計算，然後更新RD候選列表CandHadList，這裏對於ISP模式的情況下還單獨進行了處理，在當前CU是ISP模式的時候，將不加入多參考情況的ISP模式添加到水平/垂直RD候選列表中，需要定義一個獨立的定義一個指示器列表listPointer，存入非MRL的ISP模式。對每一個ISP的分區獨立定義一個MPM列表和最可能的MPM候選模式的數量numCandISP，對每個候選的MPM模式循環遍歷進行STAD的粗選，如果遍歷到的模式不包括在最可能模式中，將ISP的最可能模式push到listPointer中。
⑤獨立的對Mip模式定義一個3MPM列表，對該Mip的MPM列表中的每一個模式循環遍歷SATD粗選，將最優的模式加入到RD候選列表CandHadList。關於Mip模式的MPM列表的構造具體的技術細節我在之前的博客已經講過了。這裏直接給出鏈接：H.266/VVC代碼學習筆記7：VTM5.0中Mip模式的MPM列表構造函數——getMipMPMs()
⑥重置RD候選模式列表的數量，然後進行第二次RD候選列表CandHadList的縮減，爲了排除可能性最低的那些模式，降低編碼端RDcost的複雜度。
⑦如果當前的CU是ISP模式，從ISP專屬的列表listPointer中移除非MPM模式，對每一個子分區的ISP列表進行循環重置操作。這一步操作之後，先不要去使用numModesForFullRD作爲最終的RD候選列表的大小。
⑧如果當前的CU是ISP模式，則創造一個完整的RD候選列表，其中包括使用常規intra、mrl和isp的所有內部模式。並且使用使用快速ISP模式，這裏定義了一個indexFirstMode變量，用來找到RD候選列表CandHadList中的第一個非MRL和非Mip模式。然後對RD候選模式列表中的每個模式進行循環遍歷處理，對RD候選列表CandHadList中的模式進行排序操作，具體操作步驟如下：
（1）將indexFirstMode指示的模式移到候選列表的開頭；
（2）然後在第一個非MRL模式之後插入所有的ISP模式
（3）在當前的列表的結尾插入所有的ISP模式
在經過了兩輪非常長的SATD的粗選過程終於到了最後一步的RDcost的細選階段：
3、第三輪的RDcost細選，check 所有的RD候選模式（使用RDcost）
①首先重新定義一個臨時的RD候選列表uiRdModeListTemp，先把非ISP且非MIP的模式放到重新定義的RD候選列表的前面，再把ISP或者MIP的模式放到重新定義的RD候選列表的後面，最後將臨時的RD候選列表uiRdModeListTemp中的模式按順序放入原來的RD候選模式列表uiRdModeList中，這一步的目的就是爲了給RD候選列表重新排序。
②在確保numModesForFullRD和RD候選列表數目相等的條件下，才進入到最終對於RD候選列表中的模式的進一步精選RDcost操作，這裏有一個大循環，對每個模式進行RDcost的遍歷，這裏調用xRecurIntraCodingLumaQT函數計算預測值以及殘差值，然後對殘差值進行變換量化並且計算重建值，最後計算率失真代價，注意在這裏該函數倒數第三個參數是mtsFirstCheckId=true，表示會按照四叉樹的方式繼續向下劃分。然後比較率失真代價，check每個模式的RD cost，逐次篩選出cost最小的模式，更新最優模式。最終遍歷結束，選出最優模式。
③最後收尾工作，判斷cost最優的模式是否是Mip模式以及是否使用多參考行，並將選出的最優的模式傳給intraDir，重置上下文模型。
下面是整個函數的所有代碼流程，我都加了詳細的註釋，方便大家參考和閱讀,這裏我建議大家把代碼放到VS中去看，因爲裏面有一些宏註釋的內容，在這裏看容易混淆，但在VS裏就很清楚：
bool IntraSearch::estIntraPredLumaQT( CodingUnit &cu, Partitioner &partitioner, const double bestCostSoFar, bool mtsCheckRangeFlag, int mtsFirstCheckId, int mtsLastCheckId, bool moreProbMTSIdxFirst )
{
CodingStructure &cs = *cu.cs;//獲取當前CU的編碼結構
const SPS &sps = *cs.sps;//獲取該編碼結構的語法元素
const uint32_t uiWidthBit = g_aucLog2[partitioner.currArea().lwidth() ];
const uint32_t uiHeightBit = g_aucLog2[partitioner.currArea().lheight()];
// Lambda calculation at equivalent Qp of 4 is recommended because at that Qp, the quantization divisor is 1.
//建議在等效qp爲4時進行lambda的計算，因爲在該qp時，用於量化的除數爲1。
const double sqrtLambdaForFirstPass = m_pcRdCost->getMotionLambda(cu.transQuantBypass) / double(1 << SCALE_BITS);
//===== loop over partitions =====
//循環分區
//上下玩模型的開始
const TempCtx ctxStart ( m_CtxCache, m_CABACEstimator->getCtx() );
#if JVET_N0217_MATRIX_INTRAPRED
//Mip模式的flag上下文模型
const TempCtx ctxStartMipFlag ( m_CtxCache, SubCtx( Ctx::MipFlag, m_CABACEstimator->getCtx() ) );
//Mip模式的上下文模型
const TempCtx ctxStartMipMode ( m_CtxCache, SubCtx( Ctx::MipMode, m_CABACEstimator->getCtx() ) );
#endif
//TSP模式的上下文模型
const TempCtx ctxStartIspMode ( m_CtxCache, SubCtx( Ctx::ISPMode, m_CABACEstimator->getCtx() ) );
#if JVET_N0185_UNIFIED_MPM
//Planar模式的flag上下文模型
const TempCtx ctxStartPlanarFlag ( m_CtxCache, SubCtx( Ctx::IntraLumaPlanarFlag, m_CABACEstimator->getCtx() ) );
#endif
//其餘幀內模式的上下文模型
const TempCtx ctxStartIntraMode(m_CtxCache, SubCtx(Ctx::IntraLumaMpmFlag, m_CABACEstimator->getCtx()));
//多參考行的索引
const TempCtx ctxStartMrlIdx ( m_CtxCache, SubCtx( Ctx::MultiRefLineIdx, m_CABACEstimator->getCtx() ) );
//檢查如果CU沒有對應的PU塊的則拋出異常
CHECK( !cu.firstPU, "CU has no PUs" );
const bool keepResi = cs.pps->getPpsRangeExtension().getCrossComponentPredictionEnabledFlag() || KEEP_PRED_AND_RESI_SIGNALS;
#if JVET_N0193_LFNST
// variables for saving fast intra modes scan results across multiple LFNST passes
bool LFNSTLoadFlag = sps.getUseLFNST() && cu.lfnstIdx != 0;
bool LFNSTSaveFlag = sps.getUseLFNST() && cu.lfnstIdx == 0;
LFNSTSaveFlag &= sps.getUseIntraMTS() ? cu.mtsFlag == 0 : true;
const uint32_t lfnstIdx = cu.lfnstIdx;
#endif
#if JVET_N0193_LFNST
//當前亮度CU的寬和高
const int width = partitioner.currArea().lwidth();
const int height = partitioner.currArea().lheight();
// Marking MTS usage for faster MTS
// 0: MTS is either not applicable for current CU (cuWidth > MTS_INTRA_MAX_CU_SIZE or cuHeight > MTS_INTRA_MAX_CU_SIZE), not active in the config file or the fast decision algorithm is not used in this case
// 1: MTS fast algorithm can be applied for the current CU, and the DCT2 is being checked
// 2: MTS is being checked for current CU. Stored results of DCT2 can be utilized for speedup
uint8_t mtsUsageFlag = 0;
const int maxSizeEMT = MTS_INTRA_MAX_CU_SIZE;
if( width <= maxSizeEMT && height <= maxSizeEMT && sps.getUseIntraMTS() )
{
mtsUsageFlag = ( sps.getUseLFNST() && cu.mtsFlag == 1 ) ? 2 : 1;
}
if( width * height < 64 && !m_pcEncCfg->getUseFastLFNST() )
{
mtsUsageFlag = 0;
}
#endif
#if JVET_N0193_LFNST
#if INCLUDE_ISP_CFG_FLAG
int nOptionsForISP = ( sps.getUseISP() && cu.mtsFlag == 0 && cu.lfnstIdx == 0 ) ? NUM_INTRA_SUBPARTITIONS_MODES : 1;
#endif
#endif
//定義一個當前最優的代價
double bestCurrentCost = bestCostSoFar;
int ispOptions[NUM_INTRA_SUBPARTITIONS_MODES] = { 0 };
if( nOptionsForISP > 1 )
{
#if MAX_TB_SIZE_SIGNALLING
#else
//ISP模式的劃分
auto splitsThatCanBeUsedForISP = CU::canUseISPSplit( width, height, MAX_TB_SIZEY );
#endif
if( splitsThatCanBeUsedForISP == CAN_USE_VER_AND_HORL_SPLITS )
{
const CodingUnit* cuLeft = cu.ispMode != NOT_INTRA_SUBPARTITIONS ? cs.getCU( cs.area.blocks[partitioner.chType].pos().offset( -1, 0 ), partitioner.chType ) : nullptr;
const CodingUnit* cuAbove = cu.ispMode != NOT_INTRA_SUBPARTITIONS ? cs.getCU( cs.area.blocks[partitioner.chType].pos().offset( 0, -1 ), partitioner.chType ) : nullptr;
bool ispHorIsFirstTest = CU::firstTestISPHorSplit( width, height, COMPONENT_Y, cuLeft, cuAbove );
if( ispHorIsFirstTest )
{
ispOptions[1] = HOR_INTRA_SUBPARTITIONS;
ispOptions[2] = VER_INTRA_SUBPARTITIONS;
}
else
{
ispOptions[1] = VER_INTRA_SUBPARTITIONS;
ispOptions[2] = HOR_INTRA_SUBPARTITIONS;
}
}
else if( splitsThatCanBeUsedForISP == HOR_INTRA_SUBPARTITIONS )
{
nOptionsForISP = 2;
ispOptions[1] = HOR_INTRA_SUBPARTITIONS;
}
else if( splitsThatCanBeUsedForISP == VER_INTRA_SUBPARTITIONS )
{
nOptionsForISP = 2;
ispOptions[1] = VER_INTRA_SUBPARTITIONS;
}
else
{
nOptionsForISP = 1;
}
}
if( nOptionsForISP > 1 )
{
//variables for the full RD list without MRL modes
m_rdModeListWithoutMrl .clear();
m_rdModeListWithoutMrlHor .clear();
m_rdModeListWithoutMrlVer .clear();
//variables with data from regular intra used to skip ISP splits
m_intraModeDiagRatio .clear();
m_intraModeHorVerRatio .clear();
m_intraModeTestedNormalIntra.clear();
}
#if JVET_N0413_RDPCM
#if JVET_N0193_LFNST
const bool testBDPCM = m_pcEncCfg->getRDPCM() && CU::bdpcmAllowed( cu, ComponentID( partitioner.chType ) ) && cu.mtsFlag == 0 && cu.lfnstIdx == 0;
#else
#endif
#endif
#if JVET_N0217_MATRIX_INTRAPRED
//哈達嗎變換（即SATD）粗選後的候選列表
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> uiHadModeList;
#else
#endif
//定義代價候選列表
static_vector<double, FAST_UDI_MAX_RDMODE_NUM> CandCostList;
//定義哈達嗎變換（SATD/HAD）後的候選代價列表
static_vector<double, FAST_UDI_MAX_RDMODE_NUM> CandHadList;
auto &pu = *cu.firstPU;
#if JVET_N0193_LFNST
bool validReturn = false;
#endif
{
CandHadList.clear();
CandCostList.clear();
uiHadModeList.clear();
#if !JVET_N0217_MATRIX_INTRAPRED
extendRefList.clear();
#endif
CHECK(pu.cu != &cu, "PU is not contained in the CU");
//===== determine set of modes to be tested (using prediction signal only) =====
/******************** 確定要測試的模式集和（僅使用預測信號）**************************/
int numModesAvailable = NUM_LUMA_MODE; // 所有67種傳統幀內模式的數量，不包括MIp模式，MIp模式另行處理
#if JVET_N0217_MATRIX_INTRAPRED
//加快速編碼版本的MIp模式
const bool fastMip = sps.getUseMIP() && m_pcEncCfg->getUseFastMIP();
#if JVET_N0193_LFNST
//Mip模式被允許
const bool mipAllowed = sps.getUseMIP() && ( cu.lfnstIdx == 0 ) && isLuma( partitioner.chType ) && pu.lwidth() <= MIP_MAX_WIDTH && pu.lheight() <= MIP_MAX_HEIGHT;
#endif
//是否是Mip模式的測試
const bool testMip = mipAllowed && mipModesAvailable( pu.Y() ) && !(fastMip && (cu.lwidth() > 2 * cu.lheight() || cu.lheight() > 2 * cu.lwidth()));
//定義一個不使用哈達嗎變換（即SAD）的RD候選列表
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> uiRdModeList;
#endif
/*這裏定義的就是能夠進入到最終的RDcost環節的最優的幾種模式的數量，numModesForFullRD初始化爲3*/
int numModesForFullRD = 3;
numModesForFullRD = g_aucIntraModeNumFast_UseMPM_2D[uiWidthBit - MIN_CU_LOG2][uiHeightBit - MIN_CU_LOG2];
#if JVET_N0193_LFNST
if( mtsUsageFlag != 2 )
#endif
{
// this should always be true
//如果當前PU無效就拋出異常
CHECK( !pu.Y().valid(), "PU is not valid" );
#if ENABLE_JVET_L0283_MRL
//是否是CTU的第一行
bool isFirstLineOfCtu = (((pu.block(COMPONENT_Y).y)&((pu.cs->sps)->getMaxCUWidth() - 1)) == 0);
//使用的擴展參考行的數量，如果爲CTU的第一行，則最多隻能用1行；如果不是，則最多可以用三行
int numOfPassesExtendRef = (isFirstLineOfCtu ? 1 : MRL_NUM_REF_LINES);
#endif
pu.multiRefIdx = 0;
if( numModesForFullRD != numModesAvailable )//numModesForFullRD的數量應該屬少於所有可用的幀內模式的數量的，因此這裏是主要的RD的入口
{
CHECK( numModesForFullRD >= numModesAvailable, "Too many modes for full RD search" );
const CompArea &area = pu.Y();//獲取當前亮度PU的區域
PelBuf piOrg = cs.getOrgBuf(area);//獲取當前亮度PU區域的亮度原始值
PelBuf piPred = cs.getPredBuf(area);//當前亮度PU區域的亮度預測值，爲期分配固定大小的內存空間
#if JVET_N0363_INTRA_COST_MOD
DistParam distParamSad;//定義一個使用SAD的失真參數
DistParam distParamHad;//定義一個使用SATD（加了哈達嗎變換）的失真參數
#endif
if (cu.slice->getReshapeInfo().getUseSliceReshaper() && m_pcReshape->getCTUFlag())
{
CompArea tmpArea(COMPONENT_Y, area.chromaFormat, Position(0, 0), area.size());
PelBuf tmpOrg = m_tmpStorageLCU.getBuf(tmpArea);
tmpOrg.copyFrom(piOrg);
tmpOrg.rspSignal(m_pcReshape->getFwdLUT());
#if JVET_N0363_INTRA_COST_MOD
//設置失真參數
m_pcRdCost->setDistParam(distParamSad, tmpOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y, false); // Use SAD cost
m_pcRdCost->setDistParam(distParamHad, tmpOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y, true); // Use HAD (SATD) cost
#endif
}
else
#if JVET_N0363_INTRA_COST_MOD
{
m_pcRdCost->setDistParam(distParamSad, piOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y, false); // Use SAD cost
m_pcRdCost->setDistParam(distParamHad, piOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y, true); // Use HAD (SATD) cost
}
#endif
#if JVET_N0363_INTRA_COST_MOD
distParamSad.applyWeight = false;
distParamHad.applyWeight = false;
#endif
#if JVET_N0217_MATRIX_INTRAPRED
if( testMip)
{
numModesForFullRD += fastMip? std::max(2, g_aucLog2[std::min(pu.lwidth(), pu.lheight())] - 1) : numModesForFullRD;
}
//定義SATD變換候選的數量
const int numHadCand = (testMip ? 2 : 1) * 3;//是Mip模式爲6，非Mip模式爲3
//*** Derive (regular) candidates using Hadamard
//使用哈達嗎變換的推導角度候選（即使用SATD）
cu.mipFlag = false;
//===== init pattern for luma prediction =====
//初始化亮度預測模式
initIntraPatternChType(cu, pu.Y(), true);
#endif
bool bSatdChecked[NUM_INTRA_MODE];//定義一個存放已經被STAD粗選過的幀內模式的bool數組
memset( bSatdChecked, 0, sizeof( bSatdChecked ) );
#if JVET_N0193_LFNST
if( !LFNSTLoadFlag )
#endif
{
//首先對35種非擴展的傳統的亮度幀內模式進行第一輪SATD循環粗選
//0.1.2.4.6.8................50.............66
for( int modeIdx = 0; modeIdx < numModesAvailable; modeIdx++ )
{
uint32_t uiMode = modeIdx;
#if JVET_N0363_INTRA_COST_MOD
Distortion minSadHad = 0;
#endif
// Skip checking extended Angular modes in the first round of SATD
//這裏明確說要跳過擴展的角度模式的SATD粗選
if( uiMode > DC_IDX && ( uiMode & 1 ) )
{
continue;
}
bSatdChecked[uiMode] = true;
//intraDir中存放的就是當前CU最終選中的預測模式
pu.intraDir[0] = modeIdx;
initPredIntraParams(pu, pu.Y(), sps);
if( useDPCMForFirstPassIntraEstimation( pu, uiMode ) )
{
encPredIntraDPCM( COMPONENT_Y, piOrg, piPred, uiMode );
}
else
{
//傳統角度預測模式的函數入口
predIntraAng( COMPONENT_Y, piPred, pu);
}
#if JVET_N0363_INTRA_COST_MOD
// Use the min between SAD and HAD as the cost criterion
//使用SAD和HAD之間的最小值作爲代價標準(HAD其實就是SATD，是加了哈達嗎變換後的SAD)
// SAD is scaled by 2 to align with the scaling of HAD
//SAD的縮放比例爲2，與HAD的縮放比例一致。
minSadHad += std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif
// NB xFracModeBitsIntra will not affect the mode for chroma that may have already been pre-estimated.
//
#if JVET_N0217_MATRIX_INTRAPRED
//獲取Mip模式flag的上下文模型
m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
#endif
//獲取ISP模式的上下文模型
m_CABACEstimator->getCtx() = SubCtx( Ctx::ISPMode, ctxStartIspMode );
#if JVET_N0185_UNIFIED_MPM
//獲取亮度Planar模式的上下文模型
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
//獲取亮度MPM列表的上下文模型
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
//獲取多參考行的上下文模型
m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );
uint64_t fracModeBits = xFracModeBitsIntra(pu, uiMode, CHANNEL_TYPE_LUMA);
#if JVET_N0363_INTRA_COST_MOD
//計算每種模式的Cost，比較相互之間的Cost進行模式更新
double cost = ( double ) minSadHad + (double)fracModeBits * sqrtLambdaForFirstPass;
DTRACE(g_trace_ctx, D_INTRA_COST, "IntraHAD: %u, %llu, %f (%d)\n", minSadHad, fracModeBits, cost, uiMode);
#endif
#if JVET_N0217_MATRIX_INTRAPRED
//更新SAD之後的RD候選模式列表以及代價列表
updateCandList( ModeInfo(false, 0, NOT_INTRA_SUBPARTITIONS, uiMode), cost, uiRdModeList, CandCostList, numModesForFullRD );
#if JVET_N0363_INTRA_COST_MOD
//更新傳統幀內預測模式的候選模式列表
//就是更新SATD之後的uiHadModeList、CandHadList這些列表
updateCandList( ModeInfo(false, 0, NOT_INTRA_SUBPARTITIONS, uiMode), (double)minSadHad, uiHadModeList, CandHadList, numHadCand );
#endif
#endif
}
#if JVET_N0193_LFNST
if( LFNSTSaveFlag )
{
// save found best modes
//保存發現的最優的模式
m_uiSavedNumRdModesLFNST = numModesForFullRD;//第一輪SAD粗選後選中的相對最優模式的數量
m_uiSavedRdModeListLFNST = uiRdModeList;//被保存下來的RD模式的列表
m_dSavedModeCostLFNST = CandCostList;//被保存下來的候選代價的列表
// PBINTRA fast
m_uiSavedHadModeListLFNST = uiHadModeList;
m_dSavedHadListLFNST = CandHadList;
LFNSTSaveFlag = false;
}
#endif
} // NSSTFlag
#if JVET_N0193_LFNST
else
{
//restore saved modes
numModesForFullRD = m_uiSavedNumRdModesLFNST;
uiRdModeList = m_uiSavedRdModeListLFNST;
CandCostList = m_dSavedModeCostLFNST;
//PBINTRA fast
uiHadModeList = m_uiSavedHadModeListLFNST;
CandHadList = m_dSavedHadListLFNST;
LFNSTLoadFlag = false;
} // !LFNSTFlag
#endif
#if JVET_N0217_MATRIX_INTRAPRED
//再次檢查uiRdModeList的大小是否和numModesForFullRD相等，不相等則拋出異常
CHECK( uiRdModeList.size() != numModesForFullRD, "Error: RD mode list size" );
//將第一輪選出來的最優模式放到父模式中
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> parentCandList = uiRdModeList;
#endif
// Second round of SATD for extended Angular modes
//這裏開始第二輪對剩下的擴展角度模式的SATD粗選
for (int modeIdx = 0; modeIdx < numModesForFullRD; modeIdx++)
{
#if JVET_N0217_MATRIX_INTRAPRED
//定義父模式，父模式放入的是第一輪選出來的最優的模式列表
unsigned parentMode = parentCandList[modeIdx].modeId;
#endif
//從擴展模式開始循環處理
//模式號是從3-65
if (parentMode > (DC_IDX + 1) && parentMode < (NUM_LUMA_MODE - 1))
{
for (int subModeIdx = -1; subModeIdx <= 1; subModeIdx += 2)
{
unsigned mode = parentMode + subModeIdx;//給父模式減一或者加一，正好就是上一輪那些最優候選列表中非擴展角度模式的左右相鄰的擴展角度模式
if (!bSatdChecked[mode])
{
pu.intraDir[0] = mode;
initPredIntraParams(pu, pu.Y(), sps);
if (useDPCMForFirstPassIntraEstimation(pu, mode))
{
encPredIntraDPCM(COMPONENT_Y, piOrg, piPred, mode);
}
else
{
predIntraAng(COMPONENT_Y, piPred, pu );
}
#if JVET_N0363_INTRA_COST_MOD
// Use the min between SAD and SATD as the cost criterion
//使用SAD和SATD之間的最小值作爲代價標準
// SAD is scaled by 2 to align with the scaling of HAD
Distortion minSadHad = std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif
// NB xFracModeBitsIntra will not affect the mode for chroma that may have already been pre-estimated.
#if JVET_N0217_MATRIX_INTRAPRED
m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
#endif
m_CABACEstimator->getCtx() = SubCtx( Ctx::ISPMode, ctxStartIspMode );
#if JVET_N0185_UNIFIED_MPM
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );
uint64_t fracModeBits = xFracModeBitsIntra(pu, mode, CHANNEL_TYPE_LUMA);
#if JVET_N0363_INTRA_COST_MOD
//計算第二輪SATD每一種模式的cost
double cost = (double) minSadHad + (double) fracModeBits * sqrtLambdaForFirstPass;
#endif
#if JVET_N0217_MATRIX_INTRAPRED
updateCandList( ModeInfo( false, 0, NOT_INTRA_SUBPARTITIONS, mode ), cost, uiRdModeList, CandCostList, numModesForFullRD );
#if JVET_N0363_INTRA_COST_MOD
//更新模式候選列表，將最優的擴展角度模式加到模式列表中
updateCandList( ModeInfo( false, 0, NOT_INTRA_SUBPARTITIONS, mode ), (double)minSadHad, uiHadModeList, CandHadList, numHadCand );
#endif
#endif
bSatdChecked[mode] = true;
}
}
}
}
if( nOptionsForISP > 1 )
{
//we save the list with no mrl modes to keep only the Hadamard selected modes (no mpms)
#if JVET_N0217_MATRIX_INTRAPRED
//在不是MRL模式時將兩輪SATD後更新好的列表保存下來，存到m_rdModeListWithoutMrl中（沒有MPM模式）
m_rdModeListWithoutMrl = uiRdModeList;
#endif
}
#if ENABLE_JVET_L0283_MRL
pu.multiRefIdx = 1;//使用1參考行
const int numMPMs = NUM_MOST_PROBABLE_MODES;//定義最可能模式的數量，即MPM列表的大小
unsigned multiRefMPM [numMPMs];//定義多參考行的MPM列表
PU::getIntraMPMs(pu, multiRefMPM);//獲取該MPM列表的函數入口
//對每個擴展的參考行進行循環
for (int mRefNum = 1; mRefNum < numOfPassesExtendRef; mRefNum++)
{
//定義多參考行的索引，不同參考行的MPM列表是不同的
int multiRefIdx = MULTI_REF_LINE_IDX[mRefNum];
pu.multiRefIdx = multiRefIdx;
{
initIntraPatternChType(cu, pu.Y(), true);
}
#if JVET_N0185_UNIFIED_MPM
//對MPM列表中的6個模式進行SATD比較選擇
for (int x = 1; x < numMPMs; x++)
#endif
{
uint32_t mode = multiRefMPM[x];
{
pu.intraDir[0] = mode;
initPredIntraParams(pu, pu.Y(), sps);
if (useDPCMForFirstPassIntraEstimation(pu, mode))
{
encPredIntraDPCM(COMPONENT_Y, piOrg, piPred, mode);
}
else
{
predIntraAng(COMPONENT_Y, piPred, pu);
}
#if JVET_N0363_INTRA_COST_MOD
// Use the min between SAD and SATD as the cost criterion
// SAD is scaled by 2 to align with the scaling of HAD
Distortion minSadHad = std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif
// NB xFracModeBitsIntra will not affect the mode for chroma that may have already been pre-estimated.
#if JVET_N0217_MATRIX_INTRAPRED
m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
#endif
m_CABACEstimator->getCtx() = SubCtx( Ctx::ISPMode, ctxStartIspMode );
#if JVET_N0185_UNIFIED_MPM
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
#if !JVET_N0302_SIMPLFIED_CIIP
m_CABACEstimator->getCtx() = SubCtx( Ctx::MHIntraPredMode, ctxStartMHIntraMode );
#endif
m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );
uint64_t fracModeBits = xFracModeBitsIntra(pu, mode, CHANNEL_TYPE_LUMA);
#if JVET_N0363_INTRA_COST_MOD
double cost = (double)minSadHad + (double)fracModeBits * sqrtLambdaForFirstPass;
#endif
#if JVET_N0217_MATRIX_INTRAPRED
updateCandList( ModeInfo( false, multiRefIdx, NOT_INTRA_SUBPARTITIONS, mode ), cost, uiRdModeList, CandCostList, numModesForFullRD );
#if JVET_N0363_INTRA_COST_MOD
//更新候選列表
updateCandList( ModeInfo( false, multiRefIdx, NOT_INTRA_SUBPARTITIONS, mode ), (double)minSadHad, uiHadModeList, CandHadList, numHadCand );
#endif
#else
#endif
}
}
}
#endif
#if JVET_N0217_MATRIX_INTRAPRED
CHECKD( uiRdModeList.size() != numModesForFullRD, "Error: RD mode list size" );
//*** Derive MIP candidates using Hadamard
//在兩輪傳統角度模式的SATD以及MPM的SATD結束以後，對使用哈達嗎的MIp模式單獨進行候選模式的推導
if (testMip)
{
cu.mipFlag = true;
pu.multiRefIdx = 0;
//初始化Mip模式
initIntraMip( pu );
//getNumModesMip函數根據當前CU的塊尺寸獲取不同的Mip模式數量
for (uint32_t uiMode = 0; uiMode < getNumModesMip(pu.Y()); uiMode++)
{
pu.intraDir[CHANNEL_TYPE_LUMA] = uiMode;
//Mip預測模式的函數入口
predIntraMip(COMPONENT_Y, piPred, pu);
#if JVET_N0363_INTRA_COST_MOD
// Use the min between SAD and HAD as the cost criterion
// SAD is scaled by 2 to align with the scaling of HAD
Distortion minSadHad = std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif
m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
m_CABACEstimator->getCtx() = SubCtx( Ctx::MipMode, ctxStartMipMode );
uint64_t fracModeBits = xFracModeBitsIntra(pu, uiMode, CHANNEL_TYPE_LUMA);
#if JVET_N0363_INTRA_COST_MOD
double cost = double(minSadHad) + double(fracModeBits) * sqrtLambdaForFirstPass;
#endif
//更新候選列表，將最優的Mip模式加入到RDCost的候選列表中
updateCandList(ModeInfo(true, 0, NOT_INTRA_SUBPARTITIONS, uiMode), cost, uiRdModeList, CandCostList, numModesForFullRD);
#if JVET_N0363_INTRA_COST_MOD
updateCandList(ModeInfo(true, 0, NOT_INTRA_SUBPARTITIONS, uiMode), double(minSadHad), uiHadModeList, CandHadList, numHadCand);
#endif
}
//在總共四輪的STAD之後，最終更新好RDcost列表後，還要對該列表進行一個縮減的處理
const double thresholdHadCost = 1.0 + 1.4 / sqrt((double)(pu.lwidth()*pu.lheight()));//計算代價的閾值
reduceHadCandList(uiRdModeList, CandCostList, numModesForFullRD, thresholdHadCost, 0.0);//縮減列表的函數入口
}
#endif
//獲取FastUDI可能的MPM列表，如果滿足條件也將其加入RD候選列表
if( m_pcEncCfg->getFastUDIUseMPMEnabled() )
{
const int numMPMs = NUM_MOST_PROBABLE_MODES;
unsigned uiPreds[numMPMs];
pu.multiRefIdx = 0;
const int numCand = PU::getIntraMPMs( pu, uiPreds );//定義MPM列表中最可能的模式數量,然後只對MPM中最可能的那幾個模式進行SATD粗選，其餘模式不進行
for( int j = 0; j < numCand; j++ )
{
bool mostProbableModeIncluded = false;
#if JVET_N0217_MATRIX_INTRAPRED
ModeInfo mostProbableMode( false, 0, NOT_INTRA_SUBPARTITIONS, uiPreds[j] );
#else
int mostProbableMode = uiPreds[j];
#endif
for( int i = 0; i < numModesForFullRD; i++ )
{
#if JVET_N0217_MATRIX_INTRAPRED
mostProbableModeIncluded |= ( mostProbableMode == uiRdModeList[i] );//如果不包括在最可能模式中，則將其放入RD候選列表中
#else
mostProbableModeIncluded |= (mostProbableMode == uiRdModeList[i] && extendRefList[i] == 0);
#endif
}
if( !mostProbableModeIncluded )
{
#if !JVET_N0217_MATRIX_INTRAPRED
extendRefList.push_back(0);
#endif
numModesForFullRD++;
uiRdModeList.push_back( mostProbableMode );
#if JVET_N0217_MATRIX_INTRAPRED
CandCostList.push_back(0);
#endif
}
}
if( nOptionsForISP > 1 )
{
//we add the ISP MPMs to the list without mrl modes
//將不加入多參考情況的ISP模式添加到RD候選列表中
m_rdModeListWithoutMrlHor = m_rdModeListWithoutMrl;
m_rdModeListWithoutMrlVer = m_rdModeListWithoutMrl;
#if JVET_N0217_MATRIX_INTRAPRED
for (int k = 0; k < m_rdModeListWithoutMrl.size(); k++)
{
m_rdModeListWithoutMrlHor[k].ispMod = HOR_INTRA_SUBPARTITIONS;
m_rdModeListWithoutMrlVer[k].ispMod = VER_INTRA_SUBPARTITIONS;
}
//定義一個指示器列表，存入非MRL的ISP模式
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM>* listPointer;
#endif
//對每一個ISP的分區進行STAD的粗選
for( int k = 1; k < nOptionsForISP; k++ )
{
cu.ispMode = ispOptions[k];
listPointer = &( cu.ispMode == HOR_INTRA_SUBPARTITIONS ? m_rdModeListWithoutMrlHor : m_rdModeListWithoutMrlVer );
const int numCandISP = PU::getIntraMPMs( pu, uiPreds );//這裏再定義一個針對ISP模式的最可能的MPM模式數量
for( int j = 0; j < numCandISP; j++ )
{
bool mostProbableModeIncluded = false;
#if JVET_N0217_MATRIX_INTRAPRED
ModeInfo mostProbableMode( false, 0, cu.ispMode, uiPreds[j] );
#endif
for( int i = 0; i < listPointer->size(); i++ )
{
mostProbableModeIncluded |= ( mostProbableMode == listPointer->at( i ) );
}
if( !mostProbableModeIncluded )
{
listPointer->push_back( mostProbableMode );//如果該模式不包括在最可能模式中，將ISP的最可能模式push到listPointer中
}
}
}
cu.ispMode = NOT_INTRA_SUBPARTITIONS;
}
}
#if JVET_N0217_MATRIX_INTRAPRED
//*** Add MPMs for MIP to candidate list
//將Mip模式的MPM列表添加到RD候選列表中
if (!fastMip && testMip && pu.lwidth() < 8 && pu.lheight() < 8)//快速算法的Mip不參與其中
{
//定義一個Mip模式專屬的MPM列表，大小爲3
unsigned mpm[NUM_MPM_MIP];
//獲取當前PU的Mip模式的MPM列表
int numCandMip = PU::getMipMPMs(pu, mpm);
for( int j = 0; j < numCandMip; j++ )
{
bool mostProbableModeIncluded = false;
//判斷MPM列表中的模式是否包括在最可能模式之中，如果不包括，則將該模式加入到RD候選列表中
ModeInfo mostProbableMode(true, 0, NOT_INTRA_SUBPARTITIONS, mpm[j]);
for( int i = 0; i < numModesForFullRD; i++ )
{
mostProbableModeIncluded |= (mostProbableMode == uiRdModeList[i]);
}
if( !mostProbableModeIncluded )
{
numModesForFullRD++;
uiRdModeList.push_back( mostProbableMode );
CandCostList.push_back(0);
}
}
}
#endif
}
else
{
#if JVET_N0217_MATRIX_INTRAPRED
THROW( "Full search not supported for MIP" );
#endif
}
#if JVET_N0193_LFNST
if( sps.getUseLFNST() && mtsUsageFlag == 1 )
{
// Store the modes to be checked with RD
//保存最終用於RDCost的候選模式數量
m_savedNumRdModes[ lfnstIdx ] = numModesForFullRD;
std::copy_n( uiRdModeList.begin(), numModesForFullRD, m_savedRdModeList[ lfnstIdx ] );
}
#endif
}
#if JVET_N0193_LFNST
else //mtsUsage = 2 (here we potentially reduce the number of modes that will be full-RD checked)
{
//如果使用幀內快速LFNST
if( m_pcEncCfg->getUseFastLFNST() || !cu.slice->isIntra() )
{
//重置RD候選模式列表的數量，下面進行第二次RD候選列表的縮減
numModesForFullRD = 0;
//設置一個跳過模式閾值
double thresholdSkipMode = 1.0 + ( ( cu.lfnstIdx > 0 ) ? 0.1 : 1.0 ) * ( 1.4 / sqrt( ( double ) ( width*height ) ) );
//Skip checking the modes with much larger R-D cost than the best mode
//對所有RD候選模式進行循環處理，但是跳過檢查比最佳模式代價大得多的R-D模式，不進行RDcost檢查
for( int i = 0; i < m_savedNumRdModes[ lfnstIdx ]; i++ )
{
//如果該模式的代價小於閾值，則將其保留在RD候選模式列表中，否則將其排除
if( m_modeCostStore[ lfnstIdx ][ i ] <= thresholdSkipMode * m_bestModeCostStore[ lfnstIdx ] )
{
uiRdModeList.push_back( m_savedRdModeList[ lfnstIdx ][ i ] );
//RD候選模式列表的數量加一
numModesForFullRD++;
}
}
}
else //this is necessary because we skip the candidates list calculation, since it was already obtained for the DCT-II. Now we load it
//這裏的操作是必要的，因爲已經從DCT-2中獲得了計算數據，因此這裏跳過了候選列表的計算，現在我們加載它
{
// Restore the modes to be checked with RD
// 重新保存用於RD的候選模式數量
numModesForFullRD = m_savedNumRdModes[ lfnstIdx ];
//重置RD候選列表的大小
uiRdModeList.resize( numModesForFullRD );
std::copy_n( m_savedRdModeList[ lfnstIdx ], m_savedNumRdModes[ lfnstIdx ], uiRdModeList.begin() );
CandCostList.resize( numModesForFullRD );
}
}
#endif
if( nOptionsForISP > 1 ) // we remove the non-MPMs from the ISP lists
//從ISP列表中移除非MPM模式
{
#if JVET_N0217_MATRIX_INTRAPRED
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> uiRdModeListCopyHor = m_rdModeListWithoutMrlHor;
m_rdModeListWithoutMrlHor.clear();
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> uiRdModeListCopyVer = m_rdModeListWithoutMrlVer;
m_rdModeListWithoutMrlVer.clear();
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> *listPointerCopy, *listPointer;
#endif
for( int ispOptionIdx = 1; ispOptionIdx < nOptionsForISP; ispOptionIdx++ )
{
cu.ispMode = ispOptions[ispOptionIdx];
//we get the mpm cand list
//獲取對於ISP模式的MPM的候選列表
const int numMPMs = NUM_MOST_PROBABLE_MODES;
unsigned uiPreds[numMPMs];
pu.multiRefIdx = 0;
PU::getIntraMPMs( pu, uiPreds );
//we copy only the ISP MPMs
//僅僅複製ISP的MPM候選模式
listPointerCopy = &( cu.ispMode == HOR_INTRA_SUBPARTITIONS ? uiRdModeListCopyHor : uiRdModeListCopyVer );
listPointer = &( cu.ispMode == HOR_INTRA_SUBPARTITIONS ? m_rdModeListWithoutMrlHor : m_rdModeListWithoutMrlVer );
//對ISP列表的重置的循環操作
for( int k = 0; k < listPointerCopy->size(); k++ )
{
for( int q = 0; q < numMPMs; q++ )
{
#if JVET_N0217_MATRIX_INTRAPRED
if (listPointerCopy->at(k) == ModeInfo( false, 0, cu.ispMode, uiPreds[q] ))
#endif
{
listPointer->push_back( listPointerCopy->at( k ) );
break;
}
}
}
}
cu.ispMode = NOT_INTRA_SUBPARTITIONS;
}
CHECK( numModesForFullRD != uiRdModeList.size(), "Inconsistent state!" );
// after this point, don't use numModesForFullRD
//這一步操作之後，先不要去使用numModesForFullRD作爲最終的RD候選列表的大小
// PBINTRA fast快速幀內模式
#if JVET_N0193_LFNST
#if JVET_N0329_IBC_SEARCH_IMP
if( m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable && !cs.slice->getDisableSATDForRD() && ( mtsUsageFlag != 2 || lfnstIdx > 0 ) )
#else
if( m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable && ( mtsUsageFlag != 2 || lfnstIdx > 0 ) )
#endif
#else
#if JVET_N0329_IBC_SEARCH_IMP
if (m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable && !cs.slice->getDisableSATDForRD())
#else
if( m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable )
#endif
#endif
{
#if JVET_N0217_MATRIX_INTRAPRED
#if JVET_N0193_LFNST
double pbintraRatio = (lfnstIdx > 0) ? 1.25 : PBINTRA_RATIO;
#endif
int maxSize = -1;
const int numHadCand = (testMip ? 2 : 1) * 3;
for (int k = numHadCand - 1; k >= 0; k--)
{
#if JVET_N0193_LFNST
if (CandHadList.size() < (k + 1) || CandHadList[k] > cs.interHad * pbintraRatio) { maxSize = k; }
#else
if (CandHadList.size() < (k + 1) || CandHadList[k] > cs.interHad * PBINTRA_RATIO) { maxSize = k; }
#endif
}
if (maxSize > 0)
{
uiRdModeList.resize(std::min<size_t>(uiRdModeList.size(), maxSize));
if (nOptionsForISP > 1)
{
m_rdModeListWithoutMrlHor.resize(std::min<size_t>(m_rdModeListWithoutMrlHor.size(), maxSize));
m_rdModeListWithoutMrlVer.resize(std::min<size_t>(m_rdModeListWithoutMrlVer.size(), maxSize));
}
}
if (maxSize == 0)
{
cs.dist = std::numeric_limits<Distortion>::max();
cs.interHad = 0;
//===== reset context models =====
m_CABACEstimator->getCtx() = SubCtx(Ctx::MipFlag, ctxStartMipFlag);
m_CABACEstimator->getCtx() = SubCtx(Ctx::MipMode, ctxStartMipMode);
m_CABACEstimator->getCtx() = SubCtx(Ctx::ISPMode, ctxStartIspMode);
#if JVET_N0185_UNIFIED_MPM
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
#if !JVET_N0302_SIMPLFIED_CIIP
m_CABACEstimator->getCtx() = SubCtx(Ctx::MHIntraPredMode, ctxStartMHIntraMode);
#endif
m_CABACEstimator->getCtx() = SubCtx(Ctx::MultiRefLineIdx, ctxStartMrlIdx);
#if JVET_N0193_LFNST
return false;
#else
return;
#endif
}
#else
#if JVET_N0193_LFNST
double pbintraRatio = ( lfnstIdx > 0 ) ? 1.25 : PBINTRA_RATIO;
if( CandHadList.size() < 3 || CandHadList[ 2 ] > cs.interHad * pbintraRatio )
#else
if( CandHadList.size() < 3 || CandHadList[2] > cs.interHad * PBINTRA_RATIO )
#endif
{
uiRdModeList.resize( std::min<size_t>( uiRdModeList.size(), 2 ) );
#if !JVET_N0217_MATRIX_INTRAPRED
extendRefList.resize( std::min<size_t>( extendRefList.size(), 2 ) );
#endif
if( nOptionsForISP > 1 )
{
m_rdModeListWithoutMrlHor.resize( std::min<size_t>( m_rdModeListWithoutMrlHor.size(), 2 ) );
m_rdModeListWithoutMrlVer.resize( std::min<size_t>( m_rdModeListWithoutMrlVer.size(), 2 ) );
}
}
#if JVET_N0193_LFNST
if( CandHadList.size() < 2 || CandHadList[ 1 ] > cs.interHad * pbintraRatio )
#else
if( CandHadList.size() < 2 || CandHadList[1] > cs.interHad * PBINTRA_RATIO )
#endif
{
uiRdModeList.resize( std::min<size_t>( uiRdModeList.size(), 1 ) );
#if !JVET_N0217_MATRIX_INTRAPRED
extendRefList.resize( std::min<size_t>( extendRefList.size(), 1 ) );
#endif
if( nOptionsForISP > 1 )
{
m_rdModeListWithoutMrlHor.resize( std::min<size_t>( m_rdModeListWithoutMrlHor.size(), 1 ) );
m_rdModeListWithoutMrlVer.resize( std::min<size_t>( m_rdModeListWithoutMrlVer.size(), 1 ) );
}
}
#if JVET_N0193_LFNST
if( CandHadList.size() < 1 || CandHadList[ 0 ] > cs.interHad * pbintraRatio )
#else
if( CandHadList.size() < 1 || CandHadList[0] > cs.interHad * PBINTRA_RATIO )
#endif
{
cs.dist = std::numeric_limits<Distortion>::max();
cs.interHad = 0;
//===== reset context models =====
m_CABACEstimator->getCtx() = SubCtx(Ctx::ISPMode, ctxStartIspMode);
#if JVET_N0185_UNIFIED_MPM
m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
m_CABACEstimator->getCtx() = SubCtx( Ctx::IntraLumaMpmFlag, ctxStartIntraMode );
#if !JVET_N0302_SIMPLFIED_CIIP
m_CABACEstimator->getCtx() = SubCtx( Ctx::MHIntraPredMode, ctxStartMHIntraMode );
#endif
m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );
#if JVET_N0193_LFNST
return false;
#else
return;
#endif
}
#endif
}
if ( nOptionsForISP > 1 )
{
//we create a single full RD list that includes all intra modes using regular intra, MRL and ISP
//創造一個完整的RD候選列表，其中包括使用常規intra、mrl和isp的所有內部模式。
auto* firstIspList = ispOptions[1] == HOR_INTRA_SUBPARTITIONS ? &m_rdModeListWithoutMrlHor : &m_rdModeListWithoutMrlVer;
auto* secondIspList = ispOptions[1] == HOR_INTRA_SUBPARTITIONS ? &m_rdModeListWithoutMrlVer : &m_rdModeListWithoutMrlHor;
#if JVET_N0193_LFNST
if( !sps.getUseLFNST() && m_pcEncCfg->getUseFastISP() )//使用快速ISP模式
#else
if ( m_pcEncCfg->getUseFastISP() )
#endif
{
#if JVET_N0217_MATRIX_INTRAPRED
CHECKD( uiRdModeList.size() > CandCostList.size(), "Error: CandCostList size" );
// find the first non-MRL, non-MIP mode
//找到第一個非MRL和非Mip模式
int indexFirstMode = int(uiRdModeList.size()) - 1; // default is last mode默認是最後一個模式
//對RD候選模式列表中的每個模式進行循環遍歷處理
for (int k = 0; k < int(uiRdModeList.size()); k++)
{
if (uiRdModeList[k].mRefId == 0 && uiRdModeList[k].mipFlg == false)
{
indexFirstMode = k;
break;
}
}
// move the mode indicated by indexFirstMode to the beginning
//將indexFirstMode指示的模式移到候選列表的開頭
for (int idx = indexFirstMode - 1; idx >= 0; idx--)
{
std::swap(uiRdModeList[idx], uiRdModeList[idx + 1]);
std::swap(CandCostList[idx], CandCostList[idx + 1]);
}
//insert all ISP modes after the first non-mrl mode
//然後在第一個非MRL模式之後插入所有的ISP模式
uiRdModeList.insert(uiRdModeList.begin() + 1, secondIspList->begin(), secondIspList->end());
uiRdModeList.insert(uiRdModeList.begin() + 1, firstIspList->begin(), firstIspList->end());
#else
// find the first non-MRL mode
size_t indexFirstMode = std::find( extendRefList.begin(), extendRefList.end(), 0 ) - extendRefList.begin();
// if not found, just take the last mode
if( indexFirstMode >= extendRefList.size() ) indexFirstMode = extendRefList.size() - 1;
// move the mode indicated by indexFirstMode to the beginning
for( int idx = ((int)indexFirstMode) - 1; idx >= 0; idx-- )
{
std::swap( extendRefList[idx], extendRefList[idx + 1] );
std::swap( uiRdModeList [idx], uiRdModeList [idx + 1] );
}
//insert all ISP modes after the first non-mrl mode
uiRdModeList.insert( uiRdModeList.begin() + 1, secondIspList->begin(), secondIspList->end() );
uiRdModeList.insert( uiRdModeList.begin() + 1, firstIspList->begin() , firstIspList->end() );
extendRefList.insert( extendRefList.begin() + 1, secondIspList->size(), MRL_NUM_REF_LINES + ispOptions[2] );
extendRefList.insert( extendRefList.begin() + 1, firstIspList->size() , MRL_NUM_REF_LINES + ispOptions[1] );
#endif
}
else
{
//insert all ISP modes at the end of the current list
//在當前的列表的結尾插入所有的ISP模式
uiRdModeList.insert( uiRdModeList.end(), secondIspList->begin(), secondIspList->end() );
uiRdModeList.insert( uiRdModeList.end(), firstIspList->begin() , firstIspList->end() );
#if !JVET_N0217_MATRIX_INTRAPRED
extendRefList.insert( extendRefList.end(), secondIspList->size(), MRL_NUM_REF_LINES + ispOptions[2] );
extendRefList.insert( extendRefList.end(), firstIspList->size() , MRL_NUM_REF_LINES + ispOptions[1] );
#endif
}
}
#if !JVET_N0217_MATRIX_INTRAPRED
CHECKD(uiRdModeList.size() != extendRefList.size(),"uiRdModeList and extendRefList do not have the same size!");
#endif
//===== check modes (using r-d costs) =====
//check 所有模式（使用RDcost）
#if JVET_N0217_MATRIX_INTRAPRED
ModeInfo uiBestPUMode;//定義當前PU最優的亮度預測幀內模式
#else
uint32_t uiBestPUMode = 0;
int bestExtendRef = 0;
#endif
#if JVET_N0413_RDPCM
int bestBDPCMMode = 0;
double bestCostNonBDPCM = MAX_DOUBLE;//定義再不使用BDPCM模式的時候的最優代價
#endif
//臨時緩存CS
CodingStructure *csTemp = m_pTempCS[gp_sizeIdxInfo->idxFrom( cu.lwidth() )][gp_sizeIdxInfo->idxFrom( cu.lheight() )];
//最好的模式緩存CS
CodingStructure *csBest = m_pBestCS[gp_sizeIdxInfo->idxFrom( cu.lwidth() )][gp_sizeIdxInfo->idxFrom( cu.lheight() )];
csTemp->slice = cs.slice;
csBest->slice = cs.slice;
csTemp->initStructData();
csBest->initStructData();
#if JVET_N0217_MATRIX_INTRAPRED
//定義非Mip模式的最優的代價
m_bestCostNonMip = MAX_DOUBLE;
#if JVET_N0193_LFNST
//重新定義一個RD候選列表
static_vector<int, FAST_UDI_MAX_RDMODE_NUM> rdModeIdxList;
#endif
if (testMip)
{
//如果是Mip模式的時候，單獨爲其設置ui的RD候選臨時列表
static_vector<ModeInfo, FAST_UDI_MAX_RDMODE_NUM> uiRdModeListTemp;
//對RD候選列表中的模式進行循環遍歷處理
//先把非ISP且非MIP的模式放到重新定義的RD候選列表的前面
for( int i = 0; i < uiRdModeList.size(); i++)
{
//如果不是ISP模式且不是MIp模式
if( !uiRdModeList[i].mipFlg && uiRdModeList[i].ispMod==NOT_INTRA_SUBPARTITIONS )
{
uiRdModeListTemp.push_back( uiRdModeList[i] );//將SAD之後的RD候選模式列表中的模式push到臨時列表中保存起來
#if JVET_N0193_LFNST
rdModeIdxList.push_back( i );//給重新定義的RD候選列表中push對應的在原來RD候選模式列表中的模式號
#endif
}
}
//再把ISP或者MIP的模式放到重新定義的RD候選列表的後面
for( int i = 0; i < uiRdModeList.size(); i++)
{
//如果是Mip模式或者是ISP模式
if( uiRdModeList[i].mipFlg || uiRdModeList[i].ispMod!=NOT_INTRA_SUBPARTITIONS )
{
uiRdModeListTemp.push_back( uiRdModeList[i] );//也將SAD之後的RD候選模式列表中的模式push到臨時列表中保存起來
#if JVET_N0193_LFNST
rdModeIdxList.push_back( i );//給重新定義的RD候選列表中push對應的在原來RD候選模式列表中的模式號
#endif
}
}
//最後將原來的RD候選模式列表恢復出來
for( int i = 0; i < uiRdModeList.size(); i++)
{
uiRdModeList[i] = uiRdModeListTemp[i];//
}
}
#endif
// just to be sure
//確保numModesForFullRD和RD候選列表數目相等
numModesForFullRD = ( int ) uiRdModeList.size();
PartSplit intraSubPartitionsProcOrder = TU_NO_ISP;
int bestNormalIntraModeIndex = -1;
#if !JVET_N0217_MATRIX_INTRAPRED
uint8_t bestIspOption = NOT_INTRA_SUBPARTITIONS;
#endif
TUIntraSubPartitioner subTuPartitioner( partitioner );
#if JVET_N0193_LFNST
if( !cu.ispMode && !cu.mtsFlag )
{
m_modeCtrl->setMtsFirstPassNoIspCost( MAX_DOUBLE );
}
#endif
bool ispHorAllZeroCbfs = false, ispVerAllZeroCbfs = false;
#if JVET_N0217_MATRIX_INTRAPRED
#if JVET_N0413_RDPCM
/********************這裏才進入到最終對於RD候選列表中的模式的進一步精選RDcost操作***********/
for (int mode = -2 * int(testBDPCM); mode < (int)uiRdModeList.size(); mode++)
{
// set CU/PU to luma prediction mode
//首先設置亮度CU/PU的預測模式
ModeInfo uiOrgMode;
//如果是BDPCM模式
if ( mode < 0 )
{
//將負的模式號轉化爲正的
cu.bdpcmMode = -mode;
//定義一個MPM列表
unsigned mpm_pred[NUM_MOST_PROBABLE_MODES];
PU::getIntraMPMs(pu, mpm_pred);
uiOrgMode = ModeInfo(false, 0, NOT_INTRA_SUBPARTITIONS, mpm_pred[0]);
cu.mipFlag = uiOrgMode.mipFlg;
cu.ispMode = uiOrgMode.ispMod;
pu.multiRefIdx = uiOrgMode.mRefId;
pu.intraDir[CHANNEL_TYPE_LUMA] = uiOrgMode.modeId;
}
else//如果不是BDPCM模式
{
cu.bdpcmMode = 0;
uiOrgMode = uiRdModeList[mode];//取出RD候選列表中的模式
#else
for (uint32_t uiMode = 0; uiMode < uiRdModeList.size(); uiMode++)
{
// set CU/PU to luma prediction mode
ModeInfo uiOrgMode = uiRdModeList[uiMode];
#endif
cu.mipFlag = uiOrgMode.mipFlg;//當前候選模式是否是Mip模式
cu.ispMode = uiOrgMode.ispMod;//當前候選模式是否是ISP模式
pu.multiRefIdx = uiOrgMode.mRefId;//當前候選模式是否是使用MRL
pu.intraDir[CHANNEL_TYPE_LUMA] = uiOrgMode.modeId;//當前候選模式放入intraDir
CHECK(cu.mipFlag && pu.multiRefIdx, "Error: combination of MIP and MRL not supported");//MIP and MRL的結合不被允許
#if JVET_N0185_UNIFIED_MPM
CHECK(pu.multiRefIdx && (pu.intraDir[0] == PLANAR_IDX), "Error: combination of MRL and Planar mode not supported");//Planar and MRL的結合不被允許
#else
CHECK(pu.multiRefIdx && (pu.intraDir[0] == DC_IDX || pu.intraDir[0] == PLANAR_IDX), "Error: combination of MRL and Planar/DC mode not supported");
#endif
CHECK(cu.ispMode && cu.mipFlag, "Error: combination of ISP and MIP not supported");//ISP and MIP的結合不被允許
CHECK(cu.ispMode && pu.multiRefIdx, "Error: combination of ISP and MRL not supported");//ISP and MRL的結合不被允許
#else
#if JVET_N0413_RDPCM
for( int mode = -2 * int(testBDPCM); mode < numModesForFullRD; mode++ )
#else
for (uint32_t uiMode = 0; uiMode < numModesForFullRD; uiMode++)
#endif
{
#if JVET_N0413_RDPCM
int multiRefIdx = 0;
uint32_t uiOrgMode;
if ( mode < 0 )
{
cu.bdpcmMode = -mode;
unsigned mpm_pred[NUM_MOST_PROBABLE_MODES];
PU::getIntraMPMs(pu, mpm_pred);
pu.intraDir[0] = mpm_pred[0];
uiOrgMode = mpm_pred[0];
cu.ispMode = NOT_INTRA_SUBPARTITIONS;
}
else
{
cu.bdpcmMode = 0;
uiOrgMode = uiRdModeList[mode];
#else
// set luma prediction mode
uint32_t uiOrgMode = uiRdModeList[uiMode];
#endif
cu.ispMode = extendRefList[mode] > MRL_NUM_REF_LINES ? extendRefList[mode] - MRL_NUM_REF_LINES : NOT_INTRA_SUBPARTITIONS;
pu.intraDir[0] = uiOrgMode;
#if !JVET_N0413_RDPCM
int multiRefIdx = 0;
#endif
pu.multiRefIdx = multiRefIdx;
#endif
if( cu.ispMode )//如果當前CU是ISP模式
{
intraSubPartitionsProcOrder = CU::getISPType( cu, COMPONENT_Y );
bool tuIsDividedInRows = CU::divideTuInRows( cu );//TU是否進行行劃分
if ( ( tuIsDividedInRows && ispHorAllZeroCbfs ) || ( !tuIsDividedInRows && ispVerAllZeroCbfs ) )
{
continue;
}
if( m_intraModeDiagRatio.at( bestNormalIntraModeIndex ) > 1.25 )
{
continue;
}
if( ( m_intraModeHorVerRatio.at( bestNormalIntraModeIndex ) > 1.25 && tuIsDividedInRows ) || ( m_intraModeHorVerRatio.at( bestNormalIntraModeIndex ) < 0.8 && !tuIsDividedInRows ) )
{
continue;
}
}
#if JVET_N0413_RDPCM
}
#endif
// set context models
//設置上下文模型
m_CABACEstimator->getCtx() = ctxStart;
// determine residual for partition
//確定子分區的殘差
cs.initSubStructure( *csTemp, partitioner.chType, cs.area, true );
#if JVET_N0193_LFNST
bool tmpValidReturn = false;
#endif
if( cu.ispMode )//如果當前CU是ISP模式，則對每一個子分區用當前模式進行亮度預測
{
#if JVET_N0193_LFNST
// 該函數主要用於計算預測值以及殘差值，然後變換量化並且計算重建值，最後計算率失真代價
tmpValidReturn = xRecurIntraCodingLumaQT( *csTemp, subTuPartitioner, bestCurrentCost, 0, intraSubPartitionsProcOrder, false,
mtsCheckRangeFlag, mtsFirstCheckId, mtsLastCheckId, moreProbMTSIdxFirst );
#else
xRecurIntraCodingLumaQT( *csTemp, subTuPartitioner, bestCurrentCost, 0, intraSubPartitionsProcOrder );
#endif
}
else//如果不是ISP模式，則正常進行幀內預測
{
#if JVET_N0217_MATRIX_INTRAPRED
if( ! fastMip )
{
m_bestCostNonMip = MAX_DOUBLE;
}
#if JVET_N0193_LFNST
//幀內亮度預測主函數的入口
tmpValidReturn = xRecurIntraCodingLumaQT( *csTemp, partitioner, uiBestPUMode.ispMod ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, uiBestPUMode.ispMod,
mtsCheckRangeFlag, mtsFirstCheckId, mtsLastCheckId, moreProbMTSIdxFirst );
#else
xRecurIntraCodingLumaQT(*csTemp, partitioner, uiBestPUMode.ispMod ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, uiBestPUMode.ispMod);
#endif
#else
#if JVET_N0193_LFNST
tmpValidReturn = xRecurIntraCodingLumaQT( *csTemp, partitioner, bestIspOption ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, bestIspOption,
mtsCheckRangeFlag, mtsFirstCheckId, mtsLastCheckId, moreProbMTSIdxFirst );
#else
xRecurIntraCodingLumaQT( *csTemp, partitioner, bestIspOption ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, bestIspOption );
#endif
#endif
}
//如果當前CU是ISP模式，且第一塊TU不是亮度TU
if( cu.ispMode && !csTemp->cus[0]->firstTU->cbf[COMPONENT_Y] )
{
#if JVET_N0193_LFNST
if( !sps.getUseLFNST() )
{
#endif
if ( cu.ispMode == HOR_INTRA_SUBPARTITIONS )
{
ispHorAllZeroCbfs |= ( m_pcEncCfg->getUseFastISP() && csTemp->tus[0]->lheight() > 2 && csTemp->cost >= bestCurrentCost );
}
else
{
ispVerAllZeroCbfs |= ( m_pcEncCfg->getUseFastISP() && csTemp->tus[0]->lwidth() > 2 && csTemp->cost >= bestCurrentCost );
}
#if JVET_N0193_LFNST
}
#endif
csTemp->cost = MAX_DOUBLE;
csTemp->costDbOffset = 0;
#if JVET_N0193_LFNST
tmpValidReturn = false;
#endif
}
#if JVET_N0193_LFNST
validReturn |= tmpValidReturn;
#if JVET_N0413_RDPCM
if( sps.getUseLFNST() && mtsUsageFlag == 1 && !cu.ispMode && mode >= 0 )
{
#if JVET_N0217_MATRIX_INTRAPRED
m_modeCostStore[ lfnstIdx ][ testMip ? rdModeIdxList[ mode ] : mode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#else
m_modeCostStore[ lfnstIdx ][ mode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#endif
#else
if( sps.getUseLFNST() && mtsUsageFlag == 1 && !cu.ispMode )
{
#if JVET_N0217_MATRIX_INTRAPRED
m_modeCostStore[ lfnstIdx ][ testMip ? rdModeIdxList[ uiMode ] : uiMode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#else
m_modeCostStore[ lfnstIdx ][ uiMode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#endif
#endif
}
#endif
#if JVET_N0217_MATRIX_INTRAPRED
DTRACE( g_trace_ctx, D_INTRA_COST, "IntraCost T %f (%d) \n", csTemp->cost, uiOrgMode.modeId );
#else
DTRACE( g_trace_ctx, D_INTRA_COST, "IntraCost T %f (%d) \n", csTemp->cost, uiOrgMode );
#endif
#if JVET_N0193_LFNST
if( tmpValidReturn )
{
#endif
// check r-d cost
//check 每個模式的RD cost，逐次篩選出cost最小的模式
if( csTemp->cost < csBest->cost )//如果當前模式的cost小於目前最優的模式的cost
{
std::swap( csTemp, csBest );//當前模式成爲最新的最優模式
uiBestPUMode = uiOrgMode;//將當前模式賦值給最優模式uiBestPUMode
#if !JVET_N0217_MATRIX_INTRAPRED
bestExtendRef = multiRefIdx;
bestIspOption = cu.ispMode;
#endif
#if JVET_N0413_RDPCM
bestBDPCMMode = cu.bdpcmMode;
#endif
#if JVET_N0193_LFNST
if( sps.getUseLFNST() && mtsUsageFlag == 1 && !cu.ispMode )
{
m_bestModeCostStore[ lfnstIdx ] = csBest->cost; //cs.cost;
}
#endif
if( csBest->cost < bestCurrentCost )
{
bestCurrentCost = csBest->cost;
}
#if !JVET_N0413_RDPCM
if( !cu.ispMode )
{
bestNormalIntraModeIndex = uiMode;
}
#endif
#if JVET_N0193_LFNST
if( !cu.ispMode && !cu.mtsFlag )
{
m_modeCtrl->setMtsFirstPassNoIspCost( csBest->cost );
}
#endif
}
#if JVET_N0413_RDPCM
if( !cu.ispMode && !cu.bdpcmMode && csBest->cost < bestCostNonBDPCM )
{
bestCostNonBDPCM = csBest->cost;
bestNormalIntraModeIndex = mode;
}
#endif
#if JVET_N0193_LFNST
}
#endif
csTemp->releaseIntermediateData();
} // Mode loop//每種模式的循環
//RD候選模式的循環RDCost遍歷結束
#if JVET_N0217_MATRIX_INTRAPRED
cu.ispMode = uiBestPUMode.ispMod;//將最優的模式用到ISP中
#else
cu.ispMode = bestIspOption;
#endif
#if JVET_N0193_LFNST
if( validReturn )
{
#endif
cs.useSubStructure( *csBest, partitioner.chType, pu.singleChan( CHANNEL_TYPE_LUMA ), true, true, keepResi, keepResi );
#if JVET_N0193_LFNST
}
#endif
csBest->releaseIntermediateData();
#if JVET_N0193_LFNST
if( validReturn )
{
#endif
//=== update PU data ====
//更新PU的數據
#if JVET_N0217_MATRIX_INTRAPRED
cu.mipFlag = uiBestPUMode.mipFlg;//cost最優的模式是否是Mip模式
pu.multiRefIdx = uiBestPUMode.mRefId;//cost最優的模式是否使用多參考行
pu.intraDir[ CHANNEL_TYPE_LUMA ] = uiBestPUMode.modeId;
#else
pu.intraDir[ 0 ] = uiBestPUMode;
pu.multiRefIdx = bestExtendRef;
#endif
#if JVET_N0413_RDPCM
cu.bdpcmMode = bestBDPCMMode;
#endif
#if JVET_N0193_LFNST
}
#endif
}
//===== reset context models =====
//重置上下文模型
m_CABACEstimator->getCtx() = ctxStart;
#if JVET_N0193_LFNST
return validReturn;//返回是否可用
#endif
}