引言
在計算機視覺和圖像處理中用來進行物體檢測的特徵描述子，它通過計算和統計圖像局部區域的梯度方向直方圖構成特徵。Hog特徵結合分類算法廣泛應用於圖像識別中，尤其是在行人檢測中獲得極大成功。HOG+SVM的行人檢測方法2005年提出來之後，如今很多行人檢測算法都是以此爲思路的。
從特徵描述子說起
- Haar
- SIFT
- HOG(Histogram of Oriented Gradient)
在計算機視覺和圖像處理中用來進行物體檢測的特徵描述子，它通過計算和統計圖像局部區域
的梯度方向直方圖構成特徵。Hog特徵結合分類算法廣泛應用於圖像識別中，尤其是
在行人檢測中獲得極大成功。HOG+SVM的行人檢測方法2005年提出來之後，如今很多
行人檢測算法都是以此爲思路的。
基本的一些特徵檢測方法
- Sharpening
- High-boost filtering
- The Gradient Operator
- Digital Gradient
- Compass Gradient Operations
- Edge Detection
- The Laplace Operator
- Laplacian of Gaussian (LoG)
- Difference of Gaussian (DoG)
HOG特徵描述子有什麼特性
在一副圖像中，局部目標的表象和形狀（appearance and shape）能夠被梯度或邊緣的方向密度分佈很好地描述。（本質：梯度的統計信息，而梯度主要存在於邊緣的地方）
實現方法
- 首先將圖像分成小的連通區域，我們把它叫細胞單元。然後採集細胞單元中各像素點的梯度的或邊緣的方向直方圖。最後把這些直方圖組合起來就可以構成特徵描述器。
- 爲了提高性能，把這些局部直方圖在圖像的更大的範圍內（我們把它叫區間或block）進行對比度歸一化（contrast-normalized），所採用的方法是：先計算各直方圖在這個block中的密度，然後根據這個密度對block中的各個細胞單元做歸一化。通過這個歸一化後，能對光照變化和陰影獲得更好的效果。
算法步驟
HOG特徵提取方法就是將一個image（你要檢測的目標或者掃描窗口):
灰度化（將圖像看做一個x,y,h（灰度）的三維圖像）；
採用Gamma校正法對輸入圖像進行顏色空間的標準化（歸一化）
目的是調節圖像的對比度，降低圖像局部的陰影和光照變化所造成的影響，同時可以抑制噪音的干擾；
爲了減少光照因素的影響，首先需要將整個圖像進行規範化（歸一化）。在圖像的紋理強度中，局部的表層曝光貢獻的比重較大，所以，這種壓縮處理能夠有效地降低圖像局部的陰影和光照變化。因爲顏色信息作用不大，通常先轉化爲灰度圖；通常取0.5
計算圖像每個像素的梯度（包括大小和方向）；主要是爲了捕獲輪廓信息，同時進一步弱化光照的干擾。
圖像中像素點的梯度：
將圖像劃分成小cells（例如16*16像素/cell）；
每一個點的梯度角度可能是0~180度之間的任意值，而程序中將其離散化爲9個bin，即每個bin佔20度。所以滑動窗口中每個像素點的梯度角度如果要離散化到這9個bin中，則一般它都會有2個相鄰的bin(如果恰好位於某個bin的中心，則可認爲對該bin的權重爲1即可)。從源碼中可以看到梯度的幅值是用來計算梯度直方圖時權重投票的，所以每個像素點的梯度幅值就分解到了其角度相鄰的2個bin了，越近的那個bin得到的權重越大。因此幅度圖像用了2個通道，每個通道都是原像素點幅度的一個分量。同理，不難理解，像素點的梯度角度也用了2個通道，每個通道中存儲的是它相鄰2個bin的bin序號。序號小的放在第一通道。
其中，假設那3條半徑爲離散化後bin的中心，紅色虛線爲像素點O(像素點在圓心處)的梯度方向，梯度幅值爲A，該梯度方向與最近的相鄰bin爲bin0,這兩者之間的夾角爲a.這該像素點O處存儲的梯度幅值第1通道爲A*(1-a),第2通道爲A*a;該像素點O處存儲的角度第1通道爲0(bin的序號爲0)，第2通道爲1(bin的序號爲1)。
另外在計算圖像的梯度圖和相位圖時，如果該圖像時3通道的，則3通道分別取梯度值，並且取梯度最大的那個通道的值爲該點的梯度幅值。
統計每個cell的梯度直方圖（不同梯度的個數），即可形成每個cell的descriptor；
將每幾個cell組成一個block（例如2*2個cell/block）
一個block內所有cell的特徵descriptor串聯起來便得到該block的HOG特徵descriptor。這個在OpenCV中有HogCache中getBlock進行實現的。
如圖所示，黑色框代表1個block，紅實線隔開的爲4個cell，每個cell用綠色虛線隔開的我們稱之爲4個區域，所以該block中共有16個區域，分別爲A、B、C、…、O、P。
將這16個區域分爲4組：
第1組：A、D、M、P;該組內的像素點計算梯度方向直方圖時只對其所在的cell有貢獻。
第2組：B、C、N、O;該組內的像素點計算梯度直方圖時對其所在的左右cell有貢獻。
第3組：E、I、H、L;該組內的像素點計算梯度直方圖時對其所在的上下cell有貢獻。
第4組：F、G、J、K;該組內的像素點對其上下左右的cell計算梯度直方圖時都有貢獻。
那到底是怎麼對cell貢獻的呢？舉個例子來說，E區域內的像素點對cell0和cell2有貢獻。本來1個block對滑動窗口貢獻的向量維數爲36維，即每個cell貢獻9維，其順序分別爲cell0,cell1,cell2,cell3.而E區域內的像素由於同時對cell0和cell2有貢獻，所以在計算E區域內的像素梯度投票時，不僅要投向它本來的cell0，還要投向下面的cell2，即投向cell0和cell2有一個權重，該權重與該像素點所在位置與cell0，cell2中心位置的距離有關。具體的關係可以去查看源碼。
將圖像image內的所有block的HOG特徵descriptor串聯起來就可以得到該image（你要檢測的目標）的HOG特徵descriptor了。這個就是最終的可供分類使用的特徵向量了。
實際實現的時候，首先用[-1,0,1]梯度算子對原圖像做卷積運算，得到x方向（水平方向，以向右爲正方向）的梯度分量gradscalx，然後用[1,0,-1]T梯度算子對原圖像做卷積運算，得到y方向（豎直方向，以向上爲正方向）的梯度分量gradscaly。然後再用以上公式計算該像素點的梯度大小和方向。
HOG源碼分析
在讀源碼時，由於裏面用到了intel的ipp庫，優化了算法的速度。爲了學習方便，我對OpenCV中關於加速的
部分進行了刪減，只剩下算法的精要部分。
頭文件中有關於一些參數的默認設置：
檢測窗口大小爲12864;
Block大小爲1616；
Cell大小爲88；
Block在檢測窗口中上下移動尺寸爲88；
1個cell的梯度直方圖化成9個bin；
滑動窗口在檢測圖片中滑動的尺寸爲8*8；
頭文件
//HOG (Histogram-of-Oriented-Gradients) Descriptor and Object Detector //
//! struct for detection region of interest (ROI)
struct DetectionROI
{
//! scale(size) of the bounding box
double scale;
//! set of requrested locations to be evaluated
std::vector<cv::Point> locations;
//! vector that will contain confidence values for each location
std::vector<double> confidences;
};
struct HOGDescriptor
{
public:
enum { L2Hys = 0};
enum { DEFAULT_NLEVELS = 64};
HOGDescriptor() : winSize(64,128), blockSize(16,16), blockStride(8,8),
cellSize(8,8), nbins(9), derivAperture(1), winSigma(-1),
histogramNormType(HOGDescriptor::L2Hys), L2HysThreshold(0.2), gammaCorrection(true),
free_coef(-1.f), nlevels(HOGDescriptor::DEFAULT_NLEVELS), signedGradient(false)
{}
//! with found weights output
virtual void detect(const Mat& img, std::vector<Point>& foundLocations,
std::vector<double>& weights,
double hitThreshold = 0, Size winStride = Size(),
Size padding = Size(),
const std::vector<Point>& searchLocations = std::vector<Point>()) const;
//! without found weights output
virtual void detect(const Mat& img, std::vector<Point>& foundLocations,
double hitThreshold = 0, Size winStride = Size(),
Size padding = Size(),
const std::vector<Point>& searchLocations=std::vector<Point>()) const;
//! with result weights output
virtual void detectMultiScale(InputArray img, std::vector<Rect>& foundLocations,
std::vector<double>& foundWeights, double hitThreshold = 0,
Size winStride = Size(), Size padding = Size(), double scale = 1.05,
double finalThreshold = 2.0,bool useMeanshiftGrouping = false) const;
//! without found weights output
virtual void detectMultiScale(InputArray img, std::vector<Rect>& foundLocations,
double hitThreshold = 0, Size winStride = Size(),
Size padding = Size(), double scale = 1.05,
double finalThreshold = 2.0, bool useMeanshiftGrouping = false) const;
virtual void computeGradient(const Mat& img, Mat& grad, Mat& angleOfs,
Size paddingTL = Size(), Size paddingBR = Size()) const;
static std::vector<float> getDefaultPeopleDetector();
static std::vector<float> getDaimlerPeopleDetector();
Size winSize; // 窗口大小 64x128
Size blockSize; // block size 16x16
Size blockStride; // block 之間的stride
Size cellSize; // cell的size
int nbins; //
int derivAperture; //
double winSigma;
int histogramNormType;
double L2HysThreshold;
bool gammaCorrection;
std::vector<float> svmDetector;
UMat oclSvmDetector;
float free_coef;
int nlevels;
bool signedGradient;
//! evaluate specified ROI and return confidence value for each location
virtual void detectROI(const cv::Mat& img, const std::vector<cv::Point> &locations,
std::vector<cv::Point>& foundLocations, std::vector<double>& confidences,
double hitThreshold = 0, cv::Size winStride = Size(),
cv::Size padding = Size()) const;
//! evaluate specified ROI and return confidence value for each location in multiple scales
virtual void detectMultiScaleROI(const cv::Mat& img,
std::vector<cv::Rect>& foundLocations,
std::vector<DetectionROI>& locations,
double hitThreshold = 0,
int groupThreshold = 0) const;
};
//! @} objdetect
源文件
#include "cascadedetect.hpp"
#include "opencv2/core/core_c.h"
#include "opencl_kernels_objdetect.hpp"
#include <cstdio>
#include <iterator>
#include <limits>
/****************************************************************************************\
The code below is implementation of HOG (Histogram-of-Oriented Gradients)
descriptor and object detection, introduced by Navneet Dalal and Bill Triggs.
The computed feature vectors are compatible with the
INRIA Object Detection and Localization Toolkit
(http://pascal.inrialpes.fr/soft/olt/)
\****************************************************************************************/
namespace cv
{
#define NTHREADS 256
enum {DESCR_FORMAT_COL_BY_COL, DESCR_FORMAT_ROW_BY_ROW};
static int numPartsWithin(int size, int part_size, int stride)
{
return (size - part_size + stride) / stride;
}
static Size numPartsWithin(cv::Size size, cv::Size part_size,
cv::Size stride)
{
return Size(numPartsWithin(size.width, part_size.width, stride.width),
numPartsWithin(size.height, part_size.height, stride.height));
}
static size_t getBlockHistogramSize(Size block_size, Size cell_size, int nbins)
{
Size cells_per_block = Size(block_size.width / cell_size.width,
block_size.height / cell_size.height);
return (size_t)(nbins * cells_per_block.area());
}
size_t HOGDescriptor::getDescriptorSize() const
{
CV_Assert(blockSize.width % cellSize.width == 0 &&
blockSize.height % cellSize.height == 0);
CV_Assert((winSize.width - blockSize.width) % blockStride.width == 0 &&
(winSize.height - blockSize.height) % blockStride.height == 0 );
return (size_t)nbins*
(blockSize.width/cellSize.width)*
(blockSize.height/cellSize.height)*
((winSize.width - blockSize.width)/blockStride.width + 1)*
((winSize.height - blockSize.height)/blockStride.height + 1);
}
double HOGDescriptor::getWinSigma() const
{
return winSigma >= 0 ? winSigma : (blockSize.width + blockSize.height)/8.;
}
bool HOGDescriptor::checkDetectorSize() const
{
size_t detectorSize = svmDetector.size(), descriptorSize = getDescriptorSize();
return detectorSize == 0 ||
detectorSize == descriptorSize ||
detectorSize == descriptorSize + 1;
}
void HOGDescriptor::setSVMDetector(InputArray _svmDetector)
{
_svmDetector.getMat().convertTo(svmDetector, CV_32F);
CV_Assert(checkDetectorSize());
Mat detector_reordered(1, (int)svmDetector.size(), CV_32FC1);
size_t block_hist_size = getBlockHistogramSize(blockSize, cellSize, nbins);
cv::Size blocks_per_img = numPartsWithin(winSize, blockSize, blockStride);
for (int i = 0; i < blocks_per_img.height; ++i)
for (int j = 0; j < blocks_per_img.width; ++j)
{
const float *src = &svmDetector[0] + (j * blocks_per_img.height + i) * block_hist_size;
float *dst = detector_reordered.ptr<float>() + (i * blocks_per_img.width + j) * block_hist_size;
for (size_t k = 0; k < block_hist_size; ++k)
dst[k] = src[k];
}
size_t descriptor_size = getDescriptorSize();
free_coef = svmDetector.size() > descriptor_size ? svmDetector[descriptor_size] : 0;
detector_reordered.copyTo(oclSvmDetector);
}
#define CV_TYPE_NAME_HOG_DESCRIPTOR "opencv-object-detector-hog"
// @img [input] 計算圖像img
// @grad [output] 梯度幅度圖像`grad`
// @qangle [output] 梯度方向圖像`qangle`.
// @paddingTL爲需要在原圖像img左上角擴增的尺寸，同理paddingBR
// @paddingBR 爲需要在img圖像右下角擴增的尺寸。
void HOGDescriptor::computeGradient(const Mat& img, Mat& grad, Mat& qangle,
Size paddingTL, Size paddingBR) const
{
CV_INSTRUMENT_REGION()
CV_Assert( img.type() == CV_8U || img.type() == CV_8UC3 );
// padding之後的輸出大小
Size gradsize(img.cols + paddingTL.width + paddingBR.width,
img.rows + paddingTL.height + paddingBR.height);
grad.create(gradsize, CV_32FC2); // <magnitude*(1-alpha), magnitude*alpha>
qangle.create(gradsize, CV_8UC2); // [0..nbins-1] - quantized gradient orientation
Size wholeSize;
Point roiofs;
img.locateROI(wholeSize, roiofs);
int i, x, y;
int cn = img.channels();
Mat_<float> _lut(1, 256);
const float* const lut = &_lut(0,0);
if( gammaCorrection )
for( i = 0; i < 256; i++ )
_lut(0,i) = std::sqrt((float)i);
else
for( i = 0; i < 256; i++ )
_lut(0,i) = (float)i;
AutoBuffer<int> mapbuf(gradsize.width + gradsize.height + 4);
int* xmap = (int*)mapbuf + 1;
int* ymap = xmap + gradsize.width + 2;
const int borderType = (int)BORDER_REFLECT_101;
for( x = -1; x < gradsize.width + 1; x++ )
xmap[x] = borderInterpolate(x - paddingTL.width + roiofs.x,
wholeSize.width, borderType) - roiofs.x;
for( y = -1; y < gradsize.height + 1; y++ )
ymap[y] = borderInterpolate(y - paddingTL.height + roiofs.y,
wholeSize.height, borderType) - roiofs.y;
// x- & y- derivatives for the whole row
int width = gradsize.width;
AutoBuffer<float> _dbuf(width*4);
float* const dbuf = _dbuf;
Mat Dx(1, width, CV_32F, dbuf);
Mat Dy(1, width, CV_32F, dbuf + width);
Mat Mag(1, width, CV_32F, dbuf + width*2);
Mat Angle(1, width, CV_32F, dbuf + width*3);
if (cn == 3)
{
int end = gradsize.width + 2;
xmap -= 1, x = 0;
for ( ; x < end; ++x)
xmap[x] *= 3;
xmap += 1;
}
float angleScale = signedGradient ? (float)(nbins/(2.0*CV_PI)) : (float)(nbins/CV_PI);
for( y = 0; y < gradsize.height; y++ )
{
const uchar* imgPtr = img.ptr(ymap[y]);
//In case subimage is used ptr() generates an assert for next and prev rows
//(see http://code.opencv.org/issues/4149)
const uchar* prevPtr = img.data + img.step*ymap[y-1];
const uchar* nextPtr = img.data + img.step*ymap[y+1];
float* gradPtr = grad.ptr<float>(y);
uchar* qanglePtr = qangle.ptr(y);
if( cn == 1 )
{
for( x = 0; x < width; x++ )
{
int x1 = xmap[x];
dbuf[x] = (float)(lut[imgPtr[xmap[x+1]]] - lut[imgPtr[xmap[x-1]]]);
dbuf[width + x] = (float)(lut[nextPtr[x1]] - lut[prevPtr[x1]]);
}
}
else
{
x = 0;
for( ; x < width; x++ )
{
int x1 = xmap[x];
float dx0, dy0, dx, dy, mag0, mag;
const uchar* p2 = imgPtr + xmap[x+1];
const uchar* p0 = imgPtr + xmap[x-1];
dx0 = lut[p2[2]] - lut[p0[2]];
dy0 = lut[nextPtr[x1+2]] - lut[prevPtr[x1+2]];
mag0 = dx0*dx0 + dy0*dy0;
dx = lut[p2[1]] - lut[p0[1]];
dy = lut[nextPtr[x1+1]] - lut[prevPtr[x1+1]];
mag = dx*dx + dy*dy;
if( mag0 < mag )
{
dx0 = dx;
dy0 = dy;
mag0 = mag;
}
dx = lut[p2[0]] - lut[p0[0]];
dy = lut[nextPtr[x1]] - lut[prevPtr[x1]];
mag = dx*dx + dy*dy;
if( mag0 < mag )
{
dx0 = dx;
dy0 = dy;
mag0 = mag;
}
dbuf[x] = dx0;
dbuf[x+width] = dy0;
}
}
// computing angles and magnidutes
cartToPolar( Dx, Dy, Mag, Angle, false );
// filling the result matrix
x = 0;
for( ; x < width; x++ )
{
float mag = dbuf[x+width*2], angle = dbuf[x+width*3]*angleScale - 0.5f;
int hidx = cvFloor(angle);
angle -= hidx;
gradPtr[x*2] = mag*(1.f - angle);
gradPtr[x*2+1] = mag*angle;
if( hidx < 0 )
hidx += nbins;
else if( hidx >= nbins )
hidx -= nbins;
CV_Assert( (unsigned)hidx < (unsigned)nbins );
qanglePtr[x*2] = (uchar)hidx;
hidx++;
hidx &= hidx < nbins ? -1 : 0;
qanglePtr[x*2+1] = (uchar)hidx;
}
}
}
struct HOGCache
{
// 1個BlockData結構體是對應的一個block數據。
// 其中histOfs表示爲該block對整個滑動窗口內hog描述算子的貢獻那部分向量的起始位置；
// imgOffset爲該block在滑動窗口圖片中的座標(當然是指左上角座標)
struct BlockData
{
BlockData() :
histOfs(0), imgOffset()
{ }
int histOfs;
Point imgOffset;
};
// PixData結構體是對應的block中1個像素點的數據。
// 其中gradOfs表示該點的梯度幅度在滑動窗口圖片梯度幅度圖中的位置座標；
// qangleOfs表示該點的梯度角度在滑動窗口圖片梯度角度圖中的位置座標；
// histOfs[]表示該像素點對1個或2個或4個cell貢獻的hog描述子向量的起始位置座標（比較抽象，需要看源碼才懂）。
// histWeight[]表示該像素點對1個或2個或4個cell貢獻的權重。
// gradWeight表示該點本身由於處在block中位置的不同因而對梯度直方圖貢獻也不同，其權值按照二維高斯分佈(以block中心爲二維高斯的中心)來決定。
struct PixData
{
size_t gradOfs, qangleOfs;
int histOfs[4];
float histWeights[4];
float gradWeight;
};
HOGCache();
HOGCache(const HOGDescriptor* descriptor,
const Mat& img, const Size& paddingTL, const Size& paddingBR,
bool useCache, const Size& cacheStride);
virtual ~HOGCache() { }
virtual void init(const HOGDescriptor* descriptor,
const Mat& img, const Size& paddingTL, const Size& paddingBR,
bool useCache, const Size& cacheStride);
Size windowsInImage(const Size& imageSize, const Size& winStride) const;
Rect getWindow(const Size& imageSize, const Size& winStride, int idx) const;
const float* getBlock(Point pt, float* buf);
// 指對block獲取到的hog部分描述子進行歸一化，其實該歸一化有2層，具體看代碼。
virtual void normalizeBlockHistogram(float* histogram) const;
std::vector<PixData> pixData;
std::vector<BlockData> blockData;
bool useCache;
std::vector<int> ymaxCached;
Size winSize;
Size cacheStride;
Size nblocks, ncells;
int blockHistogramSize;
int count1, count2, count4;
Point imgoffset;
Mat_<float> blockCache;
Mat_<uchar> blockCacheFlags;
Mat grad, qangle;
const HOGDescriptor* descriptor;
};
HOGCache::HOGCache() :
blockHistogramSize(), count1(), count2(), count4()
{
useCache = false;
descriptor = 0;
}
HOGCache::HOGCache(const HOGDescriptor* _descriptor,
const Mat& _img, const Size& _paddingTL, const Size& _paddingBR,
bool _useCache, const Size& _cacheStride)
{
init(_descriptor, _img, _paddingTL, _paddingBR, _useCache, _cacheStride);
}
void HOGCache::init(const HOGDescriptor* _descriptor,
const Mat& _img, const Size& _paddingTL, const Size& _paddingBR,
bool _useCache, const Size& _cacheStride)
{
descriptor = _descriptor;
cacheStride = _cacheStride;
useCache = _useCache;
// 計算輸入圖像的權值梯度幅度圖和角度量化圖
descriptor->computeGradient(_img, grad, qangle, _paddingTL, _paddingBR);
imgoffset = _paddingTL;
winSize = descriptor->winSize;
Size blockSize = descriptor->blockSize;
Size blockStride = descriptor->blockStride;
Size cellSize = descriptor->cellSize;
int i, j, nbins = descriptor->nbins;
// rawBlockSize爲block中包含像素點的個數
int rawBlockSize = blockSize.width*blockSize.height;
// block的數目
nblocks = Size((winSize.width - blockSize.width)/blockStride.width + 1,
(winSize.height - blockSize.height)/blockStride.height + 1);
// cell的數目
ncells = Size(blockSize.width/cellSize.width, blockSize.height/cellSize.height);
// blockHistogramSize表示一個block中貢獻給hog描述子向量的長度
blockHistogramSize = ncells.width*ncells.height*nbins;
if( useCache )
{
Size cacheSize((grad.cols - blockSize.width)/cacheStride.width+1,
(winSize.height/cacheStride.height)+1);
blockCache.create(cacheSize.height, cacheSize.width*blockHistogramSize);
blockCacheFlags.create(cacheSize);
size_t cacheRows = blockCache.rows;
ymaxCached.resize(cacheRows);
for(size_t ii = 0; ii < cacheRows; ii++ )
ymaxCached[ii] = -1;
}
// weights爲一個尺寸爲blockSize的二維高斯表,下面的代碼就是計算二維高斯的係數
Mat_<float> weights(blockSize);
float sigma = (float)descriptor->getWinSigma();
float scale = 1.f/(sigma*sigma*2);
{
AutoBuffer<float> di(blockSize.height), dj(blockSize.width);
float* _di = (float*)di, *_dj = (float*)dj;
float bh = blockSize.height * 0.5f, bw = blockSize.width * 0.5f;
for (i = 0; i < blockSize.height; ++i)
{
_di[i] = i - bh;
_di[i] *= _di[i];
}
for (j = 0;; j < blockSize.width; ++j)
{
_dj[j] = j - bw;
_dj[j] *= _dj[j];
}
for(i = 0; i < blockSize.height; i++)
for(j = 0; j < blockSize.width; j++)
weights(i,j) = std::exp(-(_di[i] + _dj[j])*scale);
}
// vector<BlockData> blockData;而BlockData爲HOGCache的一個結構體成員
// nblocks.width*nblocks.height表示一個檢測窗口中block的個數，
// 而cacheSize.width*cacheSize.heigh表示一個已經擴充的圖片中的block的個數
blockData.resize(nblocks.width*nblocks.height);
// vector<PixData> pixData; 同理，Pixdata也爲HOGCache中的一個結構體成員
// rawBlockSize表示每個block中像素點的個數
// resize表示將其轉換成列向量
// rawBlockSize*3表示的是存儲同時對1個cell，2個cell，4個cell的貢獻
pixData.resize(rawBlockSize*3);
// Initialize 2 lookup tables, pixData & blockData.
// Here is why:
//
// The detection algorithm runs in 4 nested loops (at each pyramid layer):
// loop over the windows within the input image
// loop over the blocks within each window
// loop over the cells within each block
// loop over the pixels in each cell
//
// As each of the loops runs over a 2-dimensional array,
// we could get 8(!) nested loops in total, which is very-very slow.
//
// To speed the things up, we do the following:
// 1. loop over windows is unrolled in the HOGDescriptor::{compute|detect} methods;
// inside we compute the current search window using getWindow() method.
// Yes, it involves some overhead (function call + couple of divisions),
// but it's tiny in fact.
// 2. loop over the blocks is also unrolled. Inside we use **pre-computed** blockData[j]
// to set up gradient and histogram pointers.
// 3. loops over cells and pixels in each cell are merged
// (since there is no overlap between cells, each pixel in the block is processed once)
// and also unrolled. Inside we use PixData[k] to access the gradient values and
// update the histogram
//
// count1, count2, count4分別表示block中同時對1個cell，2個cell，4個cell有貢獻的像素點的個數。
count1 = count2 = count4 = 0;
for( j = 0; j < blockSize.width; j++ )
for( i = 0; i < blockSize.height; i++ )
{
PixData* data = 0;
float cellX = (j+0.5f)/cellSize.width - 0.5f;
float cellY = (i+0.5f)/cellSize.height - 0.5f;
int icellX0 = cvFloor(cellX);
int icellY0 = cvFloor(cellY);
int icellX1 = icellX0 + 1, icellY1 = icellY0 + 1;
cellX -= icellX0;
cellY -= icellY0;
if( (unsigned)icellX0 < (unsigned)ncells.width &&
(unsigned)icellX1 < (unsigned)ncells.width )
{
if( (unsigned)icellY0 < (unsigned)ncells.height &&
(unsigned)icellY1 < (unsigned)ncells.height )
{
data = &pixData[rawBlockSize*2 + (count4++)];
data->histOfs[0] = (icellX0*ncells.height + icellY0)*nbins;
data->histWeights[0] = (1.f - cellX)*(1.f - cellY);
data->histOfs[1] = (icellX1*ncells.height + icellY0)*nbins;
data->histWeights[1] = cellX*(1.f - cellY);
data->histOfs[2] = (icellX0*ncells.height + icellY1)*nbins;
data->histWeights[2] = (1.f - cellX)*cellY;
data->histOfs[3] = (icellX1*ncells.height + icellY1)*nbins;
data->histWeights[3] = cellX*cellY;
}
else
{
data = &pixData[rawBlockSize + (count2++)];
if( (unsigned)icellY0 < (unsigned)ncells.height )
{
icellY1 = icellY0;
cellY = 1.f - cellY;
}
data->histOfs[0] = (icellX0*ncells.height + icellY1)*nbins;
data->histWeights[0] = (1.f - cellX)*cellY;
data->histOfs[1] = (icellX1*ncells.height + icellY1)*nbins;
data->histWeights[1] = cellX*cellY;
data->histOfs[2] = data->histOfs[3] = 0;
data->histWeights[2] = data->histWeights[3] = 0;
}
}
else
{
if( (unsigned)icellX0 < (unsigned)ncells.width )
{
icellX1 = icellX0;
cellX = 1.f - cellX;
}
if( (unsigned)icellY0 < (unsigned)ncells.height &&
(unsigned)icellY1 < (unsigned)ncells.height )
{
data = &pixData[rawBlockSize + (count2++)];
data->histOfs[0] = (icellX1*ncells.height + icellY0)*nbins;
data->histWeights[0] = cellX*(1.f - cellY);
data->histOfs[1] = (icellX1*ncells.height + icellY1)*nbins;
data->histWeights[1] = cellX*cellY;
data->histOfs[2] = data->histOfs[3] = 0;
data->histWeights[2] = data->histWeights[3] = 0;
}
else
{
data = &pixData[count1++];
if( (unsigned)icellY0 < (unsigned)ncells.height )
{
icellY1 = icellY0;
cellY = 1.f - cellY;
}
data->histOfs[0] = (icellX1*ncells.height + icellY1)*nbins;
data->histWeights[0] = cellX*cellY;
data->histOfs[1] = data->histOfs[2] = data->histOfs[3] = 0;
data->histWeights[1] = data->histWeights[2] = data->histWeights[3] = 0;
}
}
data->gradOfs = (grad.cols*i + j)*2;
data->qangleOfs = (qangle.cols*i + j)*2;
data->gradWeight = weights(i,j);
}
assert( count1 + count2 + count4 == rawBlockSize );
// defragment pixData
for( j = 0; j < count2; j++ )
pixData[j + count1] = pixData[j + rawBlockSize];
for( j = 0; j < count4; j++ )
pixData[j + count1 + count2] = pixData[j + rawBlockSize*2];
count2 += count1;
count4 += count2;
// initialize blockData
for( j = 0; j < nblocks.width; j++ )
for( i = 0; i < nblocks.height; i++ )
{
BlockData& data = blockData[j*nblocks.height + i];
data.histOfs = (j*nblocks.height + i)*blockHistogramSize;
data.imgOffset = Point(j*blockStride.width,i*blockStride.height);
}
}
// 計算一個block中的特徵子
const float* HOGCache::getBlock(Point pt, float* buf)
{
float* blockHist = buf;
assert(descriptor != 0);
// Size blockSize = descriptor->blockSize;
pt += imgoffset;
// CV_Assert( (unsigned)pt.x <= (unsigned)(grad.cols - blockSize.width) &&
// (unsigned)pt.y <= (unsigned)(grad.rows - blockSize.height) );
if( useCache )
{
CV_Assert( pt.x % cacheStride.width == 0 &&
pt.y % cacheStride.height == 0 );
Point cacheIdx(pt.x/cacheStride.width,
(pt.y/cacheStride.height) % blockCache.rows);
if( pt.y != ymaxCached[cacheIdx.y] )
{
Mat_<uchar> cacheRow = blockCacheFlags.row(cacheIdx.y);
cacheRow = (uchar)0;
ymaxCached[cacheIdx.y] = pt.y;
}
blockHist = &blockCache[cacheIdx.y][cacheIdx.x*blockHistogramSize];
uchar& computedFlag = blockCacheFlags(cacheIdx.y, cacheIdx.x);
if( computedFlag != 0 )
return blockHist;
computedFlag = (uchar)1; // set it at once, before actual computing
}
int k, C1 = count1, C2 = count2, C4 = count4;
const float* gradPtr = grad.ptr<float>(pt.y) + pt.x*2;
const uchar* qanglePtr = qangle.ptr(pt.y) + pt.x*2;
// CV_Assert( blockHist != 0 );
memset(blockHist, 0, sizeof(float) * blockHistogramSize);
const PixData* _pixData = &pixData[0];
// 統計各個cell中的bin信息
for( k = 0; k < C1; k++ )
{
const PixData& pk = _pixData[k];
const float* const a = gradPtr + pk.gradOfs;
float w = pk.gradWeight*pk.histWeights[0];
const uchar* h = qanglePtr + pk.qangleOfs;
int h0 = h[0], h1 = h[1];
float* hist = blockHist + pk.histOfs[0];
float t0 = hist[h0] + a[0]*w;
float t1 = hist[h1] + a[1]*w;
hist[h0] = t0; hist[h1] = t1;
}
for( ; k < C2; k++ )
{
const PixData& pk = _pixData[k];
const float* const a = gradPtr + pk.gradOfs;
float w, t0, t1, a0 = a[0], a1 = a[1];
const uchar* const h = qanglePtr + pk.qangleOfs;
int h0 = h[0], h1 = h[1];
float* hist = blockHist + pk.histOfs[0];
w = pk.gradWeight*pk.histWeights[0];
t0 = hist[h0] + a0*w;
t1 = hist[h1] + a1*w;
hist[h0] = t0; hist[h1] = t1;
hist = blockHist + pk.histOfs[1];
w = pk.gradWeight*pk.histWeights[1];
t0 = hist[h0] + a0*w;
t1 = hist[h1] + a1*w;
hist[h0] = t0; hist[h1] = t1;
}
for( ; k < C4; k++ )
{
const PixData& pk = _pixData[k];
const float* a = gradPtr + pk.gradOfs;
float w, t0, t1, a0 = a[0], a1 = a[1];
const uchar* h = qanglePtr + pk.qangleOfs;
int h0 = h[0], h1 = h[1];
float* hist = blockHist + pk.histOfs[0];
w = pk.gradWeight*pk.histWeights[0];
t0 = hist[h0] + a0*w;
t1 = hist[h1] + a1*w;
hist[h0] = t0; hist[h1] = t1;
hist = blockHist + pk.histOfs[1];
w = pk.gradWeight*pk.histWeights[1];
t0 = hist[h0] + a0*w;
t1 = hist[h1] + a1*w;
hist[h0] = t0; hist[h1] = t1;
hist = blockHist + pk.histOfs[2];
w = pk.gradWeight*pk.histWeights[2];
t0 = hist[h0] + a0*w;
t1 = hist[h1] + a1*w;
hist[h0] = t0; hist[h1] = t1;
hist = blockHist + pk.histOfs[3];
w = pk.gradWeight*pk.histWeights[3];
t0 = hist[h0] + a0*w;
t1 = hist[h1] + a1*w;
hist[h0] = t0; hist[h1] = t1;
}
// 歸一化 block中的Hist
normalizeBlockHistogram(blockHist);
return blockHist;
}
void HOGCache::normalizeBlockHistogram(float* _hist) const
{
float* hist = &_hist[0], sum = 0.0f;
size_t i = 0, sz = blockHistogramSize;
for (i = 0 ; i < sz; ++i)
sum += hist[i]*hist[i];
float scale = 1.f/(std::sqrt(sum)+sz*0.1f), thresh = (float)descriptor->L2HysThreshold;
sum = 0.0f;
for(i = 0; i < sz; ++i)
{
hist[i] = std::min(hist[i]*scale, thresh);
sum += hist[i]*hist[i];
}
scale = 1.f/(std::sqrt(sum)+1e-3f), i = 0;
for ( ; i < sz; ++i)
hist[i] *= scale;
}
Size HOGCache::windowsInImage(const Size& imageSize, const Size& winStride) const
{
return Size((imageSize.width - winSize.width)/winStride.width + 1,
(imageSize.height - winSize.height)/winStride.height + 1);
}
Rect HOGCache::getWindow(const Size& imageSize, const Size& winStride, int idx) const
{
int nwindowsX = (imageSize.width - winSize.width)/winStride.width + 1;
int y = idx / nwindowsX;
int x = idx - nwindowsX*y;
return Rect( x*winStride.width, y*winStride.height, winSize.width, winSize.height );
}
static inline int gcd(int a, int b)
{
if( a < b )
std::swap(a, b);
while( b > 0 )
{
int r = a % b;
a = b;
b = r;
}
return a;
}
void HOGDescriptor::compute(InputArray _img, std::vector<float>& descriptors,
Size winStride, Size padding, const std::vector<Point>& locations) const
{
CV_INSTRUMENT_REGION()
if( winStride == Size() )
winStride = cellSize;
Size cacheStride(gcd(winStride.width, blockStride.width),
gcd(winStride.height, blockStride.height));
Size imgSize = _img.size();
size_t nwindows = locations.size();
padding.width = (int)alignSize(std::max(padding.width, 0), cacheStride.width);
padding.height = (int)alignSize(std::max(padding.height, 0), cacheStride.height);
Size paddedImgSize(imgSize.width + padding.width*2, imgSize.height + padding.height*2);
Mat img = _img.getMat();
HOGCache cache(this, img, padding, padding, nwindows == 0, cacheStride);
// 獲取圖片中windows的個數
if( !nwindows )
nwindows = cache.windowsInImage(paddedImgSize, winStride).area();
const HOGCache::BlockData* blockData = &cache.blockData[0];
int nblocks = cache.nblocks.area();
int blockHistogramSize = cache.blockHistogramSize;
size_t dsize = getDescriptorSize();
descriptors.resize(dsize*nwindows);
// for each window
for( size_t i = 0; i < nwindows; i++ )
{
float* descriptor = &descriptors[i*dsize];
Point pt0;
if( !locations.empty() )
{
pt0 = locations[i];
if( pt0.x < -padding.width || pt0.x > img.cols + padding.width - winSize.width ||
pt0.y < -padding.height || pt0.y > img.rows + padding.height - winSize.height )
continue;
}
else
{
pt0 = cache.getWindow(paddedImgSize, winStride, (int)i).tl() - Point(padding);
}
for( int j = 0; j < nblocks; j++ )
{
const HOGCache::BlockData& bj = blockData[j];
Point pt = pt0 + bj.imgOffset;
float* dst = descriptor + bj.histOfs;
const float* src = cache.getBlock(pt, dst);
if( src != dst ) memcpy(dst, src, blockHistogramSize * sizeof(float));
}
}
}
void HOGDescriptor::detect(const Mat& img,
std::vector<Point>& hits, std::vector<double>& weights, double hitThreshold,
Size winStride, Size padding, const std::vector<Point>& locations) const
{
CV_INSTRUMENT_REGION()
hits.clear();
weights.clear();
if( svmDetector.empty() )
return;
if( winStride == Size() )
winStride = cellSize;
Size cacheStride(gcd(winStride.width, blockStride.width),
gcd(winStride.height, blockStride.height));
size_t nwindows = locations.size();
padding.width = (int)alignSize(std::max(padding.width, 0), cacheStride.width);
padding.height = (int)alignSize(std::max(padding.height, 0), cacheStride.height);
Size paddedImgSize(img.cols + padding.width*2, img.rows + padding.height*2);
HOGCache cache(this, img, padding, padding, nwindows == 0, cacheStride);
if( !nwindows )
nwindows = cache.windowsInImage(paddedImgSize, winStride).area();
const HOGCache::BlockData* blockData = &cache.blockData[0];
int nblocks = cache.nblocks.area();
int blockHistogramSize = cache.blockHistogramSize;
size_t dsize = getDescriptorSize();
double rho = svmDetector.size() > dsize ? svmDetector[dsize] : 0;
std::vector<float> blockHist(blockHistogramSize);
for( size_t i = 0; i < nwindows; i++ )
{
Point pt0;
if( !locations.empty() )
{
pt0 = locations[i];
if( pt0.x < -padding.width || pt0.x > img.cols + padding.width - winSize.width ||
pt0.y < -padding.height || pt0.y > img.rows + padding.height - winSize.height )
continue;
}
else
{
pt0 = cache.getWindow(paddedImgSize, winStride, (int)i).tl() - Point(padding);
CV_Assert(pt0.x % cacheStride.width == 0 && pt0.y % cacheStride.height == 0);
}
double s = rho;
const float* svmVec = &svmDetector[0];
int j, k;
for( j = 0; j < nblocks; j++, svmVec += blockHistogramSize )
{
const HOGCache::BlockData& bj = blockData[j];
Point pt = pt0 + bj.imgOffset;
const float* vec = cache.getBlock(pt, &blockHist[0]);
for(k = 0 ; k < blockHistogramSize; k++ )
s += vec[k]*svmVec[k];
}
if( s >= hitThreshold )
{
hits.push_back(pt0);
weights.push_back(s);
}
}
}
void HOGDescriptor::detect(const Mat& img, std::vector<Point>& hits, double hitThreshold,
Size winStride, Size padding, const std::vector<Point>& locations) const
{
CV_INSTRUMENT_REGION()
std::vector<double> weightsV;
detect(img, hits, weightsV, hitThreshold, winStride, padding, locations);
}
class HOGInvoker : public ParallelLoopBody
{
public:
HOGInvoker( const HOGDescriptor* _hog, const Mat& _img,
double _hitThreshold, const Size& _winStride, const Size& _padding,
const double* _levelScale, std::vector<Rect> * _vec, Mutex* _mtx,
std::vector<double>* _weights=0, std::vector<double>* _scales=0 )
{
hog = _hog;
img = _img;
hitThreshold = _hitThreshold;
winStride = _winStride;
padding = _padding;
levelScale = _levelScale;
vec = _vec;
weights = _weights;
scales = _scales;
mtx = _mtx;
}
void operator()( const Range& range ) const
{
int i, i1 = range.start, i2 = range.end;
double minScale = i1 > 0 ? levelScale[i1] : i2 > 1 ? levelScale[i1+1] : std::max(img.cols, img.rows);
Size maxSz(cvCeil(img.cols/minScale), cvCeil(img.rows/minScale));
Mat smallerImgBuf(maxSz, img.type());
std::vector<Point> locations;
std::vector<double> hitsWeights;
for( i = i1; i < i2; i++ )
{
double scale = levelScale[i];
Size sz(cvRound(img.cols/scale), cvRound(img.rows/scale));
Mat smallerImg(sz, img.type(), smallerImgBuf.ptr());
if( sz == img.size() )
smallerImg = Mat(sz, img.type(), img.data, img.step);
else
resize(img, smallerImg, sz);
hog->detect(smallerImg, locations, hitsWeights, hitThreshold, winStride, padding);
Size scaledWinSize = Size(cvRound(hog->winSize.width*scale), cvRound(hog->winSize.height*scale));
mtx->lock();
for( size_t j = 0; j < locations.size(); j++ )
{
vec->push_back(Rect(cvRound(locations[j].x*scale),
cvRound(locations[j].y*scale),
scaledWinSize.width, scaledWinSize.height));
if (scales)
scales->push_back(scale);
}
mtx->unlock();
if (weights && (!hitsWeights.empty()))
{
mtx->lock();
for (size_t j = 0; j < locations.size(); j++)
weights->push_back(hitsWeights[j]);
mtx->unlock();
}
}
}
private:
const HOGDescriptor* hog;
Mat img;
double hitThreshold;
Size winStride;
Size padding;
const double* levelScale;
std::vector<Rect>* vec;
std::vector<double>* weights;
std::vector<double>* scales;
Mutex* mtx;
};
void HOGDescriptor::detectMultiScale(
InputArray _img, std::vector<Rect>& foundLocations, std::vector<double>& foundWeights,
double hitThreshold, Size winStride, Size padding,
double scale0, double finalThreshold, bool useMeanshiftGrouping) const
{
CV_INSTRUMENT_REGION()
double scale = 1.;
int levels = 0;
Size imgSize = _img.size();
std::vector<double> levelScale;
for( levels = 0; levels < nlevels; levels++ )
{
levelScale.push_back(scale);
if( cvRound(imgSize.width/scale) < winSize.width ||
cvRound(imgSize.height/scale) < winSize.height ||
scale0 <= 1 )
break;
scale *= scale0;
}
levels = std::max(levels, 1);
levelScale.resize(levels);
if(winStride == Size())
winStride = blockStride;
CV_OCL_RUN(_img.dims() <= 2 && _img.type() == CV_8UC1 && scale0 > 1 && winStride.width % blockStride.width == 0 &&
winStride.height % blockStride.height == 0 && padding == Size(0,0) && _img.isUMat(),
ocl_detectMultiScale(_img, foundLocations, levelScale, hitThreshold, winStride, finalThreshold, oclSvmDetector,
blockSize, cellSize, nbins, blockStride, winSize, gammaCorrection, L2HysThreshold, (float)getWinSigma(), free_coef, signedGradient));
std::vector<Rect> allCandidates;
std::vector<double> tempScales;
std::vector<double> tempWeights;
std::vector<double> foundScales;
Mutex mtx;
Mat img = _img.getMat();
Range range(0, (int)levelScale.size());
HOGInvoker invoker(this, img, hitThreshold, winStride, padding, &levelScale[0], &allCandidates, &mtx, &tempWeights, &tempScales);
parallel_for_(range, invoker);
std::copy(tempScales.begin(), tempScales.end(), back_inserter(foundScales));
foundLocations.clear();
std::copy(allCandidates.begin(), allCandidates.end(), back_inserter(foundLocations));
foundWeights.clear();
std::copy(tempWeights.begin(), tempWeights.end(), back_inserter(foundWeights));
if ( useMeanshiftGrouping )
groupRectangles_meanshift(foundLocations, foundWeights, foundScales, finalThreshold, winSize);
else
groupRectangles(foundLocations, foundWeights, (int)finalThreshold, 0.2);
clipObjects(imgSize, foundLocations, 0, &foundWeights);
}
void HOGDescriptor::detectMultiScale(InputArray img, std::vector<Rect>& foundLocations,
double hitThreshold, Size winStride, Size padding,
double scale0, double finalThreshold, bool useMeanshiftGrouping) const
{
CV_INSTRUMENT_REGION()
std::vector<double> foundWeights;
detectMultiScale(img, foundLocations, foundWeights, hitThreshold, winStride,
padding, scale0, finalThreshold, useMeanshiftGrouping);
}
void HOGDescriptor::detectROI(const cv::Mat& img, const std::vector<cv::Point> &locations,
CV_OUT std::vector<cv::Point>& foundLocations, CV_OUT std::vector<double>& confidences,
double hitThreshold, cv::Size winStride, cv::Size padding) const
{
CV_INSTRUMENT_REGION()
foundLocations.clear();
confidences.clear();
if( svmDetector.empty() || locations.empty())
return;
if( winStride == Size() )
winStride = cellSize;
Size cacheStride(gcd(winStride.width, blockStride.width),
gcd(winStride.height, blockStride.height));
size_t nwindows = locations.size();
padding.width = (int)alignSize(std::max(padding.width, 0), cacheStride.width);
padding.height = (int)alignSize(std::max(padding.height, 0), cacheStride.height);
Size paddedImgSize(img.cols + padding.width*2, img.rows + padding.height*2);
// HOGCache cache(this, img, padding, padding, nwindows == 0, cacheStride);
HOGCache cache(this, img, padding, padding, true, cacheStride);
if( !nwindows )
nwindows = cache.windowsInImage(paddedImgSize, winStride).area();
const HOGCache::BlockData* blockData = &cache.blockData[0];
int nblocks = cache.nblocks.area();
int blockHistogramSize = cache.blockHistogramSize;
size_t dsize = getDescriptorSize();
double rho = svmDetector.size() > dsize ? svmDetector[dsize] : 0;
std::vector<float> blockHist(blockHistogramSize);
for( size_t i = 0; i < nwindows; i++ )
{
Point pt0;
pt0 = locations[i];
if( pt0.x < -padding.width || pt0.x > img.cols + padding.width - winSize.width ||
pt0.y < -padding.height || pt0.y > img.rows + padding.height - winSize.height )
{
// out of image
confidences.push_back(-10.0);
continue;
}
double s = rho;
const float* svmVec = &svmDetector[0];
int j, k;
for( j = 0; j < nblocks; j++, svmVec += blockHistogramSize )
{
const HOGCache::BlockData& bj = blockData[j];
Point pt = pt0 + bj.imgOffset;
// need to devide this into 4 parts!
const float* vec = cache.getBlock(pt, &blockHist[0]);
for(k = 0 ; k < blockHistogramSize; k++ )
s += vec[k]*svmVec[k];
}
confidences.push_back(s);
if( s >= hitThreshold )
foundLocations.push_back(pt0);
}
}
void HOGDescriptor::detectMultiScaleROI(const cv::Mat& img,
CV_OUT std::vector<cv::Rect>& foundLocations, std::vector<DetectionROI>& locations,
double hitThreshold, int groupThreshold) const
{
CV_INSTRUMENT_REGION()
std::vector<Rect> allCandidates;
Mutex mtx;
parallel_for_(Range(0, (int)locations.size()),
HOGConfInvoker(this, img, hitThreshold, Size(8, 8),
&locations, &allCandidates, &mtx));
foundLocations.resize(allCandidates.size());
std::copy(allCandidates.begin(), allCandidates.end(), foundLocations.begin());
cv::groupRectangles(foundLocations, groupThreshold, 0.2);
}
}
FHOG
源代碼下載：http://www.codeforge.com/read/465952/FHOG.cpp__html
FHOG是在HOG基礎上，將冗餘計算去除之後改進的算法。下面進行介紹
參考資料
-
[hog中快速算法的三線插值將得很詳細]http://hi.baidu.com/susongzhi/item/3a3c758d7ff5cbdc5e0ec172
-
[HOG更加詳細的解釋]http://blog.csdn.net/liulina603/article/details/8291093
-
[對行人檢測任務進行了詳細分析，此外還對OpenCV中的源代碼進行了分析]http://www.cnblogs.com/tornadomeet/archive/2012/08/15/2640754.html
