指針式儀表識別（二）基於輪廓的傾斜儀表校正

本文將介紹如何利用OpenCV，提取圖片中的矩形輪廓特徵並進行圖片的傾斜校正。完成demo程序可以至：基於輪廓的傾斜儀表校正

本demo所處理的圖片是一張儀表（如下圖），欲實現的功能是將圖片中傾斜的PCB校正爲水平。基本的思路是檢測PCB的邊緣直線，而後根據邊緣直線的斜率旋轉圖片。但是由於儀表上存在指針，所以圖片在經過輪廓提取直線檢測後，會誤檢出很多條直線。demo程序中主要的算法就是從含有噪聲的直線簇中提取出直線中相互垂直的直線。 
    

Step1: 直線檢測。OpenCV提供了一個檢測直線的函數HoughLinesP()，關於此函數的API介紹，可以參考HoughLinesP。此步驟主要程序如下：

Mat imgOrigion = imread(IMAGE_PATH);
Mat imgScale;
float scaleFactor = COLSLIMIT / imgOrigion.cols;
resize(imgOrigion, imgScale, Size(imgOrigion.cols * scaleFactor, imgOrigion.rows * scaleFactor));  // reduce image size to speed up calculation

Mat imgGray;
cvtColor(imgScale, imgGray, COLOR_BGR2GRAY);  // gray scale
Mat imgCanny;
Canny(imgGray, imgCanny, 100, 200);  // use canny operator to detect contour
imshow("Contour detection", imgCanny);

std::vector<Vec4i> lineAll;
HoughLinesP(imgCanny, lineAll, 1, CV_PI / 180, 30, 50, 4);
// draw all lines detected
Mat imgAllLines;
imgScale.copyTo(imgAllLines);
for (int i = 0, steps = lineAll.size(); i < steps; i++)
{
    line(imgAllLines, Point(lineAll[i][0], lineAll[i][1]), Point(lineAll[i][2], lineAll[i][3]), Scalar(255, 255, 255), 3, 8);
}
imshow("All lines detected", imgAllLines);

    

上圖爲使用Canny算子檢測出的邊緣。儀表板上的輪廓使得算法檢測出了豐富的輪廓信息。對此輪廓圖進行直線檢測，結果如下圖所示，除了儀表的邊緣直線被檢測出來， 算法還誤檢出了很多的直線。因此，下一步算法的目的就是提取邊緣直線，濾出儀表盤內的誤檢直線。

  

 

Step2：直線濾波。儀表邊緣直線與誤檢直線所具有的一個不同特徵是邊緣直線相互垂直，誤檢直線不一定能找到與其相垂直的直線。算法利用此特徵對所有檢測出的直線進行濾波

std::list<Vec4i> linesList;
for (std::vector<Vec4i>::iterator itor = lineAll.begin(); itor != lineAll.end(); ++itor)
{
    linesList.push_back(*itor);
}
std::vector<Vec4i> lineFiltered;
for (std::list<Vec4i>::iterator itorOuter = linesList.begin(); itorOuter != linesList.end();)
{
    for (std::list<Vec4i>::iterator itorInner = linesList.begin(); itorInner != linesList.end(); ++itorInner)
    {
        if (abs(angleOfLines(*itorOuter, *itorInner) - 90) < 1)
        {
            // take out the current two perpendicular lines to reduce the size of linesList
            lineFiltered.push_back(*itorOuter);
            lineFiltered.push_back(*itorInner);
            itorInner = linesList.erase(itorInner);
            itorOuter = linesList.erase(itorOuter);
            break;
        }

        if (itorInner == --linesList.end())
        {
            if (linesList.size() > 2)
            {
                itorOuter = linesList.erase(itorOuter);  // erase current element when there is no other line perpendicular to it.
            }
            else
            {
                itorOuter = linesList.end();
                break;
            }
        }
    }
}

Mat imgLinesFiltered;
imgScale.copyTo(imgLinesFiltered);
// draw lines after filtering
for (int i = 0, steps = lineFiltered.size(); i < steps; i++)
{
    line(imgLinesFiltered, Point(lineFiltered[i][0], lineFiltered[i][1]), Point(lineFiltered[i][2], lineFiltered[i][3]), Scalar(255, 0, 0), 3, 8);
}
imshow("Lines after filtering", imgLinesFiltered);

/**
* @brief calculate the angle of two lines by using vector angle formula: cos(thea) = (a*b) / (|a||b|)
* @param line1
* @param line2
* @return result ranges from 0 to pi
*/
double angleOfLines(const cv::Vec4i& line1, const cv::Vec4i& line2)
{
    double moduleLine1 = sqrt(pow(line1[0] - line1[2], 2) + pow(line1[1] - line1[3], 2));
    double moduleLine2 = sqrt(pow(line2[0] - line2[2], 2) + pow(line2[1] - line2[3], 2));
    double dotProduct = (line1[2] - line1[0])*(line2[2] - line2[0]) + (line1[3] - line1[1])*(line2[3] - line2[1]);

    return acos(dotProduct / moduleLine1 / moduleLine2) * 180 / CV_PI;
}

    

 

Step3：傾角計算。 
濾波後的直線簇中可能會存在與邊緣直線相平行的直線，因此通過對std::vector lineFiltered進行四次排序，提取出最靠圖像邊界的直線。排序規則即爲分別對線段中點座標x,y的值升序降序排列。注意，在圖像座標系中，原點是圖像的最左上角。 
計算邊緣直線的斜率。下面的程序中angleForCorrect()函數返回的是圖像最終需要旋轉的角度。
 

double correctAngle = 0.0;  // average tilt angle of PCB 
if (lineFiltered.size() > 0)
{
    // find edge lines of PCB
    std::vector<Vec4i> lineEdge;
    sort(lineFiltered.begin(), lineFiltered.end(), getMinMidX);  // get the line at the far left of the image
    lineEdge.push_back(lineFiltered[0]);
    sort(lineFiltered.begin(), lineFiltered.end(), getMaxMidX);  // get the line at the far right of the image
    lineEdge.push_back(lineFiltered[0]);
    sort(lineFiltered.begin(), lineFiltered.end(), getMinMidY);  // get the line at the top of the image
    lineEdge.push_back(lineFiltered[0]);
    sort(lineFiltered.begin(), lineFiltered.end(), getMaxMidY);  // get the line at the buttom of the image
    lineEdge.push_back(lineFiltered[0]);

    Mat imgLinesEdge;
    imgScale.copyTo(imgLinesEdge);
    // draw lines after filtering
    for (int i = 0, steps = lineEdge.size(); i < steps; i++)
    {
        line(imgLinesEdge, Point(lineEdge[i][0], lineEdge[i][1]), Point(lineEdge[i][2], lineEdge[i][3]), Scalar(0, 0, 255), 3, 8);
    }
    imshow("PCB edge lines", imgLinesEdge);

    for (int i = 0, step = lineEdge.size(); i < step; i++)   // calcualte averge tilt angle of PCB edge lines
    {
        correctAngle += angleForCorrect(lineEdge[i]);
    }
    correctAngle /= lineEdge.size();
}

/**
* @brief comparison function for sort, sort vector<Vec4i> from small to large accodoring to x of the midpoint of each element
* @param line1
* @param line2
* @return
*/
bool getMinMidX(const cv::Vec4i& line1, const cv::Vec4i& line2)
{
    return (line1[0] + line1[2]) < (line2[0] + line2[2]); // Although middle point compared, there is no need to divide 2
}

/**
* @brief comparison function for sort, sort vector<Vec4i> from large to small accodoring to x of the midpoint of each element
* @param line1
* @param line2
* @return
*/
bool getMaxMidX(const cv::Vec4i& line1, const cv::Vec4i& line2)
{
    return (line1[0] + line1[2]) > (line2[0] + line2[2]);
}

/**
* @brief comparison function for sort, sort vector<Vec4i> from small to large accodoring to y of the midpoint of each element
* @param line1
* @param line2
* @return
*/
bool getMinMidY(const cv::Vec4i& line1, const cv::Vec4i& line2)
{
    return (line1[1] + line1[3]) < (line2[1] + line2[3]);
}

/**
* @brief comparison function for sort, sort vector<Vec4i> from large to small accodoring to y of the midpoint of each element
* @param line1
* @param line2
* @return
*/
bool getMaxMidY(const cv::Vec4i& line1, const cv::Vec4i& line2)
{
    return (line1[1] + line1[3]) > (line2[1] + line2[3]);
}

/**
* @brief rotation angle in degrees for correcting tilt
* @param line: for cv::Vec4i& line, [0] is always smaller than [2]
* @return The symbol of the result represnts the direction of rotation to correct tilt.
*         Positive values mean counter-clockwise rotation (the coordinate origin is assumed to be the top-left corner).
*/
double angleForCorrect(const cv::Vec4i& line)
{
    Vec4i unitXVector(0, 0, 1, 0);
    double angle = angleOfLines(unitXVector, line);  // here angle belongs to [0, pi/2]
    // @attention: the increment direction of X and Y axis of OpenCV is different from usual rectangular coordinate system. The origin point is in the upper left corner of the image
    if (angle < 45)
    {
        // consider in the horizontal direction
        if (line[1] > line[3])
        {
            angle = -angle;
        }
    }
    else
    {
        // consider in the vertical direction
        if (line[1] > line[3])
        {
            angle = 90 - angle;
        }
        else
        {
            angle = angle - 90;
        }
    }

    return angle;
}

    

 

Step4：圖像旋轉。 
根據上一步得到的傾斜角度，用以下函數旋轉圖片，校正傾斜。

void rotateIamge(cv::Mat& src, cv::Mat& dst, double angle)
{
    cv::Point2f center(src.cols / 2, src.rows / 2);
    cv::Mat rot = getRotationMatrix2D(center, angle, 1);
    cv::Rect box = RotatedRect(center, src.size(), angle).boundingRect(); // get circumscribed rectangle
    cv::warpAffine(src, dst, rot, box.size());
}

    

得到的圖片做進一步的裁剪，得到儀表盤圖片，利於下一步做指針式儀表識別。