【OpenPose 学习笔记-03】Example 1. Body from image

本文主要讲解OpenPose开源框架提供的example，本系列先讲解 /openpose/examples/tutorial_api_cpp/ 目录下的17个开源代码。如题，首先记录个人对第一个Example的理解。

测试图片如下：

代码功能：

读取一张图片，进行处理，进行显示以及打印pose关键点。直接上运行结果。根据测试图片，以及输出结果发现，代码实现了对输入图片中的关键点进行检测，并进行了显示以及打印出来了关键点座标。我们会发现每个人会检测出25个关键点，分别对应了身体的不同位置。关键点的输出信息如下：437.255981 123.085121 0.906523，第一个值横座标x，第二个代表纵座标y，第三个值为置信度（取值范围0到1）。备注：座标原点是图片左上角点，横座标轴为水平向右，纵座标轴为竖直向下。

那么问题来了，这25个点座标分别代表人身上的那些关键点？

0->鼻子                                                                                       15->左眼

1->脖子                                                                                       16->右眼

2->左肩                                                                                       17->左耳

3->左肘                                                                                       18->右耳

4->左手                                                                                       19-21  在左脚上的三个点

5->右肩                                                                                       23-24  在右脚上的三个点

6->右肘

7->右手

8->裤带标志的那个点，肚脐往下一点

9->左大腿根部

10->左膝盖

11->左脚

12-> 右大腿根部

13->右膝盖

14->右脚

 

如果其中一个关键点没有检测出来，输出的座标和置信度会是什么呢？

输出的是 （0.0000， 0.0000， 0.0000）。这样会发现有时候检测会出现与原点连接的直线。

 

如果你觉得输出的图不是想要的，可以在原图的基础上，根据检测出来的关键点座标重新绘制改图。

 

Starting OpenPose demo...
Configuring OpenPose...
Starting thread(s)...
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
Body keypoints: Array<T>::toString():
437.255981 123.085121 0.906523 
422.357300 175.192841 0.882059 
364.677948 171.437820 0.814926 
290.418488 227.148422 0.849849 
344.289673 286.660187 0.781581 
479.885468 182.572571 0.782618 
500.376831 253.212280 0.763941 
466.998169 303.359467 0.840747 
409.248596 348.029633 0.734004 
370.322815 349.846497 0.639143 
381.491455 496.625610 0.779169 
377.776337 619.336243 0.790947 
442.822021 348.051361 0.674805 
424.239502 476.230316 0.814295 
426.066895 611.862671 0.755358 
424.295624 110.124794 0.906900 
450.193604 111.934570 0.913698 
407.444244 111.940063 0.931875 
457.615601 113.889832 0.496016 
422.384033 630.541687 0.692417 
439.060333 630.438660 0.658018 
424.144196 619.291992 0.574923 
383.316071 632.364380 0.591681 
372.165527 632.325623 0.557938 
379.592560 630.421936 0.615449 

602.560242 186.393326 0.938347 
591.442383 245.786560 0.888576 
541.204773 245.778137 0.835189 
500.369904 327.570709 0.847654 
515.296631 394.398560 0.896734 
647.120605 245.777084 0.859364 
680.611206 318.214264 0.829531 
706.597290 383.324158 0.856119 
593.272339 388.860046 0.680254 
557.934448 390.729187 0.702126 
556.206360 526.369324 0.818692 
558.051697 632.265320 0.761809 
630.422180 385.165283 0.723132 
617.491333 496.603210 0.860527 
615.490601 615.614441 0.796588 
591.443176 175.210358 0.846590 
615.506775 180.785339 0.893679 
572.840881 184.411392 0.921707 
621.193604 186.370575 0.711615 
598.859558 634.155029 0.585496 
613.754395 634.227600 0.615592 
615.600830 623.015808 0.599560 
546.865784 643.416870 0.279773 
543.092712 641.576782 0.310959 
567.298340 637.879700 0.494990 


OpenPose demo successfully finished. Total time: 1.156978 seconds

代码如下：

// ----------------------------- OpenPose C++ API Tutorial - Example 1 - Body from image -----------------------------
// It reads an image, process it, and displays it with the pose keypoints.

// Command-line user intraface
#define OPENPOSE_FLAGS_DISABLE_POSE
#include <openpose/flags.hpp>
// OpenPose dependencies
#include <openpose/headers.hpp>

// Custom OpenPose flags
// Producer
DEFINE_string(image_path, "/home/wdong/0315/1212/test.jpg",
              "Process an image. Read all standard formats (jpg, png, bmp, etc.).");
// Display
DEFINE_bool(no_display,                 false,
            "Enable to disable the visual display.");

// This worker will just read and return all the jpg files in a directory
void display(const std::shared_ptr<std::vector<std::shared_ptr<op::Datum>>>& datumsPtr)
{
    try
    {
        // User's displaying/saving/other processing here
        // datum.cvOutputData: rendered frame with pose or heatmaps
        // datum.poseKeypoints: Array<float> with the estimated pose
        if (datumsPtr != nullptr && !datumsPtr->empty())
        {
            // Display image
            cv::imshow("01_body_from_image_default.cpp", datumsPtr->at(0)->cvOutputData);
            cv::waitKey(0);
        }
        else
            op::log("Nullptr or empty datumsPtr found.", op::Priority::High);
    }
    catch (const std::exception& e)
    {
        op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
    }
}

void printKeypoints(const std::shared_ptr<std::vector<std::shared_ptr<op::Datum>>>& datumsPtr)
{
    try
    {
        // Example: How to use the pose keypoints
        if (datumsPtr != nullptr && !datumsPtr->empty())
        {
            // Alternative 1
            op::log("Body keypoints: " + datumsPtr->at(0)->poseKeypoints.toString(), op::Priority::High);

            // // Alternative 2
            // op::log(datumsPtr->at(0).poseKeypoints, op::Priority::High);

            // // Alternative 3
            // std::cout << datumsPtr->at(0).poseKeypoints << std::endl;

            // // Alternative 4 - Accesing each element of the keypoints
            // op::log("\nKeypoints:", op::Priority::High);
            // const auto& poseKeypoints = datumsPtr->at(0).poseKeypoints;
            // op::log("Person pose keypoints:", op::Priority::High);
            // for (auto person = 0 ; person < poseKeypoints.getSize(0) ; person++)
            // {
            //     op::log("Person " + std::to_string(person) + " (x, y, score):", op::Priority::High);
            //     for (auto bodyPart = 0 ; bodyPart < poseKeypoints.getSize(1) ; bodyPart++)
            //     {
            //         std::string valueToPrint;
            //         for (auto xyscore = 0 ; xyscore < poseKeypoints.getSize(2) ; xyscore++)
            //             valueToPrint += std::to_string(   poseKeypoints[{person, bodyPart, xyscore}]   ) + " ";
            //         op::log(valueToPrint, op::Priority::High);
            //     }
            // }
            // op::log(" ", op::Priority::High);
        }
        else
            op::log("Nullptr or empty datumsPtr found.", op::Priority::High);
    }
    catch (const std::exception& e)
    {
        op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
    }
}

int tutorialApiCpp()
{
    try
    {
        op::log("Starting OpenPose demo...", op::Priority::High);
        const auto opTimer = op::getTimerInit();

        // Configuring OpenPose
        op::log("Configuring OpenPose...", op::Priority::High);
        op::Wrapper opWrapper{op::ThreadManagerMode::Asynchronous};
        // Set to single-thread (for sequential processing and/or debugging and/or reducing latency)
        if (FLAGS_disable_multi_thread)
            opWrapper.disableMultiThreading();

        // Starting OpenPose
        op::log("Starting thread(s)...", op::Priority::High);
        opWrapper.start();

        // Process and display image
        const auto imageToProcess = cv::imread(FLAGS_image_path);
        auto datumProcessed = opWrapper.emplaceAndPop(imageToProcess);
        if (datumProcessed != nullptr)
        {
            printKeypoints(datumProcessed);
            if (!FLAGS_no_display)
                display(datumProcessed);
        }
        else
            op::log("Image could not be processed.", op::Priority::High);

        // Measuring total time
        op::printTime(opTimer, "OpenPose demo successfully finished. Total time: ", " seconds.", op::Priority::High);

        // Return
        return 0;
    }
    catch (const std::exception& e)
    {
        return -1;
    }
}

int main(int argc, char *argv[])
{
    // Parsing command line flags
    gflags::ParseCommandLineFlags(&argc, &argv, true);

    // Running tutorialApiCpp
    return tutorialApiCpp();
}