Introduction

COCO的全稱是Common Object in Context, 是 MicroSoft 贊助的CV挑戰, 包含 Detections, Captions, Keypoints 三個Challenge, 及相應的數據集.
接下來根據coco 2016介紹它的Detection Challenge.
值得指出的是, coco2016 detection與coco2015一模一樣.

Dataset下載

數據集由train/val/tests三部分構成, 總共包含超過20,000張標註圖片, 80個類別. 所有的object instance都有bounding box 和segmentation groundtruth, 既可用於detection, 又可用於segmentation.

下載地址 http://mscoco.org/dataset/#download

2014 Testing Images在Captioning Challenge中使用
2015 Testing Images在Detection和 Keypoint Challenges中使用.
2014 Train 和 val Images是所有challenges共用.

所以對於Detection Challenge, 需要用到2014 Training, 2014 Val, 2015 Testing Images 和 2014 Train/Val object instances 共四部分的數據.

Competitors are recommended but not restricted to train their algorithms on COCO 2014 train and val sets. The download page contains links to all COCO 2014 train+val images and associated annotations as well as the 2015 test images. Please specify any and all external data used for training in the “method description” when uploading results to the evaluation server.

官方推薦只使用2014 train/eval訓練模型, 但也可以使用其他數據. 提交結果時需要註明.

cocoAPI

官方提供了代碼和接口用於讀取數據和生成提交結果: https://github.com/pdollar/coco

Annotation Format

Splits與文件信息

下載並解壓標註文件:

文件內容的格式相同, 都包含了info, images, annotations, licenses四部分內容, 其中images和annotations是groundtruth的主體.

一個json文件代表一個split, train/val有4個split, test 1個

image_info_test2014.json
instances_train2014.json: 82783張圖片
instances_val2014.json: 40504張圖片
instances_minival2014.json: 5000張圖片
instances_valminusminival2014.json: 35504張圖片

將val分爲minival和valminusminival, 後者可與train一起用於訓練.

Object Instance Annotations

COCO 現有三種類型的標註:
1. object instances
2. object keypoints
3. image captions
都存儲在Annotations裏面. 只有object instances與Detection相關.
instance annotation的格式如下:

每個instance 都有segment與bbox標註
當一個instance代表單個物體時, iscrowd = 0, segmentation用一個或多個多邊形(polygon)表示. 有遮擋時用多個.
當一個instance代表一羣物體時, iscrowd = 1, segmentation用RLE(Run Length Encoding) 表示.
segmentation與bbox的都是使用絕對座標, float類型, 沒有normalize到0, 1之間.

結果提交

服務器地址: https://competitions.codalab.org/competitions/5181#learn_the_details-evaluation
需要註冊.

To limit overfitting while giving researchers more flexibility to test their system, we have divided the test set into a number of splits, including test-dev, test-standard, and test-challenge. Test-dev is used for debugging and validation experiments and allows for unlimited submission to the evaluation server. Test-standard is used to maintain a public leaderboard that is updated upon submission. Finally, test-challenge is used for the workshop competition; results will be revealed during the workshop at ECCV 2016.

爲方便開發調參及防止在test上過擬合/調參, test 數據集分爲test-dev與test-std與test-challenge. 前兩者現在還可以提交結果. test-dev可以提交很多次(基本可以說不限次數), 用於調試與調參, test-std最多隻能提交5次, 用於leadboard排名: