MAP_K 檢索評估指標

1. MAP_K

MAP: mean average precision.

• precision很容易理解；precision@10表示檢索10個文檔中包含相關所佔的比例， 比如檢索了10個，有9個是相關的，那麼 precision@10=0.9

• average precision, 也容易理解。 可以看p1，p2，p3，p4，p5，p6的平均值；就是 ap6的值

• mean 則是對所有的case求mean則是結果。

[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-aG6uDOO8-1590328481054)(ap.png)]

2. 幾個公式

$precision=\frac{|\{relevant\ documents\} \cap \{retrieved\ documents\}|}{|\{retrieved\ documents\}|}$

$recall=\frac{|\{relevant\ documents\} \cap\{retrieved\ documents\}|}{|\{relevant\ documents\}|}$

$AP =\frac{1}{R}\times\sum_{r=1}^R \frac{r}{position(r)}$
r就是第1、2、3； position®則是對應的預測位置；

其中RR表示相關文檔的總個數，position®position®表示，結果列表從前往後看，第rr個相關文檔在列表中的位置。比如，有三個相關文檔，位置分別爲1、3、6，那麼AveP=1/3×(1/1+2/3+3/6); 如果再列表裏不出現則不需要計算，默認0。

3. 舉例

__Recommendations__	__Precision @k's__	__AP@3__
[0, 0, 1]	[0, 0, 1/3]	(1/3)(1/3) = 0.11
[0, 1, 1]	[0, 1/2, 2/3]	(1/3)[(1/2) + (2/3)] = 0.38
[1, 1, 1]	[1/1, 2/2, 3/3]	(1/3)[(1) + (2/2) + (3/3)] = 1

4. 實戰

! pip install ml_metrics

Looking in indexes: http://mirrors.aliyun.com/pypi/simple/, http://pypi.nexus.patsnap.local/repository/pypi-group/simple
Collecting ml_metrics
  Downloading http://mirrors.aliyun.com/pypi/packages/c1/e7/c31a2dd37045a0c904bee31c2dbed903d4f125a6ce980b91bae0c961abb8/ml_metrics-0.1.4.tar.gz
Requirement already satisfied: numpy in /Users/wangweilei/anaconda3/envs/pytorchenv/lib/python3.7/site-packages (from ml_metrics) (1.16.2)
Requirement already satisfied: pandas in /Users/wangweilei/anaconda3/envs/pytorchenv/lib/python3.7/site-packages (from ml_metrics) (0.25.1)
Requirement already satisfied: pytz>=2017.2 in /Users/wangweilei/anaconda3/envs/pytorchenv/lib/python3.7/site-packages (from pandas->ml_metrics) (2019.2)
Requirement already satisfied: python-dateutil>=2.6.1 in /Users/wangweilei/anaconda3/envs/pytorchenv/lib/python3.7/site-packages (from pandas->ml_metrics) (2.8.0)
Requirement already satisfied: six>=1.5 in /Users/wangweilei/anaconda3/envs/pytorchenv/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas->ml_metrics) (1.12.0)
Building wheels for collected packages: ml-metrics
  Building wheel for ml-metrics (setup.py) ... [?25ldone
[?25h  Stored in directory: /Users/wangweilei/Library/Caches/pip/wheels/d4/59/90/5a94d64e7ab0bb6ea388ff376b013b1a5abae15fbccc30455d
Successfully built ml-metrics
Installing collected packages: ml-metrics
Successfully installed ml-metrics-0.1.4

import numpy as np
import pandas as pd
from subprocess import check_output
import ml_metrics as metrics

actual=range(1,6)
print(actual)

range(1, 6)

predict = [6,4,7,1,2]
for i in range(1,5):
    apk = metrics.apk(actual, predict,i)
   
    print("i={},apk={}".format(i,apk))

i=1,apk=0.0
i=2,apk=0.25
i=3,apk=0.16666666666666666
i=4,apk=0.25

map = metrics.mapk([[1,3,4],[1,2,4],[1,3]],
            [range(1,6),range(1,6),range(1,6)], 3)
map

0.6851851851851851

總結

1. 瞭解MAP是檢索相關性的評估指標
2. 利用ml_metrics實戰