基於最大後驗學習分類分佈參數

最近在看計算機視覺：模型學習與推理。本篇博客介紹：使用狄利克雷分佈作爲先驗概率計算分類分佈的最大後驗概率。

基本原理如下：

$\begin{aligned} \hat{\lambda}_{1 \ldots k} &=\underset{\lambda_{1} \ldots k}{\operatorname{argmax}}\left[\prod_{i=1}^{I} \operatorname{Pr}\left(x_{i} | \lambda_{1 \ldots k}\right) \operatorname{Pr}\left(\lambda_{1 \ldots k}\right)\right] \\ &=\underset{\lambda_{1} \ldots k}{\operatorname{argmax}}\left[\prod_{i=1}^{I} \operatorname{Cat}_{x_{i}}\left[\lambda_{1 \ldots k}\right] \operatorname{Dir}_{\lambda_{1} \ldots k}\left[\alpha_{1 \ldots k}\right]\right] \\ &=\underset{\lambda_{1 \ldots k}}{\operatorname{argmax}}\left[\prod_{i=1}^{k} \lambda_{k}^{N_{k}} \prod_{k=1}^{k} \lambda^{\alpha_{k}-1}\right] \\ &=\underset{\lambda_{1 \ldots k}}{\operatorname{argmax}}\left[\prod_{k=1}^{k} \lambda_{k}^{N_{k}+\alpha_{k}-1}\right] \end{aligned}$

最終結果如下：

$\hat{\lambda}_{k}=\frac{N_{k}+\alpha_{k}-1}{\sum_{m=1}^{k}\left(N_{m}+\alpha_{m}-1\right)}$

數據生成使用的是上一篇文章的方法，這裏不再介紹。

算法流程如下：
$\begin{array}{l}{\text { Input : Binary training data }\left\{x_{i}\right\}_{i=1}^{I}, \text { Hyperparameters }\left\{\alpha_{k}\right\}_{k=1}^{K}} \\ {\text { Output: MAP estimates of parameters } \theta=\left\{\lambda_{k}\right\}_{k=1}^{K}} \\ {\text { begin }} \\ {\text { for } k=1 \text { to } K \mathrm{do}} \\ {\qquad \lambda_{k}=\left(N_{k}-1+\alpha_{k}\right) /\left(I-K+\sum_{k=1}^{K} \alpha_{k}\right)} \\ {\text { end }} \\ {\text { end }}\end{array}$

學習的代碼如下：

void MAP_categorical_distribution_parameters()
{
	vector<int> data;
	data = generate_categorical_distribution_data(100000);
	

	std::map<int, double> hist{};
	for (int i = 0; i < data.size(); i++)
	{
		++hist[data[i]];
	}

	vector<double> alpha_v;

	//set Drichilet  distribution superparameters
	for (int i = 0; i < hist.size(); i++)
	{
		alpha_v.push_back(1.0);
	}

	double total_p = 0;
	double down=0;
	for (int i = 0; i < hist.size(); i++)
	{
		down += hist.at(i) + alpha_v[i] - 1;

	}

	for (int i = 0; i < hist.size(); i++)
	{
		hist.at(i) = (hist.at(i) + alpha_v[i] - 1) / down;
		total_p += hist.at(i);
		std::cout << hist.at(i) << std::endl;
	}
	cout << "total_p: " << total_p << endl;

}

這裏Dirichlet 分佈的超參數都是設置成了1；

下面兩個圖片是書中作者的實驗對比