基於貝葉斯方法學習分類分佈參數

最近在看計算機視覺：模型學習和推理，現在在使用c++實現裏面的代碼。本篇博客使用c++實現基於貝葉斯方法學習分類分佈的參數。

$\begin{aligned} \operatorname{Pr}\left(\lambda_{1} \ldots \lambda_{k} | x_{1 \ldots I}\right) &=\frac{\prod_{i=1}^{I} \operatorname{Pr}\left(x_{i} | \lambda_{1 \ldots k}\right) \operatorname{Pr}\left(\lambda_{1 \ldots k}\right)}{\operatorname{Pr}\left(x_{1 \ldots I}\right)} \\ &=\frac{\prod_{i=1}^{I} \operatorname{Cat}_{x_{i}}\left[\lambda_{1 \ldots k}\right] \operatorname{Dir}_{\lambda_{1} \ldots k}\left[\alpha_{1 \ldots k}\right]}{\operatorname{Pr}\left(x_{1 \ldots I}\right)} \\ &=\frac{\kappa\left(\alpha_{1 \ldots k}, x_{1 \ldots I}\right) \operatorname{Dir}_{\lambda_{1 \ldots k}}\left[\tilde{\alpha}_{1 \ldots k}\right]}{\operatorname{Pr}\left(x_{1 \ldots I}\right)} \\ &=\operatorname{Dir}_{\lambda_{1 \ldots k}\left[\tilde{\alpha}_{1 \ldots k}\right]} \end{aligned}$

使用貝葉斯去做預測:

$\begin{aligned} \operatorname{Pr}\left(x^{*} | x_{1 \ldots I}\right) &=\int \operatorname{Pr}\left(x^{*} | \lambda_{1 \ldots k}\right) \operatorname{Pr}\left(\lambda_{1 \ldots k} | x_{1 \ldots I}\right) d \lambda_{1 \ldots k} \\ &=\int \operatorname{Cat}_{x^{*}}\left[\lambda_{1 \ldots k}\right] \operatorname{Dir}_{\lambda_{1 \ldots k}}\left[\tilde{\alpha}_{1 \ldots k}\right] d \lambda_{1 \ldots k} \\ &=\int \kappa\left(x^{*}, \tilde{\alpha}_{1 \ldots k}\right) \operatorname{Dir}_{\lambda_{1 \ldots k}}\left[\breve{\alpha}_{1 \ldots k}\right] d \lambda_{1 \ldots k} \\ &=\kappa\left(x^{*}, \tilde{\alpha}_{1 \ldots k}\right) \end{aligned}$

結果表示爲：
$\operatorname{Pr}\left(x^{*}=k | x_{1 \ldots I}\right)=\kappa\left(x^{*}, \tilde{\alpha}_{1 \ldots k}\right)=\frac{N_{k}+\tilde{\alpha}_{k}}{\sum_{j=1}^{k}\left(N_{j}+\tilde{\alpha}_{j}\right)}$

算法流程如下：

$\begin{array}{l}{\text { Input : Categorical training data }\left\{x_{i}\right\}_{i=1}^{I}, \text { Hyperparameters }\left\{\alpha_{k}\right\}_{k=1}^{K}} \\ {\text { Output: Posterior parameters }\left\{\tilde{\alpha}_{k}\right\}_{k=1}^{K}, \text { predictive distribution } \operatorname{Pr}\left(x^{*} | \mathbf{x}_{1} \ldots I\right)} \\ {\text { begin }} \\ {\text { l compute categsorical posterior over } \lambda} \\ {\text { for } k=l \text { to } K \text { do }} \\ {\text { Evaluate new datapoint under predictive distribution }} \\ {\text { Evaluate new datapoint under predictive distribution }} \\ {\text { for } k=1 \text { to } K \text { do }} \\ {\quad \quad \operatorname{tr}\left(x^{*}=k | \mathbf{x}_{1 \ldots I}\right)=\tilde{\alpha_{k}} /\left(\sum_{m=1}^{K} \tilde{\alpha}_{m}\right)} \\ {\text { end }}\end{array}$

代碼如下：

void Bayesian_categorical_distribution_parameters()
{
	vector<int> data;
	data = generate_categorical_distribution_data(100000);


	std::map<int, double> hist{};
	for (int i = 0; i < data.size(); i++)
	{
		++hist[data[i]];
	}

	vector<double> alpha_v;
	vector<double> alpha_v_post;

	//set Drichilet  distribution superparameters
	for (int i = 0; i < hist.size(); i++)
	{
		alpha_v.push_back(1.0);
	}

	double total_p = 0;

	for (int i = 0; i < hist.size(); i++)
	{
		alpha_v_post.push_back(alpha_v[i]+hist.at(i));
	}

	double down = 0;
	for (int i = 0; i < hist.size(); i++)
	{
		down += alpha_v_post[i];
	}
	for (int i = 0; i < hist.size(); i++)
	{
		hist.at(i) = alpha_v_post[i] / down;
		total_p += hist.at(i);
		std::cout << hist.at(i) << std::endl;
	}
	cout << "total_p: " << total_p << endl;
}

在書中作者的代碼給出瞭如下兩張圖，作爲對貝葉斯方法的解釋