個人博客地址 Glooow,歡迎光臨~~~
文章目錄
1. Binary Bayesian hypothesis testing
1.0 Problem Setting
- Hypothesis
- Hypothesis space
- Bayesian approach: Model the valid hypothesis as an RV H
- Prior
- Observation
- Observation space
- Observation Model
- Decision rule
- Cost function
- Let
- is valid if
- Optimum decision rule
1.1 Binary Bayesian hypothesis testing
Theorem: The optimal Bayes’ decision takes the form
Proof:
KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \varphi(f) &=…
Given
- if ,
- if ,
So
備註:證明過程中,注意貝葉斯檢驗爲確定性檢驗,因此對於某個確定的 y, 的概率要麼爲 0 要麼爲 1。因此對代價函數求期望時,把 H 看作是隨機變量,而把 看作是確定的值來分類討論
Special cases
- Maximum a posteriori (MAP)
- Maximum likelihood (ML)
1.2 Likelyhood Ratio Test
Generally, LRT
- Bayesian formulation gives a method of calculating
- is a sufficient statistic for the decision problem
- 的可逆函數也是充分統計量
充分統計量
1.3 ROC
- Detection probability
- False-alarm probability
性質(重要!)
- LRT 的 ROC 曲線是單調不減的
2. Non-Bayesian hypo test
- Non-Bayesian 不需要先驗概率或者代價函數
Neyman-Pearson criterion
Theorem(Neyman-Pearson Lemma):NP 準則的最優解由 LRT 得到,其中 由以下公式得到
Proof:
物理直觀:同一個 時 LRT 的 最大。物理直觀來看,LRT 中判決爲 H1 的區域中 都儘可能大,因此 相同時 可最大化
備註:NP 準則最優解爲 LRT,原因是
- 同一個 時, LRT 的 最大
- LRT 取不同的 時, 越大,則 也越大,即 ROC 曲線單調不減
3. Randomized test
3.1 Decision rule
-
Two deterministic decision rules
-
Randomized decision rule by time-sharing
- Detection prob
- False-alarm prob
-
A randomized decision rule is fully described by for m=0,1
3.2 Proposition
-
Bayesian case: cannot achieve a lower Bayes’ risk than the optimum LRT
Proof: Risk for each y is linear in , so the minima is achieved at 0 or 1, which degenerate to deterministic decision
KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \varphi(\mathb… -
Neyman-Pearson case:
- continuous-valued: For a given constraint, randomized test cannot achieve a larger than optimum LRT
- discrete-valued: For a given constraint, randomized test can achieve a larger than optimum LRT. Furthermore, the optimum rand test corresponds to simple time-sharing between the two LRTs nearby
3.3 Efficient frontier
Boundary of region of achievable operation points
- continuous-valued: ROC of LRT
- discrete-valued: LRT points and the straight line segments
Facts
- efficient frontier is concave function
4. Minmax hypo testing
prior: unknown, cost fun: known
4.1 Decision rule
-
minmax approach
-
optimal decision rule
要想證明上面的最優決策,首先引入 mismatch Bayes decision
代價函數如下,可得到 與概率 成線性關係
Lemma: Max-min inequality
Theorem:
Proof of Lemma: Let
Proof of Thm: 先取 ,可得到
由於 任取時上式都成立,因此可以取要想證明定理則只需證明
由前面可知 與 成線性關係,因此要證明上式
- 若 ,只需 ,等式自然成立
- 若 ,只需 ,最優解就是 ; 同理
根據下面的引理,可以得到最優決策就是 Bayes 決策 ,其中 滿足
Lemma:
其他內容請看:
統計推斷(一) Hypothesis Test
統計推斷(二) Estimation Problem
統計推斷(三) Exponential Family
統計推斷(四) Information Geometry
統計推斷(五) EM algorithm
統計推斷(六) Modeling
統計推斷(七) Typical Sequence
統計推斷(八) Model Selection
統計推斷(九) Graphical models
統計推斷(十) Elimination algorithm
統計推斷(十一) Sum-product algorithm