【Statistics】HYPOTHESIS TEST(SIGNIFICANCE TEST)

本文着重梳理 假設檢驗 HYPOTHESIS TEST(SIGNIFICANCE TEST)，通過 邏輯性知識 和 概念性知識 兩部分釐清該重點內容。

1.Logic of Hypothesis Test

0) What’s hypothesis test？

If we have some doubt in the origin hypothesis or assumption, then we can raise a hypothesis test to prove our doubt or said reject the origin hypothesis.

For example:

Then, we introduce some new concepts on hypothesis:

• $H_0$ (Null hypothesis): Something we doubt.

• $H_\text{a}$ (Alternative hypothesis): Our guess, or said the new hypothesis

• Single-sided and Double-sided Hypothesis :

  • If our new hypothesis is in the form of $new\ hypothesis \neq some \ number$, this is a double-sided hypothesis ;
  • If our new hypothesis is in the form of $new\ hypothesis \leq some \ number$ or $new\ hypothesis \geq some \ number$, this is a double-sided hypothesis.

PAY ATTENTION: All the hypothesis is aimed at testing the population parameter.

---

1) Set up Hypothesis

To set up a Null Hypothesis, we can just need to figure out what is the concerned problem. Or said the feature of null hpothesis is that there is no news if the null hypothesis is actually true.

To set up a lnternative Hypothesis, we use the number of null hypothesis and then choose single-sided or double-sided hypothesis to set up our internative hypothesis.

---

2) Set up Significance level

PAY ATTENTION: Before we carry out the calculation, we need to set up a significance level. It is an ethical problem if we set up a significance level to suit our calculation result in order to generate a attracting conclusion.

---

3) Take Sample and Calculate

---

4) Make Conclusion

---

2.Concepts of Hypothesis Test

1) What’s p-value and significance level？

• $p-value$: It is a probability that current sample statistic occur.

$P(observe\ current\ sample\ statistic |H_0 \ is \ True)$

• $\alpha$: We call it significance level. It is a thresold that quantify the word “extreme”. In ohter words, it’s a relatively small probability that indicates whether the $p-value$ is small enough that shake our belief on $H_0$.

• $power $: It is a probability that not making Type II error

$P(rejecet\ H_0|H_0 \ is \ false) = 1 - P(not \ rejecet\ H_0|H_0 \ is \ false) = P(not\ making \ Type \ II \ error)$

---

2) Type I Error & Type II Error

i) Understanding the concepts

Meaning of Type I Error ：The origin hypothesis ($H_0$) is actually True, but due to some extreme event happens ($p-value < \alpha$), we consider the hypothesis might be wrong and therefore reject it. It is obivous that if the $\alpha$ is too large, we can easily get a Type I Error.

Meaning of Type II Error：The origin hypothesis ($H_0$) is actually False, but nothing seems to happen ($p-value > \alpha$), then we consider the hypothesis should be true and therefore accpet it. It is obivous that if the $\alpha$ is too small, we can easily get a Type II Error. Another way to think of Type II Error is that if we can start from using the concept of $power$ instead of $p-value$.

Trade-off problems : There exists a trade-off between Type I Error and Type II Error, which means that we need to set an appropriate $\alpha$ to “balance” the error probability of these two type of error. Here is an example on trade-off problem:

Employees at a health club do a daily water quality test in the club’s swimming pool. If the level of contaminants are too high, then they temporarily close the pool to perform a water treatment.

We can state the hypotheses for their test as $H_0$ : The water quality is acceptable vs. $H_\text{a}$ : The water quality is not acceptable. Consider the following two questions:

• In terms of safety, which error has the more dangerous consequences in this setting?
• What significance level should they use to reduce the probability of the more dangerous error?

FROM Khan Academy

What will affect the error probability :

• Significance level $\alpha$: If $\alpha \uparrow$, then $power \uparrow$ and $P(type \ I \ error)\uparrow$
• Sample size n: If $n \uparrow$, then $power \uparrow$. But it doesn’t impact the likelihood of a Type I error.Larger samples are still preferred since they produce less variable results, but we’ll still reject a true $H_0$ at a rate equal to the significance level $α$.
• Statistic variability : If $Statistic\ variability \downarrow$ , then $power \uparrow$. It suits the intuition as when the statistic variablity is low, the outliner should be easier to figured out, and therefore we can hava higher probability to reject it. But we could not control this variable.
• Distance from true parameter to $H_0$ : If $distance \uparrow$ , then $power \uparrow$. It suits the intuition as when the true value far from $H_0$, we can hava higher probability to reject it. But we could not control this variable.

ii) Some associations

• 看論文經常會接觸到以下的表格，出處正是來自statistics

Table of error types	$H_0$ is True(in reality)	$H_0$ is False(in reality)
Fail to reject	Correct inference	Type II error(False Negative)
Reject	Type I error(False Positive)	Correct inference

• ROC曲線

---

3) Z-test and T-test

---

3.Others

1.Some common ideas between confidence interval and significance test