基础知识
概率的定义 Axioms of Probability:
- Sample space : The set of all the outcomes of a random experiment.
- Set of events(or event space) : where each event is a set containing zero or more outcomes (i.e., is a collection of possible outcomes of an experiment).
- Probability measure: A function that satisfies the following propeties:
- for all
- If are disjoint events (i.e., whereever ), then
理解:以掷骰子为例, 就是所有的可能出现的点数集{1, 2, 3, 4, 5, 6}, 包括了各种事件,比如{1, 2, 3, 4}, {奇数点数},{偶数点数}等等。
概率的基本属性:
- If are a set of disjoint events such that , then .
条件概率:
is the probability measure of the event A after observing the occurrence of event B. Two events are independent if and only if or .
随机变量
扔10次硬币,出现的所有10个正反面(heads and tails)组合(考虑先后顺序)便是样本空间 ,比如 . 实际问题中,我们往往不关心出现某个特定正反面序列的概率,而更关心real-valued functions of outcomes,比如十次中出现正面的次数,或者连续反面的最长长度,这些函数便是random variables随机变量。所以随机变量是一个函数!
Random variable is a function
Discrete random variable:
Continuous random variable:
CDFs, PDFs, PMFs
1.Cumulative distribution function (CDF) is a function such that
By using this function one can calculate the probability of any event in .
Properties:
- .
- .
- .
- .
2.Probability mass function (PMF) is a function such that
Properties:
- .
- , is the set of all possible values may assume.
- .
3.Probability density functions (PDF) is the derivative of the CDF:
PDF for a continuous random variable may not always exist and for very small ,
The value of PDF at any given point is not the probability of that event, i.e, and can take on values larger than one.
Properties:
- .
- .
- .
Expectation 期望
is a discrete random variable with PMF and is an arbitrary function. In this case, can be considered a random variable, and we define the expectation or expected value of as
If is a continuous random variable with PDF , then the expected value of is defined as,
The expectation of can be thought of as a “weighted average” of the values that g(x) can taken on for different values of , where the weights are given by or . is the mean of random variable .
Properties:
- for any constant .
- for any constant .
- .
- For a discrete random variable , .
Variance 方差
The variance of a random variable is a measure of how concentrated the distribution of a random variable is around its mean:
An alternate expression:
Properties:
- for any constant .
- for an constant .
Some common random variables
Discrete random variables,以扔一次硬币正面朝上的概率为 为例:
- 伯努利分布 (where ):
- 二项式分布 (where ):投掷 次,正面朝上的次数,
- 几何分布 (where ): 投掷几次第一次出现正面朝上,
- 泊松分布 (where ):a probability distribution over the nonnegative integers used for modeling the frequency of rare events,
Continuous random variable: - 均匀分布 (where ): 在 和 之间均等概率,
- 指数分布 (where ):概率密度随着 增加减弱,
- 正态(高斯)分布 (also known as the Gaussian distribution):
PDF and CDF of a couple of random variables:
Summary of some of the properties of these distributions:
双随机变量
Joint and marginal distributions 联合和边际分布
Joint cumulative distribution function 联合累积分布函数 of and :
Properties:
and are the marginal cumulative distribution functions 边际累积分布函数 of .
Joint and marginal probability mass functions
Joint probability mass function : x :
Properties:
- => marginalization
is the marginal probability mass function 边际概率质量函数 of .
Joint and marginal probability density functions
Joint probability density function:
Properties:
is the marginal probability density function 边际概率密度函数 of .
Conditional distributions and Bayes’s rule
The conditional probability mass function of Y given X, assuming that :
The conditional probability density of Y given X, assuming that :
Bayes’s rule: derive expression for the conditional probability of one variable given another. 详见贝叶斯决策理论
Discrete random variables and :
Continuous random variables and :
Independence 独立
Two random variables X and Y are independent:
- If for all values of and .
- For discrete random variables:
- for all
- whenever for all
- For continuous random variables
- for all
- whenever for all
If and are independent then for any subset :
If is independent of then any function of X is indepedent of any funciton of .
Expectation and covariance 期望和协方差
Two discrete variables , and is a function on them:
For continuous random variables , , the analogous expression is
The relationship of two random variables with each other: covariance of two variables and is defined as:
we can rewrite this as:
Properties:
- (Linearity of expectation)
- If and are independent, then
- If and are independent, then
- , we say that and are uncorrelated 不相关. This is not the same thing as stating that and are uncorrelated. For example, if and , then one can show that and are uncorrelated, even though they are not independent.
- 随机变量的 不相关 和 独立 在定义上就是不等价的。独立是不相关的充分不必要条件,即独立可以推出不相关,反之不行。独立就是两个随机变量相互独立,等价于 。随机变量uncorrelated的定义就是协方差为0。