在強化學習的世界，我們相信如果輸入是確定的，那麼輸出也一定是確定的。強化學習算法要有用，就是相信在強化學習中每一次參數的調整都會對世界造成確定性的影響。
1.on-policy和off-policy
下圖是Q-learning和Sarsa的流程圖，分別對應off-policy和on-policy
2-tradeoff variance and bias
wiki裏的解釋：
In statistics and machine learning, the bias–variance tradeoff is the property of a set of predictive models whereby models with a lower bias in parameter estimation have a higher variance of the parameter estimates across samples, and vice versa. The bias–variance dilemma or bias–variance problem is the conflict in trying to simultaneously minimize these two sources of error that prevent supervised learning algorithms from generalizing beyond their training set:[citation needed]
- The bias error is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting).
- The variance is an error from sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data, rather than the intended outputs (overfitting).
偏差和方差則需要在統計上做對應的定義。
- 偏差（bias）描述的是通過學習擬合出來的結果之期望，與真實規律之間的差距，記作 Bias(X)=E[f^(X)]−f(X)Bias(X)=E[f^(X)]−f(X)。
- 方差（variance）即是統計學中的定義，描述的是通過學習擬合出來的結果自身的不穩定性，記作 Var(X)=E[(f^(X)−E[f^(X)])2]Var(X)=E[(f^(X)−E[f^(X)])2]。