周志華Watermelon Book SVM部分公式補充、一些原理解釋

Z.F Zhou Watermelon Book SVM.This article is a supplementary material for SVM.

Watermelon Book

6.3

Suppose training data set is linear separable.Minimum margin is $\delta$.
$\begin{cases} w^Tx+b>=+\delta,y_i=+1\\ w^Tx+b<=-\delta,y_i=-1 \end{cases}\\ \rightarrow \begin{cases} \frac{w^T}{\delta}x+\frac{b}{\delta}>=+1,y_i=+1\\ \frac{w^T}{\delta}x+\frac{b}{\delta}<=-1,y_i=-1 \end{cases}$

6.8

Refer to the below link about lagrange and KKT.

6.9

$L=1/2w^Tw+\sum_{i=1}^ma_i(1-y_i(w^tx_i+b))\\ derivation\\ 0=w-\sum_{i=1}^ma_iy_ix_i\\ \rightarrow w=\sum_{i=1}^ma_iy_ix_i$

6.11

You can compute $\frac{1}{2}||w||^2$ and $\sum_{i=1}^ma_i(1-y_i(w^tx_i+b))$ respectively.I am lazy.

6.18

Because the $a_i$ may have error with theory value using SMO.

6.41

In below of 6.41,the interpretation about support vectors is based on $a_i!=0$.

[1] Partial Reference

KKT

part1

What is clear interpretation is:
Three equations base on:
$u=0\ or\ g=0\ where\ u\ and\ g\ are\ vectors.(u_i=0\ or\ g_i=0)$
For three equations,we can compute a $x$ minimizing $f(x)$ by compute another formulation,e,g,$\min_x\max_u L(x,u)$.Their $x$ and $u$ are common.

part2

Why is the $L(\hat{x},u)$ equivalent to the $\min_x L(x,u)$?
I think it can use proof by contradiction.Suppose $L(x',u)=\min_x L(x,u)$.Then $\max_u\max_xL(x,u)=\max_uL(x',u)=f(x')!=f(\hat{x})$.It is contradictory.

Lagrange

I think the link in below is very detailed.

Reference

[1]Lagrange and KKT condition.
[2]拉格朗日乘數