[scikit-learn 機器學習] 2. 簡單線性迴歸

本文爲 scikit-learn機器學習（第2版）學習筆記

1. 簡單線性迴歸

import numpy as np
import matplotlib.pyplot as plt

X = np.array([[6],[8],[10],[14],[18]])
y = np.array([7,9,13,17.5,18])
plt.title("pizza diameter vs price")
plt.xlabel('diameter')
plt.ylabel('price')
plt.plot(X,y,'r.') # r表示顏色紅

from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X,y)

test_pizza = np.array([[12]])
pred_price = model.predict(test_pizza)
pred_price
# array([13.68103448])

• 誤差 $\sum(y_i-f(x_i))^2$

print("誤差平方和：%.2f" % np.mean((model.predict(X)-y)**2))
誤差平方和：1.75

• 方差 $var(x) = \frac{\sum(x_i-\bar x)^2}{n-1}$

# 方差
x_bar = X.mean() # 11.2
variance = ((X-x_bar)**2).sum()/(len(X)-1)
variance # 23.2

np.var(X, ddof=1) # np內置的方差，ddof爲校正選項
###################
ddof : int, optional
        "Delta Degrees of Freedom": the divisor used in the calculation is
        ``N - ddof``, where ``N`` represents the number of elements. By
        default `ddof` is zero.

• 協方差 $cov(x,y) = \frac{\sum(x_i-\bar x)(y_i - \bar y)}{n-1}$

# 協方差，兩個變量之間的相關性
y_bar = y.mean()
covariance = np.multiply((X-x_bar).transpose(), y-y_bar).sum()/(len(X)-1)
covariance # 22.65

np.cov(X.transpose(), y)

array([[23.2 , 22.65],
       [22.65, 24.3 ]])

假設模型爲 $y = a+bx$

$b = \frac{cov(x,y)}{var(x)} = 22.65/23.2 = 0.98$

$a = \bar y - b \bar x = 12.9-0.98*11.2=1.92$

模型爲 $y = 1.92+0.98x$

2. 評價模型

$R^2 = 1-\frac{\sum(y_i-f(x_i))^2}{\sum(y_i-\bar y)^2}$

X_test = np.array([8,9,11,16,12]).reshape(-1,1)
y_test = [11,8.5,15,18,11]
r_squared = model.score(X_test, y_test)
r_squared # 0.6620052929422553