第2章练习题
1.使用不同的超参数,如kernel=“linear”(具有C超参数的多种值)或kernel=“rbf”(C超参数和gamma超参数的多种值),尝试一个支持向量机回归器,不用担心现在不知道这些超参数的含义。最好的SVR预测期是如何工作的?
from sklearn.model_selection import GridSearchCV
param_grid = [
{'kernel': ['linear'], 'C': [10., 30., 100., 300., 1000., 3000., 10000., 30000.0]},
{'kernel': ['rbf'], 'C': [1.0, 3.0, 10., 30., 100., 300., 1000.0],
'gamma': [0.01, 0.03, 0.1, 0.3, 1.0, 3.0]},
]
svm_reg = SVR()
grid_search = GridSearchCV(svm_reg, param_grid, cv=5, scoring='neg_mean_squared_error', verbose=2, n_jobs=4)
grid_search.fit(housing_prepared, housing_labels)
提醒:搜寻最佳参数时间有点慢~
negative_mse = grid_search.best_score_
rmse = np.sqrt(-negative_mse)
rmse
grid_search.best_params_
分析: 通过网格搜索可以看出,线性核的SVR比rbf核更适合。而且需要注意的是C的值是候选参数中最大的,所以我们可以尝试更大的C值,因为C越大越好。
2. 尝试用RandomizedSearchCV替换GridSearchCV
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import expon, reciprocal
# scipy统计分析库: https://docs.scipy.org/doc/scipy/reference/stats.html
# 有关于 `expon()` and `reciprocal()` 文档 和 更多概率分布函数.
# 注意: 当 kernel 是线性时忽略gamma
param_distribs = {
'kernel': ['linear', 'rbf'],
'C': reciprocal(20, 200000),
'gamma': expon(scale=1.0),
}
svm_reg = SVR()
rnd_search = RandomizedSearchCV(svm_reg, param_distributions=param_distribs,
n_iter=50, cv=5, scoring='neg_mean_squared_error',
verbose=2, n_jobs=4, random_state=42)
rnd_search.fit(housing_prepared, housing_labels)
negative_mse = rnd_search.best_score_
rmse = np.sqrt(-negative_mse)
rmse
这个值更接近于随机森林了。看一下最好的参数:
rnd_search.best_params_
我们的C和gamma分别是用reciprocal分布和指数分布去采样的。模拟采样过程:
expon_distrib = expon(scale=1.)
samples = expon_distrib.rvs(10000, random_state=42)
plt.figure(figsize=(10, 4))
plt.subplot(121)
plt.title("Exponential distribution (scale=1.0)")
plt.hist(samples, bins=50)
plt.subplot(122)
plt.title("Log of this distribution")
plt.hist(np.log(samples), bins=50)
plt.show()
C的选取是从给定范围的均匀分布内选取的,所以2张图看起来差别不是很大。当不知道目标比例时,这种分布非常有用。
reciprocal_distrib = reciprocal(20, 200000)
samples = reciprocal_distrib.rvs(10000, random_state=42)
plt.figure(figsize=(10, 4))
plt.subplot(121)
plt.title("Reciprocal distribution (scale=1.0)")
plt.hist(samples, bins=50)
plt.subplot(122)
plt.title("Log of this distribution")
plt.hist(np.log(samples), bins=50)
plt.show()
如上图,reciprocal 分布是一个典型的对数均匀分布。当已经知道超参数的范围大小时,指数分布是最好的。
3. 尝试再准备流水线中添加一个转换器,从而只选出最重要的属性。
首先,实现选择TopK的方法。
from sklearn.base import BaseEstimator, TransformerMixin
def indices_of_top_k(arr, k):
return np.sort(np.argpartition(np.array(arr), -k)[-k:])
class TopFeatureSelector(BaseEstimator, TransformerMixin):
def __init__(self, feature_importances, k):
self.feature_importances = feature_importances
self.k = k
def fit(self, X, y=None):
self.feature_indices_ = indices_of_top_k(self.feature_importances, self.k)
return self
def transform(self, X):
return X[:, self.feature_indices_]
如果已经以某种方式计算了特征重要度(比如RF中),那么再用TopFeatureSelector.fit()会降低网格/随机搜索的速度,除非实现缓存。
k=5
top_k_feature_indices = indices_of_top_k(feature_importances, k)
top_k_feature_indices
np.array(attributes)[top_k_feature_indices]
sorted(zip(feature_importances, attributes), reverse=True)[:k]
其次,加入到pipeline中。
preparation_and_feature_selection_pipeline = Pipeline([
('preparation', full_pipeline),
('feature_selection', TopFeatureSelector(feature_importances, k))
])
housing_prepared_top_k_features = preparation_and_feature_selection_pipeline.fit_transform(housing)
查看前3特征,并校验正确性
housing_prepared_top_k_features[0:3] # 查看
housing_prepared[0:3, top_k_feature_indices]#校验
4.尝试创建一个覆盖完整的数据准备和最终预测的流水线
prepare_select_and_predict_pipeline = Pipeline([
('preparation', full_pipeline),
('feature_selection', TopFeatureSelector(feature_importances, k)),
('svm_reg', SVR(**rnd_search.best_params_))
])
prepare_select_and_predict_pipeline.fit(housing, housing_labels)
some_data = housing.iloc[:4]
some_labels = housing_labels.iloc[:4]
print("Predictions:\t", prepare_select_and_predict_pipeline.predict(some_data))
print("Labels:\t\t", list(some_labels))
流程没问题,但是预测效果不理想,如果用RF会好很多。
5.使用GridSearchCV自动搜索一些准备选线。
param_grid = [{
'preparation__num__imputer__strategy': ['mean', 'median', 'most_frequent'],
'feature_selection__k': list(range(1, len(feature_importances) + 1))
}]
grid_search_prep = GridSearchCV(prepare_select_and_predict_pipeline, param_grid, cv=5,
scoring='neg_mean_squared_error', verbose=2, n_jobs=4)
grid_search_prep.fit(housing, housing_labels)
grid_search_prep.best_params_