前面一篇介紹了用tensorflow實現線性迴歸模型預測sklearn內置的波士頓房價，現在這一篇就記一下用邏輯迴歸分類sklearn提供的乳腺癌數據集，該數據集有569個樣本，每個樣本有30維，爲二分類數據集，212個正樣本，357個負樣本。
首先，加載數據，並劃分訓練集和測試集：
# 加載乳腺癌數據集，該數據及596個樣本，每個樣本有30維，共有兩類
cancer = skd.load_breast_cancer()
# 將數據集的數據和標籤分離
X_data = cancer.data
Y_data = cancer.target
x_train,x_test,y_train,y_test = train_test_split(X_data,Y_data,test_size=0.3,random_state=0)
這裏還要注意一下，因爲後面要用到的損失函數爲交叉熵，而數據集的標籤是一維的，所以我們需要將其轉換爲位數來表示類別，也就是[0,1]=>[[0,1],[1,0]]這種形式的標籤：
y_train_new = np.zeros([y_train.shape[0], 2], dtype='int32')
y_test_new = np.zeros([y_test.shape[0], 2], dtype='int32')
for i in range(y_train.shape[0]):
if y_train[i] == 0:
y_train_new[i,0] = 0
y_train_new[i,1] = 1
else:
y_train_new[i,0] = 1
y_train_new[i,1] = 0
for i in range(y_test.shape[0]):
if y_test[i] == 0:
y_test_new[i,0] = 0
y_test_new[i,1] = 1
else:
y_test_new[i,0] = 1
y_test_new[i,1] = 0
接着就是老套路，初始化訓練參數，設置模型輸入輸出，然後是構建模型，二分類邏輯迴歸模型的數學表達式爲：
用tensorflow實現代碼如下：
pred = tf.nn.sigmoid(tf.matmul(X, W) + b) # sigmoid
然後是構建交叉熵表達式，其數學公式爲：
用tensorflow實現代碼如下：
cost = tf.reduce_mean(-tf.reduce_sum(Y*tf.log(pred), reduction_indices=1))
對於訓練好的模型的測試性能的評價，這裏我採用計算ROC曲線的方式來展現。首先是模型對於輸入數據的預測會得到一個對應的1*2的輸出，這個輸出不一定是很好的[0 1]或者[1 0]，而是一個類似於概率的量，也就是類似於這樣[0.9527， 0.0473]，所以我們需要判斷，如果第一位小於第二位，依照前面的設定應該是0，反之爲1，當然可能會有第一位等於第二位的情況，那就只能是存在誤差了：
y_test_pred = sess.run(pred, feed_dict={X: x_test})
y_scores = np.empty((y_test_pred.shape[0]))
for i in range(y_scores.shape[0]):
if y_test_pred[i,0]<y_test_pred[i,1]:
y_scores[i]=0
else:
y_scores[i]=1
然後roc曲線和其AUC值這裏調用的是sklearn提供的函數：
fpr, tpr, thresholds = roc_curve((y_test), y_scores)
AUC_ROC = roc_auc_score(y_test, y_scores)
然後是畫出precision recall curve(精確率-召回率曲線)，同樣也是調用sklearn的函數:
precision, recall, thresholds = precision_recall_curve(y_test, y_scores)
precision = np.fliplr([precision])[0] #so the array is increasing (you won't get negative AUC)
recall = np.fliplr([recall])[0] #so the array is increasing (you won't get negative AUC) AUC_prec_rec = np.trapz(precision,recall)
接着是計算置信度矩陣，得到矩陣後順帶的就可以計算準確率、靈敏性、特異性等幾個參數了，同樣的，還是調用sklearn的函數即可：
confusion = confusion_matrix(y_test, y_pred)
accuracy = float(confusion[0,0]+confusion[1,1])/float(np.sum(confusion))
specificity = float(confusion[0,0])/float(confusion[0,0]+confusion[0,1])
sensitivity = float(confusion[1,1])/float(confusion[1,1]+confusion[1,0])
precision = float(confusion[1,1])/float(confusion[1,1]+confusion[0,1])
同樣的衡量指標還有Jaccard係數和F1分數：
jaccard_index = jaccard_similarity_score(y_test, y_pred, normalize=True)
F1_score = f1_score(y_test, y_pred, labels=None, average='binary', sample_weight=None)
這裏涉及到了很多模型性能的評價指標，具體的含義就不細究了，下一次專門寫一篇來總結吧。完整的代碼如下：
from __future__ import print_function
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets as skd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import scale
from sklearn.metrics import roc_curve
from sklearn.metrics import roc_auc_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import precision_recall_curve
from sklearn.metrics import jaccard_similarity_score
from sklearn.metrics import f1_score
# 加載乳腺癌數據集，該數據及596個樣本，每個樣本有30維，共有兩類
cancer = skd.load_breast_cancer()
# 將數據集的數據和標籤分離
X_data = cancer.data
Y_data = cancer.target
print("X_data.shape = ", X_data.shape)
print("Y_data.shape = ", Y_data.shape)
# 將數據和標籤分成訓練集和測試集
x_train,x_test,y_train,y_test = train_test_split(X_data,Y_data,test_size=0.3,random_state=0)
x_train = scale(x_train)
x_test = scale(x_test)
y_train_new = np.zeros([y_train.shape[0], 2], dtype='int32')
y_test_new = np.zeros([y_test.shape[0], 2], dtype='int32')
for i in range(y_train.shape[0]):
if y_train[i] == 0:
y_train_new[i,0] = 0
y_train_new[i,1] = 1
else:
y_train_new[i,0] = 1
y_train_new[i,1] = 0
for i in range(y_test.shape[0]):
if y_test[i] == 0:
y_test_new[i,0] = 0
y_test_new[i,1] = 1
else:
y_test_new[i,0] = 1
y_test_new[i,1] = 0
print("x_train.shape = ", x_train.shape)
print("x_test.shape = ", x_test.shape)
print("y_train.shape = ", y_train_new.shape)
print("y_test.shape = ", y_test_new.shape)
# 初始化參數
learning_rate = 0.01
training_epochs = 50000
display_step = 50
# 定義圖模型輸入
X = tf.placeholder(tf.float32, [None, 30]) # mnist data image of shape 28*28=784
Y = tf.placeholder(tf.float32, [None, 2]) # 0-9 digits recognition => 10 classes#
# 設置模型權重和偏置
W = tf.Variable(tf.random_normal([30, 2]),dtype=tf.float32, name="weight")
b = tf.Variable(tf.random_normal([2]),dtype=tf.float32, name="bias")
# 構建模型
#pred = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax
pred = tf.nn.sigmoid(tf.matmul(X, W) + b) # sigmoid
# 使用交叉熵來最小化訓練誤差
cost = tf.reduce_mean(-tf.reduce_sum(Y*tf.log(pred), reduction_indices=1))
# 梯度下降法
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)#
# 初始化器
init = tf.global_variables_initializer()
# 開始訓練
with tf.Session() as sess:#
# 初始化
sess.run(init)#
# 迭代訓練
for epoch in range(training_epochs):
avg_cost = 0.
sess.run(optimizer, feed_dict={X: x_train, Y: y_train_new})
# 顯示訓練信息
if (epoch+1) % display_step == 0:
c = sess.run(cost, feed_dict={X: x_train, Y:y_train_new})
print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c))
print("完成訓練!")
# 測試模型
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(Y, 1))
# 計算準確度
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print("Accuracy:", accuracy.eval({X: x_test, Y: y_test_new}))
# 計算模型預測
y_test_pred = sess.run(pred, feed_dict={X: x_test})
y_pred = np.empty((y_test_pred.shape[0]), dtype='int32')
# 二值化
for i in range(y_pred.shape[0]):
if y_test_pred[i,0]<y_test_pred[i,1]:
y_pred[i]=0
else:
y_pred[i]=1
# 得到ROC曲線
fpr, tpr, thresholds = roc_curve((y_test), y_pred)
AUC_ROC = roc_auc_score(y_test, y_pred)
roc_curve =plt.figure()
plt.plot(fpr,tpr,'-',label='Area Under the Curve (AUC = %0.4f)' % AUC_ROC)
plt.title('ROC curve')
plt.xlabel("FPR (False Positive Rate)")
plt.ylabel("TPR (True Positive Rate)")
plt.legend(loc="lower right")
plt.show()
# 得到precision_recall曲線
precision, recall, thresholds = precision_recall_curve(y_test, y_pred)
precision = np.fliplr([precision])[0] #so the array is increasing (you won't get negative AUC)
recall = np.fliplr([recall])[0] #so the array is increasing (you won't get negative AUC)
AUC_prec_rec = np.trapz(precision,recall)
print("\nArea under Precision-Recall curve: " +str(AUC_prec_rec))
prec_rec_curve = plt.figure()
plt.plot(recall,precision,'-',label='Area Under the Curve (AUC = %0.4f)' % AUC_prec_rec)
plt.title('Precision - Recall curve')
plt.xlabel("Recall")
plt.ylabel("Precision")
plt.legend(loc="lower right")
plt.show()
# 計算置信度矩陣
threshold_confusion = 0.5
print("\nConfusion matrix: Costum threshold (for positive) of " +str(threshold_confusion))
confusion = confusion_matrix(y_test, y_pred)
print(confusion)
accuracy = 0
if float(np.sum(confusion))!=0:
accuracy = float(confusion[0,0]+confusion[1,1])/float(np.sum(confusion))
print("Global Accuracy: " +str(accuracy))
specificity = 0
if float(confusion[0,0]+confusion[0,1])!=0:
specificity = float(confusion[0,0])/float(confusion[0,0]+confusion[0,1])
print("Specificity: " +str(specificity))
sensitivity = 0
if float(confusion[1,1]+confusion[1,0])!=0:
sensitivity = float(confusion[1,1])/float(confusion[1,1]+confusion[1,0])
print("Sensitivity: " +str(sensitivity))
precision = 0
if float(confusion[1,1]+confusion[0,1])!=0:
precision = float(confusion[1,1])/float(confusion[1,1]+confusion[0,1])
print("Precision: " +str(precision))
#Jaccard similarity index
jaccard_index = jaccard_similarity_score(y_test, y_pred, normalize=True)
print("\nJaccard similarity score: " +str(jaccard_index))
#F1 score
F1_score = f1_score(y_test, y_pred, labels=None, average='binary', sample_weight=None)
print("\nF1 score (F-measure): " +str(F1_score))
print("y_test[0:20]=", y_test[0:20])
print("y_pred[0:20]=", y_pred[0:20])
最總的結果顯示如下：
古人學問無遺力，少壯工夫老始成。
紙上得來終覺淺，絕知此事要躬行。
-- 陸游 《冬夜讀書示子聿》