keras如何可視化 模型預測時的 中間 特徵圖，網絡模型如何可視化中間層特徵圖以及如何打印網絡結構

需求：

【注意：可視化中間層特徵圖操作一般是在測試階段，因爲該文中提到的可視化的操作類似於在某一層網絡中斷，然後輸出該層之後的結果，也可以理解爲提前結束，當然也可以在訓練的時候打印，但是這樣的意義並不大，這樣終止訓練過程，訓練過程本身就是在找最優參數，所以可視化基本都是可視化訓練好的模型之後的測試結果】

使用模型文件預測一張圖片（下圖cat.jpeg爲測試圖片，超萌的小貓咪。。。有木有啊。。。。），將模型的某一層結果進行可視化。基於keras實現。

分享兩個示例，示例一爲自己建立的卷積池化結構，示例2爲VGG16（具體哪一個版本的VGG16不重要）示例。

使用的方法是：

#index = 8表示可視化第八層卷積的結果，其它層結果類似
model = Model(inputs=model.input, outputs=model.get_layer(index=8).output) 

具體示例及註釋見如下代碼部分:

示例1：手動自己建立的簡易CNN網絡

代碼中關鍵的幾點爲：(標記的位置 我都註釋在代碼裏了，不懂的可以留言)

標記0：index = ？ 我們想可視化第幾層的結果就讓index等於幾，index = 0 表示你直接沒卷積（這裏是幾就是幾，並不是 從0開始的哈，這樣說的很通俗易懂了吧。。）示例1中爲index = 8，可視化第八層[層數是相對於model.summary()結果而言的]

標記1：model.summary() 該方法的意思是打印整個的網絡結構，直接看示例1的model.summary() 結果

model.summary()的輸出結果：
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 336, 276, 9)       684       
_________________________________________________________________
activation_1 (Activation)    (None, 336, 276, 9)       0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 84, 69, 9)         0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 80, 65, 9)         2034      
_________________________________________________________________
activation_2 (Activation)    (None, 80, 65, 9)         0         
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 26, 21, 9)         0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 22, 17, 9)         2034      
_________________________________________________________________
activation_3 (Activation)    (None, 22, 17, 9)         0         
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 11, 8, 9)          0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 9, 6, 9)           738       
_________________________________________________________________
activation_4 (Activation)    (None, 9, 6, 9)           0         
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 4, 3, 9)           0         
=================================================================
Total params: 5,490
Trainable params: 5,490
Non-trainable params: 0
_________________________________________________________________
None

標記2：num_pic = feature_map.shape[2]。該方法中的 數字2 與具體的情況有關【解釋一下這裏爲什麼是2，我們看summary顯結果，第一層的輸出shape爲（336, 276, 9),很明顯我們需要打印出九張圖像，9 對應的下標正好是2，這個需要看具體情況定，具體情況在哪看，就是打印出網絡結構圖，看看每一層的輸出層數所對應的下標在shape的哪一維 】，表示我要顯示幾張圖。

標記3：該處要注意和標記2是對應的，也就是i 的位置要與標記2中的數字相匹配，可以看示例1 示例2 的代碼理解一下，

# coding: utf-8

from keras.models import Model
import cv2
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.layers import Activation
from pylab import *
from keras import backend as K


def get_row_col(num_pic):
    squr = num_pic ** 0.5
    row = round(squr)
    col = row + 1 if squr - row > 0 else row
    return row, col


def visualize_feature_map(img_batch):
    feature_map = np.squeeze(img_batch, axis=0)
    print(feature_map.shape)

    feature_map_combination = []
    plt.figure()

    num_pic = feature_map.shape[2] #標記2
    print('num_pic',num_pic)
    row, col = get_row_col(num_pic)

    for i in range(0, num_pic):
        feature_map_split = feature_map[:, :, i] #標記3
        feature_map_combination.append(feature_map_split)
        plt.subplot(row, col, i + 1)
        plt.imshow(feature_map_split)
        axis('off')
        title('feature_map_{}'.format(i))

    # plt.savefig('feature_map.png')
    plt.show()

    # 各個特徵圖按1：1 疊加
    feature_map_sum = sum(ele for ele in feature_map_combination)
    plt.imshow(feature_map_sum)
    # plt.savefig("feature_map_sum.png")


def create_model():
    model = Sequential()

    # 第一層CNN
    # 第一個參數是卷積核的數量，第二三個參數是卷積核的大小
    model.add(Convolution2D(9, (5,5),  input_shape=img.shape))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(4, 4)))
    #
    # 第二層CNN
    model.add(Convolution2D(9, (5,5),  input_shape=img.shape))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(3, 3)))

    # 第三層CNN
    model.add(Convolution2D(9, (5,5),  input_shape=img.shape))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    # 第四層CNN
    model.add(Convolution2D(9, (3,3), input_shape=img.shape))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    #model.summary()打印網絡的結構，看結果輸出
    print(model.summary()) #標記1
    return model


if __name__ == "__main__":
    img = cv2.imread('cat.jpeg')

    model = create_model()

    model = Model(inputs=model.input, outputs=model.get_layer(index=8).output) #標記0

    print('img ', img.shape)
    img_batch = np.expand_dims(img, axis=0)
    print('img_batch',img_batch.shape)
    conv_img = model.predict(img_batch)  # conv_img 卷積結果
    print('conv_img.shape', conv_img.shape)


    visualize_feature_map(conv_img)

-----------------------------------------------------------以下爲示例1的結果圖------------------------------

 

=====================分割線================================

示例2：VGG16

標記說明和上邊一樣。標記我在代碼中註釋了，這個例子需要models.py文件和示例.py兩個文件。

model.py （裏邊有很多代碼是我需要的東西，並沒有刪，使用的話直接複製即可，這個文件不需要動裏邊的東西【標記1在這這個文件裏】）

import numpy as np
import sys
import math
import operator
import csv
import glob, os
import xlrd
import cv2
import pandas as pd

from sklearn.svm import SVC
from collections import Counter
from sklearn.metrics import confusion_matrix
import scipy.io as sio
import keras
from keras.models import Sequential, Model
from keras.layers.core import Flatten, Dense, Dropout
from keras.layers.convolutional import Conv2D, MaxPooling2D, ZeroPadding2D
from keras.layers import LSTM, GlobalAveragePooling2D, GRU, Bidirectional, UpSampling2D, Input, BatchNormalization, add, \
    concatenate, Lambda, Activation, multiply
from keras.optimizers import SGD
import keras.backend as K
from keras.callbacks import Callback
from keras.engine.topology import Layer

from labelling import collectinglabel
from reordering import readinput
from evaluationmatrix import fpr
import numpy as np


def VGG_16(spatial_size, classes, channels, channel_first=True, weights_path=None):
    model = Sequential()
    if channel_first:
        model.add(ZeroPadding2D((1, 1), input_shape=(channels, spatial_size, spatial_size)))
    else:
        model.add(ZeroPadding2D((1, 1), input_shape=(spatial_size, spatial_size, channels)))

    # print("model0:" + str(model.get_layer(index=0).output_shape) + "\n")
    model.add(Conv2D(64, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(64, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))
    # print("model1:" +str(model.get_layer(index=1)) + str(model.get_layer(index=1).output_shape) + "\n")
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(128, (3, 3), activation='relu', name="conv3"))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(128, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))
    # print("model2:" + str(model.get_layer('conv3')) + str(model.get_layer('conv3').output_shape) + "\n")
    # print("model3:" + str(model.get_layer(index=3)) + str(model.get_layer(index=3).output_shape) + "\n")
    # print("model:" + str(model.get_layer))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(256, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(256, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(256, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(ZeroPadding2D((1, 1)))
    model.add(Conv2D(512, (3, 3), activation='relu'))
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))  # 33

    # print("length:"+str(len(model.get_layer)))
    model.add(Flatten())
    model.add(Dense(4096, activation='relu'))  # 34
    model.add(Dropout(0.5))
    model.add(Dense(4096, activation='relu'))  # 35
    model.add(Dropout(0.5))
    model.add(Dense(2622, activation='softmax'))  # Dropped
    # print("length:"+str(len(model.layers))+"\n")
    # # for i in range(len(model.layers)):
    # # 	print(model.get_layer(index=i).output)
    # for layer in model.layers:
    # 	print(layer.output)
    # model.add(Dense(classes, activation='softmax')) # 36

    if weights_path:
        model.load_weights(weights_path)
    print(model.summary()) #標記1
    # model.pop()
    # model.add(Dense(classes, activation='softmax')) # 36

    return model

示例2代碼：

# coding: utf-8
from keras.applications.vgg19 import VGG19
from keras.preprocessing import image
# from keras.applications.vgg19 import preprocess_input
from keras.models import Model
from models import VGG_16
import numpy as np
import matplotlib.pyplot as plt
from pylab import *
import cv2
from keras import backend as K
K.set_image_dim_ordering('th')
import os

def get_row_col(num_pic):
    squr = num_pic ** 0.5
    row = round(squr)
    col = row + 1 if squr - row > 0 else row
    return row, col


def visualize_feature_map(img_batch):
    feature_map = img_batch
    # feature_map = np.squeeze(img_batch, axis=0)
    print('this is ',feature_map.shape)

    feature_map_combination = []
    plt.figure()

    num_pic = feature_map.shape[0] #標記2
    row, col = get_row_col(num_pic)

    for i in range(0, num_pic):
        feature_map_split = feature_map[i, :, :] #標記3
        feature_map_combination.append(feature_map_split)
        plt.subplot(row, col, i + 1)
        plt.imshow(feature_map_split)
        axis('off')

    # plt.savefig('feature_map.png')
    plt.show()

    # 各個特徵圖按1：1 疊加
    feature_map_sum = sum(ele for ele in feature_map_combination)
    plt.imshow(feature_map_sum)
    # plt.savefig("feature_map_sum.png")


if __name__ == "__main__":
    base_model = VGG_16(spatial_size=224, classes=5, channels=3, weights_path="VGG_Face_Deep_16.h5")

    model = Model(inputs=base_model.input, outputs=base_model.get_layer(index=2).output) #標記0
   

    img = cv2.imread("cat.jpeg")

    img = cv2.resize(img,(224,224)) #VGG輸入要求

    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    img_batch = np.expand_dims(img, axis=0)
    block_pool_features = model.predict(x)
    feature = block_pool_features.reshape(block_pool_features.shape[1:])
    visualize_feature_map(feature)

-----------------------------------------------------------以下爲示例2的結果圖------------------------------