tensorflow自然語言處理(自動生成古詩)
在我上一篇博客當中，已經寫了CNN驗證碼識別，由此可以看出神經網絡的強大之處，所以這篇博客主要是來講解一下RNN中的LSTM網絡處理自然語言，輸入一個字就自動生成一篇優美的古詩。
RNN主要邏輯就是每個樣本之間有比較強烈的關聯性，這種關聯性比較適合自然語言的處理，因爲我們說的話都是有一定的關聯性。這裏我們不過多的講解RNN的理論基礎，因爲上百度上面搜索還是有很多的，我這篇主要是講解案例。
步驟思路
1、首先我們要先獲取訓練集，因爲這種層級比較少的神經網絡，不適合應用在比較廣的方向，一般訓練集的內容都是在一個方向上，比如只獲取唐詩三百首，或者是某個時代的詩詞，因爲我有的時候也會寫一兩首詩，所以知道每個時代的詩的風格是不一樣的，這裏我獲取了一部分的詩歌集和七言律詩和五言律詩，三個分開訓練得出三個不同的模型。古詩詞的獲取可以取網站上爬取出來。
2、然後就要處理這些古詩詞，這裏有很多人都是用分詞來做的，但是我試過分詞的效果不是很好，因爲分詞訓練出來的結果不好，因爲在用結巴分詞不是太能夠把古詩詞分的很好。所以我就把每個字分出來，因爲每個普通的詞的每個字也是有很大的聯繫在裏面的，所以我就把每個字給分出來。分詞出來後就要把這些字轉換成對應的數字，因爲機器只會識別數字。
3、建立模型，這裏要建立兩層的LSTM深層模型，兩層的效果以及比較好了，如果用三層也是可以的。
處理訓練集數據
切割古詩
start_token = 'B'
end_token = 'E'
def process_text(file_name):
poems = []
with open(file_name, "r", encoding='utf-8', ) as f:
for line in f.readlines():
try: #因爲有些詩詞不是規定格式，所以要加一個捕獲異常
title, content = line.strip().split(':') #把詩詞的標題和詩詞的內容提取出來
content = content.replace(' ', '')
#加一些判斷下去，就可以避免有些多餘的符號在裏面
if '_' in content or '(' in content or '（' in content or '《' in content or '[' in content or \
start_token in content or end_token in content:
continue
if len(content) < 5 or len(content) > 79:
continue
content = start_token + content + end_token #加上一些標記在頭尾，後期就比較好處理
poems.append(content)
except ValueError as e:
pass
#把每個字給切割下來
all_word = []
for poem in poems:
for word in poem:
all_word.append(word)
#對每個子進行計數
counter = collections.Counter(all_word)
words = sorted(counter.keys(), key=lambda x: counter[x], reverse=True) #對詞進行排序，常用的詞放在前面
words.append(' ')
L = len(words)
word_int_map = dict(zip(words, range(L))) #把詞和詞出現的次數存成字典
poems_vector = [list(map(lambda word: word_int_map.get(word, L), poem)) for poem in poems] #把每首詩的代表數字存成列表
print(word_int_map)
return poems_vector, word_int_map, words
這裏的主要邏輯是這樣的：1、先把每一首詩的內容提取出來，因爲題目不在我的考慮範圍之內。2、然後把每首詩的每個字符提取出來，存成列表。3、然後在統計每個詞出現的次數，然後按照出現最多的放在前面，存成字典。4、最後賦予每個字有一個單獨的數字代表，每首詩都變成一個列表，而列表的內容就是每個字的代表數字。
批量處理
因爲訓練的時候肯定是批量處理的，所以下面就說一下批量處理。
def get_batch(batch_size,poems_vector,word_int):
n_chunk = len(poems_vector) // batch_size #把詩詞的數量整除批量處理的量
x_batches = []
y_batches = []
#這裏定義批量處理的兩個索引
for i in range(n_chunk):
start_index = i * batch_size
end_index = start_index + batch_size
batches = poems_vector[start_index:end_index] #截取訓練集的樣本
length = max(map(len, batches)) #獲取最長的那個詩詞的長度
x_data = np.full((batch_size, length), word_int[' '], np.int32) #製造一個數組
for row, batch in enumerate(batches): #把x_data全都替換成詩詞的數字代表數組
x_data[row, :len(batch)] = batch
y_data = np.copy(x_data)
y_data[:, :-1] = x_data[:, 1:]
x_batches.append(x_data)
y_batches.append(y_data)
return x_batches, y_batches
這裏的代碼邏輯就是，先把內容分爲一批一批，這樣方便訓練的時候提取。然後根據批量內容生成一個numpy數組，這個數組是固定一個數字，然後再用每首詩的列表內容代替掉這個數組。這個數組就是特徵值。目標值就根據特徵值進行修改，調整方向如：
x_data y_data
[6,2,4,6,9] [2,4,6,9,9]
[1,4,2,8,5] [4,2,8,5,5]
這樣就可以根據模型得出的最後的結果，然後和y_data進行對比，就可以得出最後的結果是否正確了。
源碼
import collections
import numpy as np
file = "D:\\mypathon3\\python\\RNN詩詞生成\\wulv-all.txt"
#首先對詩詞的文本格式進行處理
start_token = 'B'
end_token = 'E'
def process_text(file_name):
poems = []
with open(file_name, "r", encoding='utf-8', ) as f:
for line in f.readlines():
try: #因爲有些詩詞不是規定格式，所以要加一個捕獲異常
title, content = line.strip().split(':') #把詩詞的標題和詩詞的內容提取出來
content = content.replace(' ', '')
#加一些判斷下去，就可以避免有些多餘的符號在裏面
if '_' in content or '(' in content or '（' in content or '《' in content or '[' in content or \
start_token in content or end_token in content:
continue
if len(content) < 5 or len(content) > 79:
continue
content = start_token + content + end_token #加上一些標記在頭尾，後期就比較好處理
poems.append(content)
except ValueError as e:
pass
#把每個字給切割下來
all_word = []
for poem in poems:
for word in poem:
all_word.append(word)
#對每個子進行計數
counter = collections.Counter(all_word)
words = sorted(counter.keys(), key=lambda x: counter[x], reverse=True) #對詞進行排序，常用的詞放在前面
words.append(' ')
L = len(words)
word_int_map = dict(zip(words, range(L))) #把詞和詞出現的次數存成字典
poems_vector = [list(map(lambda word: word_int_map.get(word, L), poem)) for poem in poems] #把每首詩的代表數字存成列表
print(word_int_map)
return poems_vector, word_int_map, words
#這裏批量獲取訓練樣本，並且把訓練樣本轉換成特徵值和目標值
def get_batch(batch_size,poems_vector,word_int):
n_chunk = len(poems_vector) // batch_size #把詩詞的數量整除批量處理的量
x_batches = []
y_batches = []
#這裏定義批量處理的兩個索引
for i in range(n_chunk):
start_index = i * batch_size
end_index = start_index + batch_size
batches = poems_vector[start_index:end_index] #截取訓練集的樣本
length = max(map(len, batches)) #獲取最長的那個詩詞的長度
x_data = np.full((batch_size, length), word_int[' '], np.int32) #製造一個數組
for row, batch in enumerate(batches): #把x_data全都替換成詩詞的數字代表數組
x_data[row, :len(batch)] = batch
y_data = np.copy(x_data)
y_data[:, :-1] = x_data[:, 1:]
x_batches.append(x_data)
y_batches.append(y_data)
return x_batches, y_batches
建立模型
建立模型之前我們首先要確定一些參數，LSTM的神經元輸出的特徵數，128，批量數：64，神經網絡的深度：2
定義模型的函數
def model_get(): #建立模型的函數
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=128, reuse=tf.get_variable_scope().reuse)
lstm_cell_2 = tf.nn.rnn_cell.DropoutWrapper(lstm_cell,output_keep_prob=0.6) #防止過擬合
return lstm_cell_2
這裏我用LSTM的底層細胞，然後刪除一部分值，防止過擬合，其實這裏刪不刪除都無所謂，因爲本來就是要過擬合的，因爲古詩就要過擬合纔行。
準備數據
layer_num = 2
hidden_size = 128
batch_size = 64
#首先要獲取詩詞的字典和詩詞字，和每個字對應的數字
poems_vector, word_int_map, words = process_text(file)
words_size = len(words)
batches_inputs, batches_outputs = get_batch(batch_size, poems_vector, word_int_map)
#特徵值和目標值的佔位符
input_data = tf.placeholder(tf.int32, [batch_size, None])
output_targets = tf.placeholder(tf.int32, [batch_size, None])
embedding = tf.get_variable('embedding', initializer=tf.random_uniform(
[words_size + 1, hidden_size], -1.0, 1.0))
inputs = tf.nn.embedding_lookup(embedding, input_data)
這裏主要是獲取數據，然後修改數據的形狀，這裏用tf.nn.embedding_lookup(embedding, input_data)修改數據的方式，因爲我每首詩的字符數都有可能不一樣的，那就動態修改數據，而且不清楚每層神經元層有多少個神經元，所以就動態修改形狀。
inputs的形狀就是[-1,input_size,timesept_size]。
模型生成和模型優化
mlstm_cell = tf.nn.rnn_cell.MultiRNNCell([model_get() for _ in range(layer_num)])
init_state = mlstm_cell.zero_state(batch_size, dtype=tf.float32)
outputs, last_state = tf.nn.dynamic_rnn(mlstm_cell, inputs = inputs, initial_state=init_state)
output = tf.reshape(outputs, [-1, hidden_size])
weights = tf.Variable(tf.truncated_normal([hidden_size, words_size + 1]))
bias = tf.Variable(tf.zeros(shape=[words_size + 1]))
logits = tf.nn.bias_add(tf.matmul(output, weights), bias=bias)
#模型優化
labels = tf.one_hot(tf.reshape(output_targets, [-1]), depth=words_size + 1)
loss = tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)
total_loss = tf.reduce_mean(loss)
train_op = tf.train.AdamOptimizer(0.005).minimize(total_loss)
#準確率
accuracy = tf.equal(tf.argmax(labels, 1), tf.argmax(logits, 1), name="accuracy")
accuracy = tf.cast(accuracy, tf.float32)
accuracy_mean = tf.reduce_mean(accuracy, name="accuracy_mean")
這裏如果是熟悉tensorflow的就很簡單在裏面，所以就不做過多的講解。
模型訓練和保存
saver = tf.train.Saver(tf.global_variables())
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
#開啓會話
with tf.Session() as sess:
sess.run(init_op)
start_epoch = 0
checkpoint = tf.train.latest_checkpoint("D:\\mypathon3\\python\\RNN詩詞生成\\model")
if checkpoint:
saver.restore(sess, checkpoint)
print("## restore from the checkpoint {0}".format(checkpoint))
start_epoch += int(checkpoint.split('-')[-1])
try:
n_chunk = len(poems_vector) // 64
print(n_chunk)
for i in range(start_epoch,50):
n = 0
for batch in range(n_chunk):
sess.run(train_op, feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
loss_1 = sess.run(total_loss,feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
last_state_1 = sess.run(last_state,feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
accuracys = sess.run(accuracy_mean,feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
print('Epoch: %d, batch: %d, training loss: %.6f' % (i, batch, loss_1))
print('Epoch: %d, batch: %d, training 準確率: %.6f' % (i, batch, accuracys))
if i % 6 == 0:
saver.save(sess, "D:\\mypathon3\\python\\RNN詩詞生成\\model\\model", global_step=i)
except KeyboardInterrupt:
print('## Interrupt manually, try saving checkpoint for now...')
saver.save(sess, checkpoint, global_step=i)
print('## Last epoch were saved, next time will start from epoch {}.'.format(i))
這裏使用了模型迭代的方式，就是先保存一個模型，然後在下面打開這個模型，根據我們以前訓練的步驟接着訓練，這樣就可以避免電腦死機然後重新訓練。
源碼
import tensorflow as tf
from text import process_text,get_batch
file = "D:\\mypathon3\\python\\RNN詩詞生成\\wulv-all.txt"
def model_get(): #建立模型的函數
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=128, reuse=tf.get_variable_scope().reuse)
lstm_cell_2 = tf.nn.rnn_cell.DropoutWrapper(lstm_cell,output_keep_prob=0.6) #防止過擬合
return lstm_cell_2
def train_run():
layer_num = 2
hidden_size = 128
batch_size = 64
#首先要獲取詩詞的字典和詩詞字，和每個字對應的數字
poems_vector, word_int_map, words = process_text(file)
words_size = len(words)
batches_inputs, batches_outputs = get_batch(batch_size, poems_vector, word_int_map)
#特徵值和目標值的佔位符
input_data = tf.placeholder(tf.int32, [batch_size, None])
output_targets = tf.placeholder(tf.int32, [batch_size, None])
embedding = tf.get_variable('embedding', initializer=tf.random_uniform(
[words_size + 1, hidden_size], -1.0, 1.0))
inputs = tf.nn.embedding_lookup(embedding, input_data)
#開始建立模型,建立兩層的RNN模型，每個模型的輸出的特徵數是128，神經數量是動態的，輸入的特徵值數量也是動態的
mlstm_cell = tf.nn.rnn_cell.MultiRNNCell([model_get() for _ in range(layer_num)])
init_state = mlstm_cell.zero_state(batch_size, dtype=tf.float32)
outputs, last_state = tf.nn.dynamic_rnn(mlstm_cell, inputs = inputs, initial_state=init_state)
output = tf.reshape(outputs, [-1, hidden_size])
weights = tf.Variable(tf.truncated_normal([hidden_size, words_size + 1]))
bias = tf.Variable(tf.zeros(shape=[words_size + 1]))
logits = tf.nn.bias_add(tf.matmul(output, weights), bias=bias)
#模型優化
labels = tf.one_hot(tf.reshape(output_targets, [-1]), depth=words_size + 1)
loss = tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)
total_loss = tf.reduce_mean(loss)
train_op = tf.train.AdamOptimizer(0.005).minimize(total_loss)
#準確率
accuracy = tf.equal(tf.argmax(labels, 1), tf.argmax(logits, 1), name="accuracy")
accuracy = tf.cast(accuracy, tf.float32)
accuracy_mean = tf.reduce_mean(accuracy, name="accuracy_mean")
saver = tf.train.Saver(tf.global_variables())
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
#開啓會話
with tf.Session() as sess:
sess.run(init_op)
start_epoch = 0
checkpoint = tf.train.latest_checkpoint("D:\\mypathon3\\python\\RNN詩詞生成\\model")
if checkpoint:
saver.restore(sess, checkpoint)
print("## restore from the checkpoint {0}".format(checkpoint))
start_epoch += int(checkpoint.split('-')[-1])
try:
n_chunk = len(poems_vector) // 64
print(n_chunk)
for i in range(start_epoch,50):
n = 0
for batch in range(n_chunk):
sess.run(train_op, feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
loss_1 = sess.run(total_loss,feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
last_state_1 = sess.run(last_state,feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
accuracys = sess.run(accuracy_mean,feed_dict={input_data: batches_inputs[n], output_targets: batches_outputs[n]})
print('Epoch: %d, batch: %d, training loss: %.6f' % (i, batch, loss_1))
print('Epoch: %d, batch: %d, training 準確率: %.6f' % (i, batch, accuracys))
if i % 6 == 0:
saver.save(sess, "D:\\mypathon3\\python\\RNN詩詞生成\\model\\model", global_step=i)
except KeyboardInterrupt:
print('## Interrupt manually, try saving checkpoint for now...')
saver.save(sess, checkpoint, global_step=i)
print('## Last epoch were saved, next time will start from epoch {}.'.format(i))
train_run()
開始測試
加載模型
start_token = 'B'
end_token = 'E'
file = "D:\\mypathon3\\python\\RNN詩詞生成\\wulv-all.txt"
hidden_size = 128
layer_num = 2
batch_size = 1
#先還原數據模型
def model_get(): #建立模型的函數
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=128, reuse=tf.get_variable_scope().reuse)
# lstm_cell_2 = tf.nn.rnn_cell.DropoutWrapper(lstm_cell,output_keep_prob=0.5) #防止過擬合
return lstm_cell
這裏就不用刪除一部分值了。
poems_vector, word_int_map, words = process_text(file)
words_size = len(words)
input_data = tf.placeholder(tf.int32, [batch_size, None])
embedding = tf.get_variable('embedding', initializer=tf.random_uniform(
[words_size + 1, hidden_size], -1.0, 1.0))
inputs = tf.nn.embedding_lookup(embedding, input_data)
mlstm_cell = tf.nn.rnn_cell.MultiRNNCell([model_get() for _ in range(layer_num)])
init_state = mlstm_cell.zero_state(1, dtype=tf.float32)
outputs, last_state = tf.nn.dynamic_rnn(mlstm_cell, inputs, initial_state=init_state)
output = tf.reshape(outputs, [-1, hidden_size])
weights = tf.Variable(tf.truncated_normal([hidden_size, words_size + 1]))
bias = tf.Variable(tf.zeros(shape=[words_size + 1]))
logits = tf.nn.bias_add(tf.matmul(output, weights), bias=bias)
prediction = tf.nn.softmax(logits)
這裏要注意一個點，就是init_state的形狀，因爲我要輸入一個字作爲數據，所以init_state的形狀的batch_size也是1，其他地方都沒什麼改變。
根據模型加載數據
with tf.Session() as sess:
sess.run(init_op)
checkpoint = tf.train.latest_checkpoint("D:\\mypathon3\\python\\RNN詩詞生成\\model")
saver.restore(sess, checkpoint)
x = np.array([list(map(word_int_map.get, start_token))])
#得出預測值
predict = sess.run(prediction, feed_dict={input_data: x})
ps = tf.argmax(predict, 1)
print(sess.run(ps))
last_states = sess.run(last_state, feed_dict={input_data: x})
這裏是比較重要的點，也是整個模型比較難的點。這裏要先得出一個值作爲輸入，因爲我在上面處理數據的時候，在每首詩的前面和後面都加了E和B，這樣就方便這裏處理了。
這裏首先預測得出的值作爲後面至關重要的參數，就是init_state。
得出一個接一個的字
#利用數據生成字
def to_word(predict, vocabs):
predict = predict[0]
predict /= np.sum(predict)
sample = np.random.choice(np.arange(len(predict)), p=predict)
if sample > len(vocabs):
return vocabs[-1]
else:
return vocabs[sample]
begin_word = input("輸入首個字：")
word = begin_word or to_word(predict, words)
poem_ = ''
i = 0
word_list = []
while word != end_token:
poem_ += word
i += 1
if i > 24:
break
x = np.array([[word_int_map[word]]])
# 得出預測值
predict = sess.run(prediction, feed_dict={input_data: x,init_state:last_states})
last_states = sess.run(last_state, feed_dict={input_data: x,init_state:last_states})
word = to_word(predict, words)
print(word_list)
print(poem_)
print("*"*100)
這裏就是整個代碼最難的部分了，首先我們要限制生成字的數量，用if i > 24:
break。
1、首先獲取第一個字的值，作爲模型的輸入數據輸入x = np.array([[word_int_map[word]]])
2、這裏一定要不斷更新init_state的參數，不然這首詩的關聯就只有第一個字，只要不斷拿上一個數據的參數作爲init_state的參數輸入，就會關聯上每個字了。
3、得出的預測值，不是直接用tf.argmax來直接獲取的，而是在可能性比較高的那幾個字中隨機抽取一個字，訓練過後發現，大部分的字還是跟tf.argmax得出來的字是一樣的，這也是因爲古詩的原因，某個字不是固定和某個字關聯，而是和幾個字有關聯的。
得出的字處理成詩句
def pretty_print_poem(poem_):
poem_sentences = poem_.split('。')
for s in poem_sentences:
if s != '' and len(s) > 10:
print(s + '。')
if s != "" and len(s) <= 10 and len(s) >= 5:
print(s + "。")
因爲模型的準確率和損失都訓練的非常好了，但是還會生成想這樣的例子：
松門風自埽，瀑布雪難消。秋夜聞清梵，餘音逐海潮。卻歸睦州
後面的四個字我就把他給刪除掉。
源碼
import tensorflow as tf
from text import process_text,get_batch
import numpy as np
start_token = 'B'
end_token = 'E'
file = "D:\\mypathon3\\python\\RNN詩詞生成\\wulv-all.txt"
hidden_size = 128
layer_num = 2
batch_size = 1
#利用數據生成字
def to_word(predict, vocabs):
predict = predict[0]
predict /= np.sum(predict)
sample = np.random.choice(np.arange(len(predict)), p=predict)
if sample > len(vocabs):
return vocabs[-1]
else:
return vocabs[sample]
def pretty_print_poem(poem_):
poem_sentences = poem_.split('。')
for s in poem_sentences:
if s != '' and len(s) > 10:
print(s + '。')
if s != "" and len(s) <= 10 and len(s) >= 5:
print(s + "。")
#先還原數據模型
def model_get(): #建立模型的函數
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=128, reuse=tf.get_variable_scope().reuse)
# lstm_cell_2 = tf.nn.rnn_cell.DropoutWrapper(lstm_cell,output_keep_prob=0.5) #防止過擬合
return lstm_cell
poems_vector, word_int_map, words = process_text(file)
words_size = len(words)
input_data = tf.placeholder(tf.int32, [batch_size, None])
embedding = tf.get_variable('embedding', initializer=tf.random_uniform(
[words_size + 1, hidden_size], -1.0, 1.0))
inputs = tf.nn.embedding_lookup(embedding, input_data)
mlstm_cell = tf.nn.rnn_cell.MultiRNNCell([model_get() for _ in range(layer_num)])
init_state = mlstm_cell.zero_state(1, dtype=tf.float32)
outputs, last_state = tf.nn.dynamic_rnn(mlstm_cell, inputs, initial_state=init_state)
output = tf.reshape(outputs, [-1, hidden_size])
weights = tf.Variable(tf.truncated_normal([hidden_size, words_size + 1]))
bias = tf.Variable(tf.zeros(shape=[words_size + 1]))
logits = tf.nn.bias_add(tf.matmul(output, weights), bias=bias)
prediction = tf.nn.softmax(logits)
saver = tf.train.Saver(tf.global_variables())
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
#開啓會話
with tf.Session() as sess:
sess.run(init_op)
checkpoint = tf.train.latest_checkpoint("D:\\mypathon3\\python\\RNN詩詞生成\\model")
saver.restore(sess, checkpoint)
x = np.array([list(map(word_int_map.get, start_token))])
#得出預測值
predict = sess.run(prediction, feed_dict={input_data: x})
ps = tf.argmax(predict, 1)
print(sess.run(ps))
last_states = sess.run(last_state, feed_dict={input_data: x})
begin_word = input("輸入首個字：")
word = begin_word or to_word(predict, words)
poem_ = ''
i = 0
word_list = []
while word != end_token:
poem_ += word
i += 1
if i > 24:
break
x = np.array([[word_int_map[word]]])
# 得出預測值
predict = sess.run(prediction, feed_dict={input_data: x,init_state:last_states})
last_states = sess.run(last_state, feed_dict={input_data: x,init_state:last_states})
word = to_word(predict, words)
print(word_list)
print(poem_)
print("*"*100)
pretty_print_poem(poem_)
測試
首先拿“梁”字來測試
梁法惟康富貴臣，二通標格價如泥。
非容妄聖俱當輩，作主從來作壽詩。
再用“雨”字來測試
注意
要注意一下幾點內如：
1、這裏參考模型是否優化好了要參考準確率和損失，因爲這個是古詩，所以要對比兩個方面纔可以說明模型是否已經好了。
2、訓練一定要迭代訓練，要不斷的保存模型和加載模型進行訓練。
3、訓練集的要求，訓練集一定要是某個小方面，一旦訓練集裏面的詩的風格是多變的，那麼測試出來的效果也是亂七八糟的。
這個模型總體來說要比我上一篇的模型好很多了。