tensorflow學習筆記第一部分

學習地址：Tensorflow 搭建自己的神經網絡 (莫煩 Python 教程)_嗶哩嗶哩 (゜-゜)つロ乾杯~-bilibili https://www.bilibili.com/video/av16001891/?p=16

第一課：

#-*- coding: utf-8 -*-
import tensorflow as tf
import numpy as np

#creat data
x_data=np.random.rand(100).astype(np.float32)
y_data=x_data*0.1+0.3
'''
隨機生成x和y 數據，x_data的數據類型爲32位浮點數，x爲100個數據
'''

#create tesorflow structure start#
Weights=tf.Variable(tf.random_uniform([1],-1.0,1.0))
biases=tf.Variable(tf.zeros([1]))

y=Weights*x_data+biases

loss=tf.reduce_mean(tf.square(y-y_data))
optimizer=tf.train.GradientDescentOptimizer(0.5)
train=optimizer.minimize(loss)

init=tf.initialize_all_variables()
#create tensorflow structure end#
'''
weights所表示的是隨機變量，裏面參數是隨機數列生成方式，[1]是一維的意思，生成範圍是-1.0到1.0
biases初始值是零
optimizer是gradientdecentoptimizer類型的優化器,是最基礎類型的優化器，裏面的參數0.5是learning rate
init是初始化所有的值，建立了變量
'''
#這個session很牛逼的樣子，用session來運行這個tensorflow-structure裏面的某個部分比如train，訪問Weights
sess=tf.Session()
sess.run(init)#very important


for step in range(201):
    sess.run(train)
    if step%20==0:
        print(step,sess.run(Weights),sess.run(biases))

'''
記住sess.run(train)這樣的
要訪問weights要用這樣的樣子來訪問，sess.run(Weighs)
'''

第一課運行結果：Instructions for updating:
Use `tf.global_variables_initializer` instead.
0 [-0.05082199] [0.5134365]
20 [0.03446221] [0.33414757]
40 [0.0791579] [0.3108595]
60 [0.09337187] [0.3034535]
80 [0.09789216] [0.3010983]
100 [0.09932968] [0.30034927]
120 [0.09978682] [0.3001111]
140 [0.09993222] [0.30003533]
160 [0.09997844] [0.30001125]
180 [0.09999314] [0.3000036]

200 [0.09999783] [0.30000114]

第二課：

#-*- coding: utf-8 -*-
import tensorflow as tf
import numpy as np

matrix1=tf.constant([[3,3]])
matrix2=tf.constant([[2],
                     [2]])
product=tf.matmul(matrix1,matrix2)#matmul is short of matrix mutiply
'''
mat1是1*1矩陣，mat2是2*1矩陣
product是兩個矩陣的相乘，類比到numpy中也就是np.dot(m1,m2)
'''

###method 1
##sess= tf.Session()
##result=sess.run(product)
##print(result)
#sess.close()

#method 2
with tf.Session() as sess:
    result2=sess.run(product)
    print(result2)


'''
方法1和方法2互相替代，訪問tf中的東西都需要用tf.Session來訪問，且訪問完需要close,如果使用with as，則可以省略close,記住這個with assession
'''

第二課運行結果:>>>[[12]]

第三課：

#-*- coding: utf-8 -*-
import tensorflow as tf

state=tf.Variable(0,name='counter')
#print(state.name)
one=tf.constant(1)
'''
state是tf的變量，起始數值爲0，還可以取名爲counter
one是tf的常量，數值爲1
print（state.name）出來的是name:value
'''

new_value=tf.add(state,one)
update=tf.assign(state,new_value)

init=tf.initialize_all_variables()#must have if defining variable
'''
new_value是state和one之和，tf裏的加法是tf.add(val1,val2)
tf.assign(state,new_value)是把new_value裏的變量加載到state中
等下一次運算的時候，第二個state再加上1變成new_value，說不太清，自己感受一下
assign(val1，val2)是assign val2 to val1
'''

with tf.Session() as sess:
    sess.run(init)
    for _ in range(3):
        sess.run(update)
        print(sess.run(state))

'''
只要tf中有變量，就一定要tf.initialize_all_variables
再在sess中run(inint)，不斷用session運行update,並session獲得state打印出來
'''

第三課運行結果：Instructions for updating:
Use `tf.global_variables_initializer` instead.
1
2

第四課：

#-*- coding: utf-8 -*-
import tensorflow as tf

input1=tf.placeholder(tf.float32)
input2=tf.placeholder(tf.float32)

output=tf.multiply(input1,input2)
'''
input1和2是tf的儲存數據的holder,output是tf相乘input1和input2的值
'''

with tf.Session() as sess:
    print(sess.run(output,feed_dict={input1:[7.],input2:[2.]}))
'''
因爲input1和2是placeholder的話，所以sess.run裏面需要有feed_dict
來接收傳進來的數值給input1和2，用dic的形式輸入7和2給input1和2
用placeholder的時候就要在session.run的時候feed_dic
'''

第四課運行結果：>>> [14.]

第五課：

#-*- coding: utf-8 -*-
import tensorflow as tf
import numpy as np

#添加神經層來修改激活值大小

def add_layer(inputs,in_size,out_size,activation_function=None):
    Weights=tf.Variable(tf.random_normal([in_size,out_size]))
    biases=tf.Variable(tf.zeros([1,out_size])+0.1)
    Wx_plus_b=tf.matmul(inputs,Weights)+biases
    if activation_function is None:
        outputs=Wx_plus_b
    else:
        outputs=activation_function(Wx_plus_b)
    return outputs
    
'''
如果傳入的激活方程爲None，則這一層layer相當於沒用，input=output
否則輸出等於經過activation_function變化的值
Wx_plus_b是預測值，還沒有被激活的值，爲啥裏面是random?
weights是in_size行out_size列的隨機值，biases是一行out_size列的非零列向量
（biases在機器學習中推薦不爲0）
'''

x_data=np.linspace(-1,1,300)[:,np.newaxis]
noise=np.random.normal(0,0.05,x_data.shape)
y_data=np.square(x_data)-0.5+noise

#x_data和y_data 是樣本數據，x_data 被弄成三百行一列，y則是x平方減去0.5加上噪點

#xs和ys被後面用來train當容器用的，先不用管
xs=tf.placeholder(tf.float32,[None,1])
ys=tf.placeholder(tf.float32,[None,1])


l1=add_layer(xs,1,10,activation_function=tf.nn.relu)#調用上面弄到的增加一層layer,返回激勵函數後的outputs
prediction=add_layer(l1,10,1,activation_function=None)#預測值是prediction

loss=tf.reduce_mean(tf.reduce_sum(tf.square(ys-prediction),
                                  reduction_indices=[1]))
#reduction_indices在tensorflow1.0後被改爲axis，網上有改成reduc_in=1的
train_step=tf.train.GradientDescentOptimizer(0.1).minimize(loss)#使用這個其實就在更改bias和weighs

init=tf.initialize_all_variables()
sess=tf.Session()
sess.run(init)
'''
運行1000次train_step,迭代一千次
'''
for i in range(1000):
    sess.run(train_step,feed_dict={xs:x_data,ys:y_data})
    if not i%50:
        print(sess.run(loss,feed_dict={xs:x_data,ys:y_data}))

運行結果：WARNING:tensorflow:From C:\python364\lib\site-packages\tensorflow\python\util\tf_should_use.py:118: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
0.6017571
0.008654009
0.0066226027
0.0060757496
0.005787913
0.005604301
0.0054377895
0.00529302
0.0051547633
0.005016695
0.004909197
0.004790244
0.0046654823
0.0045420392
0.0044349297
0.004340439
0.0042645116
0.004198417
0.004132898
0.0040747593

>>>

第五課還有點不太理解。。估計以後會理解，先繼續學

第六課到第十課內容簡介:

第六課:

#-*- coding: utf-8 -*-
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

#添加神經層來修改激活值大小
#add one more layer and return the outputs of this layer
def add_layer(inputs,in_size,out_size,activation_function=None):
    Weights=tf.Variable(tf.random_normal([in_size,out_size]))
    biases=tf.Variable(tf.zeros([1,out_size])+0.1)
    Wx_plus_b=tf.matmul(inputs,Weights)+biases
    if activation_function is None:
        outputs=Wx_plus_b
    else:
        outputs=activation_function(Wx_plus_b)
    return outputs
    
'''
如果傳入的激活方程爲None，則這一層layer相當於沒用，input=output
否則輸出等於經過activation_function變化的值
Wx_plus_b是預測值，還沒有被激活的值，爲啥裏面是random?
weights是in_size行out_size列的隨機值，biases是一行out_size列的非零列向量
（biases在機器學習中推薦不爲0）
'''
#make up some real data
x_data=np.linspace(-1,1,300)[:,np.newaxis]
noise=np.random.normal(0,0.05,x_data.shape)
y_data=np.square(x_data)-0.5+noise

#x_data和y_data 是樣本數據，x_data 被弄成三百行一列，y則是x平方減去0.5加上噪點

#xs和ys被後面用來train當容器用的，先不用管,define placeholder for inputs to network
xs=tf.placeholder(tf.float32,[None,1])
ys=tf.placeholder(tf.float32,[None,1])

#add hidden layer
l1=add_layer(xs,1,10,activation_function=tf.nn.relu)#調用上面弄到的增加一層layer,返回激勵函數後的outputs
#add output layer
prediction=add_layer(l1,10,1,activation_function=None)#預測值是prediction
#the error between prediction and real data
loss=tf.reduce_mean(tf.reduce_sum(tf.square(ys-prediction),
                                  reduction_indices=[1]))
#reduction_indices在tensorflow1.0後被改爲axis，網上有改成reduc_in=1的
train_step=tf.train.GradientDescentOptimizer(0.1).minimize(loss)#使用這個其實就在更改bias和weighs

init=tf.initialize_all_variables()
sess=tf.Session()
sess.run(init)


#顯示figure,內容爲x_data和y_data(make up some real data中的)
fig=plt.figure()
ax=fig.add_subplot(1,1,1)
ax.scatter(x_data,y_data)
plt.ion()#plt可以連續
plt.show()

#運行1000次train_step,迭代一千次
for i in range(1000):
    sess.run(train_step,feed_dict={xs:x_data,ys:y_data})
    if not i%50:
        #print(sess.run(loss,feed_dict={xs:x_data,ys:y_data}))
        try:
            ax.lines.remove(lines[0])#移掉之前的線，否則線會重合
        except Exception:
            pass
        prediction_value=sess.run(prediction,feed_dict={xs:x_data})
        lines=ax.plot(x_data,prediction_value,'r-',lw=5)
        plt.pause(0.1)

第七課:

#-*- coding: utf-8 -*-
import tensorflow as tf
import numpy as np

#添加神經層來修改激活值大小

def add_layer(inputs,in_size,out_size,activation_function=None):
    with tf.name_scope('layer'):
        with tf.name_scope('weights'):
            Weights=tf.Variable(tf.random_normal([in_size,out_size]),name='W')
        with tf.name_scope('biases'):
            biases=tf.Variable(tf.zeros([1,out_size])+0.1,name='b')
        with tf.name_scope('Wx_plus_b'):
            Wx_plus_b=tf.add(tf.matmul(inputs,Weights),biases)
        #先忽略activation
        if activation_function is None:
            outputs=Wx_plus_b
        else:
            outputs=activation_function(Wx_plus_b)
        return outputs
    
#詳情請見figure1,x_input對應圖中x_input,整個with name_scope對應中紅色大框架
with tf.name_scope('inputs'):
    xs=tf.placeholder(tf.float32,[None,1],name='x_input')
    ys=tf.placeholder(tf.float32,[None,1],name='y_niput')


l1=add_layer(xs,1,10,activation_function=tf.nn.relu)#調用上面弄到的增加一層layer,返回激勵函數後的outputs
prediction=add_layer(l1,10,1,activation_function=None)#預測值是prediction
with tf.name_scope('loss'):#這些後面可以加上name來描述
    loss=tf.reduce_mean(tf.reduce_sum(tf.square(ys-prediction),
                                      reduction_indices=[1]))
#reduction_indices在tensorflow1.0後被改爲axis，網上有改成reduc_in=1的
with tf.name_scope('train'):
    train_step=tf.train.GradientDescentOptimizer(0.1).minimize(loss)#使用這個其實就在更改bias和weighs

sess=tf.Session()
#把整個框架加載到文件裏，再打開瀏覽器才能觀看,graph是整個框架
#修改：tf.get_default_graph()原本是sess.graph
writer=tf.summary.FileWriter("./",tf.get_default_graph())
#或者下面兩行直接sess.run(tf.initialize_all_variables)
init=tf.initialize_all_variables()
sess.run(init)
print('ok')
'''
上面writer裏的地址是文件產生的地址，需要用TensorBoard打開則需要先cd到文件下，
接着終端輸入：tensorboard --logdir =路徑，打開瀏覽器打開即可
打開的網站裏面的graphs裏就是視圖了
'''

figure1:

第八課:

#-*- coding: utf-8 -*-
'''
從上一節課的延伸講解用視圖查看訓練過程
這節課比上節課多出histogram的部分來查看訓練過程loss的變化過程
想在視圖histogram查看到的內容，都在後面加上histogram_summary語句，詳情見代碼
'''
import tensorflow as tf
import numpy as np


def add_layer(inputs,in_size,out_size,n_layer,activation_function=None):
    #add a layer
    layer_name='layer%s'%n_layer
    #傳進來的layer_name，是這節課新加的,後面的每個layer_name都是這節課加的
    with tf.name_scope(layer_name):
        with tf.name_scope('weights'):
            Weights=tf.Variable(tf.random_normal([in_size,out_size]),name='W')
            tf.summary.histogram(layer_name+'/weights',Weights)
            #顯示layer_name和weights
            #網上說tf.summary.histogram(layer_name+"/biases",biases)
            #原來代碼：tf.histogram_summary
        with tf.name_scope('biases'):
            biases=tf.Variable(tf.zeros([1,out_size])+0.1,name='b')
            tf.summary.histogram(layer_name+'/biases',biases)
        with tf.name_scope('Wx_plus_b'):
            Wx_plus_b=tf.add(tf.matmul(inputs,Weights),biases)
        #先忽略activation
        if activation_function is None:
            outputs=Wx_plus_b
        else:
            outputs=activation_function(Wx_plus_b,)
        tf.summary.histogram(layer_name+'/outputs',outputs)
        return outputs
    
#make up some real data
#比上節課多出來的部分
x_data=np.linspace(-1,1,300)[:,np.newaxis]
noise=np.random.normal(0,0.05,x_data.shape)
y_data=np.square(x_data)-0.5+noise
    
with tf.name_scope('inputs'):
    xs=tf.placeholder(tf.float32,[None,1],name='x_input')
    ys=tf.placeholder(tf.float32,[None,1],name='y_niput')


l1=add_layer(xs,1,10,n_layer=1,activation_function=tf.nn.relu)
prediction=add_layer(l1,10,1,n_layer=2,activation_function=None)


with tf.name_scope('loss'):
    loss=tf.reduce_mean(tf.reduce_sum(tf.square(ys-prediction),
                                      reduction_indices=[1]))
    tf.summary.scalar('loss',loss)
    #這個是一個量的變換，不在histogram,而是在視圖Event裏觀看loss的變化
    #網上說現在是tf.summary.scalar('loss',loss),格式是tf.summary.**()
    #源代碼：tf.scalar_summary
#reduction_indices在tensorflow1.0後被改爲axis，網上有改成reduc_in=1的
with tf.name_scope('train'):
    train_step=tf.train.GradientDescentOptimizer(0.1).minimize(loss)

sess=tf.Session()
merged=tf.summary.merge_all()
#merge_all_summaries新版本改成了tf.summary.merge_all
#打包好後把所有summary打包到merged，弄到writer裏面
writer=tf.summary.FileWriter("./",tf.get_default_graph())
init=tf.initialize_all_variables()
sess.run(init)
'''
上面writer裏的地址是文件產生的地址，需要用TensorBoard打開則需要先cd到文件下，
接着終端輸入：tensorboard --logdir =路徑，打開瀏覽器打開即可
打開的網站裏面的graphs裏就是視圖了
'''
for i in range(1000):
    sess.run(train_step,feed_dic={xs:x_data,ys:y_data})
    if i%50==0:
        #五十步發揮一次merged作用
        result=sess.run(merged,
                        feed_dict={xs:x_data,ys:y_data})
        writer.add_summary(result,i)
        #網上說tf.summary.merge_all

'''
報錯:
    sess.run(train_step,feed_dic={xs:x_data,ys:y_data})
TypeError: run() got an unexpected keyword argument 'feed_dic'
還不知道怎麼改，先跳過
'''

第九課:

#-*- coding: utf-8 -*-
import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
#number 1 to 10 data,
#加載mnist數據，
mnist=input_data.read_data_sets('MNIST_data',one_hot=True)

def compute_accuracy(v_xs,v_ys):
    '''
    把vv_xs弄到prediction裏面生成預測值（一行十列概率值）
    correct_prediction裏面對比真實數據和預測值每個位置是否相等
    accuracy這組數據中計算有幾個對的
    result輸出的是百分比
    '''
    global prediction
    y_pre=sess.run(prediction,feed_dict={xs:v_xs})
    correct_prediction=tf.equal(tf.argmax(y_pre,1),tf.argmax(v_ys,1))
    accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
    result=sess.run(accuracy,feed_dict={xs:v_xs,ys:v_ys})
    return result

def add_layer(inputs,in_size,out_size,activation_function=None,):
    #add one more layer and return the output of this layer
    Weights=tf.Variable(tf.random_normal([in_size,out_size]))
    biases=tf.Variable(tf.zeros([1,out_size])+0.1)
    Wx_plus_b=tf.matmul(inputs,Weights)+biases
    if activation_function is None:
        outputs=Wx_plus_b
    else:
        outputs=activation_function(Wx_plus_b,)
    return outputs


#define placeholder for inputs to network
xs=tf.placeholder(tf.float32,[None,784])#784是每張圖片有784個像素點()28*28
ys=tf.placeholder(tf.float32,[None,10])#y有十個輸出（？）


#add output layer
prediction=add_layer(xs,784,10,activation_function=tf.nn.softmax)
#softmax一般是用來做classfication的

#the error between prediction and real data(loss)
#和以前的loss不太一樣，cross_enctropy組合softmax是classfication算法
cross_enctropy=tf.reduce_mean(-tf.reduce_sum(ys*tf.log(prediction),
                                             reduction_indices=[1]))

sess=tf.Session()

sess.run(tf.initialize_all_variables())
#隨機梯度下降
for i in range(1000):
    batch_xs,batch_ys=mnist.train.next_batch(100)
    sess.run(train_step,feed_dict={xs:batch_xs,ys:batch_ys})
    if i%50==0:
        print(compute_accuracy(
            mnist.test.images,mnist.test.labels))

'''
錯誤：連接不上mnist下載
演示的例子：0.2->0.4一直->0.87
'''

第十課:

#-*- coding: utf-8 -*-
'''
這一課好像是教解決overfitting的問題,用dropout的方法,過擬合的loss和訓練次數圖見figure2
實際上dropout是對add_layer中的Wx_plus_b捨棄值來,
標記有星星之前的是被運用dropout之前的程序
'''
import tensorflow as tf
from sklearn.datasets import load_digits
from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import LabelBinarizer
#load data
#X=digits.data是從零到九的數字圖片data,
digits=load_digits()
X=digits.data
y=LabelBinarizer().fit_transfrom(y)
#y是在表示數字的數據,類似於mnist裏的數據
#把數據分爲X_train和X_test
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=.3)

def add_layer(inputs,in_size,out_size,activation_function=None):
    with tf.name_scope('layer'):
        with tf.name_scope('weights'):
            Weights=tf.Variable(tf.random_normal([in_size,out_size]),name='W')
        with tf.name_scope('biases'):
            biases=tf.Variable(tf.zeros([1,out_size])+0.1,name='b')
        with tf.name_scope('Wx_plus_b'):
            Wx_plus_b=tf.add(tf.matmul(inputs,Weights),biases)
            Wx_plus_b=tf.nn.dropout(Wx_plus_b,keep_prob)#星星,這個是dropout核心
        
        if activation_function is None:
            outputs=Wx_plus_b
        else:
            outputs=activation_function(Wx_plus_b)
        tf.histogram_summary(layer_name+'/outputs',outputs)
        #這句不加up主說報錯
        return outputs

#define placeholder for inputs to network
keep_prob=tf.placeholder(tf.float32)#星星
#keep probability,保持多少的值不被dropout掉
xs=tf.placeholder(tf.float32,[None,64])#8*8
ys=tf.placeholder(tf.float32,[None,10])

#add output layer
#一個隱藏層,一個輸出層,skldata中data是64個單位,所以input_size=64
#activation_f的使用是up主的原因
l1=add_layer(xs,64,50,'l1',activation_function=tf.nn.tanh)#8*8
prediction=add_layer(l1,50,10,'l2',activation_function=tr.nn.softmax)
#the loss between prediction and real data
cross_entropy=tf.reduce_mean(-tf.reduce_sum(ys*tf.log(prediction),
                                            reduction_indices=[1]))
#sklearn.cross_validation已經被棄用 用sklearn.model_selection替代
#cross_entropy實際是loss,再把這個放入summary中,在在Tensorboard中顯示
tf.scalar_summary('loss',cross_entropy)
train_step=tf.train.GradientDescentOptimizer(0.6).minimize(cross_entropy)

sess=tf.Session()
merged=tf.merge_all_summaries()
#summary writer goes in there
train_writer=tf.train.SummaryWriter("./train",sess.graph)
test_writer=tf.train.SummaryWriter("./test",sess.graph)
#要記錄兩個summary:test 和train,
#網上說tf.train.SummaryWriter已棄用，需要改成tf.summary.FileWriter

sess.run(tf.initialize_all_variables())
#initialize_all_variables已被棄用，使用tf.global_variables_initializer()代替。

for i in range(500):
    sess.run(train_step,feed_dict={xs:X_train,ys:y_train,keep_pro:0.5})#星星,改成1可以看到dropout前的
    if i%50==0:
        #record loss
        train_result=sess.run(merged,fed_dict={xs:X_train,ys:train,keep_pro:1})#星星
        test_result=sess.run(merged,fed_dict={xs:X_train,ys:train,keep_pro:1})#星星
        train_writer,add_summary(train_result,i)
        test_writer,add_summary(test_result,i)

figure2:

tensorflow學習筆記第一部分

10分鐘搞定Mysql主從部署配置

如何使用 JS 判斷用戶是否處於活躍狀態

「Pygors跨平臺GUI」2：安裝MinGW-w64、MSYS2還是WSL2

[轉帖]

python列出centos7內存使用前50的進程信息

「Pygors跨平臺GUI」1：Pygors跨平臺GUI應用研究

一鍵自動化博客發佈工具,用過的人都說好(掘金篇)

lightdb數據庫超時相關控制參數

lightdb秒級增加列和刪除列（not null帶默認值）

Java ThreadPoolShutdown

linux進程空間的內存管理

InnoDB的隔離級別實現

linux內核的內存管理

linux的VFS

linux的high memory

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結