深度學習筆記之（四）Python數據處理庫pandas

版本信息

Python                   3.6.2

pandas                 0.23.4

本文所需文件下載地址：https://download.csdn.net/download/plsong_csdn/10764054

---

基本操作

讀取文件

mport pandas as pd
food_info = pd.read_csv('food_info.csv') #讀取數據
print(type(food_info)) #DataFrame結構
print(food_info.dtypes) #當前數據中的數據結構有哪些
food_info.head() #顯示前幾個信息
food_info.tail() #顯示後幾個信息
food_info.columns #每個列的指標，也就是說獵命
food_info.shape #樣本的大小

運行結果：

<class 'pandas.core.frame.DataFrame'>
NDB_No               int64
Shrt_Desc           object
Water_(g)          float64
Energ_Kcal           int64
......
Cholestrl_(mg)     float64
dtype: object

說明：

（1）DataFrame數據類型是pd讀取文件進行存儲的數據類型，可以近似看作一個矩陣的結構。

（2）object數據類型其實就是string類型的數據。

---

pandas索引

按行按列讀取

import pandas as pd
food_info = pd.read_csv('food_info.csv') #讀取數據
print(food_info.loc[0]) #取第一行數據
print(food_info.loc[3:6]) #取4:7行的數據
ndb_col = food_info['NDB_No'] #取'NDB_No'這一列的數據
print(ndb_col)
colums = ['NDB_No', 'Water_(g)'] #同時讀取'NDB_No'， 'Water_(g)'這一列的數據
print(food_info[colums])

運行結果：

NDB_No                         1001
Shrt_Desc          BUTTER WITH SALT
......
FA_Poly_(g)                   3.043
Cholestrl_(mg)                  215
Name: 0, dtype: object

   NDB_No         Shrt_Desc       ...        FA_Poly_(g)  Cholestrl_(mg)
3    1004       CHEESE BLUE       ...              0.800            75.0
4    1005      CHEESE BRICK       ...              0.784            94.0
5    1006       CHEESE BRIE       ...              0.826           100.0
6    1007  CHEESE CAMEMBERT       ...              0.724            72.0

[4 rows x 36 columns]
0        1001
1        1002
2        1003

        ...  

8617    93600
Name: NDB_No, Length: 8618, dtype: int64

      NDB_No  Water_(g)
0       1001      15.87
1       1002      15.87
......
8617   93600      78.50

[8618 rows x 2 columns]

讀取文件中的特定信息

import pandas as pd
food_info = pd.read_csv('food_info.csv') #讀取數據
col_name = food_info.columns.tolist() #將當前的列名作爲一個list
print(col_name)
print('---------------------')
gram_columns = []

for c in col_name:
    if c.endswith('(g)'):
        gram_columns.append(c)
print(gram_columns)
print('---------------------')
gram_df = food_info[gram_columns] #frame結構
print(gram_df.head(3))

運行結果：

['NDB_No', 'Shrt_Desc', 'Water_(g)', 'Energ_Kcal', 'Protein_(g)', 'Lipid_Tot_(g)', 'Ash_(g)', 'Carbohydrt_(g)', 'Fiber_TD_(g)', 'Sugar_Tot_(g)', 'Calcium_(mg)', 'Iron_(mg)', 'Magnesium_(mg)', 'Phosphorus_(mg)', 'Potassium_(mg)', 'Sodium_(mg)', 'Zinc_(mg)', 'Copper_(mg)', 'Manganese_(mg)', 'Selenium_(mcg)', 'Vit_C_(mg)', 'Thiamin_(mg)', 'Riboflavin_(mg)', 'Niacin_(mg)', 'Vit_B6_(mg)', 'Vit_B12_(mcg)', 'Vit_A_IU', 'Vit_A_RAE', 'Vit_E_(mg)', 'Vit_D_mcg', 'Vit_D_IU', 'Vit_K_(mcg)', 'FA_Sat_(g)', 'FA_Mono_(g)', 'FA_Poly_(g)', 'Cholestrl_(mg)']
-----------------------
['Water_(g)', 'Protein_(g)', 'Lipid_Tot_(g)', 'Ash_(g)', 'Carbohydrt_(g)', 'Fiber_TD_(g)', 'Sugar_Tot_(g)', 'FA_Sat_(g)', 'FA_Mono_(g)', 'FA_Poly_(g)']
-----------------------
   Water_(g)  Protein_(g)     ...       FA_Mono_(g)  FA_Poly_(g)
0      15.87         0.85     ...            21.021        3.043
1      15.87         0.85     ...            23.426        3.012
2       0.24         0.28     ...            28.732        3.694

[3 rows x 10 columns]

---

添加新屬性

import pandas as pd
food_info = pd.read_csv('food_info.csv') #讀取數據
iron_grams = food_info['Iron_(mg)'] / 1000 #將mg化爲g
print(food_info.shape)
food_info['Iron_(g)'] = iron_grams #將['Iron_(g)']這一列屬性添加到food_info這個frame裏面
print(food_info.shape)
food_info.head(4)

運行結果：

(8618, 36)
(8618, 37)

說明：

food_info['Iron_(g)'] = iron_grams

上面這一句代碼其實做了兩個工作

（1）爲food_info創建一個新屬性，也就是說添加了新的一列

（2）將['Iron_(g)']這個新屬性賦值給food_info

---

對某一屬性進行排序

import pandas as pd
food_info = pd.read_csv('food_info.csv') #讀取數據
food_info.sort_values('Water_(g)',inplace=True)
print(food_info['Water_(g)'])
print('--------------------------------------')
food_info.sort_values('Water_(g)',inplace=True, ascending=False) #取消升序
print(food_info['Water_(g)'])

運行結果：

676       0.00
664       0.00
         ...  
4404     99.98
4372     99.98
4378    100.00
4377    100.00
4348    100.00
4376    100.00
4209    100.00
1983       NaN
6067       NaN
6095       NaN
6113       NaN
6150       NaN
7776       NaN
Name: Water_(g), Length: 8618, dtype: float64
--------------------------------------
4209    100.00
4376    100.00
4348    100.00
4377    100.00


         ...  

743       0.00
676       0.00
1983       NaN
6067       NaN
6095       NaN
6113       NaN
6150       NaN
7776       NaN
Name: Water_(g), Length: 8618, dtype: float64

說明：NaN代表該列是缺失的。

---

求泰坦尼克號上乘客的平均年齡

方法一

import pandas as pd
import numpy as np

titanic_survival = pd.read_csv('titanic_train.csv')
age = titanic_survival['Age'] #讀取年齡
print('age[0:10]:\n',age.loc[0:10]) #輸出age的0:10列
age_is_null = pd.isnull(age) #判斷是否有空缺項
good_ages = titanic_survival["Age"][age_is_null == False]
age_mean = sum(good_ages) / len(good_ages)
print("age_mean:",age_mean)

運行結果：

age[0:10]:
 0     22.0
1     38.0
2     26.0
3     35.0
4     35.0
5      NaN
6     54.0
7      2.0
8     27.0
9     14.0
10     4.0
Name: Age, dtype: float64
age_mean: 29.69911764705882

說明：因爲有空缺的項會對結果有影響，因此，計算時應當對空缺的項目進行處理，上面採取的是去掉年齡空缺的乘客；

方法二：

import pandas as pd
import numpy as np

titanic_survival = pd.read_csv('titanic_train.csv')
age_mean = titanic_survival['Age'].mean()
print('age_mean:',age_mean)

運行結果：

age_mean: 29.69911764705882

說明：

（1）用mean()函數，採取默認去掉年齡空缺的乘客。.

（2）去掉有空缺的行，可以使用dropna()

---

兩個屬性之間的關係pivot_table

示例：求不同船艙的平均票價

import pandas as pd
import numpy as np

titanic_survival = pd.read_csv('titanic_train.csv')
fares_by_calss = titanic_survival.pivot_table(index='Pclass',values="Fare",aggfunc=np.mean)
print(fares_by_calss)

運行結果：

Pclass      Fare      
1         84.154687
2         20.662183
3         13.675550

示例：求不同船艙的乘客的平均年齡

import pandas as pd
import numpy as np

titanic_survival = pd.read_csv('titanic_train.csv')
fares_by_calss = titanic_survival.pivot_table(index='Pclass',values="Age",aggfunc=np.mean)
print(fares_by_calss)

運行結果：

              Age
Pclass           
1       38.233441
2       29.877630
3       25.140620

示例：一個量與其他兩個量之間的關係。求不同上船地點乘客的倖存總數和票價總數

import pandas as pd
import numpy as np

titanic_survival = pd.read_csv('titanic_train.csv')
fares_by_calss = titanic_survival.pivot_table(index='Embarked',values=["Age","Survived"],aggfunc=np.sum)
print(fares_by_calss)

運行結果：

               Age  Survived
Embarked                    
C          4005.92        93
Q           786.50        30
S         16312.75       217

---

pandas中的series結構

初識series

import pandas as pd
import numpy as np

fandango = pd.read_csv('fandango_score_comparison.csv')
series_film = fandango['FILM']
print(type(series_film))
print(series_film[0:5])

film_name = series_film.values
print(type(film_name))

運行結果：

<class 'pandas.core.series.Series'>
0    Avengers: Age of Ultron (2015)
1                 Cinderella (2015)
2                    Ant-Man (2015)
3            Do You Believe? (2015)
4     Hot Tub Time Machine 2 (2015)
Name: FILM, dtype: object

<class 'numpy.ndarray'>

說明：

(1)series可以看作是datafram的組成部分。有衆多series組成dataframe.

(2)series又是由ndarray構成

series索引

import pandas as pd
from pandas import Series

fandango = pd.read_csv('fandango_score_comparison.csv') #讀取文件，dataframe
series_film = fandango['FILM'] #series
series_rt = fandango['RottenTomatoes'] #series
film_name = series_film.values #ndarray
rt_scores = series_rt.values #ndarray
series_custom = Series(rt_scores, index=film_name)

print(series_custom[['Minions (2015)', 'Leviathan (2014)']])

運行結果：

Minions (2015)      54
Leviathan (2014)    99
dtype: int64

說明

series_custom = Series(rt_scores, index=film_name)

series可以用名字當索引

 

---

★finished 2018.11.4night 

★by songpl