numpy的文件存儲與處理

import numpy as np
from io import StringIO

第二部分 ndarray 文件存儲與讀取

一維二維數組 可以採用csv（Comma Separated Value 逗號分割值）

”’
1)np.savetxt np.loadtxt

numpy.savetxt(fname, X, fmt=’%.18e’, delimiter=’ ‘, newline=’\n’, header=”, footer=”, comments=’# ‘)

fname : filename or file handle 指定要存入的文件名,支持gzip （.gz .bz2）格式
If the filename ends in .gz, the file is automatically saved in compressed gzip format. loadtxt understands gzipped files transparently(易察覺的).

X : array_like 一個ndarray對象
Data to be saved to a text file.

fmt : str or sequence of strs, optional 制定每個元素要存入的格式
e.g. [‘%.3e + %.3ej’, ‘(%.15e%+.15ej)’] for 2 columns (複數可以這麼寫 多列)

delimiter : str, optional 每個元素之間的分割符
String or character separating columns.

newline : str, optional 每行的換行符
String or character separating lines.
New in version 1.5.0.
header : str, optional 文件開始時的文本
String that will be written at the beginning of the file.
New in version 1.7.0.
footer : str, optional 文件結束時的文本
String that will be written at the end of the file.
New in version 1.7.0.
comments : str, optional 將header and footer 標註爲註釋
String that will be prepended to the header and footer strings, to mark them as comments. Default: ‘# ‘, as expected by e.g. numpy.loadtxt.
New in version 1.7.0.
”’
x=np.array([[1,2,3,4],[2,3,4]])#不可存儲，用loadtxt讀取不了,因爲loadtxt必須保證列數相同
np.savetxt(fname=”test1.out”,fmt=”%s”,header=”This is test1.txt”,X=x)
y=np.array([[1,2],[3,4]])
np.savetxt(fname=”test2.out”,fmt=”%d” ,footer=”This is test2.txt”,X=y)
”’

numpy.loadtxt(fname, dtype=float, comments=’#’, delimiter=None, converters=None, skiprows=0, usecols=None, unpack=False, ndmin=0)

Parameters:
fname : file, str, or pathlib.Path
File, filename, or generator to read. If the filename extension is .gz or .bz2, the file is first decompressed. Note that generators should return byte strings for Python 3k.

dtype : data-type, optional
Data-type of the resulting array; default: float. If this is a structured data-type, the resulting array will be 1-dimensional, and each row will be interpreted as an element of the array. In this case, the number of columns used must match the number of fields in the data-type.

comments : str or sequence, optional（註釋類型 要和savetxt的相對應）
The characters or list of characters used to indicate the start of a comment; default: ‘#’.
delimiter : str, optional
The string used to separate values. By default, this is any whitespace.

converters : dict, optional(解析函數）
A dictionary mapping column number to a function that will convert that column to a float. E.g., if column 0 is a date string: converters = {0: datestr2num}. Converters can also be used to provide a default value for missing data (but see also genfromtxt): converters = {3: lambda s: float(s.strip() or 0)}. Default: None.
skiprows : int, optional
Skip the first skiprows lines; default: 0.
usecols : int or sequence, optional（要提取的列）
Which columns to read, with 0 being the first. For example, usecols = (1,4,5) will extract the 2nd, 5th and 6th columns. The default, None, results in all columns being read.
New in version 1.11.0.

Also when a single column has to be read it is possible to use an integer instead of a tuple. E.g usecols = 3 reads the fourth column the same way as usecols = (3,)` would.
unpack : bool, optional(打包？)
If True, the returned array is transposed, so that arguments may be unpacked using x, y, z = loadtxt(…). When used with a structured data-type, arrays are returned for each field. Default is False.
ndmin : int, optional
The returned array will have at least ndmin dimensions. Otherwise mono-dimensional axes will be squeezed. Legal values: 0 (default), 1 or 2.
New in version 1.6.0.

Returns:
out : ndarray
Data read from the text file.
”’

test1=np.loadtxt(fname=”test2.out”,dtype=int)
print(“test2:”,test1)
x1,x2=np.loadtxt(fname=”test2.out”,dtype=int,unpack=True)
print(“x1,x2:”,x1,x2)
str=StringIO(“Zhang 20 boy\nYu 20 girl”)
struct_dtype=np.dtype({‘names’:(‘name’,’age’,’gender’), ‘formats’:(‘S32’,’i4’,’S32’)},align=True)
str_array=np.loadtxt(fname=str,dtype=struct_dtype)
print(“str_array:”,str_array)

2)多維數組

”’

tofile(self, fid, sep=”“, format=”%s”)

fid : file or str
An open file object, or a string containing a filename.
sep : str 如果爲空，則默認爲二進制存儲
Separator between array items for text output.
If “” (empty), a binary file is written, equivalent to
file.write(a.tobytes()).
format : str 元素類型
Format string for text file output.
Each entry in the array is formatted to text by first converting
it to the closest Python type, and then using “format” % item.

            This is a convenience function for quick storage(存儲) of array data.
        Information on endianness(字節序) and precision(精確度) is lost, so this method is not a
        good choice for files intended to(打算) archive data or transport data between
        machines with different endianness. Some of these problems can be overcome
        by outputting the data as text files, at the expense of(在損失....的情況下) speed and file
        size.

”’
y.tofile(file=”tofile.txt”,format=”%d” ,sep=” “)
from_file_array=np.fromfile(file=”tofile.txt”,dtype=int,count=-1,sep=” “)#count=-1默認讀取整個文件
print(“from_file_array:”,from_file_array)
reshape_from_file_array=from_file_array.reshape((2,2))
print(“reshape_from_file_array:”,reshape_from_file_array)

3)numpy 自定義格式

np.save(file=”np_save”,arr=y)#默認.npy後綴 其實這個文件打開看一看，第一行是明文的這個ndarray數組的信息，第二行開始是二進制的數據存儲
print(“np.load:”,np.load(file=”np_save.npy”))
np.savez_compressed(file=”np_save_compressed”,Y=y)#默認爲壓縮的.npz格式 y的名字爲Y
print(“np.load:”,np.load(file=”np_save_compressed.npz”)[‘Y’])#數組以字典方式進行存儲
np.savez(file=”np_save_uncompressed”,Y=y)#默認爲非壓縮的.npz格式
print(“np.load:”,np.load(file=”np_save_uncompressed.npz”,)[‘Y’])