python struct 學習筆記

python struct 學習筆記

part 1: introduce

python 是一門簡潔的語言。爲了與其他語言或平臺（尤其在網絡傳輸過程中）進行互相轉換，有必要了解一下 python struct.

part 2: get help

NAME
    struct
FILE
    /usr/lib/python2.6/struct.py
FUNCTIONS
    calcsize(...)
        Return size of C struct described by format string fmt.
    pack(...)
        Return string containing values v1, v2, ... packed according to fmt.
    pack_into(...)
        Pack the values v1, v2, ... according to fmt.
        Write the packed bytes into the writable buffer buf starting at offset.
    unpack(...)
        Unpack the string containing packed C structure data, according to fmt.
        Requires len(string) == calcsize(fmt).
    unpack_from(...)
        Unpack the buffer, containing packed C structure data, according to
        fmt, starting at offset. Requires len(buffer[offset:]) >= calcsize(fmt).


part 3: instances

function 1: 
calcsize(...)  計算格式字符串對應的結果的長度

>>> b = struct.calcsize('HB')
>>> b
3
>>> b = struct.calcsize('!HBBBBBBIIIIBH')
>>> b
27
>>> b = struct.calcsize('HBBBBBBIIIIBH')
>>> b
28

function 2: 
pack(...) 根據格式字符串，將python值轉換爲字節流（字節數組）

>>> a = struct.pack("!HBBBBBBIIIIBH",48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> a
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01\x00\>>> repr(a)x02'

function 3:
pack_into(...) 與pack()類似。根據格式字符串，將python值轉換爲字節流（字節數組),並且將這字節流保存到一個buffer 中

>>> from ctypes import create_string_buffer
>>> buf = create_string_buffer(40)
>>> a = struct.pack_into("!HBBBBBBIIIIBH",buf,0,48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> buf
<ctypes.c_char_Array_40 object at 0x7f0eefd11dd0>

function 4：
unpack(...)  與pack() 相反。根據格式字符串，將字節流轉換成python 數據類型。

>>> b = struct.unpack("!HBBBBBBIIIIBH",a)

>>> b
(48251, 0, 24, 0, 1, 0, 0, 842003, 4294963512, 2300851747, 368142189, 1, 2)


function 5:
unpack_from(...) 與unpack()類似。根據格式字符串，從一個buffer 中將字節流轉換成python 數據類型

>>> b  = struct.unpack_from("!HBBBBBBIIIIBH",buf,0)
>>> b
(48251, 0, 24, 0, 1, 0, 0, 842003, 4294963512, 2300851747, 368142189, 1, 2)
>>> buf

<ctypes.c_char_Array_40 object at 0x7f0eefd11dd0>

# pack 之後得到的字節流（數組）也可以像序列一樣來操作：

>>> a = struct.pack("!HBBBBBBIIIIBH",48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> a
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01\x00\x02'
>>> len(a)
27
>>> head  = a[:25]
>>> head
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01'
>>> command = a[25:]
>>> command
'\x00\x02'
>>> command = a[24:]
>>> command
'\x01\x00\x02'
>>> command = a[-4:]
>>> command
'm\x01\x00\x02'
>>> command = a[-3:]
>>> command
'\x01\x00\x02'

part 4: big-endian & format

 Functions to convert between Python values and C structs represented
    as Python strings. It uses format strings (explained below) as compact
    descriptions of the lay-out of the C structs and the intended conversion
    to/from Python values.

    The optional first format char indicates byte order, size and alignment:
      @: native order, size & alignment (default)
      =: native order, std. size & alignment
      <: little-endian, std. size & alignment
      >: big-endian, std. size & alignment
      !: same as >

    The remaining chars indicate types of args and must match exactly;
    these can be preceded by a decimal repeat count:
      x: pad byte (no data); c:char; b:signed byte; B:unsigned byte;
      ?: _Bool (requires C99; if not available, char is used instead)
      h:short; H:unsigned short; i:int; I:unsigned int;
      l:long; L:unsigned long; f:float; d:double.
    Special cases (preceding decimal count indicates length):
      s:string (array of char); p: pascal string (with count byte).
    Special case (only available in native format):
      P:an integer type that is wide enough to hold a pointer.
    Special case (not in native mode unless 'long long' in platform C):
      q:long long; Q:unsigned long long
    Whitespace between formats is ignored.

大端模式：

字節的傳輸約定：按照B7~B0的順序傳輸；
字的傳輸約定： 先傳遞高8位（B15~B8），再傳遞低8位，（B7~B0）；
雙字的傳輸約定： 先傳遞高24位，（B31~B24），然後傳遞高16位（B23~B16），再傳遞高8位（B15~B8），最後傳遞低8位（B7~B0）。

>>> b  = struct.pack("!H",1)
>>> b
'\x00\x01'
>>> b  = struct.pack("H",1)
>>> b
'\x01\x00'
>>> b  = struct.pack("L",1)
>>> b
'\x01\x00\x00\x00\x00\x00\x00\x00'
>>> b  = struct.pack("!L",1)
>>> b
'\x00\x00\x00\x01'


格式化字符串：

# P     void *     long
>>> buffer = struct.pack("P",1)
>>> buffer
'\x01\x00\x00\x00\x00\x00\x00\x00'

# p     char[]     string
>>> buffer = struct.pack("p","string")
>>> buffer
'\x00'

# s     char[]     string
>>> buffer = struct.pack("ssss","string")
>>> buffer
's'
>>> buffer = struct.pack("sss","string","st","ff")
>>> buffer
'ssf'
>>> buffer = struct.pack("sss","string","j","ff")
>>> buffer
'sjf'
>>> buffer  = struct.pack("%ds"%len('jia xiaolei'),"jia xiaolei")
>>> buffer
'jia xiaolei'

# d     double     float
>>> buffer = struct.pack("d", 1.2)
>>> buffer
'333333\xf3?'

# f     float     float
>>> buffer = struct.pack("f", 1.2)
>>> buffer
'\x9a\x99\x99?'

#  這裏很有用，Q 可以匹配幾乎所有的東西 
Q     unsigned long long     long
>>> buffer = struct.pack("Q",25111)
>>> buffer
'\x17b\x00\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("Q",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'

# q     long long     long
>>> buffer = struct.pack("q",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("q",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'

# L     unsigned long     long

>>> buffer = struct.pack("L",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("L",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'

# l     long     integer
>>> buffer = struct.pack("l",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("l",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("l",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("l",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'

# I     unsigned int     integer or long

>>> buffer = struct.pack("I",2545111)
>>> buffer
'\xd7\xd5&\x00'
>>> buffer = struct.pack("I",47882545111)
__main__:1: DeprecationWarning: 'I' format requires 0 <= number <= 4294967295
#  如果傳入的值的範圍查過格式字符串說約定的範圍，會報錯

# i     int     integer
>>> buffer = struct.pack("i",5111)
>>> buffer
'\xf7\x13\x00\x00'
>>> buffer = struct.pack("i",775111)
>>> buffer
'\xc7\xd3\x0b\x00'

# H     unsigned short     integer
>>> buffer = struct.pack("H",775111)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: ushort format requires 0 <= number <= USHRT_MAX
>>> buffer = struct.pack("H",11)
>>> buffer
'\x0b\x00'
>>> buffer = struct.pack("H",711)
>>> buffer
'\xc7\x02'

# h     short     integer
>>> buffer = struct.pack("h",711)
>>> buffer
'\xc7\x02'
>>> buffer = struct.pack("",7811)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: pack requires exactly 0 arguments

# ?     _Bool     bool
>>> buffer = struct.pack("?",True)
>>> buffer
'\x01'
>>> buffer = struct.pack("?",False)
>>> buffer
'\x00'

# B     unsigned char     integer
>>> buffer = struct.pack("B",2)
>>> buffer
'\x02'
>>> buffer = struct.pack("B",15)
>>> buffer
'\x0f'

# b     signed char     integer
>>> buffer = struct.pack("b",15)
>>> buffer
'\x0f'
>>> buffer = struct.pack("b",1)
>>> buffer
'\x01'
>>> buffer = struct.pack("b",51)
>>> buffer
'3'
>>> buffer = struct.pack("b",51)
>>> buffer
'3'

# c     char     string of length 1
>>> buffer = struct.pack("c","1")
>>> buffer
'1'
>>> buffer = struct.pack("c","7")
>>> buffer
'7'

# x
>>> buffer = struct.pack("x",)
>>> buffer
'\x00'