Today is a special holiday ,1946 Valentine's Day （ The original publication date is 2021.2.14）, The first computer in the world ENIAC At the University of Pennsylvania in the United States new 了 , It marks the arrival of a new era .
Computers have gone through the process of accompanying human beings 75 A year , So today , Nothing special , Please accompany your computer more , Manual formation , Manual formation .
Network programming will be a huge knowledge system , The third chapter will start with how to encode and decode.
The data structure in Chapter 1 is provided for Chapter 3 , Then it is transmitted through the channel in Chapter 2 .
Review lessons
First address
Part two address
Tcp Network flow , The processing process can also be understood as a byte stream to a binary stream , Bytes are the ones mentioned in the first article bytes.
encode Packet compression is to convert byte stream to binary stream , that Python How did you do it .
Mainly divided into two types ,pickle（python Unique serialized byte array ）,struct（ from C The serialized byte array inherited over there ） library .
Let's see. pickle How to serialize data .
def encode_pickle(packet:bytes):
"""
Push the pickle
:param packet:
:return:
"""
return pickle.pack(">i",len(packet))+packet
Note that there is no contract out here , Review the previous , You need to create socket link , Confirm address , To contract out .
pickle.pack(fmt,*args) The first parameter of fmt It's a very important knowledge point .
Let's start with one of the most important concepts fmt, as everyone knows ,Python There is no need to declare the memory width first , Can be programmed .
Memory width refers to the length of the space occupied in the memory , The data type is marked before declaring the object （ Dynamic languages don't have this , Many static languages have automatic inference declarations such as golang var and :=）
Network programming needs to learn this , Because of the operation binary , Careful words will be marked as signed and unsigned . Do some atomic calculations with and without symbols , When subtracting , Some languages need some special processing , not so bad Python Don't worry about that , Signed generally refers to the inclusion of negative numbers .
short It's symbolic 2 Bytes of ,unsigned short It's unsigned 2 Bytes , The figure above of negative numbers needs to be memorized .
In the above code is int and unsigned int,fmt The front is the host byte order ,">i" It stands for the big end with symbols 4 Bytes ,"<" It stands for small end ,"!" Means no match .
For the time being, we will not consider the problem of byte order conversion , At present, network programming is to simulate the client sending to the server .
Byte order has big end respectively （big endian） And little end （little endian）, The contents of the program are in bytes , A byte 8 position , Each address corresponds to a byte .
The big end mode is to put the high bit of data in the low bit address , The lower address is the left . turn 16 Base is just for people to read ,int type 4 Bytes ,ff 6c 5a 4s.
The small end is the big end , yes 4s 5a 6c ff. It can be found that the size side is only related to the memory addressing order , But in the memory address 6c,5a It won't become c6 and a5 Of
This question , If you are unfamiliar in the early stage , You can confirm how to write by asking the requirements .
The number after the data type is divided by 8 Is the number of bytes , The number is bit.
take JS for instance ：buffer View processing , such as Uint32Array,U It means no sign ,.int32 yes 4 Bytes ,Array It can be understood as an array .
A byte array is just a number of bytes ,Python The data structure is bytes perhaps bytearray( The difference between this and the former is that memory is immutable )
Uint64Array That's it 8 Bytes ？ This is illegal . Here is the concept of fixed length and variable length ,int It belongs to fixed length ,int Yes, the width cannot be exceeded 4 Bytes of , So there won't be Uint64Array, But there can be Uint16Array,16/8=2, Occupy a space in the memory 2 Bytes .
The fixed length is explained in specific examples , Variable length is the core part , Look down 2 A question ：
problem 1： Reference type , For example, a pointer to an object , If you manipulate this object , The width of this object will not change .
problem 2：long type , Network programming is also used to simulate the client contracting to the server ,long The type is also variable in length , Like not 8 Multiple , yes 9 Bytes .
see 9 How is a byte written .
def encode_pack(packet:str or bytes,size:int,endian:str="big"):
"""
Jacketing
:param packet:
:param size: Fixed length
:param endian:
:return:
"""
return int(packet).to_bytes(length=size, byteorder=endian)
print(encode_pack("1256478912".encode("utf-8"),9))
Of course, this model is not recommended , More suitable fmt Inside to splice , For example, Daduan 9 Bytes - -, Direct use >9s That's it .
struct and pickle It's essentially the same thing , The essence is the same, just like Json and bjson And others json equally , The method is the same . Now you can string up the front ones .
def encode_9_buffer(packet: bytes):
"""
Push the 9 A buffer of bytes
:param packet:
:return:
"""
return struct.pack(">9s", len(packet).to_bytes(length=9, byteorder="big")) + packet
pg = "hello world".encode()
print(struct.calcsize(">9s")) # length 9
print(encode_9_buffer(pg)) #b'\x00\x00\x00\x00\x00\x00\x00\x00\x0bhello world'
front 9s Is an example of an entire package , Here's a preview , The back is not fixed , It is also the most commonly used in the game industry , The first 4 The article will mention . The current chapter can actually meet some of the requirements of the Internet TCP Pre network programming .
Use pickle Push into cache 4 Bytes big end hello world, And then use struct Unpack and restore , To verify that forward and deserialization are compatible .
therefore python If the back-end is written in pickle Of , use struct It's OK, too .
def encode_pickle(packet: bytes):
"""
Push the pickle
:param packet:
:return:
"""
return pickle.pack(">i", len(packet)) + packet
def decode_unpack(packet: bytes):
"""
unpack package It doesn't include struct.unpack
:param packet:
:return:
"""
# Because Baotou is 4 Bytes , The validity verification is to cut out first 4 Bytes .
if len(packet) >= 4:
return packet[4:].decode()
if __name__ == '__main__':
pg = "hello world".encode()
packet = encode_pickle(pg)
print(decode_unpack(packet))
Review the previous knowledge points . A packet consists of a header and a body , Baotou is 4 Bytes （ The length of the inclusion is stored inside ）.
To judge the legitimacy simply is to check whether the package is complete , The package is complete by 2 Component composition , The first part is 4 Bytes , Then it is to judge whether it is greater than or equal to 4 Bytes , Then unpack and remove the head first 4 Bytes , What's inside is the contents of the package .
The contents of the package are restored decode(), Let's take a look at how to take out the length of the package inside the package head
struct.unpack(">i",packet[:4])
See the fourth article for other contents .
In this paper 2021.02.14 First appeared in TesterHome Community , The author is Chen Ziang, a senior game testing and development engineer . use Python Write four articles on network programming , Today, I will share the third of them . Link to the original text ： https://testerhome.com/topics/27910*
The above is today's sharing , Did you stop learning ~
Want to learn more about dry goods and cutting-edge technology ？
I want to get acquainted with the test industry and the industry elite ？
Welcome to your attention 2022 MTSC The conference （ The 10th China Internet testing and Development Conference ）
Industry pairs MTSC The evaluation of the General Assembly ： to ground 、 Pragmatic 、 There is depth 、 Re share
China Internet test and Development Conference （Testing Summit China）, One of the largest test and development technology communities in China TesterHome launch . Began in 2015 year , Nine sessions have been held successfully . From the initial 300 Many people arrive at 2019 Close to the year 2000 After the man , The scale of the thousand people's Congress has been maintained .
In order to ensure the quality of the topic ,TesterHome Every year, topics will be collected half a year in advance , After several months of review , Finalize the topics to be shared at the conference .MTSC 2022 The topics are still being solicited , We sincerely invite all senior test technology experts 、 Quality management managers and test rookies recommend themselves / Recommended topics ！
Just click https://www.wjx.top/vj/wZwCju3.aspx Enter the delivery entrance , Fill in and submit according to the format , Welcome to introduce yourself / Recommended topics .
In order to facilitate the review team to have more time for review and communication with lecturers , Present better topics for the majority of participants , Topic submission （ But there is no ppt, There is an outline and other topic information introduction ） The delivery deadline is advanced to ：2022 year 3 month 30 Japan
Overall process ： Case submission > Preliminary examination and approval > PPT Submit > Confirm the topic > Conference speech
Overall time node ：
Case submission >> 3 month 30 Japan
Preliminary examination and approval >> 4 month 15 Japan
ppt Submit >> 5 month 15 Japan
Topic confirmation >> 5 month 30 Japan
Conference speech >> 7 month 22~23 Japan
MTSC 2022 Early bird tickets have been quietly on sale , Click to learn more .
