Check common encryption algorithms in Python Crawlers

Preface

1. Basic knowledge

2. Base64 Pseudo encryption

3. MD5 encryption

4. AES/DES Symmetric encryption

secret key

fill

Pattern

Today, Xiaobian will take you to review these mainstream encryption algorithms in the process of data capture , What are their characteristics 、 What are the encryption methods and so on , Knowing this will help us to reverse crack these encrypted parameters ！

I believe that when you capture data , You will encounter many encrypted parameters , Like "token"、"sign" wait , Today, Xiaobian will take you to review these mainstream encryption algorithms in the process of data capture , What are their characteristics 、 What are the encryption methods and so on , Knowing this will help us to reverse crack these encrypted parameters ！

The first thing we need to understand is , What is encryption and decryption ？ seeing the name of a thing one thinks of its function

encryption （Encryption）： The process of transforming plaintext data into ciphertext

Decrypt （Decryption）： The reverse of encryption , That is, the process of recovering the original plaintext from the ciphertext .

The operation of encryption and decryption algorithms is usually carried out under the control of a set of keys , Become the encryption key respectively （Encryption Key） And decrypt the key （Decryption Key）,

As shown in the figure below ：

The encryption algorithms are divided into symmetric encryption, asymmetric encryption and hash algorithm , among

Symmetric encryption ： That is, the same key is used for encryption and decryption , for example RC4、AES、DES Equal encryption algorithm

Asymmetric encryption ： That is, different keys are used for encryption and decryption , for example RSA Encryption algorithm, etc

Hash algorithm ： Also known as hash function . Produces a fixed output for input messages of different lengths , The output value is the hash value

Base64 Strictly speaking, it is not an encryption algorithm , Just a way of coding , It is a kind of use 64 Characters , Namely A-Z、a-z、0-9、+、/ this 64 Characters , Realize the coding of data , Can be used in HTTP In the environment, the longer identification information . use Base64 The code is unreadable , It needs to be decoded before reading . We use Python For any web address Base64 The encoding operation of , The code is as follows ：

Output ：

b'd3d3LmJhaWR1LmNvbQ=='

So again , We can also decode it , The code is as follows ：

Output ：

www.baidu.com

MD5 It is a widely used linear hash algorithm , After encryption, a fixed length is generated (32 Bit or 16 position ) The data of , It consists of letters and numbers , Unified case . The data generated by the final encryption is irreversible , That is to say, the encrypted data cannot be easily restored to the original string , Except by brute force .

We are Python To realize MD5 encryption ：

Output ：

MD5 Before encryption ：this is a md5 demo.
MD5 Encrypted as ：b2caf2a298a9254b38a2e33b75cfbe75

As mentioned above , in the light of MD5 Encryption can reduce its security by brute force cracking , Therefore, in the process of practical operation , We will add the salt value （Salt） Or double MD5 Encryption and other methods to increase its reliability , The code is as follows ：

First, let's talk about DES encryption , The full name is Data Encryption Standard, Data encryption standard , It is a common kind of symmetric encryption , That is, the keys used in the encryption and decryption processes are the same , So if you want to crack it , Through violent enumeration , As long as the computing power is strong enough, it can still be cracked .

AES The full name is Advanced Encryption Standard, yes DES The replacement of algorithm , It is also one of the most popular symmetric encryption algorithms . Want to find out AES Algorithm , First, we have to understand three basic concepts ： secret key 、 Fill and pattern .

We have talked a lot about the key before , You can think of it as a key , It can be used for locking , It can be used to unlock .AES Supports three lengths of keys ：128 position 、192 Bit and 256 position .

As for the concept of filling ,AES We need to understand the characteristics of packet encryption , The details are shown in the following figure ：

Simply speaking ,AES When the algorithm encrypts plaintext , It is not to encrypt the whole plaintext into a whole ciphertext , Instead, the plaintext is divided into independent plaintext blocks , The length of each plaintext block is 128 The bit .

These plaintext blocks go through AES After the complex processing of the encryptor , Generate independent ciphertext blocks , Putting these ciphertext blocks together is the final AES The result of encryption .

So here's a question , If the length of a plaintext is 196 The bit , If you follow every 128 Bit is a plaintext block to split , The second plaintext block is just 64 Bit , Insufficient 128 What should bit do ？ At this time, it's time for filling to play its role , The default fill method is PKCS5Padding as well as ISO10126Padding.

But in the AES When encrypting, a certain filling method is used , The same filling method must be used when decrypting .

AES Working mode of , It is embodied in the process of encrypting plaintext blocks into ciphertext blocks , There are five different working modes , Namely CBC、ECB、CTR、CFB as well as OFB Pattern , similarly , If in AES A certain working mode is used in the encryption process , The same working mode must be adopted for decryption . Finally we use Python Let's do it AES encryption .

This is about the inventory Python This is the end of the article on common encryption algorithms in crawlers , More about Python Please search the previous articles of SDN or continue to browse the relevant articles below. I hope you will support SDN more in the future ！