Why would anyone say Python Multithreading is chicken ribs ？ It seems that someone has raised such a question , In our common sense , Multi process 、 Multithreading makes full use of hardware resources through concurrency to improve the running efficiency of programs , How the Python Instead, it becomes chicken ribs ？
Some students may know the answer , because Python The infamous GIL,GIL What is it? ？ Why is there GIL？ Is multithreading really chicken ribs ？ GIL Can you remove it ？ With these questions , Let's look down together , At the same time, you need a little patience .
Multithreading is not chicken ribs , Let's do an experiment first , The experiment is very simple , It's about putting numbers "1 Billion " Decline , Reduced to 0 The program terminates , If we use a one-way process to perform this task , How long will it take to complete ？ How much will it be to use multithreading ？show me the code
# Mission def decrement(n): while n > 0: n -= 1 Copy code
Single thread
import time start = time.time() decrement(100000000) cost = time.time() - start >>> 6.541690826416016 Copy code
In my 4 nucleus CPU In the computer , The time spent on a single thread is 6.5 second . Someone might ask , Where is the thread ？ In fact, when any program runs , By default, there will be a main thread executing .（ Threads and processes are not expanded here , I'll write a separate article ）
Multithreading
import threading start = time.time() t1 = threading.Thread(target=decrement, args=[50000000]) t2 = threading.Thread(target=decrement, args=[50000000]) t1.start() # Start thread , Perform tasks t2.start() # ditto t1.join() # Main thread blocking , until t1 Execution completed , The main thread continues to execute later t2.join() # ditto cost = time.time() - start >>>6.85541033744812 Copy code
Create two sub threads t1、t2, Each thread executes 5 Tens of thousands of subtraction operations , After both threads are executed , The main thread terminates the program . result , The two threads execute in a cooperative manner is 6.8 second , It slows down . In theory , Two threads run in parallel on two threads at the same time CPU above , Time should be halved , Now, instead of decreasing, it will increase .
What causes multithreading to be slow instead of fast ？
And the reason is that GIL , stay Cpython Interpreter （Python The mainstream interpreter of language ） in , There is a global interpretation lock （Global Interpreter Lock）, Explain and execute in the interpreter Python Code , First get this lock , signify , There can only be one thread executing code at any time , Other threads want to get CPU Execute code instructions , You have to get this lock first , If the lock is occupied by another thread , Then the thread can only wait , It is not possible to execute code instructions until the thread occupying the lock releases the lock .
therefore , This is why two threads execute more slowly together , Because at the same time , Only one thread is running , Other threads can only wait , Even if it's multi-core CPU, There's no way to make multiple threads 「 parallel 」 Execute code simultaneously , Only alternate execution , Because multithreading involves online text switching 、 Lock mechanism processing （ Get the lock , Release lock, etc ）, therefore , Multithreading is not fast but slow .
When GIL Released ？
When a thread encounters I/O When the task , Will release GIL. Computationally intensive （CPU-bound） Threads execute 100 Step counting of the second interpreter （ticks） when （ Step counting can be roughly regarded as Python Virtual machine instructions ）, Will also release GIL. Can pass sys.setcheckinterval() Set the step length ,sys.getcheckinterval() Check the step length . Compared with single thread , These are the extra overhead of multithreading
CPython Why should the interpreter be designed like this ？
Multithreading is to adapt to the rapid development of modern computer hardware and make full use of multi-core processors , Make... Through multithreading CPU Resources can be used efficiently ,Python Born in 1991 year , At that time, the hardware configuration was far less luxurious than today , Now an ordinary server 32 nucleus 64G Memory is not a common thing , But multithreading has a problem , How to solve the synchronization of shared data 、 Consistency issues , because , For multiple threads accessing shared data , There may be two threads modifying one data at the same time , If there is no suitable mechanism to ensure the consistency of data , Then the program eventually leads to an exception , therefore ,Python The father of made a global thread lock , Whether you have data synchronization problems or not , One size fits all anyway , Last global lock , Ensure data security . That's why multithreading is a chicken rib , Because it has no fine-grained control over the security of data , But in a simple and rough way .
The solution lies in 90 years , In fact, there is no problem , After all , At that time, the hardware configuration was still very simple , Single core CPU Still mainstream , There are not many multithreaded application scenarios , Most of the time, it still runs as a single thread , Single thread does not involve context switching of threads , It's more efficient than multithreading （ In a multi-core environment , This rule does not apply ）. therefore , use GIL To ensure the consistency and security of data , Not necessarily undesirable , At least at that time, it was a very low-cost implementation .
Then put the GIL Is it feasible to remove ？
It's true that people do so much , But the results were disappointing , stay 1999 year Greg Stein and Mark Hammond The two brothers created one GIL Of Python Branch , Put... On all variable data structures GIL Replace with finer grained locks . However , After benchmarking , Get rid of GIL Of Python Under the condition of single thread, the execution efficiency is nearly slow 2 times .
Python The father of said ： Based on the above considerations , Get rid of GIL It doesn't have much value and doesn't have to spend too much energy .
CPython The interpreter provides GIL（ Global interpreter lock ） Ensure thread data synchronization , So there is GIL, Do we still need thread synchronization ？ Multithreading in IO In intensive tasks , What about the performance ？ Comments are welcome
