We know , The general program is executed from top to bottom . If there are two tasks , The execution time of a task needs 5 second , The execution time of another task needs 4 second , Then, as usual, we need 9 Seconds to complete the above two tasks . Can you make these two tasks go on at the same time 5 In seconds ？ Certainly. , Here we introduce today's protagonist ： Threads ——threading modular .
The emergence of multithreading is to help us solve the problem of resource hogging , Let's take a look at its basic use .
import time, datetime
import threading
def func():
""" Here is the task that the thread needs to perform """
# Get the thread name and record the task start time , thread The instance object can be named directly .name Get the thread name
print(threading.current_thread().getName(), datetime.datetime.now())
# ------ Part of the task ------
# For example, output a sentence hello, Suppose it takes two seconds to complete the task
print('hello~')
time.sleep(2)
# -------------------
# Record the end time of the task
print(threading.current_thread().getName(), datetime.datetime.now())
def func2():
""" Here is the second task """
print(threading.current_thread().getName(), datetime.datetime.now())
print('hi~')
# Assume that the output requires 3 Second time
time.sleep(3)
print(threading.current_thread().getName(), datetime.datetime.now())
# Create two threads , And bind their respective tasks , use target Note that there are no parentheses in the name of the receiving function
thread1 = threading.Thread(target=func)
thread2 = threading.Thread(target=func2)
# After binding, you can have start Launched the
thread1.start()
thread2.start()
pycharm Next run result
C:\Users\17591\.virtualenvs\test\Scripts\python.exe C:/Users/17591/PycharmProjects/test/books.py
Thread-1 2022-06-20 18:00:54.397643
hello~
Thread-2 2022-06-20 18:00:54.397643
hi~
Thread-1 2022-06-20 18:00:56.403895
Thread-2 2022-06-20 18:00:57.405731
Process finished with exit code 0
You can see , One 2 One second 3 The second task only needs 3 In seconds , It shows that these two tasks are indeed carried out at the same time .
name
Each thread name defaults to thread-xx Named , If you want to define yourself , You can use... When creating an instance object name Specify .
thread = threading.Thread(target=func,name=" This is my first thread ")
Pass parameters
When we need to pass parameters when calling a function , When creating an instance object, use args or kwargs To specify .
def func(name, age):
print(name, age)
thread = threading.Thread(target=func, name=" This is my first thread ", kwargs={
"name": "lishuaige", 'age': 21})
thread.start() # Output lishuaige 21
It can also be written as
thread = threading.Thread(target=func, name=" This is my first thread ", args=('lishuaige',21))
Daemon Threads
Daemon Threads are also called daemon threads . What is a guardian thread ？ See what others say ： The guardian thread – Also known as “ Service threads ”, stay When there is no user thread to serve, it will leave automatically . priority ： The priority of the daemons is low , Used to serve other objects and threads in the system .
In general , A program will wait for all threads to execute , Will be closed . take pycharm Come on , After executing the procedure , Will return to the console Process finished with exit code 0 The words... , This indicates that the program has been executed . The difference is , If Daemon If threads exist , After the program is run, it will be regarded as Daemon Threads also have tasks , I won't wait for it , Just shut it down , return Process finished with exit code 0, then Daemon Their own tasks will also be closed .
For example , Several students have arranged to go out to play , Everyone is here , only daemon Still on the way , I waited for a long time , Everyone is impatient to wait , Still didn't come , So a friend called daemon, hum , We've been waiting so long and haven't come yet ！ Then angrily hung up the phone , Just set out .daemon Sighed sadly , Ah , I went to buy you snacks …
To put it simply , If there is a daemon thread , After all threads except the daemon thread are executed , Will terminate the program , The task of the daemon thread will also be shut down .
stay python Each thread can be set as a daemon thread in .
import time, datetime
import threading
def func():
time.sleep(2)
print(' This is the daemon thread ')
thread = threading.Thread(target=lambda :print(" This is a thread "))
dae = threading.Thread(target=func)
dae.setDaemon(True) # True To open the daemon thread , The default is False
dae.start()
thread.start()
pycharm The output is
C:\Users\17591\.virtualenvs\test\Scripts\python.exe C:/Users/17591/PycharmProjects/test/books.py
This is a thread
Process finished with exit code 0
adopt dae.setDaemon(True) Your command will dae Set to guard threads , Because the program will be terminated after the thread outside the daemon thread is executed , therefore “ This is the daemon thread ” This output statement did not execute successfully .
join()
In thread join() Method is used to ensure the smooth execution of the thread and block the main thread . What is the main thread ？ Review the code we wrote before , Even if no import threading The module can also run , In fact, this is because the main thread is working . The main thread , It is equivalent to the thread executing the total program .
join(timeout),timeout Don't write , In that case, the code of the main thread will be executed after the thread is executed . If it's written , In seconds , Indicates how many seconds are blocked , During this period, all existing threads except the main thread will be run （ use start command ） Thread started , When the blocking time has passed, continue to execute the code of the main thread .
import time, datetime
import threading
def func():
print(" start-up ", datetime.datetime.now())
time.sleep(2)
print(" end ", datetime.datetime.now())
thread = threading.Thread(target=func)
the = threading.Thread(target=func)
the.start()
the.join(1)
thread.start()
thread.join(0.5)
pycharm The result of the operation is
C:\Users\17591\.virtualenvs\test\Scripts\python.exe C:/Users/17591/PycharmProjects/test/books.py
start-up 2022-06-20 21:04:47.814156
start-up 2022-06-20 21:04:48.826980
end 2022-06-20 21:04:49.826345
end 2022-06-20 21:04:50.834772
Process finished with exit code 0
because thread The thread did not start , therefore the A thread initiated block has only its own thread working , After blocking thread The thread starts , And initiate 0.5 Seconds of blocking , Because both threads are started , So the blockage won't affect them , Only the main thread is affected . Complete two two second tasks in the last three seconds , Due to blocking , A thread executes one second late .
isAlive()
Used to determine whether the thread is working .
thread = threading.Thread(target=func)
thread.start()
thread.join()
threading.active_count()
Number of threads currently working , Including main thread . Be careful threading Is the module name of the thread .
thread = threading.Thread(target=func)
the = threading.Thread(target=func)
thread.start()
the.start()
print(threading.active_count()) # 3
threading.enumerate()
Iterate through all the threads currently working .
print(threading.enumerate())
threading.current_thread()
Get the current working thread .
Custom thread
If you want to customize threads , Then here can also satisfy you . Just inherit threading.Thread, Call it the __init__ Method , Last in run Function to define your task .
import time, datetime
import threading
class MyThread(threading.Thread):
def __init__(self, *args, **kwargs): # It is best to bring two universal parameters
super().__init__(*args, **kwargs)
def run(self):
print('hello')
time.sleep(2)
thread = MyThread()
thread.start()
# perhaps thread.run()
Actually , Thread objects can use both start Perform tasks , Can also be used run Perform tasks . The difference is ,run Have a resemblance to join Characteristics of , Need to wait run Perform the following operations after the task is completed . in addition , Overridden thread classes can be called multiple times run Method , And the original threading.thread class , Only once run.
In the original class , When the thread finishes executing, the object will be destroyed , Recycle resources . But if we do not destroy and reuse after rewriting , It will cause unnecessary waste of resources .
When it comes to locks , Let's have a chat first ： I wonder if you have paid attention to the front “pycharm Running results ” This word has appeared many times , So you know why I emphasize pycharm？ because , stay pycharm Maybe the output can be more neat , What if you switch to a native compiler ？
import time, datetime
import threading
def func():
""" Here is the task that the thread needs to perform """
# Get the thread name and record the task start time , thread The instance object can be named directly .name Get the thread name
print(threading.current_thread().getName(), datetime.datetime.now())
# ------ Part of the task ------
# For example, output a sentence hello, Suppose it takes two seconds to complete the task
print('hello~')
time.sleep(2)
# -------------------
# Record the end time of the task
print(threading.current_thread().getName(), datetime.datetime.now())
# Create two threads , And bind their respective tasks , use target Note that there are no parentheses in the name of the receiving function
thread1 = threading.Thread(target=func)
thread2 = threading.Thread(target=func)
# After binding, you can have start Launched the
thread1.start()
thread2.start()
Take the code above as an example , Let's see the result together ！
Don't you think it's incredible ？ even to the extent that python The classic logo of >>> Run out first , perform print(2) Commands can be output normally ; Sometimes their names are merged and even there is no newline character when they are output twice , And sometimes there is no merger , Each execution will display different results . actually , Each thread cannot predict who will get the resources first and then process the data , So there will be a rush to export , This phenomenon is called racing . To avoid this phenomenon , The concept of lock also comes with it , Its appearance is not really to solve simple output problems , Maybe it's because of thread related security problems . Let me give you another example ：
import time, datetime
import threading
MONEY = 100
def withdrawMoney(amount):
global MONEY
if MONEY >= amount:
# Suppose the server has a delay , Need to wait 10 Milliseconds to continue
time.sleep(0.01)
MONEY -= amount
print(f" Taken {
amount} element , The remaining {
MONEY} element ")
else:
print(" Lack of balance ！")
# Create two threads , And bind their respective tasks , use target Note that there are no parentheses in the name of the receiving function
thread1 = threading.Thread(target=withdrawMoney,args=[100])
thread2 = threading.Thread(target=withdrawMoney,args=[50])
thread1.start()
thread2.start()
Suppose you have 100 Yuan , Want to see if you can drill a hole . Log in on your phone and computer at the same time , And withdraw cash at the same time , At this time, if the lock is not used, the following situations will occur .
The same code , I ran it three times , There will be many different results .
Now let's analyze the reasons for these situations . In the example above , There are two threads , One task is to take 50 element , Another task is to take 100 element . After the current thread enters the judgment statement , Because the server is delayed , So I waited 10 millisecond , The embodied operation is not performed , At this time MONEY still 100 element , Almost at the same time, a thread scheduling switch occurs , Another thread also goes to the judgment statement , because MONEY still 100 element , therefore , He also went in , Did not take the branch with insufficient balance . The problem is coming. , When a short server delay has passed , Because both threads enter the withdrawal step , Therefore, the operation of subtraction will be carried out , Then there will be negative numbers .
What can be done to solve the above problems ？ Some students said , Try to solve the problem of server delay ！ But there is no way to avoid occasional server delays . Some students also said to see if we can put the delay problem outside the judgment statement ？ It seems that it can , Because latency is usually a network problem . And like the logic calculation above , It won't get stuck because of network problems . But even if the delay is outside , It is also possible that two threads enter the judgment statement at almost the same time . I will time.sleep() Written in the inner layer of the judgment statement is just for the convenience of demonstration , Make sure that two threads can enter the judgment statement 100% . therefore , There are problems with the above , It needs to be solved with a lock .
First you need to get lock object
lock = threading.Lock()
Gets the lock
lock.acquire()
Release the lock
lock.release()
Write a logically indivisible block of code between the two .
Can also be used with Method .
import time, datetime
import threading
# Definite lock
lock = threading.Lock()
MONEY = 100
def withdrawMoney(amount):
global MONEY
lock.acquire()
if MONEY >= amount:
# Suppose the server has a delay , Need to wait 10 Milliseconds to continue
time.sleep(0.01)
MONEY -= amount
print(f" Taken {
amount} element , The remaining {
MONEY} element ")
else:
print(" Lack of balance ！")
lock.release()
# Create two threads , And bind their respective tasks , use target Note that there are no parentheses in the name of the receiving function
thread1 = threading.Thread(target=withdrawMoney,args=[100])
thread2 = threading.Thread(target=withdrawMoney,args=[50])
thread1.start()
thread2.start()
Creating and terminating threads will cause some overhead in time and performance , If you can reduce the operations of creating and terminating threads , It can improve the efficiency of code execution to a certain extent , And the thread pool , It is a set of optimization schemes , It contains two concepts , Task queues and thread pools . When a new task comes up , The task will be placed in the task queue , The thread pool already contains multiple pre established threads , These threads process the tasks in the queue , And pop it out of the task queue .
We will explain it in combination with the code ：
from concurrent.futures import ThreadPoolExecutor
def add(num):
num += 100
return num
lst = list(range(20))
with ThreadPoolExecutor() as pool:
res = (pool.map(add,lst))
for i in res:
print(i)
First, import. ThreadPoolExecutor Thread pool . Define an addition function , Use map Method , Let the elements in the list add 100, Final print results .ThreadPoolExecutor Under the module of map Method and common map The usage of methods is basically the same , Is to make a function act on each element of the iteratable object separately .( If you want to continue to understand map See my article for usage https://blog.csdn.net/lishuaigell/article/details/124168814)
Observations show that , The processed elements are output in sequence . Is it accidental ？ No ,map The result of method processing is output in sequence . What does that mean? ？ This means that there are some threads that finish the following tasks first , Because of the order , Results cannot be submitted , Wait until the previous task is completed , Submit results before continuing , So it's blocked ！
There are new methods to solve the above problems , submit – as_completed.as_completed Need to match submit Use it together .
from concurrent.futures import ThreadPoolExecutor,as_completed
def add(num):
num += 100
return num
lst = list(range(20))
with ThreadPoolExecutor() as pool:
futures = (pool.submit(add,l) for l in lst)
for future in as_completed(futures):
print(future.result())
The method of use is similar to the previous one , The difference is ,submit You can only have a function act on one element at a time , and map You can make the function act on every element every time , in addition , If you want to get results , Use result Method .
Be careful , The essence of thread pool is still thread , Multithreading is not suitable for dealing with CPU Intensive Computing , Only suitable for handling IO Intensive Computing . Like the addition function above , Because the order of magnitude is too small to see the effect , If the formula is a little more complicated , If the number is larger, the processing time will be much slower than that of a single thread , Because it belongs to cpu Intensive Computing . because python Yes GIL（ Global interpreter lock , It is said that python3 Every thread 15 Milliseconds will be checked and released GIL） The existence of , No matter how many you have cpu, There's only one at a time cpu, A thread is working . If the amount of calculation is large , If multiple threads are frequently scheduled , Will only improve cpu Load and waiting time , Cause reaction . It's like switching on and off lights frequently at home , If you turn the lights on and off , No more than thirty round trips , I'm afraid that lamp won't hold .
To make full use of cpu,python Relevant countermeasures have also been issued , Multi process —— multiprocessing modular .
In the next article 《python Multi process 》 in , I will explain in detail python multiprocessing Basic usage of module , Welcome to your attention .
Pythongui based drawing tool software design course paper + operation manual + demonstration video + source code and data files
Resource download address ：htt
Python crawler - climb Shanghai for 15 days with high and low temperatures
List of articles Preface One
