In the last article , We introduced in detail python Association in . Speak through python coroutines
python adopt yeild Keyword surrender CPU Implementation , Implement concurrent work similar to system interrupts , This is called “ Micro thread ” Of python Cooperative scheduling mechanism . however , This is not really concurrency , In almost all programming languages , Both provide a multi-threaded concurrency mechanism ,python It also provides a multi-threaded concurrency mechanism , In this article, we will introduce in detail python Threading mechanism in .
before , We have introduced linux Threads in the environment and related api.
One execution of a program is a process , The process may contain multiple threads , Thread is CPU Minimum unit of scheduling , Multiple threads in the same process share the program text in the process 、 Global memory and heap memory 、 Stack, file descriptor and other resources , Multiple processes on the same computer share file systems 、 disk 、 Printer and other hardware resources . Thread switching consumes less resources than process switching , So it's more efficient , Especially switching between several threads in the same process , This is because process switching requires the execution of system traps 、 Context switching and memory and cache flushing , Because multiple threads in the same process share these resources , During thread switching , The system does not need to perform any operation on these resources , Therefore, higher efficiency can be obtained . so , Thread scheduling is a very important and practical technology in programming .
python There are two modules in the standard library that maintain threads — thread and threading. because thread. The module is not satisfactory in many aspects , For example, in a multithreaded concurrent environment , When the main thread exits , All child threads exit immediately , Not even any cleanup , It is usually unacceptable , Therefore, it is generally not recommended to use . stay python3 in thread The module has been renamed _thread modular , So as to indicate by name that it is not recommended to use . If you are familiar with java Thread model of , You'll find that python The thread model and java It's very similar to , you 're right ,python The thread model of is a reference java Thread model design , but python The thread of has no priority yet , No thread groups , Threads cannot be destroyed yet 、 stop it 、 Pause 、 To resume or interrupt .
threading The module contains the following objects ：
object
describe
Thread
Execute thread object
Timer
An execution thread object that waits for a certain amount of time before running
Lock
Lock object
Condition
Conditional variable object , Used to describe condition variables in thread synchronization
Event
Event object , Used to describe events in thread synchronization
Semaphore
Semaphore object , Used to describe counters in thread synchronization
BoundedSemaphore
There is a threshold semaphore object
Barrier
Fence object , Thread synchronization allows multiple threads to execute to a specified location
threading The most important class in the module is Thread class . Every Thread An object is a thread . Here is Thread The properties and methods contained in the class .
attribute
remarks
name
Thread name
ident
Thread identifier
deamon
bool type , Indicates whether the thread is a daemon thread
start()
Start thread execution
run()
Used to define threading functions , It is usually duplicated by the developer in the subclass
join(timeout=None)
Hang until the started thread terminates or until the timeout period
is_alive()
return bool type , Indicates whether the thread is alive
There are two ways to create threads , But the second one is more recommended ：
from threading import Thread
from time import sleep, ctime
def sleep_func(i):
print('start_sleep[%s]' % i)
sleep(i+1)
print('end_sleep[%s]' % i)
if __name__ == '__main__':
print('start at %s'% ctime())
threads = list()
for i in range(3):
t = Thread(target=sleep_func, args=[i])
threads.append(t)
for i in range(3):
threads[i].start()
for i in range(3):
threads[i].join()
print('end at %s' % ctime())Printed out ：
start at Mon May 6 12:25:18 2019 start_sleep[0] start_sleep[1] start_sleep[2] end_sleep[0] end_sleep[1] end_sleep[2] end at Mon May 6 12:25:21 2019 And we observe that every time 1 Seconds will print out a end_sleep, This shows that they do execute in parallel . Should have been executed in total 1+2+3 = 6 Of a second , Due to the concurrent execution of threads , In fact, only 3 second .
from threading import Thread
from time import sleep, ctime
class myThread(Thread):
def __init__(self, nsec):
super().__init__()
self.nsec = nsec
def run(self):
print('start_sleep[%s]' % self.nsec)
sleep(self.nsec + 1)
print('end_sleep[%s]' % self.nsec)
if __name__ == '__main__':
print('start at %s'% ctime())
threads = list()
for i in range(5):
t = myThread(i)
threads.append(t)
for thread in threads:
thread.start()
for thread in threads:
thread.join()
print('end at %s' % ctime())The operation is completely consistent with the above example implemented by function . Because private members can be added to the class to save the running results of member methods or other data , This method is more flexible .
Only in Thread Object's start Method is called before starting thread activity . start Method can be called at most once in a thread , Otherwise it will throw RuntimeError. start The final logical code to execute is Thread Class run Method .
join Method has an optional timeout Parameters , This method will block the thread that calls this method , Until it's called join() Thread ends or reaches timeout Number of seconds . Of course , A thread can be join Many times , but join The current thread will cause a deadlock . If you are join Thread of is not in alive state , Will cause RuntimeError abnormal .
In the thread , Can pass sys.exit() Method or throw SystemExit An exception causes the thread to exit . however , Out of line , You cannot terminate a thread directly .
Except for the most important Thread class ,threading The following useful functions are also provided in the module .
function
explain
active_count()
Return to the currently active Thread Number of objects
current_thread()
Returns the value of the current thread Thread object
enumerate()
Return to the currently active Thread The object list
settrace(func)
Set one... For all threads trace function
setprofile(func)
Set one... For all threads profile function
local()
Create or get thread local data objects
stack_size(size=0)
Returns the stack size of the newly created thread or sets the stack size for subsequent threads by size
get_ident()
Returns the value of the current thread “ Thread identifier ”, It is a non-zero integer
main_thread()
Return to the Lord Thread object . In general , The main thread is Python Thread created at the beginning of the interpreter
We have seen from a practical example above , The three threads are sleep 1、2、3 second , The execution results only cost the phone 3 second , Enough to see the performance advantages in the concurrent environment . However , as everyone knows ,python The interpreter has multiple versions of implementation , among CPython It is widely used for its excellent execution efficiency , Also become python The default interpreter for , Another widely used is PyPy Interpreter , Both interpreters have a congenital defect , That is not thread safe , This is out of high performance And complexity . because python Interpreter CPython、PyPy Implementation limitations of , As a result, the global interpretation security lock will be set in the actual implementation （GIL）, Only one thread is allowed to execute at a time Python Bytecode , therefore , One Python Processes usually cannot use more than one at the same time CPU The core , Multithreaded programs are not always executed concurrently , But it's not python The limitation of language itself ,Jython And IronPython There is no such limitation . even so , Blocking in all standard libraries IO operation , Is released while waiting for the operating system to return results GIL, So for IO Intensive program , Using multithreading concurrency can effectively improve performance , For example, we can make multiple threads wait or receive at the same time IO The return data of the operation or when a thread performs the download task , Another thread is responsible for displaying the download progress . time.sleep The same with the operation ,time.sleep The operation will immediately release GIL lock , And let the thread block the number of seconds waiting for the parameter to be passed in , It is not until then that the request for GIL lock , This is why the multithreading concurrency in the above example reduces the execution time . But for the CPU Intensive program ,python Threads are a bit weak , However, it is not impossible to optimize , We will introduce in detail later .
The calculation of Fibonacci sequence is a typical CPU Intensive operation , The following example shows how to calculate in serial environment and concurrent environment respectively 10 The second Fibonacci sequence 100000 Time consuming of elements ：
import time
from threading import Thread
class fibThread(Thread):
def __init__(self, stop):
super().__init__()
self.stop = stop
self.result = None
def run(self):
self.result = fib(self.stop)
def fib(stop):
a = 0
b = 1
for _ in range(stop):
a, b = a + b, a
return a
if __name__ == '__main__':
stime = time.time()
for _ in range(10):
fib(100000)
print('serial time %ss' % (time.time() - stime))
stime = time.time()
threads = list()
for _ in range(10):
t = fibThread(100000)
threads.append(t)
for thread in threads:
thread.start()
for thread in threads:
thread.join()
print('concurrent time %ss' % (time.time() - stime))Printed out ：
serial time 0.9864494800567627s concurrent time 0.9564151763916016s
Although from the result , Multithreading concurrency does have a slight performance improvement , But it is far from the optimization we expected 90% How to further optimize this problem , Stay tuned for the next article .
《python Programming core 》. https://docs.python.org/zh-cn/3.6/library/threading.html.
[Python] one line of code to calculate the distance and included angle between two longitude and latitude points
Method 1 ： Call directly
This generation of workers is too difficult. How hard is it for people who dont understand Python?
As McKinsey said , Data has pe
