In the last article, we introduced Python Threads and usage in . python The thread of
Once concurrency is introduced , There may be competitive conditions , Sometimes unexpected things happen .
Above picture , Threads A Read the variable and give it a new value , Then write to memory , however , meanwhile ,B Read the same variable from memory , Maybe A The changed variable has not been written to memory , Lead to B Read the original value , May also be A Has been written to cause B The new value is read , Thus, there is uncertainty in the operation of the program . In this article, we will discuss how to solve the above problems .
When I introduced the decorator earlier , We have seen an implementation of the singleton pattern . python Magic methods （ Two ） Object creation and singleton pattern implementation
class SingleTon:
_instance = {}
def __new__(cls, *args, **kwargs):
if cls not in cls._instance:
cls._instance[cls] = super(SingleTon, cls).__new__(cls, *args, **kwargs)
return cls._instance[cls]
class TechTest(SingleTon):
testfield = 12Let's transform the code of the above singleton mode into multi-threaded mode , And join in time.sleep(1) To simulate the creation of some IO Operation scenario .
import time
from threading import Thread
class SingleTon:
_instance = {}
def __new__(cls, *args, **kwargs):
if cls not in cls._instance:
time.sleep(1)
cls._instance[cls] = super(SingleTon, cls).__new__(cls, *args, **kwargs)
return cls._instance[cls]
class TechTest(SingleTon):
testfield = 12
def createTechTest():
print(TechTest())
if __name__ == '__main__':
threads = list()
for i in range(5):
t = Thread(target=createTechTest)
threads.append(t)
for thread in threads:
thread.start()Printed out ：
<__main__.TechTest object at 0x000001F5D7E8EEF0> <__main__.TechTest object at 0x000001F5D60830B8> <__main__.TechTest object at 0x000001F5D60830F0> <__main__.TechTest object at 0x000001F5D6066048> <__main__.TechTest object at 0x000001F5D6083240>
From the running results , Our singleton pattern class creates more than one object , This is no longer a single case . Why is that ？ In our singleton class __new__ In the method , First, check whether there are objects in the dictionary , Create... If it doesn't exist , When multiple threads execute to judge at the same time , None of them execute the created statement , The result is that multiple threads judge the object that needs to create a singleton , So multiple objects are created , This constitutes a competitive condition .
The simplest way to solve the above problem is to lock .
Above picture , Threads A Will read variables 、 Write variables 、 A series of operations to write to memory are locked , And threads B Must be in the thread A Wait until all operations are completed to release the lock , Until you get the lock , Read the value after a series of operations .
threading.Lock It uses _thread Locking mechanism implemented by the module , In essence , What he actually returns is the lock provided by the operating system . The lock object does not belong to any specific thread after it is created , He has only two states — Locked and unlocked , At the same time, he has two methods to switch between these two states .
acquire(blocking=True, timeout=-1)
This method attempts to acquire the lock , If the lock state is unlocked , Return immediately , otherwise , according to blocking Parameter determines whether to block waiting . once blocking Parameter is True, And the lock is locked , Then the method will always block , Until you reach timeout Number of seconds ,timeout by -1 Indicates unlimited timeout . If successful, return True, If the lock is not obtained successfully due to timeout or non blocking lock acquisition failure , Then return to False.
release()
This method is used to release the lock , Whether the current thread holds a lock or not , He can call this method to release the lock . But if a lock is not locked , Then the method will throw RuntimeError abnormal .
With a locking mechanism , Our singleton pattern class can be transformed into the following ：
from threading import Thread, Lock
class SingleTon:
_instance_lock = Lock()
_instance = {}
def __new__(cls, *args, **kwargs):
cls._instance_lock.acquire()
try:
if cls not in cls._instance:
time.sleep(1)
cls._instance[cls] = super(SingleTon, cls).__new__(cls, *args, **kwargs)
return cls._instance[cls]
finally:
cls._instance_lock.release()In this way, the problems mentioned above will never occur again . however , Such an implementation has performance problems because the granularity of locking is too large , This is beyond the scope of this article , A separate article will be drawn to introduce the optimization of singleton mode .
Must be executed every time acquire and release Both methods seem very cumbersome , It is also very error prone , Because once due to negligence , Threads don't have release Quit , Then other threads will never be able to acquire the lock and cause serious problems . Fortunately python There is a very easy-to-use feature — Context management protocol ,threading.Lock It supports context management protocol , The above code can be modified to ：
from threading import Thread, Lock
class SingleTon:
_instance_lock = Lock()
_instance = {}
def __new__(cls, *args, **kwargs):
with cls._instance_lock:
if cls not in cls._instance:
time.sleep(1)
cls._instance[cls] = super(SingleTon, cls).__new__(cls, *args, **kwargs)
return cls._instance[cls]about threading.Lock, A deadlock occurs when the same thread acquires a lock twice , Because the previous lock is occupied by itself , And I waited for the release of the lock , Caught in a dead cycle . This deadlock situation seems easy to avoid , But in fact , In object-oriented programs , This can easily happen .
from threading import Lock
class TechlogTest:
def __init__(self):
self.lock = Lock()
def lockAndPrint(self):
with self.lock:
print('[%s] locked' % 'TechlogTest')
class TechlogTestSon(TechlogTest):
def lockAndPrint(self):
with self.lock:
print('[%s] locked' % 'TechlogTestSon')
super(TechlogTestSon, self).lockAndPrint()
if __name__ == '__main__':
son = TechlogTestSon()
son.lockAndPrint()In the example above , A subclass attempts to call a method with the same name as the parent class , Print out “[TechlogTestSon] locked” After that, I kept blocking and waiting , But in fact , The parent class locks the method just like the child class , And according to polymorphism , The lock objects obtained by the parent class and the child class are actually created by the child class , So the deadlock happened . To avoid such a situation , You need to use a reentrant lock .
And threading.Lock equally ,RLock Two methods are also provided for locking and unlocking , The locking method is also a factory method , Returns an instance of a reentrant lock in the operating system . We have studied Java Can be re-entry lock in ReentrantLock Source code . ReentrantLock Usage details
ReentrantLock Source code analysis -- ReentrantLock Lock and unlock
actually , The implementation of the reentrant lock in the operating system is the same as that in Java The implementation of reentrant locks is very similar , Usually, the current locking thread ID and a number are maintained in the lock object to indicate the number of locking times , Each time the same thread invokes the locking method, the number of times to lock ＋ 1, Unlock － 1, Only become 0 To release the lock .
acquire(blocking=True, timeout=-1) release()
You can see , The parameters of these two methods are the same as threading.Lock The methods of the same name in are identical , The usage is exactly the same , I won't go into that here .
threading.RLock It also fully implements the context management protocol , The deadlock example above , We can solve the deadlock problem with a little modification .
from threading import RLock
class TechlogTest:
def __init__(self):
self.lock = RLock()
def lockAndPrint(self):
with self.lock:
print('[%s] locked' % 'TechlogTest')
class TechlogTestSon(TechlogTest):
def lockAndPrint(self):
with self.lock:
print('[%s] locked' % 'TechlogTestSon')
super(TechlogTestSon, self).lockAndPrint()
if __name__ == '__main__':
son = TechlogTestSon()
son.lockAndPrint()Printed out ：
[TechlogTestSon] locked [TechlogTest] locked
In a multithreaded environment , While the performance is improved, there will be many thorny new problems , The above question is just the tip of the iceberg , Locking can only solve some of the most basic scenarios , There are more complex scenarios that need more appropriate tools to handle . Please look forward to the next blog , Let's introduce in detail python Other tools for thread synchronization .
https://docs.python.org/zh-cn/3.6/library/threading.html.
