1. introduction

In the last article , We introduced python Iterators and generators in . python Iterators and generators in

In the previous article , We have seen the example of the context manager . python Magic methods （ Four ） A large collection of overloaded methods of unusual methods and operators

In this paper, we analyze contextlib.contextmanager Decorator source code , Let's see how to combine them to produce more elegant code .

2. Context manager

class Test:
def __enter__(self):
print('now in __enter__')
return 'Hello World'
def __exit__(self, exc_type, exc_val, exc_tb):
print('now exit')
return True
if __name__ == '__main__':
test = Test()
with test as teststr:
print(teststr)
print('end of main')

Call to print out ：

now in __enter__ Hello World now exit end of main

When with When a block is executed , The interpreter will automatically call the... Of the object __enter__ Method . And in the with At the end of the block , The interpreter will automatically call the... Of the object __exit__ Method ,__exit__ Method can finally choose to return True Or throw an exception .

3. contextlib.contextmanager Decorator

Standard library ,contextlib.contextmanager The decorator passes through yield Keywords can reduce the amount of boilerplate code needed to create a context manager . The above example can be transformed into ：

import contextlib
class Test:
@contextlib.contextmanager
def contextmanager(self):
print('now in __enter__')
yield 'Hello World'
print('now exit')
return True
if __name__ == '__main__':
test = Test()
with test.contextmanager() as teststr:
print(teststr)
print('end of main')

Also printed out ：

now in __enter__ Hello World now exit end of main

4. principle

Essentially contextlib.contextmanager It still makes use of yield The features of the generator , He wrapped the function and added __enter__ And __exit__ Two methods .

def contextmanager(func):
@wraps(func)
def helper(*args, **kwds):
return _GeneratorContextManager(func, args, kwds)
return helper
class _GeneratorContextManager():
def __init__(self, func, args, kwds):
self.gen = func(*args, **kwds)
self.func, self.args, self.kwds = func, args, kwds
# Issue 19330: ensure context manager instances have good docstrings
doc = getattr(func, "__doc__", None)
if doc is None:
doc = type(self).__doc__
self.__doc__ = doc
def __enter__(self):
try:
return next(self.gen)
except StopIteration:
raise RuntimeError("generator didn't yield") from None
def __exit__(self, type, value, traceback):
if type is None:
try:
next(self.gen)
except StopIteration:
return False
else:
raise RuntimeError("generator didn't stop")
else:
if value is None:
# Need to force instantiation so we can reliably
# tell if we get the same exception back
value = type()
try:
self.gen.throw(type, value, traceback)
except StopIteration as exc:
# Suppress StopIteration *unless* it's the same exception that
# was passed to throw(). This prevents a StopIteration
# raised inside the "with" statement from being suppressed.
return exc is not value
except RuntimeError as exc:
# Don't re-raise the passed in exception. (issue27122)
if exc is value:
return False
# Likewise, avoid suppressing if a StopIteration exception
# was passed to throw() and later wrapped into a RuntimeError
# (see PEP 479).
if type is StopIteration and exc.__cause__ is value:
return False
raise
except:
# only re-raise if it's *not* the exception that was
# passed to throw(), because __exit__() must not raise
# an exception unless __exit__() itself failed. But throw()
# has to raise the exception to signal propagation, so this
# fixes the impedance mismatch between the throw() protocol
# and the __exit__() protocol.
#
if sys.exc_info()[1] is value:
return False
raise
raise RuntimeError("generator didn't stop after throw()")

You can see ,__enter__ Method implementation is relatively simple , Just by next Method gets the first generated data of the generator . __exit__ The method is relatively complex ：

1. Check whether the exception is passed to exc_type; If there is , call gen.throw(exception), Include... In the generator function definition body yield The line of the keyword throws an exception
2. adopt next Method call generator , Perform the next task
3. If the generator is not terminated , Throw out RuntimeError("generator didn’t stop")

5. Something to be aware of

From the above code, we can see a serious problem ：__enter__ The code does not catch exceptions , Once we're in with An exception is thrown in a block , Will lead to __exit__ The cleanup code in cannot be executed .

import contextlib
class Test:
@contextlib.contextmanager
def contextmanager(self):
print('now in __enter__')
yield self.raiseexc(1)
print('now exit')
return True
def raiseexc(self, param):
if param < 5:
raise Exception('test exception')
if __name__ == '__main__':
test = Test()
with test.contextmanager() as teststr:
print(teststr)
print('end of main')

perform , Printed out ：

now in __enter__ Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydevd.py", line 1741, in <module> main() File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydevd.py", line 1735, in main globals = debugger.run(setup[‘file’], None, None, is_module) File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydevd.py", line 1135, in run pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydev_imps\_pydev_execfile.py", line 18, in execfileexec(compile(contents+"\n", file, ’exec’), glob, loc)File "D:/Workspace/code/python/testpython/fluentpython/contextmanager.py", line 19, in <module>with test.contextmanager() as teststr:File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 81, in \_enter__ return next(self.gen) File "D:/Workspace/code/python/testpython/fluentpython/contextmanager.py", line 8, in contextmanager yield self.raiseexc(1) File "D:/Workspace/code/python/testpython/fluentpython/contextmanager.py", line 14, in raiseexc raise Exception(‘test exception’) Exception: test exception

therefore , In the use of @contextlib.contextmanager Be sure to pay attention to , Can't be in yield An exception is thrown during execution .

import contextlib
class Test:
@contextlib.contextmanager
def contextmanager(self):
print('now in __enter__')
try:
yield self.raiseexc(1)
except Exception:
print('exception happened')
print('now exit')
return True
def raiseexc(self, param):
if param < 5:
raise Exception('test exception')
if __name__ == '__main__':
test = Test()
with test.contextmanager() as teststr:
print(teststr)
print('end of main')

Printed out ：

now in __enter__ exception happened now exit Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydevd.py", line 1741, in <module> main() File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydevd.py", line 1735, in main globals = debugger.run(setup[‘file’], None, None, is_module) File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydevd.py", line 1135, in run pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Program Files\JetBrains\PyCharm 2018.3.1\helpers\pydev\pydev_imps\_pydev_execfile.py", line 18, in execfileexec(compile(contents+"\n", file, ’exec’), glob, loc)File "D:/Workspace/code/python/testpython/fluentpython/contextmanager.py", line 22, in <module>with test.contextmanager() as teststr:File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 83, in \_enter__ raise RuntimeError("generator didn’t yield") from None RuntimeError: generator didn’t yield

Although the exception is still thrown , But we see __exit__ The cleanup code in the method can still be executed .