functools Functions acting on functions
functools Modules provide tools for adjusting or extending functions and other callable objects , Without having to rewrite them completely .
partial Class is functools The main tools provided by the module , It can be used to “ packing ” The default parameter of a callable object . The object it produces is itself callable , It can be seen as a native function . All its parameters are the same as the original , And you can use additional positional or named parameters to call . Use partial Instead of lambda To provide default parameters for the function , And keep those unspecified parameters .
The following list is right myfunc Two ways partial object ,show_details() For output partial Object's func 、 args and keywords attribute ：
import functools
def myfunc(a, b=2):
"""Docstring for myfunc()."""
print(' Pass in the parameter :', (a, b))
def show_details(name, f, is_partial=False):
"""Show details of a callable object."""
print('{}:'.format(name))
print(' object:', f)
if not is_partial:
print(' __name__:', f.__name__)
if is_partial:
print(' func:', f.func)
print(' args:', f.args)
print(' keywords:', f.keywords)
return
show_details('myfunc', myfunc)
myfunc('a', 3)
print()
# # to 'b' Reset a different default parameter
# # When calling, you still need to provide parameters 'a'
p1 = functools.partial(myfunc, b=4)
show_details('partial Modify keyword parameters ', p1, True)
p1(' Pass in a')
p1(' rewrite b', b=5)
print()
#
# # to 'a' and 'b' Set default parameters .
p2 = functools.partial(myfunc, ' Default a', b=99)
show_details('partial Set default parameters ', p2, True)
p2()
p2(b=' rewrite b')
print()
print(' When the parameter is missing :')
p1()
The first... Is called at the end of the example partial Object without passing a Value , Cause exception .
myfunc:
object: <function myfunc at 0x00000180005077B8>
__name__: myfunc
Pass in the parameter : ('a', 3)
partial Modify keyword parameters :
object: functools.partial(<function myfunc at 0x00000180005077B8>, b=4)
func: <function myfunc at 0x00000180005077B8>
args: ()
keywords: {
'b': 4}
Pass in the parameter : (' Pass in a', 4)
Pass in the parameter : (' rewrite b', 5)
partial Set default parameters :
object: functools.partial(<function myfunc at 0x00000180005077B8>, ' Default a', b=99)
func: <function myfunc at 0x00000180005077B8>
args: (' Default a',)
keywords: {
'b': 99}
Pass in the parameter : (' Default a', 99)
Pass in the parameter : (' Default a', ' rewrite b')
When the parameter is missing :
Traceback (most recent call last):
File "functools_partial.py", line 51, in <module>
p1()
TypeError: myfunc() missing 1 required positional argument: 'a'
By default , partial Objects have no __name__ and __doc__ attribute . This is not conducive to the debugging of decorated functions . have access to update_wrapper() Copy or add attributes from the original function to partial object .
import functools
def myfunc(a, b=2):
"""Docstring for myfunc()."""
print(' Pass in the parameter :', (a, b))
def show_details(name, f):
"""Show details of a callable object."""
print('{}:'.format(name))
print(' object:', f)
print(' __name__:', end=' ')
try:
print(f.__name__)
except AttributeError:
print('(no __name__)')
print(' __doc__', repr(f.__doc__))
print()
show_details('myfunc', myfunc)
p1 = functools.partial(myfunc, b=4)
show_details('raw wrapper', p1)
print('Updating wrapper:')
print(' assign:', functools.WRAPPER_ASSIGNMENTS)
print(' update:', functools.WRAPPER_UPDATES)
print()
functools.update_wrapper(p1, myfunc)
show_details('updated wrapper', p1)
The attributes added to the decorator are in WRAPPER_ASSIGNMENTS In the definition of , and WRAPPER_UPDATES List the values to be modified .
myfunc:
object: <function myfunc at 0x000002315C123E18>
__name__: myfunc
__doc__ 'Docstring for myfunc().'
raw wrapper:
object: functools.partial(<function myfunc at 0x000002315C123E18>, b=4)
__name__: (no __name__)
__doc__ 'partial(func, *args, **keywords) - new function with partial application\n of the given arguments and keywords.\n'
Updating wrapper:
assign: ('__module__', '__name__', '__qualname__', '__doc__', '__annotations__')
update: ('__dict__',)
updated wrapper:
object: functools.partial(<function myfunc at 0x000002315C123E18>, b=4)
__name__: myfunc
__doc__ 'Docstring for myfunc().'
partial Applicable to all callable objects , Not only for independent functions .
import functools
class MyClass:
"""Demonstration class for functools"""
def __call__(self, e, f=6):
"Docstring for MyClass.__call__"
print(' called object with:', (self, e, f))
def show_details(name, f):
""""Show details of a callable object."""
print('{}:'.format(name))
print(' object:', f)
print(' __name__:', end=' ')
try:
print(f.__name__)
except AttributeError:
print('(no __name__)')
print(' __doc__', repr(f.__doc__))
return
o = MyClass()
show_details('instance', o)
o('e goes here')
print()
p = functools.partial(o, e='default for e', f=8)
functools.update_wrapper(p, o)
show_details('instance wrapper', p)
p()
The above example uses MyClass Of an instance of a class __call__() Method created partial object . Work as usual :
instance:
object: <__main__.MyClass object at 0x000002DE7C2CD2E8>
__name__: (no __name__)
__doc__ 'Demonstration class for functools'
called object with: (<__main__.MyClass object at 0x000002DE7C2CD2E8>, 'e goes here', 6)
instance wrapper:
object: functools.partial(<__main__.MyClass object at 0x000002DE7C2CD2E8>, e='default for e', f=8)
__name__: (no __name__)
__doc__ 'Demonstration class for functools'
called object with: (<__main__.MyClass object at 0x000002DE7C2CD2E8>, 'default for e', 8)
partial() Returns an object that can be called directly , partialmethod() Returns a callable unbound method prepared for an object . In the following example , The same independent function is added to the class twice MyClass attribute . Use partialmethod() Generate method1（）, partial() Generate method2():
import functools
def standalone(self, a=1, b=2):
""" Independent functions """
print(' called standalone with:', (self, a, b))
if self is not None:
print(' self.attr =', self.attr)
class MyClass:
""""functools Sample class """
def __init__(self):
self.attr = 'instance attribute'
method1 = functools.partialmethod(standalone)
method2 = functools.partial(standalone)
o = MyClass()
print('standalone')
standalone(None)
print()
print('method1 as partialmethod')
o.method1()
print()
print('method2 as partial')
try:
o.method2()
except TypeError as err:
print('ERROR: {}'.format(err))
method1() Can be MyClass The instance , Just like ordinary class methods , Instance is passed in as the first parameter .method2() Not successfully bound as a class method . Therefore, its self Parameters must be explicitly passed in , So this example throws TypeError abnormal :
standalone
called standalone with: (None, 1, 2)
method1 as partialmethod
called standalone with: (<__main__.MyClass object at 0x00000214B4459B70>, 1, 2)
self.attr = instance attribute
method2 as partial
ERROR: standalone() missing 1 required positional argument: 'self'
It is sometimes useful to keep the property information of a function when using decorators . However, some original function information will inevitably be lost when using the decorator . therefore functools Provides wraps() The decorator can pass through update_wrapper() Copy the specified attributes of the original function object to the wrapper function object .
from functools import wraps
def logged1(func):
def with_login(*args, **kwargs):
print(func.__name__ + "was called")
return func(*args, **kwargs)
return with_login
@logged1
def f1(x):
""" function doc"""
return x + x * 1
def logged2(func):
@wraps(func)
def with_login(*args, **kwargs):
print(func.__name__ + "was called")
return func(*args, **kwargs)
return with_login
@logged2
def f2(x):
""" function doc """
return x + x * 1
print(" Don't use functools.wraps when :")
print("__name__: " + f1.__name__)
print("__doc__: ", end=" ")
print(f1.__doc__)
print()
print(" Use functools.wraps when :")
print("__name__: " + f2.__name__)
print("__doc__: ", end=" ")
print(f2.__doc__)
Don't use functools.wraps when :
__name__: with_login
__doc__: None
Use functools.wraps when :
__name__: f2
__doc__: function doc
stay Python2 Before , Class __cmp__() Method , This method depends on whether the object is less than 、d Equal to or greater than the item being compared returns -1、0 or 1.Python2.1 It's starting to introduce Rich comparison Method API(__lt__(), __le()__, __eq__(), __ne__(), __gt__() and __ge__()), Used to perform a comparison operation and return a Boolean value .Python3 in __cmp__() Abandon support for these new methods , from functools Provide tools , So as to facilitate the compilation of the code that conforms to Python3 Compare the new required classes in .
Rich comparison API Designed to allow classes with complex comparisons to implement each calculation in the most efficient way . however , For relatively simple classes , There is no point in manually creating each rich comparison method .total_ordering() The class decorator can make the decorated class only need to define __lt__(),__le__().__gt__() and __ge__() One of them and __eq__(), The rest is automatically provided by the decorator . This simplifies the effort of defining all rich comparison operations .
''' No one answers the problems encountered in learning ？ Xiaobian created a Python Exchange of learning QQ Group ：711312441 Looking for small partners who share the same aspiration , Help each other , There are also good video tutorials and PDF e-book ！ '''
import functools
import inspect
from pprint import pprint
@functools.total_ordering
class MyObject:
def __init__(self, val):
self.val = val
def __eq__(self, other):
print(' testing __eq__({}, {})'.format(
self.val, other.val))
return self.val == other.val
def __gt__(self, other):
print(' testing __gt__({}, {})'.format(
self.val, other.val))
return self.val > other.val
print("MyObject's Methods:\n")
pprint(inspect.getmembers(MyObject, inspect.isfunction))
a = MyObject(1)
b = MyObject(2)
print('\nComparisons:')
for expr in ['a < b', 'a <= b', 'a == b', 'a >= b', 'a > b']:
print('\n{:<6}:'.format(expr))
result = eval(expr)
print(' result of {}: {}'.format(expr, result))
MyObject's Methods:
[('__eq__', <function MyObject.__eq__ at 0x0000021DE4DB4048>),
('__ge__', <function _ge_from_gt at 0x0000021DDDE5D268>),
('__gt__', <function MyObject.__gt__ at 0x0000021DE4DB40D0>),
('__init__', <function MyObject.__init__ at 0x0000021DDDE877B8>),
('__le__', <function _le_from_gt at 0x0000021DDDE5D2F0>),
('__lt__', <function _lt_from_gt at 0x0000021DDDE5D1E0>)]
Comparisons:
a < b :
testing __gt__(1, 2)
testing __eq__(1, 2)
result of a < b: True
a <= b:
testing __gt__(1, 2)
result of a <= b: True
a == b:
testing __eq__(1, 2)
result of a == b: False
a >= b:
testing __gt__(1, 2)
testing __eq__(1, 2)
result of a >= b: False
a > b :
testing __gt__(1, 2)
result of a > b: False
Although the decorator can easily create completely ordered types , But the derived comparison function may execute more slowly , And generate more complex stack traces . If the performance benchmark shows that this is the bottleneck of the program , Then implementing all six rich comparison functions may increase the speed .
stay Python3 The old comparison has been abandoned (cmp) function , So for example sorted(),min(),max() And other methods are no longer supported cmp Parameters , But still support key function .functools Provides cmp_to_key() Is used to cmp Function to key function .
For example, given a list of positive integers , Output the largest integer that can be spliced with these positive integers . If it is Python2 The program can be like this :
L = [97, 13, 4, 246]
def my_cmp(a, b):
""" Concatenate the two numbers to be compared into an integer , Compare values """
return int(str(b) + str(a)) - int(str(a) + str(b))
L.sort(cmp=my_cmp)
print(''.join(map(str, L)))
# Output 97424613
but Python3 Of sort Function is obsolete cmp Parameters , have access to cmp_to_key take cmp Function to key function :
from functools import cmp_to_key
L = [97, 13, 4, 246]
def my_cmp(a, b):
""" Concatenate the two numbers to be compared into an integer , Compare values """
return int(str(b) + str(a)) - int(str(a) + str(b))
L.sort(key=cmp_to_key(my_cmp))
print(''.join(map(str, L)))
# Output 97424613
cmp Function takes two parameters , Compare them , If less than, return a negative number , Equal return 0, Greater than return a positive number . key The function takes a parameter , Returns the key used for sorting .
lru_cache() A decorator is Cache elimination algorithm ( Recently at least use ) An implementation of . It uses the arguments of the function as key As a result value cached hash In structure ( So the arguments to the function must be hashable), If it is called again later with the same parameters, it will start from hash From the return result . At the same time, the decorator also adds a method to check the cache state transition (cache_info()) And emptying the cache (cache_clear()) To the function .
import functools
@functools.lru_cache()
def demo(a):
print('called demo with {}'.format(a))
return a ^ 2
MAX = 2
print(' First call :')
for i in range(MAX):
demo(i)
print(demo.cache_info())
print('\n Second call :')
for i in range(MAX + 1):
demo(i)
print(demo.cache_info())
print('\n After clearing the cache :')
demo.cache_clear()
print(demo.cache_info())
print('\n Call again :')
for i in range(MAX):
demo(i)
print(demo.cache_info())
Multiple calls in code demo() Method . After the first call, the result is stored in the cache .cache_info() Returns a named tuple , Include hits,misses,maxsize and currsize . When the second call hits the cache, the call will directly return the cache contents ,cache_clear() Used to clear the current cache .
First call :
called demo with 0
called demo with 1
CacheInfo(hits=0, misses=2, maxsize=128, currsize=2)
Second call :
called demo with 2
CacheInfo(hits=2, misses=3, maxsize=128, currsize=3)
After clearing the cache :
CacheInfo(hits=0, misses=0, maxsize=128, currsize=0)
Call again :
called demo with 0
called demo with 1
CacheInfo(hits=0, misses=2, maxsize=128, currsize=2)
To prevent the cache from growing indefinitely in long-running processes , Specially set maxsize Parameters , The default is 128, Set to None when , Then disable LRU function , The cache can grow indefinitely . It also provides typed Parameters , Used to set whether to distinguish parameter types , The default is Fals. If set to True, So similar as demo(1) and demo(1.0) Will be treated as different calls with different values .
Python3 In the global namespace reduce() function , take reduced() Put it in functools Module , To use reduce() Words , First of all functools Load in .
from functools import reduce
print(reduce(lambda a, b: a + b, range(11)))
# Calculation 1 Add to 10 result 55
In dynamically typed languages ( Such as Python) in , If you need to perform different operations according to the type of parameter , The simple and direct method is to check the type of the parameter . But when the behavior difference is obvious, it needs to be separated into separate functions . functools Provide singledispatch() The decorator registers a set of general-purpose functions to switch automatically based on the type of the first parameter of the function , Similar to function overloading in strongly typed languages .
''' No one answers the problems encountered in learning ？ Xiaobian created a Python Exchange of learning QQ Group ：711312441 Looking for small partners who share the same aspiration , Help each other , There are also good video tutorials and PDF e-book ！ '''
import functools
@functools.singledispatch
def myfunc(arg):
print('default myfunc({!r})'.format(arg))
@myfunc.register(int)
def myfunc_int(arg):
print('myfunc_int({})'.format(arg))
@myfunc.register(list)
def myfunc_list(arg):
print('myfunc_list({})'.format(' '.join(arg)))
myfunc('string argument')
myfunc(1)
myfunc(2.3)
myfunc(['a', 'b', 'c'])
By singledispatch() Decorated functions are implemented by default , To use its register() Properties decorate functions that receive arguments of other types . It will be called according to register() Automatically select the implementation function for the type registered in . If not, use the default implementation .
default myfunc('string argument')
myfunc_int(1)
default myfunc(2.3)
myfunc_list(a b c)
In addition, in the case of inheritance , When the type does not match exactly , According to the order of inheritance , Select the closest type .
import functools
class A:
pass
class B(A):
pass
class C(A):
pass
class D(B):
pass
class E(C, D):
pass
@functools.singledispatch
def myfunc(arg):
print('default myfunc({})'.format(arg.__class__.__name__))
@myfunc.register(A)
def myfunc_A(arg):
print('myfunc_A({})'.format(arg.__class__.__name__))
@myfunc.register(B)
def myfunc_B(arg):
print('myfunc_B({})'.format(arg.__class__.__name__))
@myfunc.register(C)
def myfunc_C(arg):
print('myfunc_C({})'.format(arg.__class__.__name__))
myfunc(A())
myfunc(B())
myfunc(C())
myfunc(D())
myfunc(E())
myfunc_A(A)
myfunc_B(B)
myfunc_C(C)
myfunc_B(D)
myfunc_C(E)
In the above code , class D and E Does not match any registered generic functions , So choose according to the inheritance order of its classes .
