Mention the word yuan , You might think of metadata , Metadata is the data that describes the data itself , Metaclasses are classes of classes , The corresponding metaprogramming is the code that describes the code itself , Metaprogramming is about creating operation source code ( For example, modify 、 Generate or wrap the original code ) Functions and classes of . The main technique is to use decorators 、 The metaclass 、 Descriptor Class .
The main purpose of this paper is to introduce these metaprogramming techniques , And give examples to demonstrate how they customize the behavior of source code .
Decorator
Decorators are functions of functions , It takes a function as an argument and returns a new function , Add new functions without changing the original function code , For example, the most commonly used timing decorator ：
from functools import wraps
def timeit(logger=None):
"""
Time consuming statistics decorator , The unit is seconds , Retain 4 Decimal place
"""
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
if logger:
logger.info(f"{func.__name__} cost {end - start :.4f} seconds")
else:
print(f"{func.__name__} cost {end - start :.4f} seconds")
return result
return wrapper
return decorator
( notes ： For example, we use @wraps(func) Annotations are important , It can retain the metadata of the original function ) Just add... To the original function @timeit() You can add new functions to it ：
@timeit()
def test_timeit():
time.sleep(1)
test_timeit()
#test_timeit cost 1.0026 seconds
The effect of the above code is the same as that written below ：
test_timeit = timeit(test_timeit)
test_timeit()
Execution sequence of decorators
When there are multiple decorators , What is their calling order ？
If there is such code , Please print first Decorator1 still Decorator2 ?
from functools import wraps
def decorator1(func):
@wraps(func)
def wrapper(*args, **kwargs):
print('Decorator 1')
return func(*args, **kwargs)
return wrapper
def decorator2(func):
@wraps(func)
def wrapper(*args, **kwargs):
print('Decorator 2')
return func(*args, **kwargs)
return wrapper
@decorator1
@decorator2
def add(x, y):
return x + y
add(1,2)
# Decorator 1
# Decorator 2
Before I answer that question , Let me give you a vivid metaphor first , The decorator is like a function wearing clothes , The closest to it is the first to wear , Far away from the last wear , In the above example decorator1 It's a coat ,decorator2 It's underwear .
add = decorator1(decorator2(add))
When the function is called , It's like taking off your clothes , First remove the outermost decorator1, That is, print first Decorator1, Execute to return func(
args, kwargs) I'll lift it when I need it decorator2, And then print Decorator2, Execute again to return func(
args, kwargs) Will really execute add() function .
It should be noted that the printing position , If the code that prints the string is after the calling function , Like this , The output is just the opposite ：
def decorator1(func):
@wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
print('Decorator 1')
return result
return wrapper
def decorator2(func):
@wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
print('Decorator 2')
return result
return wrapper
Decorators can be defined not only as functions , It can also be defined as a class , As long as you make sure it's done __call__() and __get__() Method .
The metaclass
Python All classes in （object） The metaclass of , Namely type class , in other words Python Class creation behavior is determined by the default type Class control , Make a comparison ,type Class is the ancestor of all classes . We can realize some custom object creation behaviors through programming .
Define a class to inherit type class A, Then let the metaclasses of other classes point to A, You can control A The creation behavior of . A typical example is to use a metaclass to implement a singleton ：
class Singleton(type):
def __init__(self, *args, **kwargs):
self._instance = None
super().__init__(*args, **kwargs)
def __call__(self, *args, **kwargs):
if self._instance is None:
self._instance = super().__call__(*args, **kwargs)
return self._instance
else:
return self._instance
class Spam(metaclass=Singleton):
def __init__(self):
print("Spam!!!")
The metaclass Singleton Of __init__ and __new__ Methods will be defined in Spam The period is executed , and __call__ Method will be instantiated Spam When it comes to execution .
If you want to better understand metaclasses , You can read Python Dark magic metaclass
descriptor class （ Descriptor Class ）
descriptor Is any definition __get__(),__set__() or __delete__() The object of , Descriptors enable objects to customize attribute lookup 、 Storage and deletion operations . Here are the official documents [1] An example of a custom verifier .
Define the verifier class , It is a descriptor class , It is also an abstract class ：
from abc import ABC, abstractmethod
class Validator(ABC):
def __set_name__(self, owner, name):
self.private_name = '_' + name
def __get__(self, obj, objtype=None):
return getattr(obj, self.private_name)
def __set__(self, obj, value):
self.validate(value)
setattr(obj, self.private_name, value)
@abstractmethod
def validate(self, value):
pass
Custom validators need to be from Validator Inherit , And must provide validate() Method to test various constraints as needed .
These are three practical data validation tools ：
OneOf The validation value is one of a set of constrained options .
class OneOf(Validator):
def __init__(self, *options):
self.options = set(options)
def validate(self, value):
if value not in self.options:
raise ValueError(f'Expected {value!r} to be one of {self.options!r}')
Number Verify that the value is int or float. According to the optional parameters , It can also verify that the value is between a given minimum or maximum .
class Number(Validator):
def __init__(self, minvalue=None, maxvalue=None):
self.minvalue = minvalue
self.maxvalue = maxvalue
def validate(self, value):
if not isinstance(value, (int, float)):
raise TypeError(f'Expected {value!r} to be an int or float')
if self.minvalue is not None and value < self.minvalue:
raise ValueError(
f'Expected {value!r} to be at least {self.minvalue!r}'
)
if self.maxvalue is not None and value > self.maxvalue:
raise ValueError(
f'Expected {value!r} to be no more than {self.maxvalue!r}'
)
String Verify that the value is str. According to the optional parameters , It can verify a given minimum or maximum length . It can also validate user-defined predicate.
class String(Validator):
def __init__(self, minsize=None, maxsize=None, predicate=None):
self.minsize = minsize
self.maxsize = maxsize
self.predicate = predicate
def validate(self, value):
if not isinstance(value, str):
raise TypeError(f'Expected {value!r} to be an str')
if self.minsize is not None and len(value) < self.minsize:
raise ValueError(
f'Expected {value!r} to be no smaller than {self.minsize!r}'
)
if self.maxsize is not None and len(value) > self.maxsize:
raise ValueError(
f'Expected {value!r} to be no bigger than {self.maxsize!r}'
)
if self.predicate is not None and not self.predicate(value):
raise ValueError(
f'Expected {self.predicate} to be true for {value!r}'
)
In practical application, it is written like this ：
class Component:
name = String(minsize=3, maxsize=10, predicate=str.isupper)
kind = OneOf('wood', 'metal', 'plastic')
quantity = Number(minvalue=0)
def __init__(self, name, kind, quantity):
self.name = name
self.kind = kind
self.quantity = quantity
The descriptor prevents the creation of invalid instances ：
>>> Component('Widget', 'metal', 5) # Blocked: 'Widget' is not all uppercase
Traceback (most recent call last):
...
ValueError: Expected <method 'isupper' of 'str' objects> to be true for 'Widget'
>>> Component('WIDGET', 'metle', 5) # Blocked: 'metle' is misspelled
Traceback (most recent call last):
...
ValueError: Expected 'metle' to be one of {'metal', 'plastic', 'wood'}
>>> Component('WIDGET', 'metal', -5) # Blocked: -5 is negative
Traceback (most recent call last):
...
ValueError: Expected -5 to be at least 0
>>> Component('WIDGET', 'metal', 'V') # Blocked: 'V' isn't a number
Traceback (most recent call last):
...
TypeError: Expected 'V' to be an int or float
>>> c = Component('WIDGET', 'metal', 5) # Allowed: The inputs are valid
Last words
About Python Metaprogramming of , Summarized below ：
If you want some functions to have the same function , I hope the original calling method will not be changed 、 Don't write duplicate code 、 Easy maintenance , You can use decorators to achieve .
If you want some classes to have some of the same features , Or control it in the class definition , We can customize a metaclass , Then let its metaclass point to the class .
If you want the properties of an instance to have some common characteristics , You can customize a descriptor class .
The above is all the content shared this time , Want to know more python Welcome to official account ：Python Programming learning circle , send out “J” Free access to , Daily dry goods sharing
