Python Is a scripting language , comparison C/C++ Such a compiler language , There are some deficiencies in efficiency and performance . however , A lot of times ,Python The efficiency is not as exaggerated as you think . This article is about some Python Code to speed up the operation of the skills to organize .
This article will introduce a lot of Python Tips for code acceleration . Before going into the details of code optimization , Need to understand some basic principles of code optimization .
A lot of people start writing code and they're aiming for performance optimization ,“ It's much easier to make the right program faster than to get the fast program right ”. therefore , The premise of optimization is that the code works properly . Premature optimization may neglect the overall performance indicators , Don't reverse the order until you get the global result .
Optimization comes at a cost , It's almost impossible to solve all the performance problems . Usually the choice is time for space or space for time . in addition , Development costs also need to be considered .
If you optimize every part of the code , These changes make the code difficult to read and understand . If your code is slow , First of all, find out where the code runs slowly , It's usually an internal loop , Focus on optimization where it's slow . Elsewhere , A little loss of time has no effect .
# Writing is not recommended . Code takes time ：26.8 second
import math
size = 10000
for x in range(size):
for y in range(size):
z = math.sqrt(x) + math.sqrt(y)
Many programmers are just beginning to use Python Language to write some simple scripts , When writing scripts , Usually used to write it directly as a global variable , For example, the code above .
however , Because global variables and local variables are implemented differently , Code defined globally can run much slower than defined in functions . By putting script statements into functions , It usually brings 15% - 30% Speed up .
# Recommend writing . Code takes time ：20.6 second
import math
def main(): # Define it in a function , To reduce the use of all variables
size = 10000
for x in range(size):
for y in range(size):
z = math.sqrt(x) + math.sqrt(y)
main()
# Writing is not recommended . Code takes time ：14.5 second
import math
def computeSqrt(size: int):
result = []
for i in range(size):
result.append(math.sqrt(i))
return result
def main():
size = 10000
for _ in range(size):
result = computeSqrt(size)
main()
Each use .（ Property access operator ） It triggers specific methods , Such as __getattribute__() and __getattr__(), These methods do dictionary operations , So there's an extra time cost . adopt from import sentence , Attribute access can be eliminated .
# The first optimization of writing . Code takes time ：10.9 second
from math import sqrt
def computeSqrt(size: int):
result = []
for i in range(size):
result.append(sqrt(i)) # avoid math.sqrt Use
return result
def main():
size = 10000
for _ in range(size):
result = computeSqrt(size)
main()
In the 1 In the section, we talk about , Local variables find faster than global variables , So for frequently accessed variables sqrt, By changing it to a local variable, you can speed up running .
# Second optimization of writing . Code takes time ：9.9 second
import math
def computeSqrt(size: int):
result = []
sqrt = math.sqrt # Assign to local variable
for i in range(size):
result.append(sqrt(i)) # avoid math.sqrt Use
return result
def main():
size = 10000
for _ in range(size):
result = computeSqrt(size)
main()
except math.sqrt Outside ,computeSqrt There are also . The existence of , That's called list Of append Method . By assigning the method to a local variable , It can completely eliminate computeSqrt Function for Inside the loop . Use .
# Recommend writing . Code takes time ：7.9 second
import math
def computeSqrt(size: int):
result = []
append = result.append
sqrt = math.sqrt # Assign to local variable
for i in range(size):
append(sqrt(i)) # avoid result.append and math.sqrt Use
return result
def main():
size = 10000
for _ in range(size):
result = computeSqrt(size)
main()
# Writing is not recommended . Code takes time ：10.4 second
import math
from typing import List
class DemoClass:
def __init__(self, value: int):
self._value = value
def computeSqrt(self, size: int) -> List[float]:
result = []
append = result.append
sqrt = math.sqrt
for _ in range(size):
append(sqrt(self._value))
return result
def main():
size = 10000
for _ in range(size):
demo_instance = DemoClass(size)
result = demo_instance.computeSqrt(size)
main()
avoid . The same principle applies to in class properties , visit self._value Is slower than accessing a local variable . By assigning a property in a class that needs frequent access to a local variable , It can improve the speed of code running .
# Recommend writing . Code takes time ：8.0 second
import math
from typing import List
class DemoClass:
def __init__(self, value: int):
self._value = value
def computeSqrt(self, size: int) -> List[float]:
result = []
append = result.append
sqrt = math.sqrt
value = self._value
for _ in range(size):
append(sqrt(value)) # avoid self._value Use
return result
def main():
size = 10000
for _ in range(size):
demo_instance = DemoClass(size)
demo_instance.computeSqrt(size)
main()
# Writing is not recommended , Code takes time ：0.55 second
class DemoClass:
def __init__(self, value: int):
self.value = value
@property
def value(self) -> int:
return self._value
@value.setter
def value(self, x: int):
self._value = x
def main():
size = 1000000
for i in range(size):
demo_instance = DemoClass(size)
value = demo_instance.value
demo_instance.value = i
main()
Any time you use an extra layer of processing （ Like decorators 、 Attribute access 、 Descriptor ） When it comes to wrapping code , Will slow down the code .
In most cases , It needs to be revisited to see if it's necessary to use the definition of property accessors , Use getter/setter Functions to access properties are usually C/C++ Code style left over by programmers . If there's really no need to , Just use simple properties .
# Recommend writing , Code takes time ：0.33 second
class DemoClass:
def __init__(self, value: int):
self.value = value # Avoid unnecessary property accessors
def main():
size = 1000000
for i in range(size):
demo_instance = DemoClass(size)
value = demo_instance.value
demo_instance.value = i
main()
# Writing is not recommended , Code takes time ：6.5 second
def main():
size = 10000
for _ in range(size):
value = range(size)
value_list = [x for x in value]
square_list = [x * x for x in value_list]
main()
In the above code value_list No need at all , This creates unnecessary data structures or replicates .
# Recommend writing , Code takes time ：4.8 second
def main():
size = 10000
for _ in range(size):
value = range(size)
square_list = [x * x for x in value] # Avoid meaningless duplication
main()
The other is to Python The data sharing mechanism is too paranoid , Not well understood or trusted Python Memory model , The abuse of copy.deepcopy() Functions like that . In general, copy operations can be removed from this code .
# Writing is not recommended , Code takes time ：0.07 second
def main():
size = 1000000
for _ in range(size):
a = 3
b = 5
temp = a
a = b
b = temp
main()
The above code creates a temporary variable when exchanging values temp, If you don't use intermediate variables , More concise code 、 And it runs faster .
# Recommend writing , Code takes time ：0.06 second
def main():
size = 1000000
for _ in range(size):
a = 3
b = 5
a, b = b, a # Without the aid of intermediate variables
main()
# Writing is not recommended , Code takes time ：2.6 second
import string
from typing import List
def concatString(string_list: List[str]) -> str:
result = ''
for str_i in string_list:
result += str_i
return result
def main():
string_list = list(string.ascii_letters * 100)
for _ in range(10000):
result = concatString(string_list)
main()
When using a + b When concatenating strings , because Python The string in is immutable , It will apply for a piece of memory , take a and b Copy to the memory space of the new application .
therefore , If you want to splice n A string , Will produce n-1 An intermediate result , Every time an intermediate result is generated, the memory needs to be applied and copied once , Serious impact on operational efficiency .
While using join() When concatenating strings , It will first calculate the total memory space to be applied for , Then apply for the required memory at one time , And copy each string element into the memory .
# Recommend writing , Code takes time ：0.3 second
import string
from typing import List
def concatString(string_list: List[str]) -> str:
return ''.join(string_list) # Use join instead of +
def main():
string_list = list(string.ascii_letters * 100)
for _ in range(10000):
result = concatString(string_list)
main()
# Writing is not recommended , Code takes time ：0.05 second
from typing import List
def concatString(string_list: List[str]) -> str:
abbreviations = {'cf.', 'e.g.', 'ex.', 'etc.', 'flg.', 'i.e.', 'Mr.', 'vs.'}
abbr_count = 0
result = ''
for str_i in string_list:
if str_i in abbreviations:
result += str_i
return result
def main():
for _ in range(10000):
string_list = ['Mr.', 'Hat', 'is', 'Chasing', 'the', 'black', 'cat', '.']
result = concatString(string_list)
main()
if The short-circuit characteristics of a condition refer to if a and b Such a statement , When a by False Will return directly to , No more calculations b; about if a or b Such a statement , When a by True Will return directly to , No more calculations b.
therefore , In order to save running time , about or sentence , The value should be True The more likely variables are written in or front , and and It should be pushed back .
from typing import List
def concatString(string_list: List[str]) -> str:
abbreviations = {'cf.', 'e.g.', 'ex.', 'etc.', 'flg.', 'i.e.', 'Mr.', 'vs.'}
abbr_count = 0
result = ''
for str_i in string_list:
if str_i[-1] == '.' and str_i in abbreviations: # utilize if Short circuit characteristics of conditions
result += str_i
return result
def main():
for _ in range(10000):
string_list = ['Mr.', 'Hat', 'is', 'Chasing', 'the', 'black', 'cat', '.']
result = concatString(string_list)
main()
# Writing is not recommended . Code takes time ：6.7 second
def computeSum(size: int) -> int:
sum_ = 0
i = 0
while i < size:
sum_ += i
i += 1
return sum_
def main():
size = 10000
for _ in range(size):
sum_ = computeSum(size)
main()
Python Of for Cycle ratio while The cycle is much faster .
# Recommend writing . Code takes time ：4.3 second
def computeSum(size: int) -> int:
sum_ = 0
for i in range(size): # for Loop instead of while loop
sum_ += i
return sum_
def main():
size = 10000
for _ in range(size):
sum_ = computeSum(size)
main()
For the example above , Further, we can use implicit for Loop instead of explicit for loop
# Recommend writing . Code takes time ：1.7 second
def computeSum(size: int) -> int:
return sum(range(size)) # Implicit for Loop instead of explicit for loop
def main():
size = 10000
for _ in range(size):
sum = computeSum(size)
main()
# Writing is not recommended . Code takes time ：12.8 second
import math
def main():
size = 10000
sqrt = math.sqrt
for x in range(size):
for y in range(size):
z = sqrt(x) + sqrt(y)
main()
In the above code sqrt(x) On the inside for loop , It's recalculated every time you train , Increased time cost .
# Recommend writing . Code takes time ：7.0 second
import math
def main():
size = 10000
sqrt = math.sqrt
for x in range(size):
sqrt_x = sqrt(x) # Reduce the inner layer for The calculation of the loop
for y in range(size):
z = sqrt_x + sqrt(y)
main()
Let's follow the example described above , On this basis, use numba.jit.numba Can be Python function JIT Compile to machine code execution , Greatly improve the speed of code running . About numba For more information, see the home page below ：http://numba.pydata.org/numba.pydata.org
# Recommend writing . Code takes time ：0.62 second
import numba
@numba.jit
def computeSum(size: float) -> int:
sum = 0
for i in range(size):
sum += i
return sum
def main():
size = 10000
for _ in range(size):
sum = computeSum(size)
main()
Python Built in data structures such as str, tuple, list, set, dict The bottom is C Realized , Very fast , It's almost impossible to achieve the built-in speed in terms of performance when implementing a new data structure yourself .
list Be similar to C++ Medium std::vector, It's a dynamic array . It will pre allocate a certain amount of memory space , When the pre allocated memory space runs out , When you continue to add elements to it , Will apply for a larger memory space , Then copy all the original elements , Then destroy the previous memory space , And insert new elements .
The operation is similar when deleting an element , When the used memory space is less than half of the pre allocated memory space , Will apply for another small memory , Make an element copy , Then destroy the original large memory space .
therefore , If there are frequent additions 、 Delete operation , newly added 、 When the number of deleted elements is large ,list It's not efficient .
here , Should consider using collections.deque.collections.deque It's a two terminal queue , It has stack and queue characteristics at the same time , It can be done at both ends O(1) Complexity of insert and delete operations .
list It's also very time consuming . When need is in list Look for certain elements frequently , Or frequent and orderly access to these elements , have access to bisect maintain list Objects are ordered and binary searches are performed within them , Improve the efficiency of search .
Another common requirement is to find minima or maxima , You can use heapq The module will list Into a heap , The time complexity of getting the minimum value is O(1).
The following page shows the commonly used Python The time complexity of each operation of data structure ：
TimeComplexity - Python Wikiwiki.python.org
The above summary is to be able to improve Python Skills of running speed , Hurry up and stop ！
