python generator

One 、 What is? generator（ generator ）？

A function which returns a generator iterator. It looks like a normal function except that it contains yield expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the next() function.

generator It's a use yield Keyword a function that generates a series of data , Can pass for perhaps next() Traverse all its values .generator Only when used will you try to generate data .

Two 、 How to construct and call generator？

Use... In functions yield Loop output data

def generator_function():
for i in range(10):
yield i
for item in generator_function():
print(item)
# Output: 0
# 1
# 2
# 3
# 4
# 5
# 6
# 7
# 8
# 9

perhaps , Use generator The derived type （ more list be similar , Use () replace []）

csv_gen = (row for row in open(file_name))

3、 ... and 、generator Which scenes are commonly used ？

Generate infinite data , Use at this time list Storing data may lead to insufficient memory or even system crash . Use generator Define how to generate , Data can be generated continuously by taking only one value at a time , Less memory usage .

Generators are best for calculating large sets of results (particularly calculations involving loops themselves) where you don’t want to allocate the memory for all results at the same time. Many Standard Library functions that return lists in Python 2 have been modified to return generators in Python 3 because generators require fewer resources.

For example, No generator Read large files , One possible exception is MemoryError

def csv_reader(file_name):
file = open(file_name)
result = file.read().split("\n")
return result
Traceback (most recent call last):
File "ex1_naive.py", line 22, in <module>
main()
File "ex1_naive.py", line 13, in main
csv_gen = csv_reader("file.txt")
File "ex1_naive.py", line 6, in csv_reader
result = file.read().split("\n")
MemoryError

Now , Use generator We can solve this problem

def csv_reader(file_name):
for row in open(file_name, "r"):
yield row

Four 、generator actual combat

Generate fiborache sequence

# generator version
def fibon(n):
a = b = 1
for i in range(n):
yield a
a, b = b, a + b

Generate an infinite sequence

def infinite_sequence():
num = 0
while True:
yield num
num += 1

establish Data Pipelines

file_name = "techcrunch.csv"
lines = (line for line in open(file_name))
list_line = (s.rstrip()split(",") for s in lines)
cols = next(list_line)
company_dicts = (dict(zip(cols, data)) for data in list_line)
funding = (
int(company_dict["raisedAmt"])
for company_dict in company_dicts
if company_dict["round"] == "a"
)
total_series_a = sum(funding)
print(f"Total series A fundraising: ${total_series_a}")

Reference resources

1. https://book.pythontips.com/en/latest/generators.html
2. https://www.python.org/dev/peps/pep-0255/
3. https://realpython.com/introduction-to-python-generators/