Python Generators: How to Create Iterators using Yield
There is a lot of things in building iterators in Python; we have to implement a class with __iter__() and __next__() method, keep track of internal variable states, raise StopIteration when there were no values to be returned, etc.
Python generators are used to create the iterators, but with a different approach. Generators are simple functions that return an iterable set of items, one at a time, in a unique way.
If the function contains at least one yield statement (it may include other yield or return statements, then it becomes a Generator function.
The main difference between normal function and Python Generators is that the return statement terminates the function entirely; the yield statement pauses the function saving all its states and variable values and later continues from there on successive calls.
See the following example of Generators in Python.
# app.py def createGenerator(): i = 1 print('Ronin') print(i) yield i i += 1 print('RedSkull') print(i) yield i i += 1 print('Rocket') print(i) yield i gen = createGenerator() next(gen) next(gen) next(gen)
See the output.
One interesting thing to note in the above example is that the value of the variable i is remembered between each call.
Unlike any normal functions, the local variables are not destroyed when the function yields. In addition to that, the generator object can be iterated only once.
When the yield statement is executed, the function execution is paused, and the variable’s value is not destroyed, when we call the next statement, it starts execution. When again yield statement is executed, the function execution is stopped and again next resumes the execution. That is why we can see all the values.
For Loops with Generators
We can use the for loop with Generators directly. We can display the values with executing the next() function. See the following example.
# app.py def createGenerator(): i = 1 print('Ronin') yield i i += 1 print('RedSkull') yield i i += 1 print('Rocket') yield i for g in createGenerator(): print(g)
See the output.
The for loop takes an iterator and iterates over it using the next() function. It automatically ends when StopIteration is raised.
Python Generator Expression
Simple generators can be easily created on the fly using generator expressions. It makes constructing generators easy.
Same as the lambda function creates an anonymous function, generator expression creates the anonymous generator function.
The syntax for generator expression is similar to that of the list comprehension in Python. But the square brackets are replaced with round parentheses.
The significant difference between the list comprehension and the generator expression is that while list comprehension produces the entire list, generator expression produces one item at a time.
They are kind of lazy, producing items only when asked for. For this reason, a generator expression is a much more memory efficient than list comprehension.
See the below example.
# app.py listK = [18, 19, 21, 29, 46] result = (i ** 2 for i in listK) print(next(result)) print(next(result)) print(next(result)) print(next(result))
See the output.
Generator expression can be used inside the functions. When used in such a way, the round parentheses can be used.
We can see above that the generator expression did not produce the required result immediately. Instead, it returned a generator object which produces elements on demand.
The yield statement suspends the function’s execution and sends the value back to the caller, but retains enough state to enable the function to resume where it is left off.
When resumed, a function continues execution immediately after the last yield run.
This allows its code to produce the series of values over time, rather than computing them at once and sending them back like a list.
Yield is used in Python generators.
The generator function is defined as a regular function, but whenever it needs to generate a value, it does so with a yield keyword rather than return.
If the body of the def contains yield, the function automatically becomes a generator function.
Differences between Generator and a Normal function
- The generator function contains one or more yield statements.
- When the generator function called, it returns an object (iterator) but does not start execution immediately.
- Functions like __iter__() and __next__() are implemented automatically. So we can iterate through the elements using the next() function.
- Once the function yield statement is executed, the function is paused and transfers the control to the caller.
- Local variables and their states are remembered between successive calls.
- Finally, when the function terminates, StopIteration is raised automatically on further calls.
Finally, Python Generators example is over.