Programmer's guide to python

7.7 Iterators

>An iterator is also an iterable object, but the difference is it must have both __iter__() and __next__() special methods implemented (this is called the iterator protocol). The __iter__() method as we saw earlier returns an iterator object, the __next__() method is to fetch the next element from the iterator object (we saw this method in generators).
>An iterator object represents a stream of data, when the __next__() (or the next()) method is called it returns the next consecutive element till the StopIteration is raised. It is when the iterator object is exhausted (has no values left) and so it no longer returns a value.
>One difference between iterator and iterable is that once a iterator is exhausted it stays empty even after passing it to the iter() function (as iterator object returns itself when passed to iter()), which is not the case with a iterable object (a new iterator object is created every time iter() is called).
>The iterator object is similar to a generator, it "lazy loads" data into memory. It is not required to be finite but be careful when looping over, it might lead to a RecursionError. Examples of iterators are enumerate, zip, reversed etc. The itertools module included in the standard library contains a number of commonly-used iterators as well as functions for combining several iterators.
>Limitations of an iterator is that the elements can be iterated only once and in a single direction only (can't access previous values). It needs to be re-created once it is exhausted.
>Creating a simple iterator.

## Example: Create a simple iterator from a iterable
# 'iter()' function takes a iterable/iterator object and
# returns a iterator object
my_iterator = iter([12, 34, 2, 65, 21, 65])
# iterate to next values using 'next()'
print(next(my_iterator))  # 12
# or calling the special method from a instance
print(my_iterator.__next__())  # 34
for a in my_iterator:
  print(a)  # 2 65 21 65
  # Notice: Loop started from index 2, because we already
  # called 'next()' twice

# Now as 'my_iterator' is exhausted calling 'next()' again will
# raise 'StopIteration'
# Note: Comment below line to continue further execution
print(next(my_iterator))  # StopIteration

# Calling loop will print no value
for a in my_iterator:
  print(a)
# and calling 'iter()' won't work either
my_iterator = iter(my_iterator)
print([a for a in my_iterator])  # []

>Create a custom class that has iterator functionality.

## Example: Create a iterator object which returns square of each values
class SquareIterator:
  """SquareIterator takes items and returns item's square upon called"""

  def __init__(self, *args):
      self.args = args
      # iterating limit for 'StopIteration'
      self.iter_len = len(args) - 1
      # initialize index to keep track of it
      self.idx = -1

  def __iter__(self):
      """This method returns an iterator object, which is itself."""
      return self

  def __next__(self):
      """This method is used to fetch next value, so it should return
      some value."""
      self.idx += 1
      # stop if limit is reached
      if self.idx > self.iter_len:
          raise StopIteration
      return self.args[self.idx] ** 2

my_iter = SquareIterator(2, 3, 4, 8, 9, 12)
# Iterate values using 'next()'
print(next(my_iter))  # 4

# Iterating a iterator object
# A for loop internally calls '__iter__()' function first, then '__next__()'
# on a iterable/iterator, we'll see more on the working of for loops next
for v in my_iter:
  print(v)  # 9 16 64 81 144
# now like earlier 'next()' will raise error
# Note: Comment below line to continue further execution
print(next(my_iter))  # StopIteration

# Create a new SquareIterator to reset its index
my_iter = SquareIterator(2, 3, 4, 8, 9, 12)
for a in my_iter:
  print(a)  # 4 9 16 64 81 144

>When iterating with a for loop, the iterable input object is first converted to a temporary iterator object and then this object is traversed using __next__() till StopIteration is raised. So every time a for is called a new temporary iterator object is created and removed when iteration is finished/interrupted.
>In case of iterating an iterator object, the object itself is returned (as we saw in our previous example, the __iter__() method), so once it's exhausted it stays empty. We'll check an example of the working below.

## Internal working of for loops
def for_loop(my_iterable):
  """A function to simulate a for loop"""
  # create a temporary iterator object each time starting a loop
  temp_object = iter(my_iterable)
  while True:
      try:
          # call the 'next()' function
          value = next(temp_object)
          print(value)
      except StopIteration:
          break  # stop the iteration
# so a 'for' loop is actually a 'while' loop

# Pass a iterable/iterator object to our 'for_loop()' function
my_list = [34, 32, 55, 34, 56]
for_loop(my_list)  # 34 32 55 34 56

# Now doing the same thing with a for loop
for a in my_list:
  print(a)  # 34 32 55 34 56

The difference between Iterables, Iterators and Generators.

>Iterables: Are objects that can be iterated, they can be iterated as many times as required. They need to implement the __iter__() method.
>Iterators: Are also iterable but are "lazy" and can be iterated only once, they need to be re-created for iterating again. They need to implement __iter__() and __next__() methods. They are used when it is required to have iterator functionality inside a complex class (with other functionalities).
>Generators: Are iterable and iterator (but not vice versa). Generators are an easy way to create an iterator object. They can be created using a function with a yield statement or using generator comprehension. They can be used when we need a standalone iterator object.