Is an iterator also an iterable?

Question

I found that:

>>> a={'x':42, 'y':3.14, 'z':7}
>>> b=a.__iter__()
>>> b.__dir__()
['__next__', ..., '__iter__', ...]
>>> b
<set_iterator object at 0x7efdd4e5afc0>

Does an iterator always have the __iter__ method?

According to https://stackoverflow.com/a/9884259 an iterator also an iterable. If it is true that an iterator always has __iter__ method?

Anticlinal

Yes, it always does. And it should return self. – Viridescent

Seriously, though, check out this gist. I think it should clear up some confusion with regards to for loops, what is an iterator, and what is an iterable. – Viridescent

I think it's worth noting that sometimes people's language about iterators and iterables is the reverse of the actual protocol details. That is, if somebody says "X is an iterable", they may intend to exclude iterators (despite iterators always having an __iter__ method and so being iterable in the official sense). There's no good single-word term for "non-iterator iterable", so some folks just use "iterable" for that, even though its not accurate. I elaborated a bit on this topic in a previous answer. – Frederiksen

Answer 1

An iterable needs to implement an __iter__ method or a __getitem__ method:

An object can be iterated over with for if it implements __iter__() or __getitem__().

An iterator needs a __iter__ method (that returns self) and a __next__ method (I'm not 100% sure about the __next__).

it is true that an iterator always has __iter__ method?

Yes!

This is also documented in the Data model:

object.__iter__(self)

This method is called when an iterator is required for a container. This method should return a new iterator object that can iterate over all the objects in the container. For mappings, it should iterate over the keys of the container.

Iterator objects also need to implement this method; they are required to return themselves. For more information on iterator objects, see Iterator Types.

(Emphasis mine)

As to your second question:

Is an iterator also an iterable?

Yes, because it has a __iter__ method.

Additional notes

Besides the formal implementation it's easy to check if something is iterable by just checking if iter() can be called on it:

def is_iterable(something):
    try:
        iter(something)
    except TypeError:
        return False
    else:
        return True

Likewise it's possible to check if something is an iterator by checking if iter() called on something returns itself:

def is_iterator(something):
    try:
        return iter(something) is something  # it needs to return itself to be an iterator
    except TypeError:
        return False

But don't use them in development code, these are just for "visualization". Mostly you just iterator over something using for ... in ... or if you need an iterator you use iterator = iter(...) and then process the iterator by calling next(iterator) until it throws a StopIteration.

Frazier

Thanks. Which reference can I find "An iterable has an iter method or a getitem and len method"? What are those iterables which don't have __iter__? – Anticlinal

@Ben I added some links :) – Frazier

Thanks. What are those builtin iterables which don't have __iter__? – Anticlinal

I don't think there are any. That's mostly used for user-defined sequence-like iterables. – Frazier

Is it possible to convert iterables which don't have __iter__ into iterators? – Anticlinal

@Ben Any specific use-case? Generally you don't need iterators often, you just iterate over whatever you have and then see if it works. – Frazier

@Ben: In Python 2, str doesn't have an __iter__. I'm not aware of any built-in iterable types with no __iter__ on Python 3. – Urogenital

@Ben you can always get an iterator from any iterable by using iter, i.e. iterator = iter(my_iterable) – Viridescent

Answer 2

An iterator is iterable. And yes, an iterator always has an __iter__ method.

Calling iter on an iterator, which summons the __iter__ hook, returns the same iterator:

>>> it = iter([]) # return iterator from iterable
>>> it is iter(iter(it)) is it.__iter__().__iter__().__iter__()
True

A classical example of method chaining.

As you must have also noticed, most implementations of the iterator protocol for custom classes always follows:

def __iter__(self):
    return self

That is if the iteration is not delegated to another iterator, via say return iter(...).

It would be quite counter-intuitive for an iterator to not implement the iterator protocol don't you think? The __iter__ implementation of the protocol is described below:

iterator.__iter__()

Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements.

[Emphasis mine]

This behaviour is expectedly consistent with iterator objects returned by builtins:

>>> m = map(None, [])
>>> m
<map object at 0x...>
>>> m is m.__iter__().__iter__().__iter__()
True

_{P.S: I apologise for calling dunders repeatedly; makes it look like it's the right thing to do. But it's not!}

Ow

No. An iterator could have __getitem__ instead of __iter__ so this is not true. – Counterinsurgency

No, you're thinking about "iterables". "Iterators" always have a __iter__ method. – Frazier

This is required to allow both containers and iterators to be used with the for and in statements <-- that seems to be a documentation bug. A container can just implement __contains__, and then __iter__ is not required for it to be used on the right hand side of in statement. – Garfield

@Garfield I think they mean the for-loop syntax: for ... in ... with "for and in". Not actually the in as in contains. – Frazier

I doubt it. The looping is just called a "for statement" in the grammar. And that text is from 16 years ago (!), so the intention was probably to describe old-style membership tests via iteration. Otherwise there is no good reason to explicitly write "the for and in statements" as two distinct statements like that. – Garfield

Also, the in statement in the section under question is actually linked to Membership test operations. – Garfield

Answer 3

An iterator is iterable.

That's documented and explained here:

Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable

An iterable is not necessarily an iterator

Iterators must have a __next__ method, by definition. To give a simple counterexample:

>>> ''.__next__
AttributeError: 'str' object has no attribute '__next__'

A string object is an iterable, but is not an iterator.

Garfield

Thanks. "A string object is an iterable, but is not an iterator. " but 'abc'.__iter__ exists and 'abc' is a string object. Do I miss something? – Anticlinal

Iterators return themself when __iter__ is called. But strings return an string_iterator object. So it's iterable but not an iterator (it didn't return itself). – Frazier

@Frazier Is string_iterator object not an iterator? – Anticlinal

string_iterator is an iterator but str isn't (str is just iterable). – Frazier

@Ben because to be an iterator it needs a __next__ method. – Viridescent

@Frazier Thanks . I see. What are those builtin iterables which are not iterators, besides str? – Anticlinal

All builtin containers are iterable but not iterators: list, tuple, dict, set, frozenset, bytearray, range ... But there are some functions that are iterators: map, zip, enumerate, reversed. – Frazier

@Frazier mmmm just to nitpick - the functions aren't iterators, but they return iterators. – Viridescent

@Viridescent What functions? map, etc. are classes. – Frazier

@Frazier fine, callables then. You were the one who said "but there are some functions that are iterators: map etc...". My point is, map isn't an iterator, it is a callable that returns a map-obect, which is an iterator. Similarly, one wouldn't say open is an iterator, but it returns an iterator. – Viridescent

@Viridescent No, they are really classes. Just try: map.__iter__ and map.__next__. I'm not saying open or iter are iterators. Those two are truly factory functions that return iterators. – Frazier

@Frazier oh wow, I had never noticed, but now it makes sense. – Viridescent

@Frazier I like your reverse nit-pick :) However, I think what juanpa was pointing out is that functions that are iterators should have read Built-in functions/classes that return iterators e.g. map.__new__ returns the iterator. – Ow

Well you could say that about any __new__ method. I think, except for reversed, the __new__ methods of these classes aren't used as factory methods (return not-instances) though. – Frazier

Answer 4

Does an iterator always have the __iter__ method?

Yes. All iterators have an __iter__ method that returns itself. From the docs

iterator.__iter__()

Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements.

Bogart