hasnext() for Python iterators?

Question

Do Python iterators have a hasnext method?

Margaretamargarete

Related: How do I know if a generator is empty from the start? – Proceleusmatic

There is no such thing as of now. And I think, right way to do this is to add next and has_next method in iterators itself, as these methods should not be defined independently. – Telethermometer

(For context, iterators in Java and C# have ahasNext() as well as Next(). Python obviates this by throwing an exception at generator end. And the next(gen, default_value) idiom allows you to squelch that exception without needing try..except. – Favin

Answer 1

No, there is no such method. The end of iteration is indicated by an exception. See the documentation.

Preglacial

"It's easier to ask for forgiveness than permission." – Santiagosantillan

"It's easier to ask for forgiveness than permission.": Checking whether an iterator has a next element is not asking for permission. There are situations in which you want to test for the existence of a next element without consuming it. I would accept the try catch solution if there was an unnext() method to put the first element back after I have checked that it exists by calling next(). – Odious

@Giorgio, there is no way to know whether another element exists without executing the code that generates it (you don't know whether the generator will execute yield or not). It is, of course, not difficult to write an adaptor that stores the result of next() and provides has_next() and move_next(). – Preglacial

The same idea could be used to implement the hasNext() method (to produce, cache and return true on success, or return false on failure). Then both hasNext() and next() would depend on a common underlying getNext() method and cached item. I really do not see why next() shouldn't be in the standard library if it is so easy to implement an adaptor that provides it. – Odious

@Giorgio, I don't understand what you're saying. next is the standard; it throws on end of sequence. The adaptor could use this next and cache the value. Nobody is saying that it shouldn't be in the standard library; I guess there just was no demand for it. – Preglacial

I understand, I thought the motivation for not including it in the standard was technical. Probably, as you say, there simply was not enough demand for it. – Odious

@Giorgio, I mostly agree with you, although: if hasNext() were in the standard library, all iterators would be forced to implement it (or return an error if they don't support it ... design decisions there). And for some iterators, ones that are highly time-context-sensitive, the value of the first item from the iterator could change between the call to hasNext() and the call to next() (which could be in a different module), with the result that the next() returns a stale value. I don't know, just trying to see both sides of the argument. I agree it would be useful to have. – Tad

@LarsH: You mean e.g. an iterator that reads from a file that can be changed while reading from it? I agree that this can be a problem (which affects any library providing next() and hasNext() method, not just a hypothetical Python library). So yes, next() and hasNext() becomes tricky if the content of the stream being scanned depends on when elements are read. – Odious

@Giorgio: right, that is an example of what I was thinking of. So when Python glob library has iglob() return an iterator, we run into this problem: an iterator whose return values are time-sensitive (i.e. not a pure function). So does that mean impure functions like I/O should not be allowed as iterators? Design decisions... – Tad

You're asking for permission either way. This isn't C ;) I guess there's no has_next() because can make things complicated. – Cryogenics

python often lacks constructs for basic language usage compelling such things as catching exceptions .. and then it becomes somehow justified as more or less the right way to do things. Another one is the lack of do .. while – Pony

Answer 2

The alternative to catching StopIteration is to use next(iterator, default_value).

For example:

>>> a = iter('hi')
>>> print(next(a, None))
h
>>> print(next(a, None))
i
>>> print(next(a, None))
None

This way you can check for None to see if you've reached the end of the iterator if you don't want to do it the exception way.

If your iterable can contain None values you'll have to define a sentinel value and check for it instead:

>>> sentinel = object()
>>> a = iter([None, 1, 2])
>>> elem = next(a, sentinel)
>>> if elem is sentinel:
...     print('end')
... 
>>>

Chignon

if you use None as the "sentinel", you best be sure your iterator doesn't have any Nones. you could also do sentinel = object() and next(iterator, sentinel) and test with is. – Coagulum

following @samboosalis I would rather use built-in unittest.mock.sentinel object which allows you to write an explicit next(a, sentinel.END_OF_ITERATION) and then if next(...) == sentinel.END_OF_ITERATION – Benedict

The problem is that, this way, you CONSUME the next value from the iterator as well. hasNext in Java doesn't consume the next value. – Cleaning

Answer 3

No, but you can implement your own iterable wrapper class that does:

from collections.abc import Iterator

class hn_wrapper(Iterator):
    def __init__(self, it):
        self.it = iter(it)
        self._hasnext = None
    def __iter__(self): 
        return self
    def __next__(self):
        if self._hasnext:
            result = self._thenext
        else:
            result = next(self.it)
        self._hasnext = None
        return result
    def hasnext(self):
        if self._hasnext is None:
            try: 
                self._thenext = next(self.it)
            except StopIteration: 
                self._hasnext = False
            else:
                self._hasnext = True
        return self._hasnext

Then you can use it like this:

x = hn_wrapper('ciao')
while x.hasnext():
    print(next(x))

and it will emit

c
i
a
o

Dyun

It would be a bit more pythonic to implement hasnext() like next(), i.e. renaming the method to hn_wrapper.__hasnext__() and then defining a top-level function that calls that method def hasnext(iterable): return iterable.__hasnext__(). – Thermit

Answer 4

In addition to all the mentions of StopIteration, the Python for loop does what you want:

>>> it = iter('hello')
>>> for i in it:
...     print(i)
...
h
e
l
l
o

Altigraph

Answer 5

Try the __length_hint__() method from any iterator object:

iter(...).__length_hint__() > 0

Carbonari

According to PEP 424: "It may return a value that is either larger or smaller than the actual size of the container". This is therefore a very unreliable way to determine if an iterator is empty. Moreover, there is no guarantee that this method is implemented. – Lesbianism

Answer 6

You can tee the iterator using, itertools.tee, and check for StopIteration on the teed iterator.

Entryway

Answer 7

hasNext somewhat translates to the StopIteration exception, e.g.:

>>> it = iter("hello")
>>> it.next()
'h'
>>> it.next()
'e'
>>> it.next()
'l'
>>> it.next()
'l'
>>> it.next()
'o'
>>> it.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

• StopIteration docs: http://docs.python.org/library/exceptions.html#exceptions.StopIteration
• Some article about iterators and generator in python: http://www.ibm.com/developerworks/library/l-pycon.html

Volitant

Answer 8

No. The most similar concept is most likely a StopIteration exception.

Guyot

What Python uses exceptions for control flow ? Sounds pretty nafted. – Justinn

Right: exceptions should be used to handle errors, not to define the normal flow of control. – Odious

Answer 9

I believe python just has next() and according to the doc, it throws an exception is there are no more elements.

http://docs.python.org/library/stdtypes.html#iterator-types

Ita

Answer 10

The use case that lead me to search for this is the following

def setfrom(self,f):
    """Set from iterable f"""
    fi = iter(f)
    for i in range(self.n):
        try:
            x = next(fi)
        except StopIteration:
            fi = iter(f)
            x = next(fi)
        self.a[i] = x 

where hasnext() is available, one could do

def setfrom(self,f):
    """Set from iterable f"""
    fi = iter(f)
    for i in range(self.n):
        if not hasnext(fi):
            fi = iter(f) # restart
        self.a[i] = next(fi)

which to me is cleaner. Obviously you can work around issues by defining utility classes, but what then happens is you have a proliferation of twenty-odd different almost-equivalent workarounds each with their quirks, and if you wish to reuse code that uses different workarounds, you have to either have multiple near-equivalent in your single application, or go around picking through and rewriting code to use the same approach. The 'do it once and do it well' maxim fails badly.

Furthermore, the iterator itself needs to have an internal 'hasnext' check to run to see if it needs to raise an exception. This internal check is then hidden so that it needs to be tested by trying to get an item, catching the exception and running the handler if thrown. This is unnecessary hiding IMO.

Getaway

For this use case, you can use itertools.cycle – Fluecure

Answer 11

Maybe it's just me, but while I like https://stackoverflow.com/users/95810/alex-martelli 's answer, I find this a bit easier to read:

from collections.abc import Iterator  # since python 3.3 Iterator is here

class MyIterator(Iterator):  # need to subclass Iterator rather than object
  def __init__(self, it):
    self._iter = iter(it)
    self._sentinel = object()
    self._next = next(self._iter, self._sentinel)
    
  def __iter__(self): 
    return self
  
  def __next__(self):        # __next__ vs next in python 2
    if not self.has_next():
      next(self._iter)  # raises StopIteration

    val = self._next
    self._next = next(self._iter, self._sentinel)
    return val
  
  def has_next(self):
    return self._next is not self._sentinel

Phonics

Answer 12

No, there is no such method. The end of iteration is indicated by a StopIteration (more on that here).

---

This follows the python principle EAFP (easier to ask for forgiveness than permission). A has_next method would follow the principle of LBYL (look before you leap) and contradicts this core python principle.

This interesting article explains the two concepts in more detail.

Psyche

Answer 13

It is also possible to implement a helper generator that wraps any iterator and answers question if it has next value:

Try it online!

def has_next(it):
    first = True
    for e in it:
        if not first:
            yield True, prev
        else:
            first = False
        prev = e
    if not first:
        yield False, prev

for has_next_, e in has_next(range(4)):
    print(has_next_, e)

Which outputs:

True 0
True 1
True 2
False 3

The main and probably only drawback of this method is that it reads ahead one more element, for most of tasks it is totally alright, but for some tasks it may be disallowed, especially if user of has_next() is not aware of this read-ahead logic and may missuse it.

Code above works for infinite iterators too.

Actually for all cases that I ever programmed such kind of has_next() was totally enough and didn't cause any problems and in fact was very helpful. You just have to be aware of its read-ahead logic.

Lamarlamarck

Answer 14

The way has solved it based on handling the "StopIteration" execption is pretty straightforward in order to read all iterations :

    end_cursor = False
    while not end_cursor:
        try:
            print(cursor.next())
        except StopIteration:
            print('end loop')
            end_cursor = True
        except:
            print('other exceptions to manage')
            end_cursor = True

Sunbathe

Answer 15

I think there are valid use cases for when you may want some sort of has_next functionality, in which case you should decorate an iterator with a has_next defined.

Combining concepts from the answers to this question here is my implementation of that which feels like a nice concise solution to me (python 3.9):

_EMPTY_BUF = object()


class BufferedIterator(Iterator[_T]):
    def __init__(self, real_it: Iterator[_T]):
        self._real_it = real_it
        self._buf = next(self._real_it, _EMPTY_BUF)

    def has_next(self):
        return self._buf is not _EMPTY_BUF

    def __next__(self) -> _T_co:
        v = self._buf
        self._buf = next(self._real_it, _EMPTY_BUF)
        if v is _EMPTY_BUF:
            raise StopIteration()
        return v

The main difference is that has_next is just a boolean expression, and also handles iterators with None values.

Added this to a gist here with tests and example usage.

Avalokitesvara

Answer 16

There is no has_next in Python, but you can achieve has_next functionality for iterators like this:

class BSTIterator:

    def __init__(self, root: Optional[TreeNode]):
        def inorder_dfs(root):
            if root is None:
                return
            yield from inorder_dfs(root.left)
            yield root.val
            yield from inorder_dfs(root.right)
        self.inorder_generator = inorder_dfs(root)
        self.has_next = None

    def next(self) -> int:
        if self.has_next is not None:
            temp = self.has_next
            self.has_next = None
            return temp
        return next(self.inorder_generator)

    def hasNext(self) -> bool:
        if self.has_next is not None:
            return True
        try:
            self.has_next = next(self.inorder_generator)
            return True
        except StopIteration:
            return False

Icefall

Answer 17

The more-itertools library implements peekable iterators where you can peek into the next value(s) without consuming them.

Check the truth value of a peekable to test if it is exhausted:

p1, p2 = peekable([]), peekable([1, 2])
assert not p1
assert p2

Use peek() (also supports a default value) to return the next item without consuming it:

assert p2.peek() == 1
assert next(p2) == 1

Seine

Answer 18

This will fail in many cases (mostly if the iterator ever intentionally yields False), but can be useful.

gen = iter(...)
thing_of_interest = next(gen)
while thing_of_interest:
    # operations on or pertaining to thing_of_interest
    thing_of_interest = next(gen, False)

I just used:

gen = iter(['a', 'b', 'c', 'd'])
data = ''
newData = next(gen)
while newData:
    data += ' {}'.format(newData)
    newData = next(gen, False)
print(data) 

outputs: 'a b c d'

I had additional surrounding operations such that ' '.join(...) was not appropriate.

Talky

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Nautilus

Answer 19

The way I solved my problem is to keep the count of the number of objects iterated over, so far. I wanted to iterate over a set using calls to an instance method. Since I knew the length of the set, and the number of items counted so far, I effectively had an hasNext method.

A simple version of my code:

class Iterator:
    # s is a string, say
    def __init__(self, s):
        self.s = set(list(s))
        self.done = False
        self.iter = iter(s)
        self.charCount = 0

    def next(self):
        if self.done:
            return None
        self.char = next(self.iter)
        self.charCount += 1
        self.done = (self.charCount < len(self.s))
        return self.char

    def hasMore(self):
        return not self.done

Of course, the example is a toy one, but you get the idea. This won't work in cases where there is no way to get the length of the iterable, like a generator etc.

Slaphappy

Answer 20

very interesting question, but this "hasnext" design had been put into leetcode: https://leetcode.com/problems/iterator-for-combination/

here is my implementation:

class CombinationIterator:

def __init__(self, characters: str, combinationLength: int):
    from itertools import combinations
    from collections import deque
    self.iter = combinations(characters, combinationLength)
    self.res = deque()


def next(self) -> str:
    if len(self.res) == 0:
        return ''.join(next(self.iter))
    else:
        return ''.join(self.res.pop())


def hasNext(self) -> bool:
    try:
        self.res.insert(0, next(self.iter))
        return True
    except:
        return len(self.res) > 0

Hakluyt