How to convert an Iterator to a Stream?

Question

I am looking for a concise way to convert an Iterator to a Stream or more specifically to "view" the iterator as a stream.

For performance reason, I would like to avoid a copy of the iterator in a new list:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Collection<String> copyList = new ArrayList<String>();
sourceIterator.forEachRemaining(copyList::add);
Stream<String> targetStream = copyList.stream();

---

Based on the some suggestions in the comments, I have also tried to use Stream.generate:

public static void main(String[] args) throws Exception {
    Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
    Stream<String> targetStream = Stream.generate(sourceIterator::next);
    targetStream.forEach(System.out::println);
}

However, I get a NoSuchElementException (since there is no invocation of hasNext)

Exception in thread "main" java.util.NoSuchElementException
    at java.util.AbstractList$Itr.next(AbstractList.java:364)
    at Main$$Lambda$1/1175962212.get(Unknown Source)
    at java.util.stream.StreamSpliterators$InfiniteSupplyingSpliterator$OfRef.tryAdvance(StreamSpliterators.java:1351)
    at java.util.Spliterator.forEachRemaining(Spliterator.java:326)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at Main.main(Main.java:20)

I have looked at StreamSupport and Collections but I didn't find anything.

Copartner

possible duplicate of How to create an infinite Stream<E> out of an Iterator<E>? – Pariah

@DmitryGinzburg euh i don't want to create an "Infinite" Stream. – Copartner

@Copartner that wouldn't matter in the case. – Pariah

@DmitryGinzburg Stream.generate(iterator::next) works ? – Copartner

@DmitryGinzburg That won't work for a finite iterator. – Uric

See #23114515 – Sedlik

Answer 1

One way is to create a Spliterator from the Iterator and use that as a basis for your stream:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Stream<String> targetStream = StreamSupport.stream(
          Spliterators.spliteratorUnknownSize(sourceIterator, Spliterator.ORDERED),
          false);

An alternative which is maybe more readable is to use an Iterable - and creating an Iterable from an Iterator is very easy with lambdas because Iterable is a functional interface:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();

Iterable<String> iterable = () -> sourceIterator;
Stream<String> targetStream = StreamSupport.stream(iterable.spliterator(), false);

Uric

Thanks, is it really a view of the iterator or a copy ? – Copartner

Stream are lazy: the code is only linking the Stream to the Iterator but the actual iteration won't happen until a terminal operation. If you use the iterator in the meantime you won't get the expected result. For example you can introduce a sourceIterator.next() before using the stream and you will see the effect (the first item will not be seen by the Stream). – Uric

Would Spliterator.ORDERED also be a valid charasteric for any iterator? – Alicealicea

@Alicealicea yes you are right - and actually SIZED is removed by the spliteratorUnknownSize method. I have amended. – Uric

@assylias, yeah it is really nice ! Maybe you could explains for future readers this quite magic line Iterable<String> iterable = () -> sourceIterator;. I have to admit that it took me some time to understand. – Copartner

@Copartner The line is still magical to me. Can you tell me what you found? Is it a Supplier<Iterable<String>>? – Weinreb

I should tell what I found. Iterable<T> is a FunctionalInterface which has only one abstract method iterator(). So () -> sourceIterator is a lambda expression instantiating an Iterable instance as an anonymous implementation. – Weinreb

Again, () -> sourceIterator; is a shortened form of new Iterable<>() { @Override public Iterator<String> iterator() { return sourceIterator; } } – Weinreb

@JinKwon It is not really a shortened form of an anonymous class (there are a few subtle differences, such as scope and how it is compiled) but it does behave similarly in this case. – Uric

The tricky bit with () -> sourceIterator is that if it's called more than once, it'll return the same iterator each time. This leads to unexpected results. So you're depending on the implementation of StreamSupport.stream() and spliterator(_, false) here. – Milliemillieme

@Milliemillieme the stream could be use only once, so it is not a problem. – Copartner

To prevent the problems above with reusing smae iterator, use the following: Iterable<String> iterable = Arrays.asList("A", "B", "C")::iterator; Stream<String> targetStream = StreamSupport.stream(iterable.spliterator(), false); – Ewing

@Stephan I don't think it works (it will throw an exception at the end of the iterator). – Uric

@Uric I tried with an ArrayList iterator and it worked. However I passed Arraylist.size() method to limit() method. – Dravidian

@Dravidian if you receive the iterator from a collection it works indeed, but you could have an iterator with an unknown​ size, for example a database cursor like in https://mcmap.net/q/65297/-java-util-stream-with-resultset or https://mcmap.net/q/65297/-java-util-stream-with-resultset – Uric

Lists.newArrayList(your iterator here).stream() – Partin

What happens when you go parallel in the stream? Must the iterator not be threadsafe to avoid concurrency issues? – Remind

@Remind You should check Spliterator's javadoc. In particular "Despite their obvious utility in parallel algorithms, spliterators are not expected to be thread-safe; instead, implementations of parallel algorithms using spliterators should ensure that the spliterator is only used by one thread at a time." – Uric

Seems like Java features get "part way there" and then just stop. This is such an obvious omission from the Stream API! – Multipara

@Uric guava streams implementation does not set the Spliterator.ORDERED flag. Would this make any difference for the specific solution? – Prance

@MarinosAn in my example the source is a list so ORDERED makes sense, but I guess that if the source if a set for example, it doesn't. – Uric

@JoshM. for whatever it's worth, they did it deliberately: #23114515 – Hexone

Answer 2

Since version 21, Guava library provides Streams.stream(iterator)

It does what @assylias's answer shows.

Verify

Javadoc: static.javadoc.io/com.google.guava/guava/21.0/com/google/common/… – Delija

Far better to use this consistently until such time as the JDK supports a native one-liner. It will be far simpler to find (hence refactor) this in future than the pure-JDK solutions shown elsewhere. – Putnam

This is excellent but… how does Java have native iterators and streams… but no built-in, straightforward way to go from one to the other!? Quite an omission in my opinion. – Redeemable

Answer 3

Great suggestion! Here's my reusable take on it:

public class StreamUtils {

    public static <T> Stream<T> asStream(Iterator<T> sourceIterator) {
        return asStream(sourceIterator, false);
    }

    public static <T> Stream<T> asStream(Iterator<T> sourceIterator, boolean parallel) {
        Iterable<T> iterable = () -> sourceIterator;
        return StreamSupport.stream(iterable.spliterator(), parallel);
    }
}

And usage (make sure to statically import asStream):

List<String> aPrefixedStrings = asStream(sourceIterator)
                .filter(t -> t.startsWith("A"))
                .collect(toList());

Lithophyte

Answer 4

This is possible in Java 9.

Stream.generate(() -> null)
    .takeWhile(x -> iterator.hasNext())
    .map(n -> iterator.next())
    .forEach(System.out::println);

Belshazzar

Simple, efficient and without resorting to subclassing - this should be the accepted answer! – Camm

Unfortunately these don't seem to work with .parallel() streams. They also appear a bit slower than going over Spliterator, even for sequential use. – Paleoecology

Also, the first method throws up if the iterator is empty. The second method does work for now, but it does violate the requirement of the functions in map and takeWhile to be stateless, so I'd hesitate to do that in production code. – Anachronous

Really, this should be an accepted answer. Even though parallel might be funky the simpleness is amazing. – Uella

You really shouldn't be working with side effects and mutation. – Thersathersites

While takeWhile() is generally a cheap operation on sequential stream pipelines, it can be quite expensive on ordered parallel pipelines. Reference: docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/… – Mccormick

Answer 5

import com.google.common.collect.Streams;

and use Streams.stream(iterator) :

Streams.stream(iterator)
       .map(v-> function(v))
       .collect(Collectors.toList());

Meingolda

Answer 6

Create Spliterator from Iterator using Spliterators class contains more than one function for creating spliterator, for example here am using spliteratorUnknownSize which is getting iterator as parameter, then create Stream using StreamSupport

Spliterator<Model> spliterator = Spliterators.spliteratorUnknownSize(
        iterator, Spliterator.NONNULL);
Stream<Model> stream = StreamSupport.stream(spliterator, false);

Lorylose

Answer 7

If you are using Spring (Data) then there is a utility class StreamUtils which has two methods:

createStreamFromIterator(Iterator<T> iterator) 

createStreamFromIterator(CloseableIterator<T> iterator)

https://docs.spring.io/spring-data/commons/docs/3.0.0/api/org/springframework/data/util/StreamUtils.html#createStreamFromIterator(java.util.Iterator)

Georgetown

Answer 8

1 assylias's solution wrapped in a method:

public static <T> Stream<T> toStream(Iterator<T> iterator) {
    return StreamSupport.stream(((Iterable<T>)() -> iterator).spliterator(), false);
}

2 guava Streams implementation (marked with @Beta):

public static <T> Stream<T> stream(Iterator<T> iterator) {
    return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, 0), false);
}

Prance

Answer 9

I think that one answer using Apache Commons Lang is missing.

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Stream<String> targetStream = org.apache.commons.lang3.stream.Streams.of(sourceIterator);

This works since Apache Commons Lang 3.13.0 (July 2023).

Entourage

Answer 10

if iteration size it's known this is possible:

public static void main(String[] args) throws Exception {
    Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
    Stream<String> targetStream = Stream.generate(sourceIterator::next);
    targetStream.**limit(3)**.forEach(System.out::println);
}

Giule

Answer 11

Stream.iterate does the job for you. Simply iterate while it has elements and mapped them to next.

public <X> Stream<X> toStream(Iterator<X> iter) {
    return Stream.iterate(iter, Iterator::hasNext, UnaryOperator.identity()).map(Iterator::next);
}

Acth

Answer 12

Use Collections.list(iterator).stream()...

Charleen

While short, this is very underperforming. – Dumbstruck

This will unfold the whole iterator to java object then convert it to stream. I dont suggest it – Milieu

This seems to be only for enumerations, not iterators. – Gondola

Not a terrible answer in general, useful in a pinch, but the question mentions performance and the answer is not performant. – Prolix

Iterators and Streams are both lazy. Collecting the whole thing into a list before processing begins seems to miss the whole point. – Opener