Adding elements to a collection during iteration
Asked Answered
E

18

87

Is it possible to add elements to a collection while iterating over it?

More specifically, I would like to iterate over a collection, and if an element satisfies a certain condition I want to add some other elements to the collection, and make sure that these added elements are iterated over as well. (I realise that this could lead to an unterminating loop, but I'm pretty sure it won't in my case.)

The Java Tutorial from Sun suggests this is not possible: "Note that Iterator.remove is the only safe way to modify a collection during iteration; the behavior is unspecified if the underlying collection is modified in any other way while the iteration is in progress."

So if I can't do what I want to do using iterators, what do you suggest I do?

Eighteen answered 14/6, 2009 at 15:32 Comment(0)
S
67

How about building a Queue with the elements you want to iterate over; when you want to add elements, enqueue them at the end of the queue, and keep removing elements until the queue is empty. This is how a breadth-first search usually works.

Svensen answered 14/6, 2009 at 15:35 Comment(8)
This is a good way to do things if it fits the model the OP is coding for. This way you don't use an iterator -- just a while loop. while there are elements in the queue, process the first element. You could do this with a List as well, however.Neology
ListIterator iter = list.listIterator() has both add() and remove() methods, so you can add and remove elements during iterationPelisse
@Pelisse Are you sure about this? If I try to do so I get a ConcurrentModificationException.Zerlina
I think you're correct, but there is another option, use thread-safe collections like LinkedBlockingQueuePelisse
I believe there's still a gap here. For example, if I have a backtracking algorithm, then I won't be able to ( or how ? ) handle it using a Set unless I do need to use siblings of List interface as @Pelisse suggested.Bartholemy
@Pelisse But what if the Collection is a Set?Fain
@RaffiKhatchadourian Set is still a Iterable class, however, Set has only .iterator(), no .listIterator()Pelisse
For those of you (@NiekeAerts) who are getting the ConcurrentModificationException while using the ListIterator: You have to use the remove() and add() methods on the ListIterator, not on the List.Fahland
T
48

There are two issues here:

The first issue is, adding to an Collection after an Iterator is returned. As mentioned, there is no defined behavior when the underlying Collection is modified, as noted in the documentation for Iterator.remove:

... The behavior of an iterator is unspecified if the underlying collection is modified while the iteration is in progress in any way other than by calling this method.

The second issue is, even if an Iterator could be obtained, and then return to the same element the Iterator was at, there is no guarantee about the order of the iteratation, as noted in the Collection.iterator method documentation:

... There are no guarantees concerning the order in which the elements are returned (unless this collection is an instance of some class that provides a guarantee).

For example, let's say we have the list [1, 2, 3, 4].

Let's say 5 was added when the Iterator was at 3, and somehow, we get an Iterator that can resume the iteration from 4. However, there is no guarentee that 5 will come after 4. The iteration order may be [5, 1, 2, 3, 4] -- then the iterator will still miss the element 5.

As there is no guarantee to the behavior, one cannot assume that things will happen in a certain way.

One alternative could be to have a separate Collection to which the newly created elements can be added to, and then iterating over those elements:

Collection<String> list = Arrays.asList(new String[]{"Hello", "World!"});
Collection<String> additionalList = new ArrayList<String>();

for (String s : list) {
    // Found a need to add a new element to iterate over,
    // so add it to another list that will be iterated later:
    additionalList.add(s);
}

for (String s : additionalList) {
    // Iterate over the elements that needs to be iterated over:
    System.out.println(s);
}

Edit

Elaborating on Avi's answer, it is possible to queue up the elements that we want to iterate over into a queue, and remove the elements while the queue has elements. This will allow the "iteration" over the new elements in addition to the original elements.

Let's look at how it would work.

Conceptually, if we have the following elements in the queue:

[1, 2, 3, 4]

And, when we remove 1, we decide to add 42, the queue will be as the following:

[2, 3, 4, 42]

As the queue is a FIFO (first-in, first-out) data structure, this ordering is typical. (As noted in the documentation for the Queue interface, this is not a necessity of a Queue. Take the case of PriorityQueue which orders the elements by their natural ordering, so that's not FIFO.)

The following is an example using a LinkedList (which is a Queue) in order to go through all the elements along with additional elements added during the dequeing. Similar to the example above, the element 42 is added when the element 2 is removed:

Queue<Integer> queue = new LinkedList<Integer>();
queue.add(1);
queue.add(2);
queue.add(3);
queue.add(4);

while (!queue.isEmpty()) {
    Integer i = queue.remove();
    if (i == 2)
        queue.add(42);

    System.out.println(i);
}

The result is the following:

1
2
3
4
42

As hoped, the element 42 which was added when we hit 2 appeared.

Tensimeter answered 14/6, 2009 at 15:49 Comment(3)
I think that Avi's point was that if you have a queue you don't need to iterate over it. You just dequeue elements from the front while it's not empty and enqueue new elements onto the back.Giffie
@Nat: You're right, thank you for pointing that out. I've edited my answer to reflect that.Tensimeter
@Tensimeter For some reason your answer is truncated. [...] in order to go through all the elements along with additional el— and that's all I can see, however if I try and edit the answer everything is there. Any idea on what's going on?Antisepsis
P
9

You may also want to look at some of the more specialised types, like ListIterator, NavigableSet and (if you're interested in maps) NavigableMap.

Pavid answered 14/6, 2009 at 16:20 Comment(0)
T
5

Actually it is rather easy. Just think for the optimal way. I beleive the optimal way is:

for (int i=0; i<list.size(); i++) {
   Level obj = list.get(i);

   //Here execute yr code that may add / or may not add new element(s)
   //...

   i=list.indexOf(obj);
}

The following example works perfectly in the most logical case - when you dont need to iterate the added new elements before the iteration element. About the added elements after the iteration element - there you might want not to iterate them either. In this case you should simply add/or extend yr object with a flag that will mark them not to iterate them.

Trin answered 16/1, 2010 at 22:15 Comment(3)
The indexOf is not required for add and could be confusing if you have duplicates.Serviette
Yes, indeed, duplicates are a problem. Thanx for adding that.Trin
It should be added that, depending on the actual list implementation, list.get(i) can be much more expensive than using an iterator. There might be a considerable performance penalty at least for larger linked lists, e.g.)Madeleinemadelena
O
5

Use ListIterator as follows:

List<String> l = new ArrayList<>();
l.add("Foo");
ListIterator<String> iter = l.listIterator(l.size());
while(iter.hasPrevious()){
    String prev=iter.previous();
    if(true /*You condition here*/){
        iter.add("Bah");
        iter.add("Etc");
    }
}

The key is to iterate in reverse order - then the added elements appear on the next iteration.

Overstudy answered 15/11, 2017 at 16:22 Comment(0)
P
2

I know its been quite old. But thought of its of any use to anyone else. Recently I came across this similar problem where I need a queue that is modifiable during iteration. I used listIterator to implement the same much in the same lines as of what Avi suggested -> Avi's Answer. See if this would suit for your need.

ModifyWhileIterateQueue.java

import java.util.ArrayList;
import java.util.List;
import java.util.ListIterator;

public class ModifyWhileIterateQueue<T> {
        ListIterator<T> listIterator;
        int frontIndex;
        List<T> list;

        public ModifyWhileIterateQueue() {
                frontIndex = 0;
                list =  new ArrayList<T>();
                listIterator = list.listIterator();
        }

        public boolean hasUnservicedItems () {
                return frontIndex < list.size();  
        }

        public T deQueue() {
                if (frontIndex >= list.size()) {
                        return null;
                }
                return list.get(frontIndex++);
        }

        public void enQueue(T t) {
                listIterator.add(t); 
        }

        public List<T> getUnservicedItems() {
                return list.subList(frontIndex, list.size());
        }

        public List<T> getAllItems() {
                return list;
        }
}

ModifyWhileIterateQueueTest.java

    @Test
    public final void testModifyWhileIterate() {
            ModifyWhileIterateQueue<String> queue = new ModifyWhileIterateQueue<String>();
            queue.enQueue("one");
            queue.enQueue("two");
            queue.enQueue("three");

            for (int i=0; i< queue.getAllItems().size(); i++) {
                    if (i==1) {
                            queue.enQueue("four");
                    }
            }

            assertEquals(true, queue.hasUnservicedItems());
            assertEquals ("[one, two, three, four]", ""+ queue.getUnservicedItems());
            assertEquals ("[one, two, three, four]", ""+queue.getAllItems());
            assertEquals("one", queue.deQueue());

    }
Postrider answered 14/10, 2018 at 3:46 Comment(0)
B
1

Using iterators...no, I don't think so. You'll have to hack together something like this:

    Collection< String > collection = new ArrayList< String >( Arrays.asList( "foo", "bar", "baz" ) );
    int i = 0;
    while ( i < collection.size() ) {

        String curItem = collection.toArray( new String[ collection.size() ] )[ i ];
        if ( curItem.equals( "foo" ) ) {
            collection.add( "added-item-1" );
        }
        if ( curItem.equals( "added-item-1" ) ) {
            collection.add( "added-item-2" );
        }

        i++;
    }

    System.out.println( collection );

Which yeilds:
[foo, bar, baz, added-item-1, added-item-2]

Botanical answered 14/6, 2009 at 16:40 Comment(0)
N
1

Besides the solution of using an additional list and calling addAll to insert the new items after the iteration (as e.g. the solution by user Nat), you can also use concurrent collections like the CopyOnWriteArrayList.

The "snapshot" style iterator method uses a reference to the state of the array at the point that the iterator was created. This array never changes during the lifetime of the iterator, so interference is impossible and the iterator is guaranteed not to throw ConcurrentModificationException.

With this special collection (usually used for concurrent access) it is possible to manipulate the underlying list while iterating over it. However, the iterator will not reflect the changes.

Is this better than the other solution? Probably not, I don't know the overhead introduced by the Copy-On-Write approach.

Nelson answered 14/6, 2009 at 21:21 Comment(0)
C
1
public static void main(String[] args)
{
    // This array list simulates source of your candidates for processing
    ArrayList<String> source = new ArrayList<String>();
    // This is the list where you actually keep all unprocessed candidates
    LinkedList<String> list = new LinkedList<String>();

    // Here we add few elements into our simulated source of candidates
    // just to have something to work with
    source.add("first element");
    source.add("second element");
    source.add("third element");
    source.add("fourth element");
    source.add("The Fifth Element"); // aka Milla Jovovich

    // Add first candidate for processing into our main list
    list.addLast(source.get(0));

    // This is just here so we don't have to have helper index variable
    // to go through source elements
    source.remove(0);

    // We will do this until there are no more candidates for processing
    while(!list.isEmpty())
    {
        // This is how we get next element for processing from our list
        // of candidates. Here our candidate is String, in your case it
        // will be whatever you work with.
        String element = list.pollFirst();
        // This is where we process the element, just print it out in this case
        System.out.println(element);

        // This is simulation of process of adding new candidates for processing
        // into our list during this iteration.
        if(source.size() > 0) // When simulated source of candidates dries out, we stop
        {
            // Here you will somehow get your new candidate for processing
            // In this case we just get it from our simulation source of candidates.
            String newCandidate = source.get(0);
            // This is the way to add new elements to your list of candidates for processing
            list.addLast(newCandidate);
            // In this example we add one candidate per while loop iteration and 
            // zero candidates when source list dries out. In real life you may happen
            // to add more than one candidate here:
            // list.addLast(newCandidate2);
            // list.addLast(newCandidate3);
            // etc.

            // This is here so we don't have to use helper index variable for iteration
            // through source.
            source.remove(0);
        }
    }
}
Coldiron answered 2/11, 2011 at 20:33 Comment(0)
F
1

For examle we have two lists:

  public static void main(String[] args) {
        ArrayList a = new ArrayList(Arrays.asList(new String[]{"a1", "a2", "a3","a4", "a5"}));
        ArrayList b = new ArrayList(Arrays.asList(new String[]{"b1", "b2", "b3","b4", "b5"}));
        merge(a, b);
        a.stream().map( x -> x + " ").forEach(System.out::print);
    }
   public static void merge(List a, List b){
        for (Iterator itb = b.iterator(); itb.hasNext(); ){
            for (ListIterator it = a.listIterator() ; it.hasNext() ; ){
                it.next();
                it.add(itb.next());

            }
        }

    }

a1 b1 a2 b2 a3 b3 a4 b4 a5 b5

Foursquare answered 25/11, 2015 at 0:59 Comment(0)
G
0

I prefer to process collections functionally rather than mutate them in place. That avoids this kind of problem altogether, as well as aliasing issues and other tricky sources of bugs.

So, I would implement it like:

List<Thing> expand(List<Thing> inputs) {
    List<Thing> expanded = new ArrayList<Thing>();

    for (Thing thing : inputs) {
        expanded.add(thing);
        if (needsSomeMoreThings(thing)) {
            addMoreThingsTo(expanded);
        }
    }

    return expanded;
}
Giffie answered 14/6, 2009 at 16:52 Comment(0)
K
0

IMHO the safer way would be to create a new collection, to iterate over your given collection, adding each element in the new collection, and adding extra elements as needed in the new collection as well, finally returning the new collection.

Kobe answered 14/6, 2009 at 18:58 Comment(0)
U
0

Given a list List<Object> which you want to iterate over, the easy-peasy way is:

while (!list.isEmpty()){
   Object obj = list.get(0);

   // do whatever you need to
   // possibly list.add(new Object obj1);

   list.remove(0);
}

So, you iterate through a list, always taking the first element and then removing it. This way you can append new elements to the list while iterating.

Unlettered answered 8/2, 2013 at 7:20 Comment(0)
E
0

Forget about iterators, they don't work for adding, only for removing. My answer applies to lists only, so don't punish me for not solving the problem for collections. Stick to the basics:

    List<ZeObj> myList = new ArrayList<ZeObj>();
    // populate the list with whatever
            ........
    int noItems = myList.size();
    for (int i = 0; i < noItems; i++) {
        ZeObj currItem = myList.get(i);
        // when you want to add, simply add the new item at last and
        // increment the stop condition
        if (currItem.asksForMore()) {
            myList.add(new ZeObj());
            noItems++;
        }
    }
Embolden answered 22/7, 2013 at 14:59 Comment(1)
Thanks Stefan. Fixed it.Embolden
G
0

I tired ListIterator but it didn't help my case, where you have to use the list while adding to it. Here's what works for me:

Use LinkedList.

LinkedList<String> l = new LinkedList<String>();
l.addLast("A");

while(!l.isEmpty()){
    String str = l.removeFirst();
    if(/* Condition for adding new element*/)
        l.addLast("<New Element>");
    else
        System.out.println(str);
}

This could give an exception or run into infinite loops. However, as you have mentioned

I'm pretty sure it won't in my case

checking corner cases in such code is your responsibility.

Grimace answered 21/6, 2014 at 18:47 Comment(0)
Z
0

This is what I usually do, with collections like sets:

Set<T> adds = new HashSet<T>, dels = new HashSet<T>;
for ( T e: target )
  if ( <has to be removed> ) dels.add ( e );
  else if ( <has to be added> ) adds.add ( <new element> )

target.removeAll ( dels );
target.addAll ( adds );

This creates some extra-memory (the pointers for intermediate sets, but no duplicated elements happen) and extra-steps (iterating again over changes), however usually that's not a big deal and it might be better than working with an initial collection copy.

Zyrian answered 14/1, 2015 at 11:35 Comment(0)
C
0

Even though we cannot add items to the same list during iteration, we can use Java 8's flatMap, to add new elements to a stream. This can be done on a condition. After this the added item can be processed.

Here is a Java example which shows how to add to the ongoing stream an object depending on a condition which is then processed with a condition:

List<Integer> intList = new ArrayList<>();
intList.add(1);
intList.add(2);
intList.add(3);

intList = intList.stream().flatMap(i -> {
    if (i == 2) return Stream.of(i, i * 10); // condition for adding the extra items
    return Stream.of(i);
}).map(i -> i + 1)
        .collect(Collectors.toList());

System.out.println(intList);

The output of the toy example is:

[2, 3, 21, 4]

Celeski answered 21/8, 2017 at 16:24 Comment(0)
H
-1

In general, it's not safe, though for some collections it may be. The obvious alternative is to use some kind of for loop. But you didn't say what collection you're using, so that may or may not be possible.

Hyperbola answered 14/6, 2009 at 15:36 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.