Iteratively compute the Cartesian product of an arbitrary number of sets

Question

I want to compute the cartesian product of an arbitrary number of nonempty sets in Java.

I've wrote that iterative code...

public static <T> List<Set<T>> cartesianProduct(List<Set<T>> list) {
    List<Iterator<T>> iterators = new ArrayList<Iterator<T>>(list.size());
    List<T> elements = new ArrayList<T>(list.size());
    List<Set<T>> toRet = new ArrayList<Set<T>>();
    for (int i = 0; i < list.size(); i++) {
        iterators.add(list.get(i).iterator());
        elements.add(iterators.get(i).next());
    }
    for (int j = 1; j >= 0;) {
        toRet.add(Sets.newHashSet(elements));
        for (j = iterators.size()-1; j >= 0 && !iterators.get(j).hasNext(); j--) {
            iterators.set(j, list.get(j).iterator());
            elements.set(j, iterators.get(j).next());
        }
        elements.set(Math.abs(j), iterators.get(Math.abs(j)).next());
    }
    return toRet;
}

...but I found it rather inelegant. Someone has a better, still iterative solution? A solution that uses some wonderful functional-like approach? Otherwise... suggestion about how to improve it? Errors?

Mapp

Answer 1

I've written a solution that doesn't require you to fill up a large collection in memory. Unfortunately, the code required is hundreds of lines long. You may have to wait until it appears in the Guava project (https://github.com/google/guava), which I hope will be by the end of the year. Sorry. :(

Note that you may not need such a utility if the number of sets you're cartesian-producting is a fixed number known at compile time -- you could just use that number of nested for loops.

EDIT: the code is released now.

Sets.cartesianProduct()

I think you'll be very happy with it. It only creates the individual lists as you ask for them; doesn't fill up memory with all MxNxPxQ of them.

If you want to inspect the source, it's here.

Enjoy!

Comet

What is the reason to implement this just for sets, and not generally for Iterables (i.e. given a list of Iterables, return an Iterable of lists)? Of course for Sets you can do a bit more like easy checking for contains, but I just needed this when I had no sets available (and had to implement it myself). – Boneset

@PaŭloEbermann sorry I never saw this, but there's been a List version of the method sitting next to that one all this time (in the Lists class). – Comet

Answer 2

Using Google Guava 19 and Java 8 is very simple:

Say you have the List of all arrays you want to associate...

public static void main(String[] args) {
  List<String[]> elements = Arrays.asList(
    new String[]{"John", "Mary"}, 
    new String[]{"Eats", "Works", "Plays"},
    new String[]{"Food", "Computer", "Guitar"}
  );

  // Create a list of immutableLists of strings
  List<ImmutableList<String>> immutableElements = makeListofImmutable(elements);

  // Use Guava's Lists.cartesianProduct, since Guava 19
  List<List<String>> cartesianProduct = Lists.cartesianProduct(immutableElements);

  System.out.println(cartesianProduct);
}

The method to make the list of immutable lists is as follows:

/**
 * @param values the list of all profiles provided by the client in matrix.json
 * @return the list of ImmutableList to compute the Cartesian product of values
 */
private static List<ImmutableList<String>> makeListofImmutable(List<String[]> values) {
  List<ImmutableList<String>> converted = new LinkedList<>();
  values.forEach(array -> {
    converted.add(ImmutableList.copyOf(array));
  });
  return converted;
}

The output is as follows:

[
  [John, Eats, Food], [John, Eats, Computer], [John, Eats, Guitar],
  [John, Works, Food], [John, Works, Computer], [John, Works, Guitar], 
  [John, Plays, Food], [John, Plays, Computer], [John, Plays, Guitar],
  [Mary, Eats, Food], [Mary, Eats, Computer], [Mary, Eats, Guitar],
  [Mary, Works, Food], [Mary, Works, Computer], [Mary, Works, Guitar],
  [Mary, Plays, Food], [Mary, Plays, Computer], [Mary, Plays, Guitar]
]

Triceps

Answer 3

The following answer uses iteration and not recursion. It uses the same Tuple class from my previous answer.

It is a separate answer because IMHO both are valid, different approaches.

Here is the new main class:

public class Example {
    public static <T> List<Tuple<T>> cartesianProduct(List<Set<T>> sets) {
        List<Tuple<T>> tuples = new ArrayList<Tuple<T>>();
        for (Set<T> set : sets) {
            if (tuples.isEmpty()) {
                for (T t : set) {
                    Tuple<T> tuple = new Tuple<T>();
                    tuple.add(t);
                    tuples.add(tuple);
                }
            } else {
                List<Tuple<T>> newTuples = new ArrayList<Tuple<T>>();
                for (Tuple<T> subTuple : tuples) {
                    for (T t : set) {
                        Tuple<T> tuple = new Tuple<T>();
                        tuple.addAll(subTuple);
                        tuple.add(t);
                        newTuples.add(tuple);
                    }
                }
                tuples = newTuples;
            }
        }
        return tuples;
    }
}

Yap

Interesting and clean approach, but I've some doubts about memory consuption with all those intermediate tuples lost "in the time, like tears in rain" :P – Mapp

Agreed, the performance could be wretched. I guess you are really asking for an algorithm rather than a coding style? – Yap

Answer 4

Here's an iterative, lazy implementation I wrote. The interface is very similar to Google's Sets.cartesianProduct, but it's a bit more flexible: it deals in Iterables instead of Sets. This code and its unit tests are at https://gist.github.com/1911614.

/* Copyright 2012 LinkedIn Corp.

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
 */

import com.google.common.base.Function;
import com.google.common.collect.Iterables;
import java.lang.reflect.Array;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.NoSuchElementException;

/**
 * Implements the Cartesian product of ordered collections.
 * 
 * @author <a href="mailto:[email protected]">John Kristian</a>
 */
public class Cartesian {
  /**
   * Generate the <a href="http://en.wikipedia.org/wiki/Cartesian_product">Cartesian
   * product</a> of the given axes. For axes [[a1, a2 ...], [b1, b2 ...], [c1, c2 ...]
   * ...] the product is [[a1, b1, c1 ...] ... [a1, b1, c2 ...] ... [a1, b2, c1 ...] ...
   * [aN, bN, cN ...]]. In other words, the results are generated in same order as these
   * nested loops:
   * 
   * <pre>
   * for (T a : [a1, a2 ...])
   *   for (T b : [b1, b2 ...])
   *     for (T c : [c1, c2 ...])
   *       ...
   *         result = new T[]{ a, b, c ... };
   * </pre>
   * 
   * Each result is a new array of T, whose elements refer to the elements of the axes. If
   * you prefer a List, you can call asLists(product(axes)).
   * <p>
   * Don't change the axes while iterating over their product, as a rule. Changes to an
   * axis can affect the product or cause iteration to fail (which is usually bad). To
   * prevent this, you can pass clones of your axes to this method.
   * <p>
   * The implementation is lazy. This method iterates over the axes, and returns an
   * Iterable that contains a reference to each axis. Iterating over the product causes
   * iteration over each axis. Methods of each axis are called as late as practical.
   */
  public static <T> Iterable<T[]> product(Class<T> resultType,
                                          Iterable<? extends Iterable<? extends T>> axes) {
    return new Product<T>(resultType, newArray(Iterable.class, axes));
  }

  /** Works like product(resultType, Arrays.asList(axes)), but slightly more efficient. */
  public static <T> Iterable<T[]> product(Class<T> resultType, Iterable<? extends T>... axes) {
    return new Product<T>(resultType, axes.clone());
  }

  /**
   * Wrap the given arrays in fixed-size lists. Changes to the lists write through to the
   * arrays.
   */
  public static <T> Iterable<List<T>> asLists(Iterable<? extends T[]> arrays) {
    return Iterables.transform(arrays, new AsList<T>());
  }

  /**
   * Arrays.asList, represented as a Function (as used in Google collections).
   */
  public static class AsList<T> implements Function<T[], List<T>> {
    @Override
    public List<T> apply(T[] array) {
      return Arrays.asList(array);
    }
  }

  /** Create a generic array containing references to the given objects. */
  private static <T> T[] newArray(Class<? super T> elementType, Iterable<? extends T> from) {
    List<T> list = new ArrayList<T>();
    for (T f : from)
      list.add(f);
    return list.toArray(newArray(elementType, list.size()));
  }

  /** Create a generic array. */
  @SuppressWarnings("unchecked")
  private static <T> T[] newArray(Class<? super T> elementType, int length) {
    return (T[]) Array.newInstance(elementType, length);
  }

  private static class Product<T> implements Iterable<T[]> {
    private final Class<T> _resultType;
    private final Iterable<? extends T>[] _axes;

    /** Caution: the given array of axes is contained by reference, not cloned. */
    Product(Class<T> resultType, Iterable<? extends T>[] axes) {
      _resultType = resultType;
      _axes = axes;
    }

    @Override
    public Iterator<T[]> iterator() {
      if (_axes.length <= 0) // an edge case
        return Collections.singleton(newArray(_resultType, 0)).iterator();
      return new ProductIterator<T>(_resultType, _axes);
    }

    @Override
    public String toString() {
      return "Cartesian.product(" + Arrays.toString(_axes) + ")";
    }

    private static class ProductIterator<T> implements Iterator<T[]> {
      private final Iterable<? extends T>[] _axes;
      private final Iterator<? extends T>[] _iterators; // one per axis
      private final T[] _result; // a copy of the last result
      /**
       * The minimum index such that this.next() will return an array that contains
       * _iterators[index].next(). There are some special sentinel values: NEW means this
       * is a freshly constructed iterator, DONE means all combinations have been
       * exhausted (so this.hasNext() == false) and _iterators.length means the value is
       * unknown (to be determined by this.hasNext).
       */
      private int _nextIndex = NEW;
      private static final int NEW = -2;
      private static final int DONE = -1;

      /** Caution: the given array of axes is contained by reference, not cloned. */
      ProductIterator(Class<T> resultType, Iterable<? extends T>[] axes) {
        _axes = axes;
        _iterators = Cartesian.<Iterator<? extends T>> newArray(Iterator.class, _axes.length);
        for (int a = 0; a < _axes.length; ++a) {
          _iterators[a] = axes[a].iterator();
        }
        _result = newArray(resultType, _iterators.length);
      }

      private void close() {
        _nextIndex = DONE;
        // Release references, to encourage garbage collection:
        Arrays.fill(_iterators, null);
        Arrays.fill(_result, null);
      }

      @Override
      public boolean hasNext() {
        if (_nextIndex == NEW) { // This is the first call to hasNext().
          _nextIndex = 0; // start here
          for (Iterator<? extends T> iter : _iterators) {
            if (!iter.hasNext()) {
              close(); // no combinations
              break;
            }
          }
        } else if (_nextIndex >= _iterators.length) {
          // This is the first call to hasNext() after next() returned a result.
          // Determine the _nextIndex to be used by next():
          for (_nextIndex = _iterators.length - 1; _nextIndex >= 0; --_nextIndex) {
            Iterator<? extends T> iter = _iterators[_nextIndex];
            if (iter.hasNext()) {
              break; // start here
            }
            if (_nextIndex == 0) { // All combinations have been generated.
              close();
              break;
            }
            // Repeat this axis, with the next value from the previous axis.
            iter = _axes[_nextIndex].iterator();
            _iterators[_nextIndex] = iter;
            if (!iter.hasNext()) { // Oops; this axis can't be repeated.
              close(); // no more combinations
              break;
            }
          }
        }
        return _nextIndex >= 0;
      }

      @Override
      public T[] next() {
        if (!hasNext())
          throw new NoSuchElementException("!hasNext");
        for (; _nextIndex < _iterators.length; ++_nextIndex) {
          _result[_nextIndex] = _iterators[_nextIndex].next();
        }
        return _result.clone();
      }

      @Override
      public void remove() {
        for (Iterator<? extends T> iter : _iterators) {
          iter.remove();
        }
      }

      @Override
      public String toString() {
        return "Cartesian.product(" + Arrays.toString(_axes) + ").iterator()";
      }
    }
  }
}

Robison

Answer 5

Index-based solution

Working with the indices is a simple alternative that is fast and memory-efficient and can handle any number of sets. Implementing Iterable allows easy use in a for-each loop. See the #main method for a usage example.

public class CartesianProduct implements Iterable<int[]>, Iterator<int[]> {
    private final int[] _lengths;
    private final int[] _indices;
    private boolean _hasNext = true;

    public CartesianProduct(int[] lengths) {
        _lengths = lengths;
        _indices = new int[lengths.length];
    }

    public boolean hasNext() {
        return _hasNext;
    }

    public int[] next() {
        int[] result = Arrays.copyOf(_indices, _indices.length);
        for (int i = _indices.length - 1; i >= 0; i--) {
            if (_indices[i] == _lengths[i] - 1) {
                _indices[i] = 0;
                if (i == 0) {
                    _hasNext = false;
                }
            } else {
                _indices[i]++;
                break;
            }
        }
        return result;
    }

    public Iterator<int[]> iterator() {
        return this;
    }

    public void remove() {
        throw new UnsupportedOperationException();
    }

    /**
     * Usage example. Prints out
     *
     * <pre>
     * [0, 0, 0] a, NANOSECONDS, 1
     * [0, 0, 1] a, NANOSECONDS, 2
     * [0, 0, 2] a, NANOSECONDS, 3
     * [0, 0, 3] a, NANOSECONDS, 4
     * [0, 1, 0] a, MICROSECONDS, 1
     * [0, 1, 1] a, MICROSECONDS, 2
     * [0, 1, 2] a, MICROSECONDS, 3
     * [0, 1, 3] a, MICROSECONDS, 4
     * [0, 2, 0] a, MILLISECONDS, 1
     * [0, 2, 1] a, MILLISECONDS, 2
     * [0, 2, 2] a, MILLISECONDS, 3
     * [0, 2, 3] a, MILLISECONDS, 4
     * [0, 3, 0] a, SECONDS, 1
     * [0, 3, 1] a, SECONDS, 2
     * [0, 3, 2] a, SECONDS, 3
     * [0, 3, 3] a, SECONDS, 4
     * [0, 4, 0] a, MINUTES, 1
     * [0, 4, 1] a, MINUTES, 2
     * ...
     * </pre>
     */
    public static void main(String[] args) {
        String[] list1 = {"a", "b", "c",};
        TimeUnit[] list2 = TimeUnit.values();
        int[] list3 = new int[]{1, 2, 3, 4};

        int[] lengths = new int[]{list1.length, list2.length, list3.length};
        for (int[] indices : new CartesianProduct(lengths)) {
            System.out.println(Arrays.toString(indices) //
                    + " " + list1[indices[0]] //
                    + ", " + list2[indices[1]] //
                    + ", " + list3[indices[2]]);
        }
    }
}

Prodigal

Huh, this breaks if you try to iterate over this object twice. – Boneset

Answer 6

Here is a lazy iterator approach that uses a function to produce an appropriate output type.

public static <T> Iterable<T> cartesianProduct(
        final Function<Object[], T> fn, Object[]... options) {
    final Object[][] opts = new Object[options.length][];
    for (int i = opts.length; --i >= 0; ) {
        // NPE on null input collections, and handle the empty output case here
        // since the iterator code below assumes that it is not exhausted the
        // first time through fetch.
        if (options[i].length == 0) {
            return Collections.emptySet();
        }
        opts[i] = options[i].clone();
    }
    return new Iterable<T>() {
        public Iterator<T> iterator() {
            return new Iterator<T>() {
                final int[] pos = new int[opts.length];
                boolean hasPending;
                T pending;
                boolean exhausted;

                public boolean hasNext() {
                    fetch();
                    return hasPending;
                }

                public T next() {
                    fetch();
                    if (!hasPending) {
                        throw new NoSuchElementException();
                    }
                    T out = pending;
                    pending = null;  // release for GC
                    hasPending = false;
                    return out;
                }

                public void remove() {
                    throw new UnsupportedOperationException();
                }

                private void fetch() {
                    if (hasPending || exhausted) {
                        return;
                    }
                    // Produce a result.
                    int n = pos.length;
                    Object[] args = new Object[n];
                    for (int j = n; --j >= 0; ) {
                        args[j] = opts[j][pos[j]];
                    }
                    pending = fn.apply(args);
                    hasPending = true;
                    // Increment to next.
                    for (int i = n; --i >= 0; ) {
                        if (++pos[i] < opts[i].length) {
                            for (int j = n; --j > i; ) {
                                pos[j] = 0;
                            }
                            return;
                        }
                    }
                    exhausted = true;
                }
            };
        }
    };
}

Kinesics

Answer 7

I wrote an recursive cartesian product algorithm for table of Strings. You can modify it to have sets istead. Below is the algorithm. It's also explained in my article

public class Main {
    public static void main(String[] args) {
        String[] A = new String[]{"a1", "a2", "a3"};
        String[] B = new String[]{"b1", "b2", "b3"};
        String[] C = new String[]{"c1"};

        String[] cp = CartesianProduct(0, A, B, C);

        for (String s : cp) {
            System.out.println(s);
        }
    }

    public static String[] CartesianProduct(int prodLevel, String[] res, String[]... s) {
        if (prodLevel < s.length) {
            int cProdLen = res.length * s[prodLevel].length;
            String[] tmpRes = new String[cProdLen];

            for (int i = 0; i < res.length; i++) {
                for (int j = 0; j < s[prodLevel].length; j++) {
                    tmpRes[i * res.length + j] = res[i] + s[prodLevel][j];
                }
            }
            res = Main.CartesianProduct(prodLevel + 1, tmpRes, s);
        }
        return res;
    }
}

Abdicate

Answer 8

You can use Stream.reduce method.

Java 9 without additional libraries.

public static <U> List<Set<U>> cartesianProduct(List<Set<? extends U>> sets) {
    // incorrect incoming data
    if (sets == null) return Collections.emptyList();
    return sets.stream()
            // non-null and non-empty sets
            .filter(set -> set != null && set.size() > 0)
            // represent each set element as Set<U>
            .map(set -> set.stream().map(Set::<U>of)
                    // Stream<List<Set<U>>>
                    .collect(Collectors.toList()))
            // summation of pairs of inner sets
            .reduce((set1, set2) -> set1.stream()
                    // combinations of inner sets
                    .flatMap(inner1 -> set2.stream()
                            // merge two inner sets into one
                            .map(inner2 -> Stream.of(inner1, inner2)
                                    .flatMap(Set::stream)
                                    .collect(Collectors.toSet())))
                    // list of combinations
                    .collect(Collectors.toList()))
            // List<Set<U>>
            .orElse(Collections.emptyList());
}

public static void main(String[] args) {
    Set<Integer> set1 = Set.of(1, 2);
    Set<Double> set2 = Set.of(3.0, 4.0);
    Set<Long> set3 = Set.of(5L, 6L);

    List<Set<Number>> sets = cartesianProduct(List.of(set1, set2, set3));
    // output
    sets.forEach(System.out::println);
}

Output (the order of the elements may differ):

[1, 3.0, 5]
[1, 3.0, 6]
[1, 4.0, 5]
[1, 4.0, 6]
[2, 3.0, 5]
[2, 3.0, 6]
[2, 4.0, 5]
[2, 4.0, 6]

---

^{See also: Cartesian product of an arbitrary number of sets}

Flews

Answer 9

You might be interested in Another question about cartesian products (edit: removed to conserve hyperlinks, search for the tag cartesian products). That answer has a nice recursive solution that I'd be hard pressed to improve on. Do you specifically want an iterative solution instead of recursive solution?

---

After looking at another iterative solution on stack overflow in perl and a clean explanation, here is another solution:

public static <T> List<Set<T>> uglyCartesianProduct(List<Set<T>> list) {
    List<Iterator<T>> iterators = new ArrayList<Iterator<T>>(list.size());
    List<T> elements = new ArrayList<T>(list.size());
    List<Set<T>> toRet = new ArrayList<Set<T>>();

    for (int i = 0; i < list.size(); i++) {
        iterators.add(list.get(i).iterator());
        elements.add(iterators.get(i).next());
    }

    for (int i = 0; i < numberOfTuples(list); i++) {
        toRet.add(new HashSet<T>());
    }

    int setIndex = 0;
    for (Set<T> set : list) {
        int index = 0;
        for (int i = 0; i < numberOfTuples(list); i++) {
            toRet.get(index).add((T) set.toArray()[index % set.size()]);
            index++;
        }
        setIndex++;
    }
    return toRet;
}

private static <T> int numberOfTuples(List<Set<T>> list) {
    int product = 1;
    for (Set<T> set : list) {
        product *= set.size();
    }
    return product;
}

Guacin

Answer 10

I believe this is correct. It is not seeking efficiency, but a clean style through recursion and abstraction.

The key abstraction is to introduce a simple Tuple class. This helps the generics later:

class Tuple<T> {
    private List<T> list = new ArrayList<T>();

    public void add(T t) { list.add(t); }

    public void addAll(Tuple<T> subT) {
        for (T t : subT.list) {
            list.add(t);
        }
    }

    public String toString() {
        String result = "(";

        for (T t : list) { result += t + ", "; }

        result = result.substring(0, result.length() - 2);
        result += " )";

        return result;
    }
}

With this class, we can write a class like so:

public class Example {
    public static <T> List<Tuple<T>> cartesianProduct(List<Set<T>> sets) {
        List<Tuple<T>> tuples = new ArrayList<Tuple<T>>();

        if (sets.size() == 1) {
            Set<T> set = sets.get(0);
            for (T t : set) {
                Tuple<T> tuple = new Tuple<T>();
                tuple.add(t);
                tuples.add(tuple);
            }
        } else {
            Set<T> set = sets.remove(0);
            List<Tuple<T>> subTuples = cartesianProduct(sets);
            System.out.println("TRACER size = " + tuples.size());
            for (Tuple<T> subTuple : subTuples) {
                for (T t : set) {
                    Tuple<T> tuple = new Tuple<T>();
                    tuple.addAll(subTuple);
                    tuple.add(t);
                    tuples.add(tuple);
                }
            }
        }
        return tuples;
    }
}

I have a decent example of this working, but it is omitted for brevity.

Yap