Java Generics: chaining together generic function object

Question

I've been struggling with the following problem. I have a series of function objects, each with it's own input and output types defined via generic type arguments in java. I would like to arrange these in a chain so that raw data is input to the first function, transformed to the into the output type, which is the input type of the next object, and so on. of course this would be trivial to hard-code, but i'd like to have the code be pluggable to new function objects. if i just leave out type arguments (only the final output type), this is how things look:

    public T process() {
        Iterator<Context> it = source.provideData();
        for(Pipe pipe : pipeline) {
            it = pipe.processIterator(it);
        }
        return sink.next(it);
    }

here an iterator over the data is passed between function objects, and context should be Context. is there a way to keep the following kind of pipe pluggable and still maintain type safety?

edit: for clarity, i have a series of function objects, pipes. each takes as input a particular type and outputs another type. (actually an iterators over these types) these will be chained together, eg, Pipe<A,B> -> Pipe<B,C> -> Pipe<C,D> -> ..., so that the output of one pipe is the input type for the next pipe. There is also a source here that outputs an iterator of type A, and a sink that would accept type (the output of the past pipe). does this make things clearer? The question is, because there is critical dependence on the compatibility of input and output types, is there a way to ensure this?

I am starting to think that on insert of the function objects into the pipeline may be the best time to ensure type safety, but i'm not sure how to do this.

edit 2: i have a adder method for the function objects that currently looks like below:

public void addPipe(Pipe<?,?> pipe) {
    pipeline.add(pipe);
}

i'd like to check if the first type parameter is the same as the "end" of the current pipe, and throw an exception if not? i dont think there is a good way to get compile time safety here. the "end" of the current pipe can then be set to the second type param of the input pipe. I can't think of how to do this with generics, and passing around the class information seems pretty hideous.

Hadji

Type erasure may make your life harder for fully generic functions. The JVM byte compiler removes the type from instance of generic classes so List<String> becomes List<Object>. – Erotogenic

Can you explain the requirement little more? Unable to figure out what you have and what you want to have. – Finegan

so would the best approach be to try to ensure type safety when inserting the pipe function object (how??) and just suppress warnings on this above method? – Hadji

tried to clarify. hope that helps. – Hadji

Answer 1

Here's a way to do it. The run method is not typesafe, but given that the only way to append a pipe is to do it in a type-safe way, the whole chain is type-safe.

public class Chain<S, T> {
    private List<Pipe<?, ?>> pipes;

    private Chain() {
    }

    public static <K, L> Chain<K, L> start(Pipe<K, L> pipe) {
        Chain<K, L> chain = new Chain<K, L>();
        chain.pipes = Collections.<Pipe<?, ?>>singletonList(pipe);;
        return chain;
    }

    public <V> Chain<S, V> append(Pipe<T, V> pipe) {
        Chain<S, V> chain = new Chain<S, V>();
        chain.pipes = new ArrayList<Pipe<?, ?>>(pipes);
        chain.pipes.add(pipe);
        return chain;
    }

    @SuppressWarnings({ "rawtypes", "unchecked" })
    public T run(S s) {
        Object source = s;
        Object target = null;
        for (Pipe p : pipes) {
            target = p.transform(source);
            source = target;
        }
        return (T) target;
    }

    public static void main(String[] args) {
        Pipe<String, Integer> pipe1 = new Pipe<String, Integer>() {
            @Override
            public Integer transform(String s) {
                return Integer.valueOf(s);
            }
        };
        Pipe<Integer, Long> pipe2 = new Pipe<Integer, Long>() {
            @Override
            public Long transform(Integer s) {
                return s.longValue();
            }
        };
        Pipe<Long, BigInteger> pipe3 = new Pipe<Long, BigInteger>() {
            @Override
            public BigInteger transform(Long s) {
                return new BigInteger(s.toString());
            }
        };
        Chain<String, BigInteger> chain = Chain.start(pipe1).append(pipe2).append(pipe3);
        BigInteger result = chain.run("12");
        System.out.println(result);
    }
}

Surmise

really cool, and like all good solutions, it's obvious once it is known :) – Hadji

Thanks for your answer :) I create a github project based on your answer that add async functionality and parallel execution. Hope it would help someone. – Ryun

Answer 2

Here's another way to do it: this way allows for a transformation step to result in a list. For example, a transformation could split a string into multiple substrings. Moreover, it allows for common exception handling code if transforming any of the values produces an exception. It also allows the use of an empty List as a return value instead of an ambiguous null value that has to be tested for to avoid NullPointerException. The main problem with this one is that it does each transformation step in its entirety before moving to the next step, which may not be memory efficient.

public class Chain<IN, MEDIAL, OUT> {
    private final Chain<IN, ?, MEDIAL> head;
    private final Transformer<MEDIAL, OUT> tail;

    public static <I, O> Chain<I, I, O> makeHead(@Nonnull Transformer<I, O> tail) {
        return new Chain<>(null, tail);
    }

    public static <I, M, O> Chain<I, M, O> append(@Nonnull Chain<I, ?, M> head, @Nonnull Transformer<M, O> tail) {
        return new Chain<>(head, tail);
    }

    private Chain(@Nullable Chain<IN, ?, MEDIAL> head, @Nonnull Transformer<MEDIAL, OUT> tail) {
        this.head = head;
        this.tail = tail;
    }

    public List<OUT> run(List<IN> input) {
        List<OUT> allResults = new ArrayList<>();

        List<MEDIAL> headResult;
        if (head == null) {
            headResult = (List<MEDIAL>) input;
        } else {
            headResult = head.run(input);
        }

        for (MEDIAL in : headResult) {
            // try/catch here
            allResults.addAll(tail.transform(in));
        }

        return allResults;
    }

    public static void main(String[] args) {

        Transformer<String, Integer> pipe1 = new Transformer<String, Integer>() {
            @Override
            public List<Integer> transform(String s) {
                return Collections.singletonList(Integer.valueOf(s) * 3);
            }
        };
        Transformer<Integer, Long> pipe2 = new Transformer<Integer, Long>() {
            @Override
            public List<Long> transform(Integer s) {
                return Collections.singletonList(s.longValue() * 5);
            }
        };
        Transformer<Long, BigInteger> pipe3 = new Transformer<Long, BigInteger>() {
            @Override
            public List<BigInteger> transform(Long s) {
                return Collections.singletonList(new BigInteger(String.valueOf(s * 7)));
            }
        };
        Chain<String, ?, Integer> chain1 = Chain.makeHead(pipe1);
        Chain<String, Integer, Long> chain2 = Chain.append(chain1, pipe2);
        Chain<String, Long, BigInteger> chain3 = Chain.append(chain2, pipe3);
        List<BigInteger> result = chain3.run(Collections.singletonList("1"));
        System.out.println(result);
    }
}

Vindicable

Obviously, if you're able to use Java 8, then you'd want to use Streams instead. Or, something like RxJava. – Vindicable