Composing Enumeratees in Enumerator

Question

Disclaimer: this was asked recently on the haskell-cafe list. My apologies to anyone bothered by the double post.

All of the iteratee-implementing packages that I know of (e.g. iteratee, iterIO, and conduit) define an enumeratee composition function, except for the enumerator package. This seems to me like a serious limitation, and yet it also seems relatively straightforward to implement:

import Data.Enumerator
import Data.Enumerator.Internal

(=$=) :: Monad m
      => Enumeratee a0 a1 m (Step a2 m b) -> Enumeratee a1 a2 m b
      -> Enumeratee a0 a2 m b
(=$=) e01 e12 step = Iteratee $ do
    step' <- runIteratee $ e12 step
    runIteratee . joinI $ e01 step'

Is there some gotcha here that I'm missing? Or some other reason for enumerator not to define enumeratee composition?

Touber

I've emailed the author/maintainer of the enumerator package, John Millikin, linking him to this question. – Hellgrammite

Answer 1

There's now a new release (0.4.17) of enumerator that includes a (=$=) operator with the signature I gave above. I emailed the package's author and he makes a good case against including a lot of simplified operators (like ($=), (=$), and now (=$=)) in the package.

Basically, the problem is that of handling left-over input. The joinI combinator

joinI :: Monad m => Iteratee a m (Step a' m b) -> Iteratee a m b

discards the left-over Stream a' that's yielded by the inner Iteratee. This is not a problem if one uses a style like

joinI (foo $$ (bar $$ baz))

where the left-over data is only discarded at the end of the computation. However, using the simplified operators results in multiple implicit joins and the left-over data becomes much harder to keep track of. If the iteratees being used are simple (i.e. they don't yield left-over data) then this is not a problem and the simplified operators make sense to use.

Touber