What about this combination of gulp-concat and lazypipe is causing an error using gulp 4?

Question

I'm upgrading from Gulp 3 to 4, and I'm running into an error:

The following tasks did not complete: build
Did you forget to signal async completion?

I understand what it's saying, but can't understand why this code is triggering it.

Error or not, the task completes (the files are concatenated and written to dest). Executing the same code without lazypipe results in no error, and removing the concatenation within lazypipe also fixes the error.

Wrapping the whole thing in something that creates a stream (like merge-stream) fixes the issue. I guess something about the interaction between gulp-concat and lazypipe is preventing a stream from being correctly returned.

Here's the (simplified) task:

gulp.task('build', function() {
    var dest = 'build';

    var buildFiles = lazypipe()
        .pipe(plugins.concat, 'cat.js') // Task will complete if I remove this
        .pipe(gulp.dest, dest);

    // This works
    // return gulp.src(src('js/**/*.js'))
    //     .pipe(plugins.concat('cat.js'))
    //     .pipe(gulp.dest(dest));

    // This doesn't (unless you wrap it in a stream-making function)
    return gulp.src(src('js/**/*.js'))
        .pipe(buildFiles());
});

Any advice appreciated!

Sperry

Answer 1

This is a known issue when using lazypipe with gulp 4 and it's not going to be fixed in the near future. Quote from that issue:

OverZealous commented on 20 Dec 2015
As of now, I have no intention of making lazypipe work on Gulp 4.

As far as I can tell this issue is caused by the fact that gulp 4 uses async-done which has this to say about its stream support:

Note: Only actual streams are supported, not faux-streams; Therefore, modules like event-stream are not supported.

When you use lazypipe() as the last pipe what you get is a stream that doesn't have a lot of the properties that you usually have when working with streams in gulp. You can see this for yourself by logging the streams:

// console output shows lots of properties
console.log(gulp.src(src('js/**/*.js'))
  .pipe(plugins.concat('cat.js'))
  .pipe(gulp.dest(dest))); 

// console output shows much fewer properties
console.log(gulp.src(src('js/**/*.js'))
  .pipe(buildFiles())); 

This is probably the reason why gulp considers the second stream to be a "faux-stream" and doesn't properly detect when the stream has finished.

Your only option at this point is some kind of workaround. The easiest workaround (which doesn't require any additional packages) is to just add a callback function cb to your task and listen for the 'end' event:

gulp.task('build', function(cb) {
  var dest = 'build';

  var buildFiles = lazypipe()
   .pipe(plugins.concat, 'cat.js') 
   .pipe(gulp.dest, dest);

  gulp.src(src('js/**/*.js'))
   .pipe(buildFiles())
   .on('end', cb);
});

Alternatively, adding any .pipe() after buildFiles() should fix this, even one that doesn't actually do anything like gutil.noop():

var gutil = require('gulp-util');

gulp.task('build', function() {
  var dest = 'build';

  var buildFiles = lazypipe()
    .pipe(plugins.concat, 'cat.js') 
    .pipe(gulp.dest, dest);

  return gulp.src(src('js/**/*.js'))
    .pipe(buildFiles())
    .pipe(gutil.noop());
});

Silverfish

Thanks for the clear and detailed explanation, really appreciate it! – Sperry

Answer 2

So the error is clear. I had to do some refactoring to make things work again for gulp 4. I ended up making some extra methods that take a source and destination and perform the tasks previously done by my lazypipe implementation.

I have to say I don't miss lazypipe now. It's just a different approach. I did end up with some extra tasks but they use a standard method like in the example below:

// previously a lazypipe, now just a method to return from a gulp4 task 
const _processJS = (sources, destination) => {

  return src(sources)
    .pipe(minify(...))
    .pipe(uglify(...))
    .pipe(obfuscate(...))
    .pipe(whatever())
    .pipe(dest(destination));

};

const jsTaskXStep1 = ()=>{
  return src(...).pipe(...).pipe(...).pipe(dest(...));
};

const jsTaskXStep2 = ()=>{
  return _processJS(['./src/js/x/**/*.js'], './dist/js');
};

const jsTaskYStep1 = ()=>{
  return src(...).pipe(...).pipe(...).pipe(dest(...));
};

const jsTaskYStep2 = ()=>{
  return _processJS(['./src/js/y/**/*.js'], './dist/js');
};

const jsTaskX = series(jsTaskXStep1, jsTaskXStep2);
const jsTaskY = series(jsTaskYStep1, jsTaskYStep2);

module.exports = {
  js: parallel(jsTaskX, jsTaskY),
  css: ...,
  widgets: ...,
  ...
  default: parallel(js, css, widgets, series(...), ...); 
}

So basically you can put your lazypipe stuff in methods like _processJS in this example. And then create tasks that use it and combine everything with gulp series and parallel. Hope this helps out some of you who are strugling with this.

Elixir