My Notebook

How to synchronize a Gulp-task that starts multiple streams in a loop

Author
Date
Category
Web Development/Gulp

How to create one Gulp-task that depends on another task is well documented in the Gulp API Reference. Gulp offers three options for doing this, namely accept a callback function, return a stream or return a promise. But what if you have a really complex Gulp-task that starts multiple asynchronous streams in a loop and you need to synchronize that with other tasks? Unfortunately there seems to be no simple solution for that at the moment, but I found a reasonable work-around:

Scenario

I have a very complex Gulp-task that renders multiple static HTML pages from a JSON file. The JSON file contains an array of articles and Gulp creates static pages for them. I have another task that depends on this one. Here is my solution to the problem. I used a simple but ingenious hack, which I got from this blog post. It simply creates a dummy stream that only executes a callback function.

Source Code

var gulp = require('gulp'),
    fs = require('fs'),
    rename = require('gulp-rename'),
    concat = require('gulp-concat'),
    htmlmin = require('gulp-html-minifier'),
    order = require("gulp-order"),
    template = require('gulp-template'),
    through2 = require('through2');

// Simple callback stream used to synchronize stuff
// Source: http://unobfuscated.blogspot.co.at/2014/01/executing-asynchronous-gulp-tasks-in.html
function synchro(done) {
    return through2.obj(function (data, enc, cb) {
        cb();
    },
    function (cb) {
        cb();
        done();
    });
}

var articles = JSON.parse(fs.readFileSync('articles.json', 'utf8'));

gulp.task('compile_html', function (done) {
    /* ... */

    var doneCounter = 0;
    function incDoneCounter() {
        doneCounter += 1;
        if (doneCounter >= articles.length) {
            done();
        }
    }

    for (var i = 0; i < articles.length; ++i) {
        gulp.src([ './src/templates/article/index.html', './src/templates/common/*.html'])
            .pipe(order([
                '**/header.html',
                '**/nav.html',
                'article/index.html',
                '**/aside.html',
                '**/footer.html',
            ], {base: './src/templates/'}))
            .pipe(concat('index.html'))
            .pipe(template({
                article: articles[i]
            }))
            .pipe(htmlmin({collapseWhitespace: true}))
            .pipe(rename(articles[i].path))
            .pipe(gulp.dest('dist'))
            .pipe(synchro(incDoneCounter));
    }
});

gulp.task('dependent_task', ['compile_html'], function () {
    /* Do something */
});

The function incDoneCounter() is called at the end of every stream. It increments the variable doneCounter. If all streams have finished doneCounter is equal to articles.length and the callback function done() is executed to signal to Gulp that the task 'compile_html' has finished. Since 'dependent_task' has 'compile_html' listed as a dependency, it is executed after 'compile_html'.

References