Polling with JQuery deferred (en)

• • Problem
  • Objective
  • Ver 0.1
  • Ver 0.2 (Generalization)

Problem

Imagine you want to get some web pages via JQuery Ajax, but without knowing how many pages until you actually fetch them. In such case, you can’t aggregate deferred objects before fetching. Available solution is polling pages instead. That is, fetch a page and see if more pages to go. If so, continue to retrieve another one, otherwise stop polling.

Let’s assume an API for polling page. e.g.

http://example.com/api/page?p=n`

and request it several time with consecutive n. i.e.

var api = 'http://example.com/api/page';
$.get(api, { p : 0 })
.then(function(data) {
    if(is_more(data)) {
        return $.get(api, {p : 1});
    }
    return data;
})
.then(...)
;

Now you see the problem I’m gonna shoot it - how to resolve the last then() with polled pages?

Objective

Prototype:

/**
 * @param api  URL for retrieving page.
 * @param next Function for advancing parameter for next retrieving which returns
 *             false if no more page.
 * @param fold Function returns accumulating pages.
 */
poll_pages(api, next, fold)
{ ... }

Usage:

var api = 'http://example.com/api/page';

// Generate iterator for polling
function make_next(initial) { 
    return function(data, status, xhr) {
        return ( xhr.status == 200 ) ? 
            { n : initial++ } :
            null ;
    }; 
};

// Closure for accumulating pages
function make_fold() {
    var pages = [];
    return function(obj) {
        if(obj)
            pages.push(obj);
        return pages;
    };
};

poll_pages(api, make_next(0), make_fold())
    .then(function(all_pages) {
        console.log(all_pages);
    });

Ver 0.1

The key idea is creating another deferred object to guard polling operation.

function poll_pages(api, next, fold) {
    // Our guardian
    var deferred = $.Deferred();
    function polling(data, status, xhr) {
        var next_param = next(data, status, xhr);
        if( xhr.status >= 400 ) {
            // Error handling
            deferred.reject(fold());
        } else if( next_param ) {
            fold(data);
            // Recursive polling
            return $.get(api, next_param).then(polling);
        } else {
            // No more pages
            deferred.resolve(fold());
        }
        return deferred.promise();
    }
    return $.get(api, next()).then(polling);
}

Note Above code hadn’t been tested. Please refer to following section for tested implementation.

Ver 0.2 (Generalization)

For generalization, we can decouple terminating condition from poll_pages(), and put it in into next(). As such, clients can craft the make_next() whatever they want. Though, a simple protocol is required in between next() and poll_general(), for differentiating error, termination, and continuation:

1. If some error occurred, then return false.
2. If an expected end of iteration, then return true.
3. Otherwise return deferred object for polling.

Prototype:

/**
 * @param next Function returns deferred object or boolean (for stop polling).
 * @param fold Function returns accumulated data.
 * @param this_arg Context for the `next' function.
 */
poll_general(next, fold, this_arg)
{ ... }

Usage:

function make_next(api, initial) {
    return function(data, status, xhr) {
        // error
        if(xhr && xhr.status >= 400) {
            return false;
        }
        // termination
        if(initial > 2) {
            return true;
        }
        // continue
        if (arguments.length != 0) {
            initial += 1;
        }
        return $.get(api, { p: initial });
    };
}

function make_fold() {
    var accumulated = []
    return function(data, status, xhr) {
        if(data) 
            accumulated.push(data);
        return accumulated;
    }
}

poll_general(make_next('http://fiddle.jshell.net',0), make_fold())
.then(function(accumulated){
    console.log(accumulated);
});

Implementation:

function poll_general(next, fold, this_arg) {
    var deferred = $.Deferred();
    function polling() {
        // Now we resolve/reject deferred per next()
        var iteration = next.apply(this_arg, arguments);
        if(iteration && iteration.then) {
            fold.apply(null, arguments);
            return iteration.then(polling);
        } else if (iteration) {
            deferred.resolve(fold());
        } else {
            deferred.reject(fold());
        }
        return deferred.promise();
    }
    return next().then(polling);
}

Live demo in JSFiddle.

Note In above example, we stop polling after get 2 pages due to we are polling the same page in fact.

---

Written with StackEdit.