Using Crossfilter, is it possible to track max/min when grouping?

Question

When using Crossfilter (https://github.com/square/crossfilter), I specify functions to use when adding and removing data from a group. It's fairly trivial to keep track of a running average (using CoffeeScript):

reduceAdd = (p, v) ->
  ++p.count;
  p.sum += v.digit;
  p

reduceRemove = (p, v) ->
  --p.count;
  p.sum -= v.digit;
  p

reduceInitial = ->
  {
    count: 0
    sum: 0
    average: ->
      return 0 if this.count == 0
      return this.sum / this.count
  }

Is it possible to keep track of the max and min of each group? I can't figure out a way short of keeping all elements in a huge array and doing a d3.min / d3.max. It seems that adding/removing data would be extremely inefficient.

I also looked for a way to tell Crossfilter to completely rebuild the group from scratch, rather than removing items from an existing group. If a filter is applied, the group is reset and rebuilt. Nothing obvious.

Grimalkin

Just ran into the same problem. BTW, I opened an issue in github, github.com/square/crossfilter/issues/25 – Gersham

Answer 1

After playing around with this for a bit, you can rebuild the group by just calling the group method again.

Gersham

Interesting, I'll check it out.BTW, I couldn't make a 'crossfilter' tag in SO without more reputation. If anyone stumbles upon this and can create a tag, I'm happy to re-tag my question. – Grimalkin

Just created the tag. Crossfilter is a great library hopefully it will get more attention. – Adelladella

Can you please explain the solution in more detail? When/how do you rebuild the group? – Wilhite

Answer 2

You can use dimension.top(1) and dimension.bottom(1) to retrieve the current min and max. These methods respect any filters that may be active on the crossfilter.

Jonme

How would you use dimension.top and dimension.bottom to get the max and min per group? – Wilhite

Answer 3

The best solution I came up with, was to keep track of all values in an ordered list and to add elements with a simple quicksort-style insertion function (cp. how to insert a number into a sorted array) and to remove them using indexOf.

Common functions:

function insertElement(element, array) {
    array.splice(locationOfElement(element, array) + 1, 0, element);
    return array;
}

function removeElement(element, array) {
    var index = array.indexOf(element);
    if (index >= 0) array.splice(index, 1);
    return array;
}

function locationOfElement(element, array, start, end) {
    start = start || 0;
    end = end || array.length;
    var pivot = parseInt(start + (end - start) / 2, 10);
    if (array[pivot] === element) return pivot;
    if (end - start <= 1)
        return array[pivot] > element ? pivot - 1 : pivot;
    if (array[pivot] < element) {
        return locationOfElement(element, array, pivot, end);
    } else {
        return locationOfElement(element, array, start, pivot);
    }
}

function maxElement(array) {
    return (array.length > 0) ? 
        array[array.length - 1] : null;
}

function minElement(array) {
    return (array.length > 0) ? 
        array[0] : null;
}

Functions to use when adding and removing data from a group to track min / max:

minMaxDimension = cf.dimension(function (d) {
    return d.key; 
});

var reduceAdd = function(p, v) {
    insertElement(v.value, p.elements);
    return p;
};

var reduceRemove = function(p, v) {
    removeElement(v.value, p.elements);
    return p;
};

var reduceInitial = function() {
    return { 
        elements: [],
        max: function() { return maxElement(elements); },
        min: function() { return minElement(elements); }
    }
}

minMaxGroup = minMaxDimension
    .group()
    .reduce(reduceAdd, reduceRemove, reduceInitial)
    .orderNatural()
    .top(Infinity);

Richmond

this is the more efficient solution by far. – Dumanian

I think you can use d3.bisectRight instead of locationOfElement – Dumanian