How to make ng-repeat filter out duplicate results

Question

I'm running a simple ng-repeat over a JSON file and want to get category names. There are about 100 objects, each belonging to a category - but there are only about 6 categories.

My current code is this:

<select ng-model="orderProp" >
  <option ng-repeat="place in places" value="{{place.category}}">{{place.category}}</option>
</select>

The output is 100 different options, mostly duplicates. How do I use Angular to check whether a {{place.category}} already exists, and not create an option if it's already there?

edit: In my javascript, $scope.places = JSON data, just to clarify

Thistledown

Why don't you just dedup your $scope.places? use jquery map api.jquery.com/map – Ideal

what was your final working solution? IS that it above or is there some JS doing the de-duping – Bathulda

I want to know the solution to this. Please post a follow-up. Thanks! – Autocephalous

@Autocephalous I'll start with a TLDR: Use the answers below, or use vanilla Javascript to filter out only unique values in an array. What I did: In the end, it was a problem with my logic and my understanding of MVC. I was loading in data from MongoDB asking for a dump of data and wanted Angular to magically filter it down to just unique places. The solution, as if often the case, was to stop being lazy and fix my DB model - for me, it was calling Mongo's db.collection.distinct("places"), which was far, far better than doing it in within Angular! Sadly this won't work for everyone. – Thistledown

thanks for the update! – Autocephalous

I have a problem like this but my array is small so the "unique" filter is a great solution. – Autocephalous

Answer 1

You could use the unique filter from AngularUI (source code available here: AngularUI unique filter) and use it directly in the ng-options (or ng-repeat).

<select ng-model="orderProp" ng-options="place.category for place in places | unique:'category'">
    <option value="0">Default</option>
    // unique options from the categories
</select>

Apomict

For those who can't get the unique filter of AngularUI working: the filters are in a separate module: E.g you must include it as an additional reference in your module angular.module('yourModule', ['ui', 'ui.filters']);. Was stumped until I took a look inside the AngularUI js file. – Ventre

The unique filter can currently be found as part of AngularJs UI Utils – Pickwickian

In the new version of ui utils, you can just include ui.unique on its own. use bower install just for the module. – Psychologist

If you don't want to include AngularUI and its entirety, but want to use the unique filter, you can just copy paste the unique.js source into your app, then change angular.module('ui.filters') to your app name. – Tetrabrach

Answer 2

Or you can write your own filter using lodash.

app.filter('unique', function() {
    return function (arr, field) {
        return _.uniq(arr, function(a) { return a[field]; });
    };
});

Ganesha

Hello Mike, looks really elegant but doesn't seem to work as expected when I pass a filter like unique:status[my field name] it only return the first result as opposed to the desired list of all unique statues available. Any guesses why? – Airtight

Although I used underscore, this solution worked like a charm. Thanks! – Piperine

Very elegant solution. – Butanone

who anyone faces with retruning first name with ng-repeat use it like this way ng-repeat="elm in data.content | unique:'title' " – Basement

I was wondering : Differences between lodash and underscore ? – Urbani

lodash is a fork of underscore. It has better performance and more utilities. – Ganesha

in lodash v4 _.uniqBy(arr, field); should work for nested properties – Robby

Answer 3

You can use 'unique'(aliases: uniq) filter in angular.filter module

usage: colection | uniq: 'property'
you can also filter by nested properties: colection | uniq: 'property.nested_property'

What you can do, is something like that..

function MainController ($scope) {
 $scope.orders = [
  { id:1, customer: { name: 'foo', id: 10 } },
  { id:2, customer: { name: 'bar', id: 20 } },
  { id:3, customer: { name: 'foo', id: 10 } },
  { id:4, customer: { name: 'bar', id: 20 } },
  { id:5, customer: { name: 'baz', id: 30 } },
 ];
}

HTML: We filter by customer id, i.e remove duplicate customers

<th>Customer list: </th>
<tr ng-repeat="order in orders | unique: 'customer.id'" >
   <td> {{ order.customer.name }} , {{ order.customer.id }} </td>
</tr>

result
Customer list:
foo 10
bar 20
baz 30

Arbela

like your answer, but having a hard time translating it to my problem. How would you handle a nested list. For example, a list of orders that contained a list of items for each order. In your example, you have one customer per order. I would like to see a unique list of items across all orders. Not to belabor the point, but you can also think of it as a customer has many orders and an order has many items. How can I show all the previous items that a customer has ordered? Thoughts? – Spherule

@GregGrater Can you provide example object that similar to your problem ? – Arbela

With which versions of angular it is compatible ? – Wurth

Answer 4

this code works for me.

app.filter('unique', function() {

  return function (arr, field) {
    var o = {}, i, l = arr.length, r = [];
    for(i=0; i<l;i+=1) {
      o[arr[i][field]] = arr[i];
    }
    for(i in o) {
      r.push(o[i]);
    }
    return r;
  };
})

and then

var colors=$filter('unique')(items,"color");

Telethon

The Definition of l should be l = arr != undefined ? arr.length : 0 since otherwise there is an parsing error in angularjs – Chili

Answer 5

If you want to list categories, I think you should explicitly state your intention in the view.

<select ng-model="orderProp" >
  <option ng-repeat="category in categories"
          value="{{category}}">
    {{category}}
  </option>
</select>

in the controller:

$scope.categories = $scope.places.reduce(function(sum, place) {
  if (sum.indexOf( place.category ) < 0) sum.push( place.category );
  return sum;
}, []);

Sudor

could you explain this a bit more. I want to repeat th elements in a row for each unique category. Does the JS just go in the JS file where the controller is defined? – Bathulda

If you want to group the options, it is different question. You may want to look into optgroup element. Angular supports optgroup. search for group by expression in select directive. – Sudor

Will will happen in the case that a place is added/removed? Will the associated category be added/removed if it is the only instance in places? – Spherule

Answer 6

Here's a straightforward and generic example.

The filter:

sampleApp.filter('unique', function() {

  // Take in the collection and which field
  //   should be unique
  // We assume an array of objects here
  // NOTE: We are skipping any object which
  //   contains a duplicated value for that
  //   particular key.  Make sure this is what
  //   you want!
  return function (arr, targetField) {

    var values = [],
        i, 
        unique,
        l = arr.length, 
        results = [],
        obj;

    // Iterate over all objects in the array
    // and collect all unique values
    for( i = 0; i < arr.length; i++ ) {

      obj = arr[i];

      // check for uniqueness
      unique = true;
      for( v = 0; v < values.length; v++ ){
        if( obj[targetField] == values[v] ){
          unique = false;
        }
      }

      // If this is indeed unique, add its
      //   value to our values and push
      //   it onto the returned array
      if( unique ){
        values.push( obj[targetField] );
        results.push( obj );
      }

    }
    return results;
  };
})

The markup:

<div ng-repeat = "item in items | unique:'name'">
  {{ item.name }}
</div>
<script src="your/filters.js"></script>

Pardew

This works fine but it throws an error in console Cannot read property 'length' of undefined in the line l = arr.length – Peeress

Answer 7

I decided to extend @thethakuri's answer to allow any depth for the unique member. Here's the code. This is for those who don't want to include the entire AngularUI module just for this functionality. If you're already using AngularUI, ignore this answer:

app.filter('unique', function() {
    return function(collection, primaryKey) { //no need for secondary key
      var output = [], 
          keys = [];
          var splitKeys = primaryKey.split('.'); //split by period


      angular.forEach(collection, function(item) {
            var key = {};
            angular.copy(item, key);
            for(var i=0; i<splitKeys.length; i++){
                key = key[splitKeys[i]];    //the beauty of loosely typed js :)
            }

            if(keys.indexOf(key) === -1) {
              keys.push(key);
              output.push(item);
            }
      });

      return output;
    };
});

Example

<div ng-repeat="item in items | unique : 'subitem.subitem.subitem.value'"></div>

Tartarus

Answer 8

UPDATE

I was recomending the use of Set but sorry this doesn't work for ng-repeat, nor Map since ng-repeat only works with array. So ignore this answer. anyways if you need to filter out duplicates one way is as other has said using angular filters, here is the link for it to the getting started section.

---

Old answer

Yo can use the ECMAScript 2015 (ES6) standard Set Data structure, instead of an Array Data Structure this way you filter repeated values when adding to the Set. (Remember sets don't allow repeated values). Really easy to use:

var mySet = new Set();

mySet.add(1);
mySet.add(5);
mySet.add("some text");
var o = {a: 1, b: 2};
mySet.add(o);

mySet.has(1); // true
mySet.has(3); // false, 3 has not been added to the set
mySet.has(5);              // true
mySet.has(Math.sqrt(25));  // true
mySet.has("Some Text".toLowerCase()); // true
mySet.has(o); // true

mySet.size; // 4

mySet.delete(5); // removes 5 from the set
mySet.has(5);    // false, 5 has been removed

mySet.size; // 3, we just removed one value

Stillbirth

Answer 9

I had an array of strings, not objects and i used this approach:

ng-repeat="name in names | unique"

with this filter:

angular.module('app').filter('unique', unique);
function unique(){
return function(arry){
        Array.prototype.getUnique = function(){
        var u = {}, a = [];
        for(var i = 0, l = this.length; i < l; ++i){
           if(u.hasOwnProperty(this[i])) {
              continue;
           }
           a.push(this[i]);
           u[this[i]] = 1;
        }
        return a;
    };
    if(arry === undefined || arry.length === 0){
          return '';
    }
    else {
         return arry.getUnique(); 
    }

  };
}

Kilovolt

Answer 10

It seems everybody is throwing their own version of the unique filter into the ring, so I'll do the same. Critique is very welcome.

angular.module('myFilters', [])
  .filter('unique', function () {
    return function (items, attr) {
      var seen = {};
      return items.filter(function (item) {
        return (angular.isUndefined(attr) || !item.hasOwnProperty(attr))
          ? true
          : seen[item[attr]] = !seen[item[attr]];
      });
    };
  });

Eugine

Answer 11

Here's a template-only way to do it (it's not maintaining the order, though). Plus, the result will be ordered as well, which is useful in most cases:

<select ng-model="orderProp" >
   <option ng-repeat="place in places | orderBy:'category' as sortedPlaces" data-ng-if="sortedPlaces[$index-1].category != place.category" value="{{place.category}}">
      {{place.category}}
   </option>
</select>

Protasis

Answer 12

None of the above filters fixed my issue so I had to copy the filter from official github doc. And then use it as explained in the above answers

angular.module('yourAppNameHere').filter('unique', function () {

return function (items, filterOn) {

if (filterOn === false) {
  return items;
}

if ((filterOn || angular.isUndefined(filterOn)) && angular.isArray(items)) {
  var hashCheck = {}, newItems = [];

  var extractValueToCompare = function (item) {
    if (angular.isObject(item) && angular.isString(filterOn)) {
      return item[filterOn];
    } else {
      return item;
    }
  };

  angular.forEach(items, function (item) {
    var valueToCheck, isDuplicate = false;

    for (var i = 0; i < newItems.length; i++) {
      if (angular.equals(extractValueToCompare(newItems[i]), extractValueToCompare(item))) {
        isDuplicate = true;
        break;
      }
    }
    if (!isDuplicate) {
      newItems.push(item);
    }

  });
  items = newItems;
}
return items;
  };

});

Presentiment

Answer 13

If you want to get unique data based on the nested key:

app.filter('unique', function() {
        return function(collection, primaryKey, secondaryKey) { //optional secondary key
          var output = [], 
              keys = [];

          angular.forEach(collection, function(item) {
                var key;
                secondaryKey === undefined ? key = item[primaryKey] : key = item[primaryKey][secondaryKey];

                if(keys.indexOf(key) === -1) {
                  keys.push(key);
                  output.push(item);
                }
          });

          return output;
        };
    });

Call it like this :

<div ng-repeat="notify in notifications | unique: 'firstlevel':'secondlevel'">

Heptarchy

Answer 14

Add this filter:

app.filter('unique', function () {
return function ( collection, keyname) {
var output = [],
    keys = []
    found = [];

if (!keyname) {

    angular.forEach(collection, function (row) {
        var is_found = false;
        angular.forEach(found, function (foundRow) {

            if (foundRow == row) {
                is_found = true;                            
            }
        });

        if (is_found) { return; }
        found.push(row);
        output.push(row);

    });
}
else {

    angular.forEach(collection, function (row) {
        var item = row[keyname];
        if (item === null || item === undefined) return;
        if (keys.indexOf(item) === -1) {
            keys.push(item);
            output.push(row);
        }
    });
}

return output;
};
});

Update your markup:

<select ng-model="orderProp" >
   <option ng-repeat="place in places | unique" value="{{place.category}}">{{place.category}}</option>
</select>

Babushka

Answer 15

This might be overkill, but it works for me.

Array.prototype.contains = function (item, prop) {
var arr = this.valueOf();
if (prop == undefined || prop == null) {
    for (var i = 0; i < arr.length; i++) {
        if (arr[i] == item) {
            return true;
        }
    }
}
else {
    for (var i = 0; i < arr.length; i++) {
        if (arr[i][prop] == item) return true;
    }
}
return false;
}

Array.prototype.distinct = function (prop) {
   var arr = this.valueOf();
   var ret = [];
   for (var i = 0; i < arr.length; i++) {
       if (!ret.contains(arr[i][prop], prop)) {
           ret.push(arr[i]);
       }
   }
   arr = [];
   arr = ret;
   return arr;
}

The distinct function depends on the contains function defined above. It can be called as array.distinct(prop); where prop is the property you want to be distinct.

So you could just say $scope.places.distinct("category");

Escort

Answer 16

Create your own array.

<select name="cmpPro" ng-model="test3.Product" ng-options="q for q in productArray track by q">
    <option value="" >Plans</option>
</select>

 productArray =[];
angular.forEach($scope.leadDetail, function(value,key){
    var index = $scope.productArray.indexOf(value.Product);
    if(index === -1)
    {
        $scope.productArray.push(value.Product);
    }
});

Tomahawk