Get all unique values in a JavaScript array (remove duplicates) [duplicate]

Question

I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.

So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?

Array.prototype.getUnique = function() {
 var o = {}, a = [], i, e;
 for (i = 0; e = this[i]; i++) {o[e] = 1};
 for (e in o) {a.push (e)};
 return a;
}

Veldaveleda

For future readers, when start finding that you have to algorithmically modify the contents of your data structure all the time, (order them, remove repeating elements, etc.) or search for elements inside it at every iteration, it's safe to assume that you're using the wrong data structure in the first place and start using one that is more appropriate for the task at hand (in this case a hash set instead of array). – Wilder

Just wanted to point out, a lot of people have suggested using JavaScript Set as a solution, proceed with caution because it is not supported in Internet Explorer. If you have to support IE, then use a polyfill. – Photocathode

For those who want to return an array of objects with all properties unique by key: #15126420 – Modigliani

Related: Showing unique characters in a string only once. – Abrahamsen

Answer 1

With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:

function onlyUnique(value, index, array) {
  return array.indexOf(value) === index;
}

// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);

console.log(unique); // ['a', 1, 2, '1']

The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.

onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.

This solution works without any extra library like jQuery or prototype.js.

It works for arrays with mixed value types too.

For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.

If you want to keep the last occurrence of a value, simply replace indexOf with lastIndexOf.

With ES6 this can be shorten to:

// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);

console.log(unique); // unique is ['a', 1, 2, '1']

Thanks to Camilo Martin for hint in comment.

ES6 has a native object Set to store unique values. To get an array with unique values you could now do this:

var myArray = ['a', 1, 'a', 2, '1'];

let unique = [...new Set(myArray)];

console.log(unique); // unique is ['a', 1, 2, '1']

The constructor of Set takes an iterable object, like an Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.

Jonquil

This solution will run much slower, unfortunately. You're looping twice, once with filter and once with index of – Perceptible

In modern JS: .filter((v,i,a)=>a.indexOf(v)==i) (fat arrow notation). – Jethro

let unique_values = [...new Set(random_array)]; developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… – Arabic

Performance wise, set is the way to go medium.com/@jakubsynowiec/… It's only downside IMO is that it does not make the programmers intent clear. I mean once you know what it does, you get it, but it's nowhere near as clear as something like myArray.unique() – Penzance

Why is [... ... ] necessary? new Set(myArray) on its own seems to have the same behaviour – Avouch

...in fact using the square brackets seems to create an array of size 1 containing the set – Avouch

npmjs.com/package/array-uniq uses the Set approach – Penzance

Does anyone understand how to type the 'self' parameter in the function if using typescript? – Fritter

Lest any newcomer be scared away by the performance implications, consider "much slower" may very well be nanoseconds. If your array is relatively small (hundreds or less), using the nice concise one off version like this is good for reasons outside of performance, i.e. readability and maintainability. But the set version is quite nice and concise. – Islean

Late to the party, but let unique_values = Array.from(new Set(random_array)); is IMO easier to read. – Reinaldoreinaldos

Comparing performance of .filter+indexOf and the Set-approach in Chrome 100 revealed that for an array with numbers and length 0 to 120 the filter-approach is even faster. With array length 200 the filter-approach takes 50% more time than with a Set (6 vs. 9 microseconds). For 5000 elements the filter operation takes over 3 milliseconds while Set still copes with my example in 173 microseconds. So "much slower", really depends on the use case. If someone is interested I can post the data as a separate Q&A. – Sigrid

Honestly? neither method is somthing I'd rely on. If you have an array with objects for example, then it will not get rid of duplicates (because every object ref. is unique). You are going to have to loop the array once per lookup anyway (unless you have sorted all your duplicates to align one after another, in which case you only need to compare with the current index minus 1), so I think a more robust approach would be to simply filter out the duplicate values in a for loop when you're building your array. Just makes more resource sense than to build another array for filtering. – Coroneted

i wish someone would show how to use the set method in an array prototype extender function. – Charlton

@Fritter use the array version of whatever you have in value, e.g.: function onlyUnique(value: string | number, index: number, self: (string | number)[]) – Slumlord

const cars = ["Saab", "Volvo", "BMW", "BMW"]; for(var i=0;i<cars.length;i++){ if(cars[i] != cars[i+1]){ alert(cars[i]); } } – Dasilva

self is not working. – Fotinas

error, guess new things are not backward compatible: { "message": "Uncaught TypeError: self.indexOf is not a function", "filename": "stacksnippets.net/js", "lineno": 13, "colno": 15 } – Drice

[...new Set(myArray)]; is cool 😎 – Foret

The only answer is the set. Everything else would be super slow for a decently sized array! – Chemist

right, you should not use the indexOf method, because that's O(n²), so if your n happens to be 2000, you are looking at 4 million iterations potentially. You probably don't want that to happen in the browser to make the UI sluggish – Expectorant

@Sigrid How do you measure the timing? with console.time()? – Tawannatawdry

Sorry, @Cemstrian, it's over a year ago and I don't have the code available any more. – Sigrid

Answer 2

Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:

let uniqueItems = [...new Set(items)]

Which returns

[4, 5, 6, 3, 2, 23, 1]

Primal

Notice, that inner array wouldn't work Array.from(new Set([[1,2],[1,2],[1,2,3]])) – Untwist

Please note that if you use the Set and add objects instead of primitive values it will contain unique references to the objects. Thus the set s in let s = new Set([{Foo:"Bar"}, {Foo:"Bar"}]); will return this: Set { { Foo: 'Bar' }, { Foo: 'Bar' } } which is a Set with unique object references to objects that contain the same values. If you write let o = {Foo:"Bar"}; and then create a set with two references like so: let s2 = new Set([o,o]);, then s2 will be Set { { Foo: 'Bar' } } – Blondy

In case anyone was wondering, this works well for strings too, e.g. [...new Set(["apple","apple","orange"])] results in ['apple', 'orange'] . Great! – Bunnell

In Typescript, use Array.from( new Set( items ) ) – Alamo

Does using Set preserve the item order? – Shel

Answer 3

I split all answers to 4 possible solutions:

1. Use object { } to prevent duplicates
2. Use helper array [ ]
3. Use filter + indexOf
4. Bonus! ES6 Set constructor.

Here's sample codes found in answers:

Use object { } to prevent duplicates

function uniqueArray1( ar ) {
  var j = {};

  ar.forEach( function(v) {
    j[v+ '::' + typeof v] = v;
  });

  return Object.keys(j).map(function(v){
    return j[v];
  });
} 

Use helper array [ ]

function uniqueArray2(arr) {
    var a = [];
    for (var i=0, l=arr.length; i<l; i++)
        if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
            a.push(arr[i]);
    return a;
}

Use filter + indexOf

function uniqueArray3(a) {
  function onlyUnique(value, index, self) { 
      return self.indexOf(value) === index;
  }
  
  // usage
  var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
  
  return unique;
}

Use ES6 [...new Set(a)]

function uniqueArray4(a) {
  return [...new Set(a)];
}

And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.

Here's the result of tests:

I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to @rab for the code.

Update 2020

Google Script enabled ES6 Engine. Now I tested the last code with Set and it appeared faster than the object method.

Mccomas

Makrov, so the uniqueItems = [...new Set(items)] appears to be the fastest and the most succinct of all the approaches? – Fantail

Your solution only handles primitives, it won't work with objects, you'd need to JSON.stringify the v in the hash – Ewall

you are GREAT for making this time test comparison, thanks, I tested now with an Array with hundreds of thousands of entries and indeed uniqueItems = [...new Set(items)] seems the fastest – Significant

"Object.keys(j).map ..." could be changed to use Object.values(j); – Heyward

Answer 4

You can also use underscore.js.

console.log(_.uniq([1, 2, 1, 3, 1, 4]));

<script src="http://underscorejs.org/underscore-min.js"></script>

which will return:

[1, 2, 3, 4]

Watersick

Please do this folks. Don't jack something onto to the Array prototype. Please. – Abirritant

@JacobDalton Please don't do this. There's no need to add an extra library just for a small job that can be done with array = [...new Set(array)] – Elijah

@JacobDalton why not? Is there a downside to "jacking something" onto the array? – Galicia

If you want to find unique objects using object equality semantics, Lodash offers _.uniqWith(arrayOfObjects, _.isEqual). _.uniqWith([{a: 2}, {b: 3}, {a: 2}], _.isEqual) gives [{a: 2}, {b: 3}]. – Squadron

This is a good solution overall, but surely the best solution if you already have a library to do that in your dependencies. This answer inspired me to look up something among my dependencies and, lo and behold, we already use lodash, which already has an uniq() funciton. So, even if it is not the same lib, thanks for the heads-up! – Rhodes

Answer 5

One Liner, Pure JavaScript

With ES6 syntax

list = list.filter((x, i, a) => a.indexOf(x) == i)

x --> item in array
i --> index of item
a --> array reference, (in this case "list")

With ES5 syntax

list = list.filter(function (x, i, a) { 
    return a.indexOf(x) == i; 
});

Browser Compatibility: IE9+

Joaquinajoash

Answer 6

Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?

In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.

const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);

The Array.from is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from at all, as Sets have plenty of useful features like forEach.

If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.

// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];

// Go through each car, one at a time.
cars.forEach(function (car) {
    // The code within the following block runs only if the
    // current car does NOT exist in the uniqueCars list
    // - a.k.a. prevent duplicates
    if (uniqueCars.indexOf(car) === -1) {
        // Since we now know we haven't seen this car before,
        // copy it to the end of the uniqueCars list.
        uniqueCars.push(car);
    }
});

To make this instantly reusable, let's put it in a function.

function deduplicate(data) {
    if (data.length > 0) {
        var result = [];

        data.forEach(function (elem) {
            if (result.indexOf(elem) === -1) {
                result.push(elem);
            }
        });

        return result;
    }
}

So to get rid of the duplicates, we would now do this.

var uniqueCars = deduplicate(cars);

The deduplicate(cars) part becomes the thing we named result when the function completes.

Just pass it the name of any array you like.

Durazzo

How would this work if I wanted the new array to not be uniques, but be an array of values that were duplicated? So using the above example, the array I'm looking for is ["volvo","lincoln"] – Donaldson

@Donaldson I'd probably create a Map to store previously seen items and an array to store the duplicate items. Then loop through the cars array and check if the Map has the current item, if it does then push it to the duplicates array, if not then add it to the Map. I'd be happy to create a code example for you if you create a new question and we can continue the discussion there. – Durazzo

Just note that if the array is empty, the function does not return anything. – Vinylidene

too bad you didn't show the set method as an extension of array.prototype – Charlton

I deliberately did not show that. I consider it bad practice to modify Array.prototype and would strongly advise against doing so. Among other reasons, it can lead to future bugs if new methods are added to Array as part of the language. In fact, if your site is popular, it could even prevent new methods from being added to the language. That has actually happened. See: 2ality.com/2022/03/naming-conflicts.html – Durazzo

Answer 7

I have since found a nice method that uses jQuery

arr = $.grep(arr, function(v, k){
    return $.inArray(v ,arr) === k;
});

Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P

Veldaveleda

A concise solution, but calling inArray is way less efficient than calling hasOwnProperty. – Primogenitor

This is also O(N^2), right? Whereas the dictionary or hasOwnProperty approach would likely be O(N*logN). – Isauraisbel

Answer 8

Magic

a.filter( e => !(t[e]=e in t) ) 

O(n) performance - we assume your array is in a and t={}. Explanation here (+Jeppe impr.)

let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));

// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));

// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];

// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))

Stereogram

Not magic, but is much like the "Set"-answers, using O(1) key-lookups in the dictionary. Do you need to increment the counters though? How about "e=>!(t[e]=e in t)". Nice answer though. – Sonneteer

@Sonneteer when I run your improvement then I experience aha effect (before I don't know that I can use in operator outside the other construction than for loop :P) - Thank you - I appreciate it and will give +2 to your other good answers. – Beshore

Well, the solution is so great indeed However it works well for tuples only The examples below work incorrect: unique(['2', 2]) // ['2']; unique([[1, 7], [1, '7'], ['1', 7], ['1', '7']]) // [1, 7] So be careful using this – Content

Answer 9

We can do this using ES6 sets:

var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];

console.log(uniqueArray); // [1,2,3,4,5]

Misgiving

Answer 10

The simplest, and fastest (in Chrome) way of doing this:

Array.prototype.unique = function() {
    var a = [];
    for (var i=0, l=this.length; i<l; i++)
        if (a.indexOf(this[i]) === -1)
            a.push(this[i]);
    return a;
}

Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.

According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.

The non-prototype version:

function uniques(arr) {
    var a = [];
    for (var i=0, l=arr.length; i<l; i++)
        if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
            a.push(arr[i]);
    return a;
}

Sorting

When also needing to sort the array, the following is the fastest:

Array.prototype.sortUnique = function() {
    this.sort();
    var last_i;
    for (var i=0;i<this.length;i++)
        if ((last_i = this.lastIndexOf(this[i])) !== i)
            this.splice(i+1, last_i-i);
    return this;
}

or non-prototype:

function sortUnique(arr) {
    arr.sort();
    var last_i;
    for (var i=0;i<arr.length;i++)
        if ((last_i = arr.lastIndexOf(arr[i])) !== i)
            arr.splice(i+1, last_i-i);
    return arr;
}

This is also faster than the above method in most non-Chrome browsers.

Whereat

On Linux, Chrome 55.0.2883 prefers your arr.unique() and swilliams' arrclone2.sortFilter() is slowest (78% slower). However, Firefox 51.0.0 (with lots of addons) has swilliams as fastest (yet still slower by Ops/sec than any other Chrome result) with mottie's jQuery $.grep(arr, jqFilter) being slowest (46% slower). Your arr.uniq() was 30% slower. I ran each test twice and got consistent results. Rafael's arr.getUnique() got second place in both browsers. – Insistent

jsPerf is buggy at the moment, so my edit to this test didn't commit everything, but it did result in adding two tests: Cocco's toUnique() beats Vamsi's ES6 list.filter() on both browsers, beating swilliams' sortFilter() for #1 on FF (sortFilter was 16% slower) and beating your sorted testing (which was slower by 2%) for #3 on Chrome. – Insistent

Ah, I hadn't caught that those tests were trivially small and don't really matter. A comment to the accepted answer describes that problem and offers a correction in a revision to the test, in which Rafael's code is easily the fastest and Joetje50's arr.unique code is 98% slower. I've also made another revision as noted in this comment. – Insistent

Well, actually the algorithm you implemented in unique function has O(n^2) complexity while the one in getUnique is O(n). The first one may be faster on small data sets, but how can you argue with the maths :) You can make sure the latter one is faster if you run it on an array of, say, 1e5 unique items – Warbler

also used by lodash.uniq for input_array.length < 200, otherwise uses the [...new Set(input_array)] method. expressed as reducer: input_array.reduce((c, v) => {if (!c.includes(v)) c.push(v); return c;}, []) – Iceboat

unique shouldn't use lastIndexOf, as that causes O(n^2) performance. Instead, have var last_value = null;... if (arr[i] !== last_value) { if (last_i !== null) arr.splice(last_i+1, i - last_i); last_i = i; last_value = arr[i]; } - not tested, i - last_i might be off by 1. – Simba

how is this the fastest way? who in the world would upvote this answer? – Chemist

@Chemist To be honest, I'm still not that surprised it was the fastest out of the answers given at the time, because there were just some very inefficient answers. Just using an ES6 approach such as filters or Sets is probably much more efficient nowadays. – Whereat

@Whereat but this is n^2 – Chemist

Answer 11

["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
  return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
 }, []);

[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
  return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
 }, []);

Expel

This worked very well with an array of objects! [].reduce((p, c) => (p.some((item) => c.id === item.id) ? p : p.concat(c)), []) – Windham

Answer 12

This has been answered a lot, but it didn't address my particular need.

Many answers are like this:

a.filter((item, pos, self) => self.indexOf(item) === pos);

But this doesn't work for arrays of complex objects.

Say we have an array like this:

const a = [
 { age: 4, name: 'fluffy' },
 { age: 5, name: 'spot' },
 { age: 2, name: 'fluffy' },
 { age: 3, name: 'toby' },
];

If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:

a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);

Magnify

Great solution, beware that a new array will return from a function. (it doesn't modify itself) – Rhododendron

Works will with a complex array of objets – Autoharp

@EdgarQuintero only if the elements are actually the exact same object, so the array [ { a: 2 }, { a: 2 } ] won't work as many people might expect if you use the indexOf solution, but the findIndex solution might be useful – Magnify

Answer 13

Primitive values

With Set (Recommended)

var array = ["FreePhoenix888", "FreePhoenix888", "konard", "FreePhoenix888"];

let set = [...new Set(array)];

console.log(set); // ["FreePhoenix888", "konard"]

Without Set

function filterUniqueObjects(value, index, array) {
  return array.indexOf(value) === index;
}

// usage example:
var array = ["FreePhoenix888", "FreePhoenix888", "konard", "FreePhoenix888"];
var arrayOfUniqueItems = array.filter(filterUniqueObjects);

console.log(arrayOfUniqueItems); // ["FreePhoenix888", "konard"]

Objects

This example shows how you can filter not just an array of primitive values but an array of objects. I have added comments to make it easier to understand what you can change there depending on your requirements.

let array = [
  { name: '@deep-foundation/core', version: '0.0.2' },
  { name: '@deep-foundation/capacitor-device', version: '10.0.1' },
  { name: '@deep-foundation/capacitor-device', version: '10.0.2' },
];

// Of course you can inline this function as filter argument uniqueArray.filter((item, index, self) => self.findIndex(innerItem => innerItem.name === item.name) === index);
function filterUniqueObjects(value, index, self) {
  return (
    self.findIndex(
      // Modify this function as you desire. You may want to calculate uniqueness depending only on specific fields, not all
      (obj) => obj.name === value.name
    ) === index
  );
};

let uniqueArray = array
  .reverse() // If you want latest duplicates to remain
  .filter(filterUniqueObjects)
  .reverse(); // To get back to original order after first reverse

console.log(uniqueArray)

Tomtit

Answer 14

After looking into all the 90+ answers here, I saw there is room for one more:

Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter callback method will search the array, starting from [current index] + 1 which guarantees not to include currently filtered item in the lookup and also saves time.

Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the Set trick if the Array is a collection of Objects.

//                🚩              🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']

console.log( 
  list.filter((v,i) => !list.includes(v,i+1))
)

// [0,1,3,"b",4,5,2,"a"]

Explanation:

For example, lets assume the filter function is currently iterating at index 2) and the value at that index happens to be 2. The section of the array that is then scanned for duplicates (includes method) is everything after index 2 (i+1):

           👇                    👇
[0, 1, 2,   2 ,3 ,'a', 'b', 4, 5, 2, 'a']
       👆   |---------------------------|

And since the currently filtered item's value 2 is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.

---

If order is important, use this method:

//                🚩              🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']

console.log( 
  // Initialize with empty array and fill with non-duplicates
  list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)

// [0,1,2,3,"a","b",4,5]

Defazio

Unfortunately, this keeps the LAST instance of each value, not the first. (which might be ok, but I think keeping the first is generally what's expected) – Voltage

Although this is a nice optimization idea, I don't think people would want to use this since this's keeping the last item which basically mess up the array order. – Romanfleuve

Answer 15

This prototype getUnique is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"] it will return ["1","2","3","4"] and "1" is string and 1 is a integer; they are different.

Here is a correct solution:

Array.prototype.unique = function(a){
    return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });

using:

var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();

The above will produce ["1",2,3,4,1,"foo"].

Engraving

Note that $foo = 'bar' is the PHP way of declaring variables. It will work in javascript, but will create an implicit global, and generally shouldn't be done. – Jethro

@CamiloMartin sorry but you're wrong, $foo is global because the example is not in a closure and he's missing the var keyword. Nothing to do with the dollar jsfiddle.net/robaldred/L2MRb – Closeknit

@Closeknit that's exactly what I'm saying, PHP people will think $foo is the way of declaring variables in javascript while actually var foo is. – Jethro

Answer 16

[...new Set(duplicates)]

This is the simplest one and referenced from MDN Web Docs.

const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]

Broke

While this code may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please edit your answer to add explanation, and give an indication of what limitations and assumptions apply. – Scummy

Identical to previous answer which dates a year prior to this one – Defazio

Answer 17

That's because 0 is a falsy value in JavaScript.

this[i] will be falsy if the value of the array is 0 or any other falsy value.

Bifoliate

Ahhhh, ok I see now... but would there be an easy fix to make it work? – Veldaveleda

Answer 18

Array.prototype.getUnique = function() {
    var o = {}, a = []
    for (var i = 0; i < this.length; i++) o[this[i]] = 1
    for (var e in o) a.push(e)
    return a
}

Deandeana

I think this won't work if the array contains objects/arrays, and I'm not sure if it will preserve the type of scalars. – Jethro

Yes, everything gets stringified. That could be fixed by storing the original value in o instead of just a 1, although equality comparison would still be stringwise (although, out of all the possible Javascript equalities, it doesn't seem too unreasonable). – Deandeana

The Array.prototype could be extended only with non enumerable methods .... Object.defineProperty(Array.prototype,"getUnique",{}) ... but the idea of using a helper object is very nice – Tillion

Answer 19

Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter the array.

By keeping last occurrence:

    function arrayLastUnique(array) {
        return array.filter(function (a, b, c) {
            // keeps last occurrence
            return c.indexOf(a, b + 1) < 0;
        });
    },

or first occurrence:

    function arrayFirstUnique(array) {
        return array.filter(function (a, b, c) {
            // keeps first occurrence
            return c.indexOf(a) === b;
        });
    },

Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).

Chiaki

The fact that it's js 1.6 does not mean you can't use filter. At the MDN page they have an implementation for Internet Explorer, I mean, older browsers. Also: JS 1.6 refers only to Firefox's js engine, but the right thing to say it's that it is ECMAScript 5. – Jethro

Answer 20

Now using sets you can remove duplicates and convert them back to the array.

var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];

console.log([...new Set(names)])

Another solution is to use sort & filter

var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);

Farant

Answer 21

If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:

var a = Array.uniq();  

Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq() I used size() and there was my simple result. p.s. Sorry if i mistyped something

edit: if you want to escape undefined records you may want to add compact() before, like this:

var a = Array.compact().uniq();  

Milner

because i found a better answer, i think about topics are for all people not just for the one who asked – Milner

Thanks for time machine yet iirc around 15 years ago JS community had debate and result is - do not extend prototype cause of side effects and cause you pollute all JS arrays this way. – Arabic

Answer 22

I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.

let objArr = [{
  id: '123'
}, {
  id: '123'
}, {
  id: '456'
}];

objArr = objArr.reduce((acc, cur) => [
  ...acc.filter((obj) => obj.id !== cur.id), cur
], []);

console.log(objArr);

Donaugh

Answer 23

If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).

Usage

If you don't have it in your codebase already, install it using npm:

npm install lodash

Then use it as follows:

import _ from 'lodash';
let idArray = _.uniq ([
    1,
    2,
    3,
    3,
    3
]);
console.dir(idArray);

Out:

[ 1, 2, 3 ]

Hallie

You can also use lodash to remove objects with duplicate properties from an array: _.uniqWith(objectArray, _.isEqual). – Foodstuff

Answer 24

In ES6/Later

Get Only Unique Values

  let a = [
           { id: 1, name: "usman" },
           { id: 2, name: "zia" },
           { id: 3, name: "usman" },
          ];
const unique = [...new Set(a.map((item) => item.name))];
console.log(unique); // ["usman", "zia"]

Get Unique Objects

const myObjArray = [
                       { id: 1, name: "usman" },
                       { id: 2, name: "zia" },
                       { id: 3, name: "usman" },
                   ];
// Creates an array of objects with unique "name" property values.
let uniqueObjArray = [
  ...new Map(myObjArray.map((item) => [item["name"], item])).values(),
];

console.log("uniqueObjArray", uniqueObjArray);

Inclinatory

Answer 25

I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:

Array.prototype.unique = function() {
  return this.filter(function(value, index, array) {
    return array.indexOf(value, index + 1) < 0;
  });
};

or in CoffeeScript:

Array.prototype.unique = ->
  this.filter( (value, index, array) ->
    array.indexOf(value, index + 1) < 0
  )

Mango

Answer 26

It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:

Array.prototype.getUnique = function(){
    var u = {}, a = [];
    for (var i = 0, l = this.length; i < l; ++i) {
        if (u.hasOwnProperty(this[i])) {
            continue;
        }
        a.push(this[i]);
        u[this[i]] = 1;
    }
return a;
}

If you do have a mixed-type array, you can serialize the hash key:

Array.prototype.getUnique = function() {
    var hash = {}, result = [], key; 
    for ( var i = 0, l = this.length; i < l; ++i ) {
        key = JSON.stringify(this[i]);
        if ( !hash.hasOwnProperty(key) ) {
            hash[key] = true;
            result.push(this[i]);
        }
    }
    return result;
}

Donaldson

Answer 27

Finding unique Array values in simple method

function arrUnique(a){
  var t = [];
  for(var x = 0; x < a.length; x++){
    if(t.indexOf(a[x]) == -1)t.push(a[x]);
  }
  return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]

Guarded

How can this answer be correct? Expected result for unique array is [5,9] as per the input given [1,4,2,7,1,5,9,2,4,7,2] – Alginate

@PoojaThapa this will extract the unique values from input array. – Guarded

Answer 28

strange this hasn't been suggested before.. to remove duplicates by object key (id below) in an array you can do something like this:

const uniqArray = array.filter((obj, idx, arr) => (
  arr.findIndex((o) => o.id === obj.id) === idx
)) 

Ineslta

Don't both filter() and findIndex() have to iterate through the array? That would make this a double-loop and therefore twice as expensive to run as any other answer here. – Insistent

@AdamKatz yes it will iterate over the array n+1 times. Please be aware that the other answers here using a combination of map, filter, indexOf, reduce etc. also must do this, it's sort of inherent in the problem. To avoid, you could use new Set(), or a lookup object similar to the answer by Grozz. – Ineslta

Answer 29

Simple.

return Array.from(new Set(a));

Diastema

Answer 30

For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity

function getUniqueArr(arr){
    const mapObj = {};
    arr.forEach(a => { 
       mapObj[a.id] = a
    })
    return Object.values(mapObj);
}

Foliate

Answer 31

Using object keys to make unique array, I have tried following

function uniqueArray( ar ) {
  var j = {};

  ar.forEach( function(v) {
    j[v+ '::' + typeof v] = v;
  });


  return Object.keys(j).map(function(v){
    return j[v];
  });
}   

uniqueArray(["1",1,2,3,4,1,"foo", false, false, null,1]);

Which returns ["1", 1, 2, 3, 4, "foo", false, null]

Pseudohermaphroditism

I think, your answer is the fastest solution because it uses hash. – Mccomas

Answer 32

The task is to get a unique array from an array consisted of arbitrary types (primitive and non primitive).

The approach based on using new Set(...) is not new. Here it is leveraged by JSON.stringify(...), JSON.parse(...) and [].map method. The advantages are universality (applicability for an array of any types), short ES6 notation and probably performance for this case:

const dedupExample = [
    { a: 1 },
    { a: 1 },
    [ 1, 2 ],
    [ 1, 2 ],
    1,
    1,
    '1',
    '1'
]

const getUniqArrDeep = arr => {
    const arrStr = arr.map(item => JSON.stringify(item))
    return [...new Set(arrStr)]
        .map(item => JSON.parse(item))
}

console.info(getUniqArrDeep(dedupExample))
   /* [ {a: 1}, [1, 2], 1, '1' ] */

Intuit

since you're stringifying and then reparsing, positive performance is definitely not an attribute here. – Furlana

What do you mean "positive performance" and "attribute here"? Need some elaboration. – Intuit

stringifying an object and then parsing it again is one of the most expensive operations you can do in JS apparently. jsben.ch/wQ9RU – Furlana

@airtonix, It is true, we need to take performance into an account. Some cases are like "Tough times call for tough decisions" though :-) – Intuit

Answer 33

Finding unique in Array of objects using One Liner

const uniqueBy = (x,f)=>Object.values(x.reduce((a,b)=>((a[f(b)]=b),a),{}));
// f -> should must return string because it will be use as key

const data = [
  { comment: "abc", forItem: 1, inModule: 1 },
  { comment: "abc", forItem: 1, inModule: 1 },
  { comment: "xyz", forItem: 1, inModule: 2 },
  { comment: "xyz", forItem: 1, inModule: 2 },
];

uniqueBy(data, (x) => x.forItem +'-'+ x.inModule); // find unique by item with module
// output
// [
//   { comment: "abc", forItem: 1, inModule: 1 },
//   { comment: "xyz", forItem: 1, inModule: 2 },
// ];

// can also use for strings and number or other primitive values

uniqueBy([1, 2, 2, 1], (v) => v); // [1, 2]
uniqueBy(["a", "b", "a"], (v) => v); // ['a', 'b']

uniqueBy(
  [
    { id: 1, name: "abc" },
    { id: 2, name: "xyz" },
    { id: 1, name: "abc" },
  ],
  (v) => v.id
);
// output
// [
//   { id: 1, name: "abc" },
//   { id: 2, name: "xyz" },
// ];

Rourke

use can use uniqBy as well instead of uniqueBy – Seattle

Answer 34

As explained already, [...new Set(values)] is the best option, if that's available to you.

Otherwise, here's a one-liner that doesn't iterate the array for every index:

values.sort().filter((val, index, arr) => index === 0 ? true : val !== arr[index - 1]);

That simply compares each value to the one before it. The result will be sorted.

Example:

let values = [ 1, 2, 3, 3, 4, 5, 5, 5, 4, 4, 4, 5, 1, 1, 1, 3, 3 ];
let unique = values.sort().filter((val, index, arr) => index === 0 ? true : val !== arr[index - 1]);
console.log(unique);

Indiana

Not working when multiple same values are in row – Phantasm

Added code snippet, seems to work. – Indiana

Answer 35

Building on other answers, here's another variant that takes an optional flag to choose a strategy (keep first occurrence or keep last):

Without extending Array.prototype

function unique(arr, keepLast) {
  return arr.filter(function (value, index, array) {
    return keepLast ? array.indexOf(value, index + 1) < 0 : array.indexOf(value) === index;
  });
};

// Usage
unique(['a', 1, 2, '1', 1, 3, 2, 6]); // -> ['a', 1, 2, '1', 3, 6]
unique(['a', 1, 2, '1', 1, 3, 2, 6], true); // -> ['a', '1', 1, 3, 2, 6]

Extending Array.prototype

Array.prototype.unique = function (keepLast) {
  return this.filter(function (value, index, array) {
    return keepLast ? array.indexOf(value, index + 1) < 0 : array.indexOf(value) === index;
  });
};

// Usage
['a', 1, 2, '1', 1, 3, 2, 6].unique(); // -> ['a', 1, 2, '1', 3, 6]
['a', 1, 2, '1', 1, 3, 2, 6].unique(true); // -> ['a', '1', 1, 3, 2, 6]

Moulin

Answer 36

let ar = [1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 2, 1];
let unique = ar.filter((value, index) => {
        return ar.indexOf(value) == index;
      });
console.log(unique);

Madelene

Answer 37

You can use a Set to eliminate the duplicates.

const originalNumbers = [1, 2, 2, 3, 3, 4, 4, 4, 4, 5, 1, 2, 9];
const uniqueNumbersSet = new Set(originalNumbers);

/** get the array back from the set */
const uniqueNumbersArray = Array.from(uniqueNumbersSet);

/** uniqueNumbersArray outputs to: [1, 2, 3, 4, 5, 9] */

Learn more about set: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set

Quint

JavaScript Set() is really a good way to generate unique array – Aquatint

Answer 38

A simple code as this:

let arr = [1,'k',12,1,1,'k','12'];
let distictArr=arr.filter((item, index, arr) => arr.indexOf(item) === index);

console.log(distictArr); // [1, 'k', 12, '12']

Hoxie

Answer 39

You can also use jQuery

var a = [1,5,1,6,4,5,2,5,4,3,1,2,6,6,3,3,2,4];

// note: jQuery's filter params are opposite of javascript's native implementation :(
var unique = $.makeArray($(a).filter(function(i,itm){ 
    // note: 'index', not 'indexOf'
    return i == $(a).index(itm);
}));

// unique: [1, 5, 6, 4, 2, 3]

Originally answered at: jQuery function to get all unique elements from an array?

Traceytrachea

This one seems only to work for arrays of integers. When I include some strings they all get stripped out of the result. – Thermionic

Answer 40

If anyone using knockoutjs

ko.utils.arrayGetDistinctValues()

BTW have look at all ko.utils.array* utilities.

Street

Answer 41

Look at this. Jquery provides uniq method: https://api.jquery.com/jQuery.unique/

var ids_array = []

$.each($(my_elements), function(index, el) {
    var id = $(this).attr("id")
    ids_array.push(id)
});

var clean_ids_array = jQuery.unique(ids_array)

$.each(clean_ids_array, function(index, id) {
   elment = $("#" + id)   // my uniq element
   // TODO WITH MY ELEMENT
});

Bisulfate

If you read the description on the page you linked: Description: Sorts an array of DOM elements, in place, with the duplicates removed. Note that this only works on arrays of DOM elements, not strings or numbers. – Veldaveleda

Answer 42

Do it with lodash and identity lambda function, just define it before use your object

const _ = require('lodash');
...    
_.uniqBy([{a:1,b:2},{a:1,b:2},{a:1,b:3}], v=>v.a.toString()+v.b.toString())
_.uniq([1,2,3,3,'a','a','x'])

and will have:

[{a:1,b:2},{a:1,b:3}]
[1,2,3,'a','x']

(this is the simplest way )

Footslog

Answer 43

Deduplication usually requires an equality operator for the given type. However, using an eq function stops us from utilizing a Set to determine duplicates in an efficient manner, because Set falls back to ===. As you know for sure, === doesn't work for reference types. So we're kind if stuck, right?

The way out is simply using a transformer function that allows us to transform a (reference) type into something we can actually lookup using a Set. We could use a hash function, for instance, or JSON.stringify the data structure, if it doesn't contain any functions.

Often we only need to access a property, which we can then compare instead of the Object's reference.

Here are two combinators that meet these requirements:

const dedupeOn = k => xs => {
  const s = new Set();

  return xs.filter(o =>
    s.has(o[k])
      ? null
      : (s.add(o[k]), o[k]));
};

const dedupeBy = f => xs => {
  const s = new Set();

  return xs.filter(x => {
    const r = f(x);
    
    return s.has(r)
      ? null
      : (s.add(r), x);
  });
};

const xs = [{foo: "a"}, {foo: "b"}, {foo: "A"}, {foo: "b"}, {foo: "c"}];

console.log(
  dedupeOn("foo") (xs)); // [{foo: "a"}, {foo: "b"}, {foo: "A"}, {foo: "c"}]

console.log(
  dedupeBy(o => o.foo.toLowerCase()) (xs)); // [{foo: "a"}, {foo: "b"}, {foo: "c"}]

With these combinators we're extremely flexible in handling all kinds of deduplication issues. It's not the fastes approach, but the most expressive and most generic one.

Filth

Answer 44

Here is an almost one-liner that is O(n), keeps the first element, and where you can keep the field you are uniq'ing on separate.

This is a pretty common technique in functional programming - you use reduce to build up an array that you return. Since we build the array like this, we guarantee that we get a stable ordering, unlike the [...new Set(array)] approach. We still use a Set to ensure we don't have duplicates, so our accumulator contains both a Set and the array we are building.

const removeDuplicates = (arr) =>
  arr.reduce(
    ([set, acc], item) => set.has(item) ? [set, acc] : [set.add(item), (acc.push(item), acc)],
    [new Set(), []]
  )[1]

The above will work for simple values, but not for objects, similarly to how [...new Set(array)] breaks down. If the items are objects that contain an id property, you'd do:

const removeDuplicates = (arr) =>
  arr.reduce(
    ([set, acc], item) => set.has(item.id) ? [set, acc] : [set.add(item.id), (acc.push(item), acc)],
    [new Set(), []]
  )[1]

Lexi

Answer 45

For removing the duplicates there could be 2 situations. first, all the data are not objects, secondly all the data are objects.

If all the data are any kind of primitive data type like int, float, string etc then you can follow this one

const uniqueArray = [...new Set(oldArray)]

But suppose your array consist JS objects like bellow

{
    id: 1,
    name: 'rony',
    email: '[email protected]'
}

then to get all the unique objects you can follow this

let uniqueIds = [];
const uniqueUsers = oldArray.filter(item => {
    if(uniqueIds.includes(item.id)){
        return false;
    }else{
        uniqueIds.push(item.id);
        return true;
    }
})

You can also use this method to make any kind of array to make unique. Just keep the tracking key on the uniqueIds array.

Gerdes

Answer 46

  var myArray = ["a",2, "a", 2, "b", "1"];
  const uniques = [];
  myArray.forEach((t) => !uniques.includes(t) && uniques.push(t));
  console.log(uniques);

Roscoe

Answer 47

Not really a direct literal answer to the original question, because I preferred to have the duplicate values never in the array in the first place. So here's my UniqueArray:

class UniqueArray extends Array {
    constructor(...args) {
        super(...new Set(args));
    }
    push(...args) {
        for (const a of args) if (!this.includes(a)) super.push(a);
        return this.length;
    }
    unshift(...args) {
        for (const a of args.reverse()) if (!this.includes(a)) super.unshift(a);
        return this.length;
    }
    concat(...args) {
        var r = new UniqueArray(...this);
        for (const a of args) r.push(...a);
        return r;
    }
}

> a = new UniqueArray(1,2,3,1,2,4,5,1)
UniqueArray(5) [ 1, 2, 3, 4, 5 ]
> a.push(1,4,6)
6
> a
UniqueArray(6) [ 1, 2, 3, 4, 5, 6 ]
> a.unshift(1)
6
> a
UniqueArray(6) [ 1, 2, 3, 4, 5, 6 ]
> a.unshift(0)
7
> a
UniqueArray(7) [
  0, 1, 2, 3,
  4, 5, 6
]
> a.concat(2,3,7)
UniqueArray(8) [
  0, 1, 2, 3,
  4, 5, 6, 7
]

Landed

Answer 48

Using ES6 (one-liner)

Array of Primitive values

let originalArr= ['a', 1, 'a', 2, '1'];

let uniqueArr = [...new Set(originalArr)];

Array of Objects

let uniqueObjArr = [...new Map(originalObjArr.map((item) => [item["propertyName"], item])).values()];

const ObjArray = [
    {
        name: "Eva Devore",
        character: "Evandra",
        episodes: 15,
    },
    {
        name: "Alessia Medina",
        character: "Nixie",
        episodes: 15,
    },
    {
        name: "Kendall Drury",
        character: "DM",
        episodes: 15,
    },
    {
        name: "Thomas Taufan",
        character: "Antrius",
        episodes: 14,
    },
    {
        name: "Alessia Medina",
        character: "Nixie",
        episodes: 15,
    },
];

let uniqueObjArray = [...new Map(ObjArray.map((item) => [item["id"], item])).values()];

Stimulus

Answer 49

You can also use sugar.js:

[1,2,2,3,1].unique() // => [1,2,3]

[{id:5, name:"Jay"}, {id:6, name:"Jay"}, {id: 5, name:"Jay"}].unique('id') 
  // => [{id:5, name:"Jay"}, {id:6, name:"Jay"}]

Watersick

Answer 50

Yet another solution for the pile.

I recently needed to make a sorted list unique and I did it using filter that keeps track of the previous item in an object like this:

uniqueArray = sortedArray.filter(function(e) { 
    if(e==this.last) 
      return false; 
    this.last=e; return true;  
  },{last:null});

Simonsimona

Answer 51

The version that accepts selector, should be pretty fast and concise:

function unique(xs, f) {
  var seen = {};
  return xs.filter(function(x) {
    var fx = (f && f(x)) || x;
    return !seen[fx] && (seen[fx] = 1);
  });
}

Ildaile

Answer 52

This one is not pure, it will modify the array, but this is the fastest one. If yours is faster, then please write in the comments ;)

http://jsperf.com/unique-array-webdeb

Array.prototype.uniq = function(){
    for(var i = 0, l = this.length; i < l; ++i){
        var item = this[i];
        var duplicateIdx = this.indexOf(item, i + 1);
        while(duplicateIdx != -1) {
            this.splice(duplicateIdx, 1);
            duplicateIdx = this.indexOf(item, duplicateIdx);
            l--;
        }
    }

    return this;
}

[
 "",2,4,"A","abc",
 "",2,4,"A","abc",
 "",2,4,"A","abc",
 "",2,4,"A","abc",
 "",2,4,"A","abc",
 "",2,4,"A","abc",
 "",2,4,"A","abc",
 "",2,4,"A","abc"
].uniq() //  ["",2,4,"A","abc"]

Stalnaker

Answer 53

var a = [1,4,2,7,1,5,9,2,4,7,2]
var b = {}, c = {};
var len = a.length;
for(var i=0;i<len;i++){
  a[i] in c ? delete b[a[i]] : b[a[i]] = true;
  c[a[i]] = true;
} 

// b contains all unique elements

Generable

b returns {5: true, 9: true} in the example above (demo). – Veldaveleda

Answer 54

For a array of strings:

function removeDuplicatesFromArray(arr) {
  const unique = {};
  arr.forEach((word) => {
    unique[word] = 1; // it doesn't really matter what goes here
  });
  return Object.keys(unique);
}

Antepenult

why is this answer down voted? It does work as expect! – Abhorrence

Thank you. That would also be my approach here as one-liner, with a as the array with duplicates and h as helper array: var h = {}; for (var k in a) { h[a[k]] = true;}; var unique = Object.keys(h); I like using the Object's internal hashtable. – Mahdi

Answer 55

You can use Ramda.js, a functional javascript library to do this:

var unique = R.uniq([1, 2, 1, 3, 1, 4])
console.log(unique)

<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.25.0/ramda.js"></script>

Lasonde

Answer 56

A lot of people have already mentioned using...

[...new Set(arr)];

And this is a great solution, but my preference is a solution that works with .filter. In my opinion filter is a more natural way to get unique values. You're effectively removing duplicates, and removing elements from an array is exactly what filter is meant for. It also lets you chain off of .map, .reduce and other .filter calls. I devised this solution...

const unique = () => {
  let cache;  
  return (elem, index, array) => {
    if (!cache) cache = new Set(array);
    return cache.delete(elem);
  };
};

myArray.filter(unique());

The caveat is that you need a closure, but I think this is a worthy tradeoff. In terms of performance, it is more performant than the other solutions I have seen posted that use .filter, but worse performing than [...new Set(arr)].

See also my github package youneek

Pechora

Answer 57

If you want to only get the unique elements and remove the elements which repeats even once, you can do this:

let array = [2, 3, 4, 1, 2, 8, 1, 1, 2, 9, 3, 5, 3, 4, 8, 4];

function removeDuplicates(inputArray) {
  let output = [];
  let countObject = {};

  for (value of array) {
    countObject[value] = (countObject[value] || 0) + 1;
  }

  for (key in countObject) {
    if (countObject[key] === 1) {
      output.push(key);
    }
  }

  return output;
}

console.log(removeDuplicates(array));

Galore

Answer 58

You don't need .indexOf() at all; you can do this O(n):

function SelectDistinct(array) {
    const seenIt = new Set();

    return array.filter(function (val) {
        if (seenIt.has(val)) { 
            return false;
        }

        seenIt.add(val);

        return true;
    });
}

var hasDuplicates = [1,2,3,4,5,5,6,7,7];
console.log(SelectDistinct(hasDuplicates)) //[1,2,3,4,5,6,7]

If you don't want to use .filter():

function SelectDistinct(array) {
    const seenIt = new Set();
    const distinct = [];

    for (let i = 0; i < array.length; i++) {
        const value = array[i];

        if (!seenIt.has(value)) {
            seenIt.add(value);
            distinct.push(value);
        }
    }
    
    return distinct; 
    /* you could also drop the 'distinct' array and return 'Array.from(seenIt)', which converts the set object to an array */
}

Wonderful

Answer 59

in my solution, I sort data before filtering :

const uniqSortedArray = dataArray.sort().filter((v, idx, t) => idx==0 || v != t[idx-1]); 

Checkoff

Answer 60

If you want to remove duplicates, return the whole objects and want to use ES6 Set and Map syntax, and also run only one loop, you can try this, to get unique ids:

const collection = [{id:3, name: "A"}, {id:3, name: "B"}, {id:4, name: "C"}, {id:5, name: "D"}]

function returnUnique(itemsCollection){
  const itemsMap = new Map();

  itemsCollection.forEach(item => {
    if(itemsMap.size === 0){
      itemsMap.set(item.id, item)       
    }else if(!itemsMap.has(item.id)){
      itemsMap.set(item.id, item)
    }
  });
  
    return [...new Set(itemsMap.values())];
 }

console.log(returnUnique(collection));

Osteoblast

Answer 61

For an array of tuples, I'll throw things into a Map and let it do the work. With this approach, you have to be mindful about the key you want to be using:

const arrayOfArraysWithDuplicates = [
    [1, 'AB'],
    [2, 'CD'],
    [3, 'EF'],
    [1, 'AB'],
    [2, 'CD'],
    [3, 'EF'],
    [3, 'GH'],
]

const uniqueByFirstValue = new Map();
const uniqueBySecondValue = new Map();

arrayOfArraysWithDuplicates.forEach((item) => {
    uniqueByFirstValue.set(item[0], item[1]);
    uniqueBySecondValue.set(item[1], item[0]);
});

let uniqueList = Array.from( uniqueByFirstValue, ( [ value, name ] ) => ( [value, name] ) );

console.log('Unique by first value:');
console.log(uniqueList);

uniqueList = Array.from( uniqueBySecondValue, ( [ value, name ] ) => ( [value, name] ) );

console.log('Unique by second value:');
console.log(uniqueList);

Output:

Unique by first value:
[ [ 1, 'AB' ], [ 2, 'CD' ], [ 3, 'GH' ] ]

Unique by second value:
[ [ 'AB', 1 ], [ 'CD', 2 ], [ 'EF', 3 ], [ 'GH', 3 ] ]

Giliane

Answer 62

Always remember, The build-in methods are easy to use. But keep in mind that they have a complexity.

The basic logic is best. There is no hidden complexity.

let list = [1, 1, 2, 100, 2] // your array
let check = {}
list = list.filter(item => {
    if(!check[item]) {
        check[item] = true
        return true;
    }
})

or use, let check = [] if you need future traverse to checked items (waste of memory though)

Camarilla

Maybe filter() was not released at time when this question was asked. – Clinquant

Does this return the new array without duplicates? – Jaddo

Yes @Jaddo – Camarilla

Answer 63

In ES6/ES2015 and later, you can achieve a succinct one-liner for filtering out duplicate items in an array using the Set and the Spread Operator:

let uniqueItems = Array.from(new Set(items));

This solution maintains clarity by explicitly converting the Set back into an array using Array.from(). The end result is an array containing only unique items:

[4, 5, 6, 3, 2, 23, 1]

Tommietommy

Answer 64

If order is not important then we can make an hash and get the keys to make unique array.

var ar = [1,3,4,5,5,6,5,6,2,1];
var uarEle = {};
links.forEach(function(a){ uarEle[a] = 1; });
var uar = keys(uarEle)

uar will be having the unique array elements.

Slumber

Answer 65

I looked at Joeytje50's code on jsperf who has compared a number of alternatives. His code had many minor typos, which made a difference in the performance and the correctness.

More importantly, he is testing on a very small array. I made an array with 1000 integers. Each integer was 100 times a random integer between 0 and 1000. This makes for about 1000/e = 368 duplicates on the average. The results are at jsperf.

This is a much more realistic scenario of where efficiency might be needed. These changes make dramatic changes in the claims (specifically the code touted as fastest is nowhere near fast). The obvious winners are where hashing techniques are used, with the best one being

Array.prototype.getUnique3 = function(){
   var u = Object.create(null), a = [];
   for(var i = 0, l = this.length; i < l; ++i){
      if(this[i] in u) continue;
      a.push(this[i]);
      u[this[i]] = 1;
   }
   return a.length;
}

Unconquerable

Answer 66

I know this has been answered to death already... but... no one has mentioned the javascript implementation of linq. Then the .distinct() method can be used - and it makes the code super easy to read.

var Linq = require('linq-es2015');
var distinctValues =  Linq.asEnumerable(testValues)
            .Select(x)
            .distinct()
            .toArray();

var testValues = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 1];

var distinctValues = Enumerable.asEnumerable(testValues)
  .distinct()
  .toArray();

console.log(distinctValues);

<script src="https://npmcdn.com/linq-es5/dist/linq.js"></script>

Diandre

Answer 67

If you have an array of objects, and you want a uniqueBy function, say, by an id field:

function uniqueBy(field, arr) {
   return arr.reduce((acc, curr) => {
     const exists = acc.find(v => v[field] === curr[field]);
     return exists ? acc : acc.concat(curr);
   }, [])
}

Adornment

Answer 68

Making an array of unique arrays, using field[2] as an Id:

const arr = [
  ['497', 'Q0', 'WTX091-B06-138', '0', '1.000000000', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-92', '1', '0.866899288', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-92', '2', '0.846036819', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-57', '3', '0.835025326', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B43-79', '4', '0.765068215', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B43-56', '5', '0.764211464', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B44-448', '6', '0.761701704', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B44-12', '7', '0.761701704', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B49-128', '8', '0.747434800', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B18-17', '9', '0.746724770', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B19-374', '10', '0.733379549', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B19-344', '11', '0.731421782', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-92', '12', '0.726450470', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B19-174', '13', '0.712757036', 'GROUP001']
];


arr.filter((val1, idx1, arr) => !!~val1.indexOf(val1[2]) &&
  !(arr.filter((val2, idx2) => !!~val2.indexOf(val1[2]) &&
    idx2 < idx1).length));

console.log(arr);

Hobo

Answer 69

This is an ES6 function which removes duplicates from an array of objects, filtering by the specified object property

function dedupe(arr = [], fnCheck = _ => _) {
  const set = new Set();
  let len = arr.length;

  for (let i = 0; i < len; i++) {
    const primitive = fnCheck(arr[i]);
    if (set.has(primitive)) {
      // duplicate, cut it
      arr.splice(i, 1);
      i--;
      len--;
    } else {
      // new item, add it
      set.add(primitive);
    }
  }

  return arr;
}

const test = [
    {video:{slug: "a"}},
    {video:{slug: "a"}},
    {video:{slug: "b"}},
    {video:{slug: "c"}},
    {video:{slug: "c"}}
]
console.log(dedupe(test, x => x.video.slug));

// [{video:{slug: "a"}}, {video:{slug: "b"}}, {video:{slug: "c"}}]

Anatto

Answer 70

I have a solution that uses es6 reduce and find array helper methods to remove duplicates.

let numbers = [2, 2, 3, 3, 5, 6, 6];

const removeDups = array => {
  return array.reduce((acc, inc) => {
    if (!acc.find(i => i === inc)) {
      acc.push(inc);
    }
    return acc;
  }, []);
}

console.log(removeDups(numbers)); /// [2,3,5,6]

Newspaper

Answer 71

The Object answer above does not seem to work for me in my use case with Objects.

I have modified it as follows:

var j = {};

this.forEach( function(v) {
   var typ = typeof v;
   var v = (typ === 'object') ? JSON.stringify(v) : v;

   j[v + '::' + typ] = v;
});

return Object.keys(j).map(function(v){
  if ( v.indexOf('::object') > -1 ) {
    return JSON.parse(j[v]);
  }

  return j[v];
});

This seems to now work correctly for objects, arrays, arrays with mixed values, booleans, etc.

Holeandcorner

Answer 72

var numbers = [1, 1, 2, 3, 4, 4];

function unique(dupArray) {
  return dupArray.reduce(function(previous, num) {

    if (previous.find(function(item) {
        return item == num;
      })) {
      return previous;
    } else {
      previous.push(num);
      return previous;
    }
  }, [])
}

var check = unique(numbers);
console.log(check);

Demp

Answer 73

To filter-out undefined and null values because most of the time you do not need them.

const uniques = myArray.filter(e => e).filter((e, i, a) => a.indexOf(e) === i);

or

const uniques = [...new Set(myArray.filter(e => e))];

Parol

Answer 74

Sometimes I need to get unique occurrences from an array of objects. Lodash seems like a nice helper but I don't think filtering an array justifies adding a dependency to a project.

Let's assume the comparison of two objects poses on comparing a property, an id for example.

const a = [{id: 3}, {id: 4}, {id: 3}, {id: 5}, {id: 5}, {id: 5}];

Since we all love one line snippets, here is how it can be done:

a.reduce((acc, curr) => acc.find(e => e.id === curr.id) ? acc : [...acc, curr], [])

Uncircumcision

Answer 75

This solution should be very fast, and will work in many cases.

1. Convert the indexed array items to object keys

2. Use Object.keys function

   var indexArray = ["hi","welcome","welcome",1,-9];
   var keyArray = {};
   indexArray.forEach(function(item){ keyArray[item]=null; });
   var uniqueArray = Object.keys(keyArray);
   

Caliginous

Answer 76

I have a simple example where we can remove objects from array having repeated id in objects,

  let data = new Array({id: 1},{id: 2},{id: 3},{id: 1},{id: 3});
  let unique = [];
  let tempArr = [];
  console.log('before', data);
  data.forEach((value, index) => {
    if (unique.indexOf(value.id) === -1) {
      unique.push(value.id);
    } else {
      tempArr.push(index);    
    }
  });
  tempArr.reverse();
  tempArr.forEach(ele => {
    data.splice(ele, 1);
  });
  console.log(data);

Lobscouse

Answer 77

The easiest way is to transform values into strings to filter also nested objects values.

const uniq = (arg = []) => {
  const stringifyedArg = arg.map(value => JSON.stringify(value))
  return arg.filter((value, index, self) => {
    if (typeof value === 'object')
      return stringifyedArg.indexOf(JSON.stringify(value)) === index
    return self.indexOf(value) === index
  })
}

    console.log(uniq([21, 'twenty one', 21])) // [21, 'twenty one']
    console.log(uniq([{ a: 21 }, { a: 'twenty one' }, { a: 21 }])) // [{a: 21}, {a: 'twenty one'}]

Karyolymph

Answer 78

For my part this was the easiest solution

// A way to check if the arrays are equal
const a = ['A', 'B', 'C'].sort().toString()
const b = ['A', 'C', 'B'].sort().toString()

console.log(a === b); // true


// Test Case
const data = [
  { group: 'A', name: 'SD' },
  { group: 'B', name: 'FI' },
  { group: 'A', name: 'SD' },
  { group: 'B', name: 'CO' }
];

// Return a new Array without dublocates
function unique(data) {
  return data.reduce(function (accumulator, currentValue) {
    // Convert to string in order to check if they are the same value.
    const currentKeys = Object.keys(currentValue).sort().toString();
    const currentValues = Object.values(currentValue).sort().toString();

    let hasObject = false
    
    for (const obj of accumulator) {
      // Convert keys and values into strings so we can
      // see if they are equal with the currentValue
      const keys = Object.keys(obj).sort().toString();
      const values = Object.values(obj).sort().toString();
      // Check if keys and values are equal
      if (keys === currentKeys && values === currentValues) {
        hasObject = true
      }
    }

    // Push the object if it does not exist already.
    if (!hasObject) {
      accumulator.push(currentValue)
    }

    return accumulator
  }, []);
}

// Run Test Case
console.log(unique(data)); // [ { group: 'A', name: 'SD' }, { group: 'B', name: 'FI' }, { group: 'B', name: 'CO' } ]

Chopping

Answer 79

Here is another approach using comparators (I care more about clean code than performance):

const list = [
    {name: "Meier"},
    {name: "Hans"},
    {name: "Meier"},
]
const compare = (a, b) => a.name.localeCompare(b.name);
const uniqueNames = list.makeUnique(compare);
uniqueNames.pushIfAbsent({name: "Hans"}, compare);

Prototype declaration:

declare global {
    interface Array<T>  {
        pushIfAbsent(item: T, compare:(a:T, b:T)=>number): number;
    }
    interface Array<T>  {
        makeUnique(compare:(a:T, b:T)=>number): Array<T>;
    }
}
Array.prototype.pushIfAbsent = function <T>(this:T[], item:T, compare:(a:T, b:T)=>number) {
    if (!this.find(existing => compare(existing, item)===0)) {
        return this.push(item)
    } else {
        return this.length;
    }
}
Array.prototype.makeUnique = function <T>(this:T[], compare:(a:T, b:T)=>number) {
    return this.filter((existing, index, self) => self.findIndex(item => compare(existing, item) == 0) == index);
}

Haden

Answer 80

A modern approach that's extensible, fast, efficient and easy to read, using iter-ops library:

import {pipe, distinct} from 'iter-ops';

const input = [1, 1, 2, 2, 2, 3]; // our data

const i = pipe(input, distinct()); // distinct iterable

console.log([...i]); //=> [1, 2, 3]

And if your input is an array of objects, you will just provide a key selector for the distinct operator.

Adamski

Answer 81

There's already bunch of great answers. Here's my approach.

var removeDuplicates = function(nums) {
    let filteredArr = [];
    nums.forEach((item) => {
        if(!filteredArr.includes(item)) {
            filteredArr.push(item);
        }
    })

  return filteredArr;
}

Alephnull

Answer 82

ES2016 .includes() One Method Simple Answer:

var arr = [1,5,2,4,1,6]
function getOrigs(arr) {
  let unique = []
  arr && arr.forEach(number => {
    !unique.includes(number) && unique.push(number)
    if (number === arr[arr.length - 1]) {
      console.log('unique: ', unique)
    }
  })
}
getOrigs(arr)

Use this instead:

• later ES version
• Simple question shouldn't use multiple advanced JS methods and push(), length() and forEach() are common
• Easier readability utilizing a closure
• Seems better on memory, garbage collection, and performance than the others
• Less lines of code: if you separated lines based on where there is a line ending, you would only need one line of logic (so you can call or refactor this one-liner however you want):

var arr = [1,5,2,4,1,6];
function getOrigs(arr) {let unique = []; 
  arr && arr.forEach(number => !unique.includes(number) && unique.push(number) && ((number === arr[arr.length - 1]) && console.log('unique: ', unique)))};
getOrigs(arr);

Jaddo

Answer 83

Try doing this:

let d_array = [1, 2, 2, 3, 'a', 'b', 'b', 'c'];
d_array = d_array.filter((x,i)=>d_array.indexOf(x)===i);
console.log(d_array); // [1, 2, 3, "a", "b", "c"]

This loops through the array, checks if the first found result of the same entry in the array is the current index, and if so, allows it to be in the array.

Fledgy

Answer 84

I wanted to remove duplicates from an array of objects. Duplicates had the same ids. Here is what I did.

// prev data
const prev = [
  {
    id: 1,
    name: "foo",
  },
  {
    id: 2,
    name: "baz",
  },
  {
    id: 1,
    name: "foo",
  },
];

// method:
// Step 1: put them in an object with the id as the key. Value of same id would get overriden.
// Step 2: get all the values.

const tempObj = {};
prev.forEach((n) => (tempObj[n.id] = n));
const next = Object.values(tempObj);

// result
[
  {
    id: 1,
    name: "foo",
  },
  {
    id: 2,
    name: "baz",
  }
];

Ride

Answer 85

Simple

const listWithDupes = [1, 2, 3, 4, 5, 6, 7, 7, 8, 8, 8, 8, 8, 8, 9];

const uniqueList = Object.values(listWithDupes.reduce((acc, next) => ({ ...acc,
  [next]: next
}), {}));

console.log(uniqueList);

Chemist

Answer 86

An old way With 2 Loops.

let a = ["Messi", 23, false, 11, 17, true, 23, "Messi", "Sachin", "2023", true, "Sachin"];

let b = [];
    for (let i in a) {
        let found = false;
        for (let j in b) {
            if (a[i] === b[j]) {
                found = true;
                break;
            }
        }
        if (found === false) {
            b.push(a[i]);
        }
    }

console.log(b); //['Messi', 23, false, 11, 17, true, 'Sachin', '2023']

Solangesolano

Answer 87

Yet another answer, just because I wrote one for my specific use case. I happened to be sorting the array anyway, and given I'm sorting I can use that to deduplicate.

Note that my sort deals with my specific data types, you might need a different sort depending on what sort of elements you have.

var sortAndDedup = function(array) {
  array.sort(function(a,b){
    if(isNaN(a) && isNaN(b)) { return a > b ? 1 : (a < b ? -1 : 0); }
    if(isNaN(a)) { return 1; }
    if(isNaN(b)) { return -1; }
    return a-b;
  });

  var newArray = [];
  var len = array.length;
  for(var i=0; i<len; i++){
    if(i === 0 || array[i] != array[i-1]){
      newArray.push(array[i]);
    }
  }
};

Martinamartindale

Answer 88

This script modify the array, filtering out duplicated values. It works with numbers and strings.

https://jsfiddle.net/qsdL6y5j/1/

    Array.prototype.getUnique = function () {
        var unique = this.filter(function (elem, pos) {
            return this.indexOf(elem) == pos;
        }.bind(this));
        this.length = 0;
        this.splice(0, 0, unique);
    }

    var duplicates = [0, 0, 1, 1, 2, 3, 1, 1, 0, 4, 4];
    duplicates.getUnique();
    alert(duplicates);

---

This version instead, allow you to return a new array with unique value keeping the original (just pass true).

https://jsfiddle.net/dj7qxyL7/

    Array.prototype.getUnique = function (createArray) {
        createArray = createArray === true ? true : false;
        var temp = JSON.stringify(this);
        temp = JSON.parse(temp);
        if (createArray) {
            var unique = temp.filter(function (elem, pos) {
                return temp.indexOf(elem) == pos;
            }.bind(this));
            return unique;
        }
        else {
            var unique = this.filter(function (elem, pos) {
                return this.indexOf(elem) == pos;
            }.bind(this));
            this.length = 0;
            this.splice(0, 0, unique);
        }
    }

    var duplicates = [0, 0, 1, 1, 2, 3, 1, 1, 0, 4, 4];
    console.log('++++ ovveride')
    duplicates.getUnique();
    console.log(duplicates);
    console.log('++++ new array')
    var duplicates2 = [0, 0, 1, 1, 2, 3, 1, 1, 0, 4, 4];
    var unique = duplicates2.getUnique(true);
    console.log(unique);
    console.log('++++ original')
    console.log(duplicates2);

---

Browser support:

Feature Chrome  Firefox (Gecko)     Internet Explorer   Opera   Safari
Basic support   (Yes)   1.5 (1.8)   9                   (Yes)   (Yes)

Meditate

In the first comment why did you add this.length = 0; and this.splice(0, 0, unique);? Also would this not be enough? Array.prototype.uniques = function() { return this.filter(function(value, index){ return this.indexOf(value) === index; }.bind(this)); }; – Coplanar

Answer 89

You can try this:

function removeDuplicates(arr){
  var temp = arr.sort();
  for(i = 0; i < temp.length; i++){
    if(temp[i] == temp[i + 1]){
      temp.splice(i,1);
      i--;
    }
  }
  return temp;
}

Dreg

Answer 90

Using mongoose I had an array of ObjectIds to work with.

I had a array/list of Object Ids to work with which first needed to be set to an string and after the unique set, amended back to Object Ids.

var mongoose = require('mongoose')

var ids = [ObjectId("1"), ObjectId("2"), ObjectId("3")]

var toStringIds = ids.map(e => '' + e)
let uniqueIds = [...new Set(toStringIds)]
uniqueIds = uniqueIds.map(b => mongoose.Types.ObjectId(b))


console.log("uniqueIds :", uniqueIds)

Plano

Answer 91

let arr = [1,5,1,2,2,1,3,2,3,3,3,1,10]

for(let i = 0; i<arr.length; i++) {

for(let j = i+1; j<arr.length;j++) {

    if(arr[i] === arr[j]) {
        arr[j] = 0;
    }
    
}
if(arr[i] === 0) {
   
  arr.splice(i,1)
  i--
  

}

}

console.log(arr)

Tadzhik

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Daytime

Hello, please don't post code only and add an explantation as to why you think that this is the optimal solution. People are supposed to learn from your answer, which might not occur if they just copy paste code without knowing why it should be used. – Trooper

Answer 92

(function() {
    "use strict";

    Array.prototype.unique = function unique() {
        var self = this;
        return self.filter(function(a) {
            var that = this;
            // console.log(that);
            return !that[a] ? that[a] = true : false;
        }, {});
    }

    var sampleArray = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
    var distinctArray = sampleArray.unique();
    console.log(distinctArray);
})();

Here is the simple way to solve this problem...

Coulisse

Answer 93

Short and sweet prototype solution:

Array.prototype.unique = function() {
  return [...new Set(this)]
}

Mariande

Answer 94

I would sort the array, then all duplicates are neighbours. Then walk once through the array and eliminate all duplicates.

function getUniques(array) {
  var l = array.length
  if(l > 1) {
    // get a cloned copy and sort it
    array = [...array].sort();
    var i = 1, j = 0;
    while(i < l) {
      if(array[i] != array[j]) {
        array[++j] = array[i];
      }
      i++;
    }
    array.length = j + 1;
  }
  return array;
}

Settera