Removing duplicate elements from an array in Swift

Question

I might have an array that looks like the following:

[1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]

Or, really, any sequence of like-typed portions of data. What I want to do is ensure that there is only one of each identical element. For example, the above array would become:

[1, 4, 2, 6, 24, 15, 60]

Notice that the duplicates of 2, 6, and 15 were removed to ensure that there was only one of each identical element. Does Swift provide a way to do this easily, or will I have to do it myself?

Dorfman

The easiest way is to convert the array in an NSSet, NSSet is an unordered collection of objects, if need to keep order NSOrderedSet. – Maleki

You could use the intersection function as you can find in this class with functions for arrays: github.com/pNre/ExSwift/blob/master/ExSwift/Array.swift – Mocha

Not part of Swift but I use Dollar. $.uniq(array) github.com/ankurp/Dollar#uniq---uniq – Mixologist

Probably the most elegant, smartest and fastest answer is provided by mxcl's answer below. Which also helps maintain order – Dogoodism

Why don't you just use Set from Swift ? You'll be able to provide a list of unordered and unique elements. – Shareeshareholder

I agree with @Andrea. You need to use NSOrderedSet if you want to remove duplicated elements in O(N) time and keep the order. NSOrderedSet holds a set and an array inside it: the set is for checking duplication, and the array is for keeping order. If you don't want to use NSOrderedSet, you can just use another Set for checking duplication. Hope this will help. – Predatory

Apple finally realized this is a common problem and added their own solution in swift-algorithms. See MH175's answer. – Posterior

You can try uniqued() method. The uniqued() method returns a sequence, dropping duplicate elements from a sequence. The uniqued(on:) method does the same, using the result of the given closure to determine the "uniqueness" of each element. – Vulturine

Answer 1

You can roll your own, e.g. like this:

func unique<S : Sequence, T : Hashable>(source: S) -> [T] where S.Iterator.Element == T {
    var buffer = [T]()
    var added = Set<T>()
    for elem in source {
        if !added.contains(elem) {
            buffer.append(elem)
            added.insert(elem)
        }
    }
    return buffer
}

let vals = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let uniqueVals = uniq(vals) // [1, 4, 2, 6, 24, 15, 60]

And as an extension for Array:

extension Array where Element: Hashable {
    func uniqued() -> Array {
        var buffer = Array()
        var added = Set<Element>()
        for elem in self {
            if !added.contains(elem) {
                buffer.append(elem)
                added.insert(elem)
            }
        }
        return buffer
    }
}

Or more elegantly (Swift 4/5):

extension Sequence where Element: Hashable {
    func uniqued() -> [Element] {
        var set = Set<Element>()
        return filter { set.insert($0).inserted }
    }
}

Which would be used:

[1,2,4,2,1].uniqued()  // => [1,2,4]

Exergue

You could also implement the body of that function as var addedDict = [T:Bool](); return filter(source) { addedDict(true, forKey: $0) == nil } – Lattermost

Very nice. Your solution will be my preferred approach for this kind of situations from now on. – Exergue

@AirspeedVelocity: Did you mean updateValue(true, forKey: $0)... instead of addedDict(true, forKey: $0)... – Inpatient

Oops yes sorry I accidentally the method! Should be return filter(source) { addedDict.updateValue(true, forKey: $0) == nil } as you say. – Lattermost

Is this answer, or converting it to a set faster? – Masochism

Converting to a set, I suppose. I don't see how this one could possibly be faster, as it indeed adds each element to the set, with additional logic on top. But this approach preserves the initial order, which may be a necessity. – Exergue

Just a word of caution: Avoid discussing performance for simple functions like this until you provably depend on their performance, at which point the only thing you should do is benchmark. Too often have I seen unmaintainable code or even less performant code due to making assumptions. :) Also, this is probably easier to grasp: let uniques = Array(Set(vals)) – Zajac

@Zajac Agreed. Once again, here the advantage lies in respecting the order of elements of the original array. – Exergue

What would be safest way to define this method as extension of SequenceType? Does it need to be mutating? I tried myself, but failed :/ – Domeniga

Why do you limit T to be Hashable? I'm assuming it's because if it's not Hashable you're not allowed to use on Sets. I'm trying to remove duplicates from an multi-demensional array, but since arrays aren't hashable...I'm thinking I should simply just append them to an array, rather than inserting. Obviously using sets are faster because it benefits from hashing... – Dogoodism

i have [[1,2,4],[1,2,3],[2,3,1],[2,3,4]] this kind of array so what i have to change in your code – Participle

@Jean-PhilippePellet @Zajac Some cursory benchmarking: For a large number of elements, observed all Set based variants to benchmark roughly the same - just minor differences over the implementations here (5-10%). Only the use of a Dictionary over Set benchmarks consistently slower (~0.5x). Release -O, swift 4.1 – Ciaphus

@Cœur I wrote that comment a while ago. Wouldn't it be better to limit it Equatable. That way it be more inclusive? – Dogoodism

@Honey if you use Equatable, then you can't use a Set (you'd need an Array) and you lose in performances. So you should only add an Equatable overload if you really need it. I would be curious to know your use case. – Opacity

@Cœur I don't remember my case. Merely pointing out that this implementation is restricting, but possibly more optimized. – Dogoodism

I don't see any use of Set<T> – Kekkonen

Converting to a set does NOT guarantee order. – Anaphora

Answer 2

You can convert to a Set and back to an Array again quite easily:

let unique = Array(Set(originals))

This is not guaranteed to maintain the original order of the array.

Chrysolite

Yup, after Swift 1.2, this is the way to roll this issue. – Elsyelton

Is there a way to use a set while preserving the original order of the array? – Probity

@Probity See my answer. – Exergue

If you need to keep the objects unique by a specific property, than also implement the Hashable and Equatable protocol on that class, instead of just using the Array->Set->Array transformation – Winkle

What if you wanted to remove the duplicates altogether (rather than keeping one of them) ? – Fiord

Nice!! What's the time complexity of this solution please? – Dime

Does this makes it change the original order? – Rozanne

For Swift3, I needed to set the type for the Set ... and make my struct conform to Hashable – Assuntaassur

This solution is simple but: You have to conform to equatable protocol. Changing Set to Array also doesn't guarantee keeping the same order of elements. – Selemas

This probably doesn't preserve the order! – Cyprus

This gives problems on iOS 12 – Forejudge

@RudolfJ What kind of problems? – Chrysolite

Fails if the elements in originals are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array. – Machine

I don't understand why this answer has so many upvotes. Seems like maintaining the order of the array is almost certainly a requirement. Otherwise you might as well just use a Set instead of an Array in the first place. – Postpaid

Based on the comments to this Answer, I do think the Ordered Set edit is quite essential... even if that's not explicitly asked in the question, that's likely what many/most people finding this page want. Good to have that related option in the high-voted answer rather than much further down the page. – Ewaewald

Answer 3

Swift 4

public extension Array where Element: Hashable {
    func uniqued() -> [Element] {
        var seen = Set<Element>()
        return filter{ seen.insert($0).inserted }
    }
}

---

every attempt to insert will also return a tuple: (inserted: Bool, memberAfterInsert: Set.Element). See documentation.

Using the returned value means we can avoid doing more than one loop, so this is O(n).

Gwendolin

After simple profiling, this method is really fast. Its hundreds times faster then using reduce( _: _:), or even reduce(into: _:) – Oregano

@Oregano Because all those other algorithm were O(n^2), and nobody noticed. – Bensky

@Oregano this answer is identical to Eneko Alonso answer + my comment (Jun 16 '17). – Opacity

Beware that duplicity is determined by Hash values of two objects/structs and not the == equality in Equatable protocol. If you want to determine duplicity lets say in struct hased only on some properties (not all properties), you need to use custom Hashable implementation with func hash(into hasher: inout Hasher) – Heldentenor

We will use Set<Element>(minimumCapacity: count) initialiser in this answer. – Amora

Answer 4

Use a Set or NSOrderedSet to remove duplicates, then convert back to an Array:

let uniqueUnordered = Array(Set(array))
let uniqueOrdered = Array(NSOrderedSet(array: array))

Stepp

let uniqueOrderedNames = Array(NSOrderedSet(array: userNames)) as! [String] if u have array of String, not of Any – Unfruitful

Fails if the elements in array are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array. – Machine

Tested in Swift 5.1b5, given that the elements are Hashable and a desire to retain ordering, the NSOrderedSet(array: array).array is marginally faster than the pure swift func uniqued() using a set with filter. I tested with 5100 strings that resulted in 13 unique values. – Jobye

Array(NSOrderedSet(array: array)) is not working in Swift 5. Use NSOrderedSet(array: array).array as! [String] instead. – Anus

The second one only works for "primitive" types – Tearing

Answer 5

Many answers available here, but I missed this simple extension, suitable for Swift 2 and up:

extension Array where Element:Equatable {
    func removeDuplicates() -> [Element] {
        var result = [Element]()

        for value in self {
            if result.contains(value) == false {
                result.append(value)
            }
        }

        return result
    }
}

Makes it super simple. Can be called like this:

let arrayOfInts = [2, 2, 4, 4]
print(arrayOfInts.removeDuplicates()) // Prints: [2, 4]

Filtering based on properties

To filter an array based on properties, you can use this method:

extension Array {

    func filterDuplicates(@noescape includeElement: (lhs:Element, rhs:Element) -> Bool) -> [Element]{
        var results = [Element]()

        forEach { (element) in
            let existingElements = results.filter {
                return includeElement(lhs: element, rhs: $0)
            }
            if existingElements.count == 0 {
                results.append(element)
            }
        }

        return results
    }
}

Which you can call as followed:

let filteredElements = myElements.filterDuplicates { $0.PropertyOne == $1.PropertyOne && $0.PropertyTwo == $1.PropertyTwo }

Treblinka

@Treblinka Thank you for the Filtering based on properties extension. It's really useful. But can you please explain how it works. It's too hard to understand for me. Thank you – Alfalfa

Updates for swift 3: func filterDuplicates(_ includeElement: (_ lhs:Element, _ rhs:Element) -> Bool) -> [Element]{ – Geodesy

The first part of this answer (extension Array where Element: Equatable) is being superseded by https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift which offers a more powerful solution (extension Sequence where Iterator.Element: Equatable). – Opacity

This will have O(n²) time performance, which is really bad for large arrays. – Hallette

You should use a set to keep track of elements seen so far, to bring this terrible O(n²) complexity back down to O(n) – Bensky

Answer 6

If you put both extensions in your code, the faster Hashable version will be used when possible, and the Equatable version will be used as a fallback.

public extension Sequence where Element: Hashable {
  /// The elements of the sequence, with duplicates removed.
  /// - Note: Has equivalent elements to `Set(self)`.
  var firstUniqueElements: [Element] {
    let getSelf: (Element) -> Element = \.self
    return firstUniqueElements(getSelf)
  }
}

public extension Sequence where Element: Equatable {
  /// The elements of the sequence, with duplicates removed.
  /// - Note: Has equivalent elements to `Set(self)`.
  var firstUniqueElements: [Element] {
    let getSelf: (Element) -> Element = \.self
    return firstUniqueElements(getSelf)
  }
}

public extension Sequence {
  /// The elements of the sequences, with "duplicates" removed
  /// based on a closure.
  func firstUniqueElements<Hashable: Swift.Hashable>(
    _ getHashable: (Element) -> Hashable
  ) -> [Element] {
    var set: Set<Hashable> = []
    return filter { set.insert(getHashable($0)).inserted }
  }

  /// The elements of the sequence, with "duplicates" removed,
  /// based on a closure.
  func firstUniqueElements<Equatable: Swift.Equatable>(
    _ getEquatable: (Element) -> Equatable
  ) -> [Element] {
    reduce(into: []) { uniqueElements, element in
      if zip(
        uniqueElements.lazy.map(getEquatable),
        AnyIterator { [equatable = getEquatable(element)] in equatable }
      ).allSatisfy(!=) {
        uniqueElements.append(element)
      }
    }
  }
}

If order isn't important, then you can always just use this Set initializer.

Taxexempt

okay, got it. i wasnt able to call it because my array is an array of structs... how would i handle it in my case? struct of 20 different variables, string and [string] – Lanell

@David Seek It sounds like you haven't made your strict hashable or equatable. Is that correct? – Taxexempt

@DavidSeek like this, uniqueArray = nonUniqueArray.uniqueElements – Bucket

yeah dont worry. got it working right after. been almost 2 years now :P – Lanell

This will have O(n²) time performance, which is really bad for large arrays. – Hallette

The hahsable version will have better performance, but won't preserve the order of elements in the original array. Leo's answer will give both O(n) performance AND preserve object ordering. – Hallette

More precisely, it preserves the original order of the array (whatever that is) and removes all but the first occurrence of duplicate items. – Hallette

@Jessy There are already multiple O(1) answers, but they have way less votes than most of the naive O(n^2) solutions. This one is particularly good for its simplicity: https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Bensky

@Bensky That requires Hashable. As I state in my answer, my Hashable implementation from Swift 3 translates directly to Swift 4. You need to include solutions for both Hashable and Equatable for the beginning of my answer to be accurate. – Taxexempt

Answer 7

edit/update Swift 4 or later

We can also extend RangeReplaceableCollection protocol to allow it to be used with StringProtocol types as well:

extension RangeReplaceableCollection where Element: Hashable {
    var orderedSet: Self {
        var set = Set<Element>()
        return filter { set.insert($0).inserted }
    }
    mutating func removeDuplicates() {
        var set = Set<Element>()
        removeAll { !set.insert($0).inserted }
    }
}

---

let integers = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let integersOrderedSet = integers.orderedSet // [1, 4, 2, 6, 24, 15, 60]

---

"abcdefabcghi".orderedSet  // "abcdefghi"
"abcdefabcghi".dropFirst(3).orderedSet // "defabcghi"

---

Mutating method:

var string = "abcdefabcghi"
string.removeDuplicates() 
string  //  "abcdefghi"

var substring = "abcdefabcdefghi".dropFirst(3)  // "defabcdefghi"
substring.removeDuplicates()
substring   // "defabcghi"

---

For Swift 3 click here

Chomp

I like this, it works with an array of dictionarys too! – Qua

@Bensky Leo Dabus has replaced the reduce implementation, so now the complexity is different. – Opacity

What is the expression .inserted above? I can't find it anywhere. – Hallette

@DuncanC it is the return value from the insert method.developer.apple.com/documentation/swift/set/1541375-insert it returns a tuple. (inserted: Bool, memberAfterInsert: Element) if newMember was not contained in the set. If an element equal to newMember was already contained in the set, the method returns (false, oldMember), where oldMember is the element that was equal to newMember. In some cases, oldMember may be distinguishable from newMember by identity comparison or some other means. – Chomp

Yup, I found it finally. That's very cool. What's the time complexity of your solution? At a glance I think it would be n • log(n), since sets use hashes to determine membership. The naive solution of using array.contains yields n^2 performance, which is really bad. – Hallette

@DuncanC I don't know the complexity but I think it can't get any faster than that – Chomp

I just did a quick test on 1m and 8m items, and it seems to run in O(n) time! – Hallette

Which one is faster flatmap or filter ? Or is it the same? – Chomp

The results are interesting. For both 1 million unique items and 8 million, the filter version is faster. However, the filter-based version takes 8.38x longer for 8 million unique items (a hair over O(n) time), where the flatmap-based version takes 7.47x longer for 8 million unique entries than 1 million, suggesting that the flatmap based version scales better. Somehow the flatmap based version does slightly better than O(n) time! – Hallette

In fact, when I run the test with 64x more items in the array, the flatmap based version is faster. – Hallette

@Jessy yes it keeps the original order. It is not called sortedSet. Btw it uses the same name Apple uses in NSOrderedSet just dropping the NS – Chomp

"Original order" is an undefined concept, but wow, you are correct about Apple naming it wrong too. NSOrderedSet(array: [1, 3, 2, 3]) does not enforce order; it's [1, 3, 2]. – Taxexempt

kkkk Why do you think your way is correct? How would you name it? Note that this method it is not restricted to numbers. Its is constrained to Hashable not Comparable – Chomp

@Jessy, ordered collection means that no matter how many times you gonna access it, it will have same order of elements. Like Array, each element will stay on it's place. For Set and Dictionary it's not true, if you will try to iterate over or print content you may have elements in different order each time. And that type of collections is called unordered. – Kentish

@DuncanC a filter { expression } is a flatmap { expression ? $0 : nil }, so there shouldn't be any difference and I recommend filter for readability. – Opacity

Answer 8

Swift 4

Guaranteed to keep ordering.

extension Array where Element: Equatable {
    func removingDuplicates() -> Array {
        return reduce(into: []) { result, element in
            if !result.contains(element) {
                result.append(element)
            }
        }
    }
}

Advocaat

I use this now, only changed the method name to removeDuplicates :) – Dibucaine

I guess this solution is compact, but I believe that deanWombourne solution posted a year earlier may be slightly more efficient than a reduce: overall, it's just one more line in your whole project to write your function as: var unique: [Iterator.Element] = []; for element in self where !unique.contains(element) { unique.append(element) }; return unique. I admit I haven't tested the relative performances yet. – Opacity

This will have O(n²) time performance, which is really bad for large arrays. – Hallette

@NickGaens No it's not, it's O(n²). There's nothing swift about this. – Bensky

@Cœur reduce or reduce(into:) wouldn't make a critical difference. Rewriting this to not repeatedly call contains would make a MUCH bigger difference. – Bensky

@Cœur Of course you can. It's actually quite simple. You just filter array elements according to whether or not you have seen them already (which you maintain using a set for O(1) insert and contains). See https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Bensky

@Bensky sorry, yes, I was distracted. I even found that solution BEFORE mxcl. – Opacity

Yes, as many comments pointed out, this is not very time efficient and, if the Elements is also Hashable, other solutions are more efficient. – Advocaat

Answer 9

Inspired by https://www.swiftbysundell.com/posts/the-power-of-key-paths-in-swift, we can declare a more powerful tool that is able to filter for unicity on any keyPath. Thanks to Alexander comments on various answers regarding complexity, the below solutions should be near optimal.

Non-mutating solution

We extend with a function that is able to filter for unicity on any keyPath:

extension RangeReplaceableCollection {
    /// Returns a collection containing, in order, the first instances of
    /// elements of the sequence that compare equally for the keyPath.
    func unique<T: Hashable>(for keyPath: KeyPath<Element, T>) -> Self {
        var unique = Set<T>()
        return filter { unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Note: in the case where your object doesn't conform to RangeReplaceableCollection, but does conform to Sequence, you can have this additional extension, but the return type will always be an Array:

extension Sequence {
    /// Returns an array containing, in order, the first instances of
    /// elements of the sequence that compare equally for the keyPath.
    func unique<T: Hashable>(for keyPath: KeyPath<Element, T>) -> [Element] {
        var unique = Set<T>()
        return filter { unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Usage

If we want unicity for elements themselves, as in the question, we use the keyPath \.self:

let a = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let b = a.unique(for: \.self)
/* b is [1, 4, 2, 6, 24, 15, 60] */

If we want unicity for something else (like for the id of a collection of objects) then we use the keyPath of our choice:

let a = [CGPoint(x: 1, y: 1), CGPoint(x: 2, y: 1), CGPoint(x: 1, y: 2)]
let b = a.unique(for: \.y)
/* b is [{x 1 y 1}, {x 1 y 2}] */

---

Mutating solution

We extend with a mutating function that is able to filter for unicity on any keyPath:

extension RangeReplaceableCollection {
    /// Keeps only, in order, the first instances of
    /// elements of the collection that compare equally for the keyPath.
    mutating func uniqueInPlace<T: Hashable>(for keyPath: KeyPath<Element, T>) {
        var unique = Set<T>()
        removeAll { !unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Usage

If we want unicity for elements themselves, as in the question, we use the keyPath \.self:

var a = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
a.uniqueInPlace(for: \.self)
/* a is [1, 4, 2, 6, 24, 15, 60] */

If we want unicity for something else (like for the id of a collection of objects) then we use the keyPath of our choice:

var a = [CGPoint(x: 1, y: 1), CGPoint(x: 2, y: 1), CGPoint(x: 1, y: 2)]
a.uniqueInPlace(for: \.y)
/* a is [{x 1 y 1}, {x 1 y 2}] */

Sink

Now that's a good implementation! I only with that key paths were convertible to closures, so that you can use a closure arg to support both arbitrary code (in closures) and mere property look ups (via key paths). Only change I would make is to make keyPath default to \.self, because that's probably the majority of use cases. – Bensky

@Bensky I tried to default to Self, but then I would need to make Element always Hashable. An alternative to a default value is adding a simple overload without parameters: extension Sequence where Element: Hashable { func unique() { ... } } – Opacity

Ah yes, makes sense! – Bensky

Brilliant ... simple, and best of all 'flexible'. Thx. – Ruttger

@Alexander-ReinstateMonica: This looks very similar to your own solution from March 2018: gist.github.com/amomchilov/fbba1e58c91fbd4b5b767bcf8586112b 👏👏👏 – Bersagliere

Ah, @FizzBuzz, that's a good find. Congrats to Martin R and Alexander for being, like John Sundell, avant-gardists of those techniques. – Opacity

@Bersagliere I have no idea how you found that, but that's been kind of interesting to see. I forget that all this random crap I put out on the internet sticks around and get eventually seened by people. I can only hope it helped :p – Bensky

Answer 10

Here's a category on SequenceType which preserves the original order of the array, but uses a Set to do the contains lookups to avoid the O(n) cost on Array's contains(_:) method.

public extension Sequence where Element: Hashable {

    /// Return the sequence with all duplicates removed.
    ///
    /// i.e. `[ 1, 2, 3, 1, 2 ].uniqued() == [ 1, 2, 3 ]`
    ///
    /// - note: Taken from https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift, as 
    ///         per @Alexander's comment.
    func uniqued() -> [Element] {
        var seen = Set<Element>()
        return self.filter { seen.insert($0).inserted }
    }
}

If you aren't Hashable or Equatable, you can pass in a predicate to do the equality check:

extension Sequence {

    /// Return the sequence with all duplicates removed.
    ///
    /// Duplicate, in this case, is defined as returning `true` from `comparator`.
    ///
    /// - note: Taken from https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift
    func uniqued(comparator: @escaping (Element, Element) throws -> Bool) rethrows -> [Element] {
        var buffer: [Element] = []

        for element in self {
            // If element is already in buffer, skip to the next element
            if try buffer.contains(where: { try comparator(element, $0) }) {
                continue
            }

            buffer.append(element)
        }

        return buffer
    }
}

Now, if you don't have Hashable, but are Equatable, you can use this method:

extension Sequence where Element: Equatable {

    /// Return the sequence with all duplicates removed.
    ///
    /// i.e. `[ 1, 2, 3, 1, 2 ].uniqued() == [ 1, 2, 3 ]`
    ///
    /// - note: Taken from https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift
    func uniqued() -> [Element] {
        return self.uniqued(comparator: ==)
    }
}

Finally, you can add a key path version of uniqued like this:

extension Sequence {

    /// Returns the sequence with duplicate elements removed, performing the comparison using the property at
    /// the supplied keypath.
    ///
    /// i.e.
    ///
    /// ```
    /// [
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "World")
    ///  ].uniqued(\.value)
    /// ```
    /// would result in
    ///
    /// ```
    /// [
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "World")
    /// ]
    /// ```
    ///
    /// - note: Taken from https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift
    ///
    func uniqued<T: Equatable>(_ keyPath: KeyPath<Element, T>) -> [Element] {
        self.uniqued { $0[keyPath: keyPath] == $1[keyPath: keyPath] }
    }
}

You can stick both of these into your app, Swift will choose the right one depending on your sequence's Iterator.Element type.

---

For El Capitan, you can extend this method to include multiple keypaths like this:

    /// Returns the sequence with duplicate elements removed, performing the comparison using the property at
    /// the supplied keypaths.
    ///
    /// i.e.
    ///
    /// ```
    /// [
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Bean"),
    ///   MyStruct(value1: "World", value2: "Sigh")
    ///  ].uniqued(\.value1, \.value2)
    /// ```
    /// would result in
    ///
    /// ```
    /// [
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Bean"),
    ///   MyStruct(value1: "World", value2: "Sigh")
    /// ]
    /// ```
    ///
    /// - note: Taken from https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift
    ///
    func uniqued<T: Equatable, U: Equatable>(_ keyPath1: KeyPath<Element, T>, _ keyPath2: KeyPath<Element, U>) -> [Element] {
        self.uniqued {
            $0[keyPath: keyPath1] == $1[keyPath: keyPath1] && $0[keyPath: keyPath2] == $1[keyPath: keyPath2]
        }
    }

but (imho) you're probably better off just passing in your own block to self.uniqued.

Tridactyl

Heyyy finally someone with an O(n) solution. You can combine the "check" and "insert" set operations into one, by the way. See https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Bensky

Oh, that's clever :) – Tridactyl

@Tridactyl How to distinct elements by multiple keypaths? – Debauchee

@EICaptainv2.0 You can just extend the uniqued method to take two generic parameters and check them both for equality - check out the edit I just made. The items are only duplicates if both of the values specified by the keypaths are the same. – Tridactyl

Cool. Thanks @Tridactyl – Debauchee

Answer 11

No need to write any extensions.

Apple has finally introduced uniqued() method in its Algorithms package.

Points to consider:

1. Not just Arrays. It can be used on any type conforming to Sequence protocol.
2. The Element type in Sequence must conform to Hashable protocol

---

Example:

import Algorithms

let numbers = [1, 2, 3, 3, 2, 3, 3, 2, 2, 2, 1]
print(numbers.uniqued()) // prints [1, 2, 3]

More info https://github.com/apple/swift-algorithms/blob/main/Guides/Unique.md

Peck

Algorithms not from Apple i guess. – Chinfest

@UmitKaya It is absolutely from Apple, just published as a separate package to avoid littering Foundation. – Indole

Note that this still requires that the elements conform to Hashable – Shrubby

Answer 12

Think like a functional programmer :)

To filter the list based on whether the element has already occurred, you need the index. You can use enumerated to get the index and map to return to the list of values.

let unique = myArray
    .enumerated()
    .filter{ myArray.firstIndex(of: $0.1) == $0.0 }
    .map{ $0.1 }

This guarantees the order. If you don't mind about the order then the existing answer of Array(Set(myArray)) is simpler and probably more efficient.

---

UPDATE: Some notes on efficiency and correctness

A few people have commented on the efficiency. I'm definitely in the school of writing correct and simple code first and then figuring out bottlenecks later, though I appreciate it's debatable whether this is clearer than Array(Set(array)).

This method is a lot slower than Array(Set(array)). As noted in comments, it does preserve order and works on elements that aren't Hashable.

However, @Alain T's method also preserves order and is also a lot faster. So unless your element type is not hashable, or you just need a quick one liner, then I'd suggest going with their solution.

Here are a few tests on a MacBook Pro (2014) on Xcode 11.3.1 (Swift 5.1) in Release mode.

The profiler function and two methods to compare:

func printTimeElapsed(title:String, operation:()->()) {
    var totalTime = 0.0
    for _ in (0..<1000) {
        let startTime = CFAbsoluteTimeGetCurrent()
        operation()
        let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
        totalTime += timeElapsed
    }
    let meanTime = totalTime / 1000
    print("Mean time for \(title): \(meanTime) s")
}

func method1<T: Hashable>(_ array: Array<T>) -> Array<T> {
    return Array(Set(array))
}

func method2<T: Equatable>(_ array: Array<T>) -> Array<T>{
    return array
    .enumerated()
    .filter{ array.firstIndex(of: $0.1) == $0.0 }
    .map{ $0.1 }
}

// Alain T.'s answer (adapted)
func method3<T: Hashable>(_ array: Array<T>) -> Array<T> {
    var uniqueKeys = Set<T>()
    return array.filter{uniqueKeys.insert($0).inserted}
}

And a small variety of test inputs:

func randomString(_ length: Int) -> String {
  let letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
  return String((0..<length).map{ _ in letters.randomElement()! })
}

let shortIntList = (0..<100).map{_ in Int.random(in: 0..<100) }
let longIntList = (0..<10000).map{_ in Int.random(in: 0..<10000) }
let longIntListManyRepetitions = (0..<10000).map{_ in Int.random(in: 0..<100) }
let longStringList = (0..<10000).map{_ in randomString(1000)}
let longMegaStringList = (0..<10000).map{_ in randomString(10000)}

Gives as output:

Mean time for method1 on shortIntList: 2.7358531951904296e-06 s
Mean time for method2 on shortIntList: 4.910230636596679e-06 s
Mean time for method3 on shortIntList: 6.417632102966309e-06 s
Mean time for method1 on longIntList: 0.0002518167495727539 s
Mean time for method2 on longIntList: 0.021718120217323302 s
Mean time for method3 on longIntList: 0.0005312927961349487 s
Mean time for method1 on longIntListManyRepetitions: 0.00014377200603485108 s
Mean time for method2 on longIntListManyRepetitions: 0.0007293639183044434 s
Mean time for method3 on longIntListManyRepetitions: 0.0001843773126602173 s
Mean time for method1 on longStringList: 0.007168249964714051 s
Mean time for method2 on longStringList: 0.9114790915250778 s
Mean time for method3 on longStringList: 0.015888616919517515 s
Mean time for method1 on longMegaStringList: 0.0525397013425827 s
Mean time for method2 on longMegaStringList: 1.111266262292862 s
Mean time for method3 on longMegaStringList: 0.11214958941936493 s

Shavian

unlike Array(Set(myArray)), this works for things that aren't Hashable – Trainband

... and unlike Array(Set(myArray)) the order of your array is maintained. – Calve

It looks like the best answer to me, at least at the present when Swift 5 is already the current version. – Diggins

This is a very elegant solution; unfortunately, it's also rather slow. – Swansea

This is really slow. If you wanted to keep the last of each element, you would be better users a better implementation, and doing array.reversed().unique().reversed(). – Bensky

@Alexander-ReinstateMonica I don't understand your suggestion of array.reversed().unique().reversed(). The above solution keeps the first rather than the last of each element, and as far as I know there is no unique() method - that is the whole point of the question. Anyway, I don't doubt it's probably not the fastest method but unless it's in a performance sensitive place I'd go with readability and clarity of correctness over optimisation. – Shavian

@TimMB Oh I misread your post. I saw someone's adaptation that usedthe lastIndex(of:). I totally disagree over the clarity vs optimization point in this case. I don't think this implementation is particularly clear, especially compared to a simple set-based solution. In any case, such code should be extracted out to an extension function. This algorithm becomes basically unusable at even a low input size, like in the thousands to tens of thousands. It's not hard to find such data sets, people can have thousands of songs, files, contacts, etc. – Bensky

@TimMB You can achieve the same goal of uniquing non-Hashable elements, keeping the first ones, by using Array for contains checks, similar to how the set is being used. That removes the need for enumerated (which doesn't really cost much) and the final map call (which does one full sweep of the full array). Furthermore, you can use .reserveCapacity on the set/array, to tradeoff some extra memory use for removing the need for reallocations mid-way through the algorithm. – Bensky

@TimMB Other things to consider: method3 allows for an implementation that removes elements inline from the existing buffer using removeAll, which isn't possible with methods 1 or 2. Method 2 and 3 are also generalizable to sequences that aren't Collection (which might not be re-iterable) – Bensky

Check out my bench mark results: drive.google.com/a/ryerson.ca/file/d/… full code: gist.github.com/amomchilov/299d012dccba375bf15880355684ebed – Bensky

This is absolute gold – Nakasuji

It should be noted that the fastest solution (after an unsorted set) might be indeed Alain T. approach together your addition of reserveCapacity as linked in the gist. – Edger

Answer 13

An alternate (if not optimal) solution from here using immutable types rather than variables:

func deleteDuplicates<S: ExtensibleCollectionType where S.Generator.Element: Equatable>(seq:S)-> S {
    let s = reduce(seq, S()){
        ac, x in contains(ac,x) ? ac : ac + [x]
    }
    return s
}

Included to contrast Jean-Pillippe's imperative approach with a functional approach.

As a bonus this function works with strings as well as arrays!

Edit: This answer was written in 2014 for Swift 1.0 (before Set was available in Swift). It doesn't require Hashable conformance & runs in quadratic time.

Bushwa

Beware, there are not one, but two ways this runs in quadratic time – both contains and array append run in O(n). Though it does have the benefit of only requiring equatable, not hashable. – Lattermost

this is a really complicated way of writing filter. It's O(n^2) (which is required if you don't want to require Hashable conformance), but you should at least call that out explicitly – Bensky

Answer 14

One more Swift 3.0 solution to remove duplicates from an array. This solution improves on many other solutions already proposed by:

• Preserving the order of the elements in the input array
• Linear complexity O(n): single pass filter O(n) + set insertion O(1)

Given the integer array:

let numberArray = [10, 1, 2, 3, 2, 1, 15, 4, 5, 6, 7, 3, 2, 12, 2, 5, 5, 6, 10, 7, 8, 3, 3, 45, 5, 15, 6, 7, 8, 7]

Functional code:

func orderedSet<T: Hashable>(array: Array<T>) -> Array<T> {
    var unique = Set<T>()
    return array.filter { element in
        return unique.insert(element).inserted
    }
}

orderedSet(array: numberArray)  // [10, 1, 2, 3, 15, 4, 5, 6, 7, 12, 8, 45]

Array extension code:

extension Array where Element:Hashable {
    var orderedSet: Array {
        var unique = Set<Element>()
        return filter { element in
            return unique.insert(element).inserted
        }
    }
}

numberArray.orderedSet // [10, 1, 2, 3, 15, 4, 5, 6, 7, 12, 8, 45]

This code takes advantage of the result returned by the insert operation on Set, which executes on O(1), and returns a tuple indicating if the item was inserted or if it already existed in the set.

If the item was in the set, filter will exclude it from the final result.

Fee

Not to be picky but you will be performing the insert and the membership test as many times as there are elements so you should count their cost as O(n) as well. This doesn't mean 3xO(n) however because these O's and don't have equal cost with the filter so the addition of O(n)'s is apples to oranges. If we consider set operations to be a O(1) part of the filter cost, the complexity is merely O(n), albeit with a larger "O". Pushing this to the limit, you could also avoid the insertions when the element is already in the set. – Squeak

You are right, using defer the code would do the set test operation twice, one with contains and one with insert. Further reading Swift documentation, I found that insert returns a tuple indicating if the element was inserted or not, so I've simplified the code removing the contains check. – Fee

Nice. Your extension could be optimal by doing it on extension Sequence where Iterator.Element: Hashable { ... } – Opacity

@AlainT. Nope. Both insert and contains have O(1) complexity. O(1) + O(1) = O(1). These two operations are then done n times (once per call of the closure passed to filter, which is called once per element) I.e. if an operation takes a constant amount of time regardless of input size, then doing it twice still makes it take a constant time that is regardless of input size. The total complexity of this is O(n). – Bensky

Answer 15

Swift 5

extension Sequence where Element: Hashable {
    func unique() -> [Element] {
        NSOrderedSet(array: self as! [Any]).array as! [Element]
    }
}

Barbur

I did some variation so I could select a key to compare. extension Sequence { // Returns distinct elements based on a key value. func distinct<key: Hashable>(by: ((_ el: Iterator.Element) -> key)) -> [Iterator.Element] { var existing = Set<key>() return self.filter { existing.insert(by($0)).inserted } } } – Caerphilly

There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift Please delete this answer. – Bensky

if your element is Hasable, you can directly use Array(Set(yourElements) – Tearing

This changes the order of the array. – Townie

Answer 16

swift 2

with uniq function answer:

func uniq<S: SequenceType, E: Hashable where E==S.Generator.Element>(source: S) -> [E] {
    var seen: [E:Bool] = [:]
    return source.filter({ (v) -> Bool in
        return seen.updateValue(true, forKey: v) == nil
    })
}

use:

var test = [1,2,3,4,5,6,7,8,9,9,9,9,9,9]
print(uniq(test)) //1,2,3,4,5,6,7,8,9

Pelite

The Bool value obviously is redundant, as your code never reads it. Use a Set instead of a Dictionary and you get my upvote. – Queenqueena

Answer 17

In Swift 5

 var array: [String] =  ["Aman", "Sumit", "Aman", "Sumit", "Mohan", "Mohan", "Amit"]

 let uniq = Array(Set(array))
 print(uniq)

Output Will be

 ["Sumit", "Mohan", "Amit", "Aman"]

Jilolo

This is a repeat of many of the answers already here, and it doesn't preserve ordering. – Bensky

Answer 18

Swift 4.x:

extension Sequence where Iterator.Element: Hashable {
  func unique() -> [Iterator.Element] {
    return Array(Set<Iterator.Element>(self))
  }

  func uniqueOrdered() -> [Iterator.Element] {
    return reduce([Iterator.Element]()) { $0.contains($1) ? $0 : $0 + [$1] }
  }
}

usage:

["Ljubljana", "London", "Los Angeles", "Ljubljana"].unique()

or

["Ljubljana", "London", "Los Angeles", "Ljubljana"].uniqueOrdered()

Metachromatism

This is O(n^2). Don't do this. – Bensky

Answer 19

For arrays where the elements are neither Hashable nor Comparable (e.g. complex objects, dictionaries or structs), this extension provides a generalized way to remove duplicates:

extension Array
{
   func filterDuplicate<T:Hashable>(_ keyValue:(Element)->T) -> [Element]
   {
      var uniqueKeys = Set<T>()
      return filter{uniqueKeys.insert(keyValue($0)).inserted}
   }

   func filterDuplicate<T>(_ keyValue:(Element)->T) -> [Element]
   { 
      return filterDuplicate{"\(keyValue($0))"}
   }
}

// example usage: (for a unique combination of attributes):

peopleArray = peopleArray.filterDuplicate{ ($0.name, $0.age, $0.sex) }

or...

peopleArray = peopleArray.filterDuplicate{ "\(($0.name, $0.age, $0.sex))" }

You don't have to bother with making values Hashable and it allows you to use different combinations of fields for uniqueness.

Note: for a more robust approach, please see the solution proposed by Coeur in the comments below.

https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift

[EDIT] Swift 4 alternative

With Swift 4.2 you can use the Hasher class to build a hash much easier. The above extension could be changed to leverage this :

extension Array
{
    func filterDuplicate(_ keyValue:((AnyHashable...)->AnyHashable,Element)->AnyHashable) -> [Element]
    {
        func makeHash(_ params:AnyHashable ...) -> AnyHashable
        { 
           var hash = Hasher()
           params.forEach{ hash.combine($0) }
           return hash.finalize()
        }  
        var uniqueKeys = Set<AnyHashable>()
        return filter{uniqueKeys.insert(keyValue(makeHash,$0)).inserted}     
    }
}

The calling syntax is a little different because the closure receives an additional parameter containing a function to hash a variable number of values (which must be Hashable individually)

peopleArray = peopleArray.filterDuplicate{ $0($1.name, $1.age, $1.sex) } 

It will also work with a single uniqueness value (using $1 and ignoring $0).

peopleArray = peopleArray.filterDuplicate{ $1.name } 

Squeak

This could give random results depending on the behavior of "\()", as it may not gives you unique values like conforming to Hashable should. Example, if your elements conform to Printable by all returning the same description, then your filtering fails. – Opacity

Agreed. Selection of the fields (or formula) that will produce the desired uniqueness pattern will have to take this into account. For many use cases, this provides a simple ad-hoc solution that requires no alteration of the element's class or struct. – Squeak

@AlainT. Don't do this, really. String's purpose is not to be some ghetto ad-hoc key generation mechanism. Just constrain T to being Hashable. – Bensky

@Bensky I've applied this idea in a new answer: https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Opacity

Perfect answer as i want. Thank you so much. – Fetishist

Answer 20

As was noted at WWDC 2021, Swift has community-developed Algorithms, Collections, and Numerics Packages. The Algorithms package features a uniqued() algorithm.

These are not yet part of the Swift Standard library. You can currently download them from Apple's Github page and/or install them via Swift Package Manager.

WWDC Video:

https://developer.apple.com/videos/play/wwdc2021/10256/

Github page:

https://github.com/apple/swift-algorithms

uniqued() and uniqued(on:) documentation:

https://github.com/apple/swift-algorithms/blob/main/Guides/Unique.md

Ninetieth

This should be the top answer – Ernestoernestus

How do I do this in iOS? – Illstarred

Answer 21

1. First add all the elements of an array to NSOrderedSet.
2. This will remove all the duplicates in your array.
3. Again convert this orderedset to an array.

Done....

Example

let array = [1,1,1,1,2,2,2,2,4,6,8]

let orderedSet : NSOrderedSet = NSOrderedSet(array: array)

let arrayWithoutDuplicates : NSArray = orderedSet.array as NSArray

output of arrayWithoutDuplicates - [1,2,4,6,8]

Haslam

Answer 22

You can use directly a set collection to remove duplicate, then cast it back to an array

var myArray = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
var mySet = Set<Int>(myArray)

myArray = Array(mySet) // [2, 4, 60, 6, 15, 24, 1]

Then you can order your array as you want

myArray.sort{$0 < $1} // [1, 2, 4, 6, 15, 24, 60]

Pegasus

"Then you can order your array as you want" What if I want the same ordering as from the original array? It's not that easy. – Bensky

Answer 23

In case you need values sorted, this works (Swift 4)

let sortedValues = Array(Set(array)).sorted()

Hurty

You loosing elements order in this case. – Jacquez

Not at all, that's what the .sorted() at the end is for. Regards. – Hurty

@MauricioChirino And if your original array was [2, 1, 1]? It would come out [1, 2], that's not ordered :p – Bensky

@MauricioChirino No, it wouldn't. The correct answer would be [2, 1], but this would return [1, 2], which is wrong. – Bensky

I think you're confusing sorted with reversed @Alexander. – Hurty

@MauricioChirino No, I'm not. If the goal is to remove duplicate values from a sequence, while retaining the order in which elements uniquely appeared, this doesn't do that. The very clear counter example is [2, 1, 1]. The first appearance of unique elements, in order is [2, 1]. That's the correct answer. But using your (incorrect) algorithm, you get [1, 2], which is sorted, but is not in the correct, original, order. – Bensky

You're right, I've edited the answer. Regards @Bensky – Hurty

Fails if the elements in array are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array. – Machine

Answer 24

Here is a solution that

• Uses no legacy NS types
• Is reasonably fast with O(n)
• Is concise
• Preserves element order

extension Array where Element: Hashable {

    var uniqueValues: [Element] {
        var allowed = Set(self)
        return compactMap { allowed.remove($0) }
    }
}

Cyprus

This is nice but would only works for Hashable elements – Tearing

Answer 25

Slightly more succinct syntax version of Daniel Krom's Swift 2 answer, using a trailing closure and shorthand argument name, which appears to be based on Airspeed Velocity's original answer:

func uniq<S: SequenceType, E: Hashable where E == S.Generator.Element>(source: S) -> [E] {
  var seen = [E: Bool]()
  return source.filter { seen.updateValue(true, forKey: $0) == nil }
}

Example of implementing a custom type that can be used with uniq(_:) (which must conform to Hashable, and thus Equatable, because Hashable extends Equatable):

func ==(lhs: SomeCustomType, rhs: SomeCustomType) -> Bool {
  return lhs.id == rhs.id // && lhs.someOtherEquatableProperty == rhs.someOtherEquatableProperty
}

struct SomeCustomType {

  let id: Int

  // ...

}

extension SomeCustomType: Hashable {

  var hashValue: Int {
    return id
  }

}

In the above code...

id, as used in the overload of ==, could be any Equatable type (or method that returns an Equatable type, e.g., someMethodThatReturnsAnEquatableType()). The commented-out code demonstrates extending the check for equality, where someOtherEquatableProperty is another property of an Equatable type (but could also be a method that returns an Equatable type).

id, as used in the hashValue computed property (required to conform to Hashable), could be any Hashable (and thus Equatable) property (or method that returns a Hashable type).

Example of using uniq(_:):

var someCustomTypes = [SomeCustomType(id: 1), SomeCustomType(id: 2), SomeCustomType(id: 3), SomeCustomType(id: 1)]

print(someCustomTypes.count) // 4

someCustomTypes = uniq(someCustomTypes)

print(someCustomTypes.count) // 3

Cayuga

There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Bensky

Answer 26

here I've done some O(n) solution for objects. Not few-lines solution, but...

struct DistinctWrapper <T>: Hashable {
    var underlyingObject: T
    var distinctAttribute: String
    var hashValue: Int {
        return distinctAttribute.hashValue
    }
}
func distinct<S : SequenceType, T where S.Generator.Element == T>(source: S,
                                                                distinctAttribute: (T) -> String,
                                                                resolution: (T, T) -> T) -> [T] {
    let wrappers: [DistinctWrapper<T>] = source.map({
        return DistinctWrapper(underlyingObject: $0, distinctAttribute: distinctAttribute($0))
    })
    var added = Set<DistinctWrapper<T>>()
    for wrapper in wrappers {
        if let indexOfExisting = added.indexOf(wrapper) {
            let old = added[indexOfExisting]
            let winner = resolution(old.underlyingObject, wrapper.underlyingObject)
            added.insert(DistinctWrapper(underlyingObject: winner, distinctAttribute: distinctAttribute(winner)))
        } else {
            added.insert(wrapper)
        }
    }
    return Array(added).map( { return $0.underlyingObject } )
}
func == <T>(lhs: DistinctWrapper<T>, rhs: DistinctWrapper<T>) -> Bool {
    return lhs.hashValue == rhs.hashValue
}

// tests
// case : perhaps we want to get distinct addressbook list which may contain duplicated contacts like Irma and Irma Burgess with same phone numbers
// solution : definitely we want to exclude Irma and keep Irma Burgess
class Person {
    var name: String
    var phoneNumber: String
    init(_ name: String, _ phoneNumber: String) {
        self.name = name
        self.phoneNumber = phoneNumber
    }
}

let persons: [Person] = [Person("Irma Burgess", "11-22-33"), Person("Lester Davidson", "44-66-22"), Person("Irma", "11-22-33")]
let distinctPersons = distinct(persons,
    distinctAttribute: { (person: Person) -> String in
        return person.phoneNumber
    },
    resolution:
    { (p1, p2) -> Person in
        return p1.name.characters.count > p2.name.characters.count ? p1 : p2
    }
)
// distinctPersons contains ("Irma Burgess", "11-22-33") and ("Lester Davidson", "44-66-22")

Humanize

Rather than using a Set with a custom DistinctWrapper, you should use a Dictionary from distinctAttributes to objects. When you follow that logic through, you would eventually end up implementing [Dictionary.init(_:uniquingKeysWith:)]pastebin.com/w90pVe0p(https://developer.apple.com/documentation/…, which is now built into the standard library. Check out how simple this is pastebin.com/w90pVe0p – Bensky

Answer 27

I used @Jean-Philippe Pellet's answer and made an Array extension that does set-like operations on arrays, while maintaining the order of elements.

/// Extensions for performing set-like operations on lists, maintaining order
extension Array where Element: Hashable {
  func unique() -> [Element] {
    var seen: [Element:Bool] = [:]
    return self.filter({ seen.updateValue(true, forKey: $0) == nil })
  }

  func subtract(takeAway: [Element]) -> [Element] {
    let set = Set(takeAway)
    return self.filter({ !set.contains($0) })
  }

  func intersect(with: [Element]) -> [Element] {
    let set = Set(with)
    return self.filter({ set.contains($0) })
  }
}

Cislunar

There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Bensky

Answer 28

This is just a very simple and convenient implementation. A computed property in an extension of an Array that has equatable elements.

extension Array where Element: Equatable {
    /// Array containing only _unique_ elements.
    var unique: [Element] {
        var result: [Element] = []
        for element in self {
            if !result.contains(element) {
                result.append(element)
            }
        }

        return result
    }
}

Joh

This is also O(n^2). – Bensky

Answer 29

func removeDublicate (ab: [Int]) -> [Int] {
var answer1:[Int] = []
for i in ab {
    if !answer1.contains(i) {
        answer1.append(i)
    }}
return answer1
}

Usage:

let f = removeDublicate(ab: [1,2,2])
print(f)

Unalloyed

I think this is the simplest – Unalloyed

it keeps the order and gives you an array you want – Unalloyed

This is also O(n²). – Bensky

Answer 30

Swift 3/ Swift 4/ Swift 5

Just one line code to omit Array duplicates without effecting order:

let filteredArr = Array(NSOrderedSet(array: yourArray))

Upward

Here we are typecasting an array into Orderedset. Definition of "Sets" - sets allow only distinct values (It does not allow duplicates). Hence duplicates will be omitted As we are typecasting with NSOrderedSet hence array order will not be disturbed. – Upward

Answer 31

Swift 5.7

Using Ordered Set

You can pass an array with repeating elements to the following generic function, which processes an ordered Set and returns a new array with no duplicates.

import Foundation

internal func withoutDuplicates<T>(_ array: [T]) -> [T] {
    
    let orderedSet: NSMutableOrderedSet = []
    var modifiedArray = [T]()
    
    orderedSet.addObjects(from: array)
    
    for i in 0...(orderedSet.count - 1) {
        modifiedArray.append(orderedSet[i] as! T)
    }
    return modifiedArray
}

////////////////////////////////////////////////////////////

let arrayOfStrings: [String] = ["A","A","A","B","B","C","C"]
let arrayOfIntegers: [UInt8] = [1, 1, 1, 2, 2, 2, 3, 3]
let arrayOfBooleans: [Bool] = [true, false, false, true]

let ordered_01 = withoutDuplicates(arrayOfStrings)
let ordered_02 = withoutDuplicates(arrayOfIntegers)
let ordered_03 = withoutDuplicates(arrayOfBooleans)

Results:

// ordered_01  –––>  ["A","B","C"]
// ordered_02  –––>  [1, 2, 3]
// ordered_03  –––>  [true, false]

Using Unordered Set

If the order of the elements in the new array doesn't really matter to you, use an unordered Set when processing. Elements' type in unordered Set must conform to Hashable protocol.

import UIKit

fileprivate func noDuplicates<T: Hashable>(_ array: [T]) -> [T] {
    
    var unorderedSet = Set<T>()
    var modifiedArray: [T] = []
    
    for i in 0...(array.count - 1) {
        unorderedSet.insert(array[i])
    }
    for i in unorderedSet.indices {
        modifiedArray.append(unorderedSet[i])
    }
    return modifiedArray
}

////////////////////////////////////////////////////////////

let arrayOfInts: [Int] = [10, 5, 7, 200, -500, 10, 7, 5]
let arrayOfStrs: [String] = ["A","A","A","B","B","C","C"]
let arrayOfBools: [Bool] = [true, false, false, true]

let unordered_01 = noDuplicates(arrayOfInts)
let unordered_02 = noDuplicates(arrayOfStrs)
let unordered_03 = noDuplicates(arrayOfBools)

Results:

// unordered_01  –––>  [200, 7, 10, -500, 5]
// unordered_02  –––>  ["B", "C", "A"]
// unordered_03  –––>  [false, true]

Perplexed

don't force unwrap code, modifiedArray.append(orderedSet[i] as! T) – Borneol

Answer 32

You can always use a Dictionary, because a Dictionary can only hold unique values. For example:

var arrayOfDates: NSArray = ["15/04/01","15/04/01","15/04/02","15/04/02","15/04/03","15/04/03","15/04/03"]

var datesOnlyDict = NSMutableDictionary()
var x = Int()

for (x=0;x<(arrayOfDates.count);x++) {
    let date = arrayOfDates[x] as String
    datesOnlyDict.setValue("foo", forKey: date)
}

let uniqueDatesArray: NSArray = datesOnlyDict.allKeys // uniqueDatesArray = ["15/04/01", "15/04/03", "15/04/02"]

println(uniqueDatesArray.count)  // = 3

As you can see, the resulting array will not always be in 'order'. If you wish to sort/order the Array, add this:

var sortedArray = sorted(datesOnlyArray) {
(obj1, obj2) in

    let p1 = obj1 as String
    let p2 = obj2 as String
    return p1 < p2
}

println(sortedArray) // = ["15/04/01", "15/04/02", "15/04/03"]

.

Harlequin

Answer 33

I believe it would be good to offer a uniq() and uniqInPlace() function to mutate an Array by removing it's values. This works similar as the sort() and sortInPlace() function provided by Swift. Also since it's an Array it should keep it's original order of elements.

extension Array where Element: Equatable {

    public func uniq() -> [Element] {
        var arrayCopy = self
        arrayCopy.uniqInPlace()
        return arrayCopy
    }

    mutating public func uniqInPlace() {
        var seen = [Element]()
        var index = 0
        for element in self {
            if seen.contains(element) {
                removeAtIndex(index)
            } else {
                seen.append(element)
                index++
            }
        }
    }
}

You can only use uniqInPlace() on a variable Array (i.e. var) since you cannot mutate a constant Array (i.e. let).

Some usage examples:

var numbers = [1, 6, 2, 2, 4, 1, 5]
numbers.uniqInPlace() // array is now [1, 6, 2, 4, 5]

let strings = ["Y", "Z", "A", "Y", "B", "Y", "Z"]
let uniqStrings = strings.uniq() // uniqStrings is now ["Y", "Z", "A", "B"]

Tantalus

An Array<Element> is not an appropriate choice for the type of seen. The repeated contains calls (each of which is O(n)) makes this algorithm at least O(n^2). Also, removeAtIndex) is also O(n) (because it causes every element after the remove element to have to be shifted left by 1). Instead, an algorithm like this would work better using var seen = Set<Element>(), and rather than "removing" elements, you "overwrite" them, by reading ahead until you see the next element that you need to keep. – Bensky

That way, you keep all the elements you want, and end up with a set of empty space at the end of array that can just be trimmed in one shot – Bensky

Answer 34

The easiest way would be to use NSOrderedSet, that stores unique elements and preserves the elements order. Like:

func removeDuplicates(from items: [Int]) -> [Int] {
    let uniqueItems = NSOrderedSet(array: items)
    return (uniqueItems.array as? [Int]) ?? []
}

let arr = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
removeDuplicates(from: arr)

Osbourn

I wonder how this performance compares to the better answers on here. Have you compared? – Bensky

Answer 35

I think this the better way with knowing the logic itself

var arrayOfInts = [2, 2, 4, 4]
var mainArray = [Int]()

for value in arrayOfInts {

if mainArray.contains(value) != true  {
    
    mainArray.append(value)
    print("mainArray:\(mainArray)")
}}

Transversal

This is quadratic behavior. Each iteration of your loop call contains, which itself uses a loop over all elements. Really slow. – Bensky

mainArray.contains(value) == false can be simplified to mainArray.contains(value) != true – Incapable

Answer 36

var numbers = [1,2,3,4,5,10,10, 12, 12, 6,6,6,7,8,8, 8, 8, 8 , 7 , 1 , 1, 2 , 9]

var newArr : [Int] = []
for n in numbers {
    if !newArr.contains(n) {
        newArr.append(n)
    }
}

output - [1, 2, 3, 4, 5, 10, 12, 6, 7, 8, 9]

The above solution maintains order but very slow as .contains iterates again and again. Thus use the ordered set.

This will print the ordered array.

Array(NSOrderedSet.init(array: numbers))

output - [1, 2, 3, 4, 5, 10, 12, 6, 7, 8, 9]

This Will Print an unordered array.

let uniqueUnordered = Array(Set(numbers))

output - [4, 2, 1, 9, 10, 3, 5, 6, 8, 12, 7]

Metronymic

Answer 37

contains checks for equality while insert checks for hash, its safest to check in the following way:

extension Array where Element: Hashable {

    /// Big O(N) version. Updated since @Adrian's comment. 
    var uniques: Array {
        // Go front to back, add element to buffer if it isn't a repeat.
         var buffer: [Element] = []
         var dictionary: [Element: Int] = [:]
         for element in self where dictionary[element] == nil {
             buffer.append(element)
             dictionary[element] = 1
         }
         return buffer
    }
}

Townie

This might get the job done on smaller arrays, but I tried it on a large data set and it was extremely slow. – Ahola

Thank you for the input! Ah yes, the method contains makes this an O(N^2) operation...Good catch. – Townie

This doesn't work if there are collisions in the hashValue. Having collision should be handled by checking for equality. This is why the Hashable protocol inherits from Equatable. – Holiday

Just updated to another attempt – Townie

@Adrian, can you check this one please? – Townie

@Gigisommo, is this implementation better Can you check? – Townie

@Townie this is better, but you can improve it further by using a set instead of a dictionary. In this implementation you're already using the dictionary as a set. They both have a constant lookup time and they both allow only unique keys (in the case of the dictionary) / item (in the case of the set). There is no reason to use a dictionary associated with an arbitrary integer, you can just use a set instead. But this is already an improvement and it should work in all scenarios, even with conflicting hashes. :) – Holiday

Answer 38

easy way to remove duplicates from array

extension Array where Element: Equatable {
mutating func removeDuplicates() {
    var result = [Element]()
    for value in self {
        if !result.contains(value) {
            result.append(value)
        }
    }
    self = result
}}

Fondly

While this method is, indeed, simple, it is really not optimal. It has quadratic algorithmic complexity because you keep looping the result temporary array over and over for each element of the original array. The other solutions involving using a Set will be much faster because the contains lookup is very efficient. – Lorenzalorenzana

Answer 39

Let me suggest an answer similar to Scott Gardner's answer but with more laconic syntax using reduce. This solution removes duplicates from an array of custom objects (keeping the initial order)

// Custom Struct. Can be also class. 
// Need to be `equitable` in order to use `contains` method below
struct CustomStruct : Equatable {
      let name: String
      let lastName : String
    }

// conform to Equatable protocol. feel free to change the logic of "equality"
func ==(lhs: CustomStruct, rhs: CustomStruct) -> Bool {
  return (lhs.name == rhs.name && lhs.lastName == rhs.lastName)
}

let categories = [CustomStruct(name: "name1", lastName: "lastName1"),
                  CustomStruct(name: "name2", lastName: "lastName1"),
                  CustomStruct(name: "name1", lastName: "lastName1")]
print(categories.count) // prints 3

// remove duplicates (and keep initial order of elements)
let uniq1 : [CustomStruct] = categories.reduce([]) { $0.contains($1) ? $0 : $0 + [$1] }
print(uniq1.count) // prints 2 - third element has removed

And just if you are wondering how this reduce magic works - here is exactly the same, but using more expanded reduce syntax

let uniq2 : [CustomStruct] = categories.reduce([]) { (result, category) in
  var newResult = result
  if (newResult.contains(category)) {}
  else {
    newResult.append(category)
  }
  return newResult
}
uniq2.count // prints 2 - third element has removed

You can simply copy-paste this code into a Swift Playground and play around.

Doth

For Swift 4 you need to replace $0 with $0.0 and $1 with $0.1. – Opacity

Note that prior to Swift 4 (i.e. in Swift 3) it is discouraged to use reduce with a mutable struct type (here it's []), as the mutability is not preserved and reduce is creating an overhead of copying the struct each time. – Opacity

Generally you would want to encapsulate code like this into a function, rather than copy pasting it everywhere it's used. And reduce isn't an appropriate choice for this algorithm. It makes it O(n²) – Bensky

Answer 40

In Swift 3.0 the simplest and fastest solution I've found to eliminate the duplicated elements while keeping the order:

extension Array where Element:Hashable {
    var unique: [Element] {
        var set = Set<Element>() //the unique list kept in a Set for fast retrieval
        var arrayOrdered = [Element]() //keeping the unique list of elements but ordered
        for value in self {
            if !set.contains(value) {
                set.insert(value)
                arrayOrdered.append(value)
            }
        }

        return arrayOrdered
    }
}

Zymogenic

You can do the contains and insert in one shot: https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift – Bensky

Answer 41

Here's a more flexible way to make a sequence unique with a custom matching function.

extension Sequence where Iterator.Element: Hashable {

    func unique(matching: (Iterator.Element, Iterator.Element) -> Bool) -> [Iterator.Element] {

        var uniqueArray: [Iterator.Element] = []
        forEach { element in
            let isUnique = uniqueArray.reduce(true, { (result, item) -> Bool in
                return result && matching(element, item)
            })
            if isUnique {
                uniqueArray.append(element)
            }
        }
        return uniqueArray
    }
}

Rearm

That is an interesting demonstration of how contains(where:) can be implemented with only reduce, but in production code like this, it's better to just use contains(where:) directly. For one, it can short circuit the search if an element is found early on (no need to keep searching after that, but this reduce method does that anyway) – Bensky

Answer 42

this is the simplest way in swift 4.2 onwards the code like below

let keyarray:NSMutableArray = NSMutableArray()

for  object in dataArr
{
    if !keysArray.contains(object){
        keysArray.add(object)
    }
}

print(keysArray)

Transversal

Ooof. Don't do this. That's an O(n^2) algorithm (because contains is O(n), which itself runs n times). And don't use NSMutableArray in Swift – Bensky

Answer 43

Xcode 10.1 - Swift 4.2 Simple and Powerful Solution

func removeDuplicates(_ nums: inout [Int]) -> Int {
    nums = Set(nums).sorted()
    return nums.count
}

Example

var arr = [1,1,2,2,3,3,4,4,5,5,6,6,7,7,8,8,9,9]
removeDuplicates(&arr)

print(arr) // [1,2,3,4,5,6,7,8,9]

Armijo

This does not keep the original order: it applies a new order, which may or may not be the same. Even if it's the same order, it's not as performant as a solution that would simply keep the existing order without adding an extra sorted. – Opacity

Yeah, the sorted() call is a totally misplaced responsibility. The caller asked for deduplication. If they also wanted sorting they can already do that themselves. I would suggest deleting this answer. – Bensky

Answer 44

My solution, it seems it can be in O(n) time as Hash map access is O(1), and filter is O(n). It also uses by closure to select property by which to make the distinction of elements in sequence.

extension Sequence {

    func distinct<T: Hashable>(by: (Element) -> T) -> [Element] {
        var seen: [T: Bool] = [:]
        return self.filter { seen.updateValue(true, forKey: by($0)) == nil }
    }
}

Ovenware

There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift Please delete this answer. – Bensky

Answer 45

I have created a higher-order function that has time complexity is o(n). Also, capability like the map to return any type you want.

extension Sequence {
    func distinct<T,U>(_ provider: (Element) -> (U, T)) -> [T] where U: Hashable {
        var uniqueKeys = Set<U>()
        var distintValues = [T]()
        for object in self {
            let transformed = provider(object)
            if !uniqueKeys.contains(transformed.0) {
                distintValues.append(transformed.1)
                uniqueKeys.insert(transformed.0)
            }
        }
        return distintValues
    }
}

Amy

This does two hashing operations per element, which is unnecessary. insert returns a tuple that tells you whether the element was already there, or was added for the first time. https://mcmap.net/q/92418/-removing-duplicate-elements-from-an-array-in-swift Please delete this answer. – Bensky

Answer 46

if you want to keep the order as well then use this

let fruits = ["apple", "pear", "pear", "banana", "apple"] 
let orderedNoDuplicates = Array(NSOrderedSet(array: fruits).map({ $0 as! String }))

Arleen

Answer 47

Add this extension to Array to remove duplicates and maintain the original order of elements:

extension Array {

    var removingDuplicates: [Element] {
        NSOrderedSet(array: self).array.compactMap({ $0 as? Element })
    }
}

The first occurrence of the value is kept, subsequent duplicate values are removed.

Usage:

let nums = [1, 2, 1, 3, 2, 3, 2]
nums.removingDuplicates // [1, 2, 3]

Rheo

Answer 48

Simple solution using dictionary. Time: O(m+n), Space: O(m+n).

var array = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]

func removeDuplicate(array: [Int]) -> [Int] {
  var dict: [Int: Bool] = [Int: Bool]()

  for value in array { 
      dict[value] = true
  }

  return  dict.keys.map { $0 }//.sorted()
}

Clemmie

Answer 49

Preserve unique values and preserve sorting in an array.

(using Swift 3)

    var top3score: [Int] = []


    outerLoop: for i in 0..<top10score.count {
        dlog(message: String(top10score[i]))

        if top3score.count == 3 {
            break
        }

        for aTop3score in top3score {
            if aTop3score == top10score[i] {
                continue outerLoop
            }
        }

        top3score.append(top10score[i])

    }

    print("top10score is \(top10score)")  //[14, 5, 5, 5, 3, 3, 2, 2, 2, 2]
    print("top3score is \(top3score)")   //[14, 5, 3]

Norword

This is a good first approach, but it's quite complex and verbose. This code does too many things (deduplicating, truncating to 3 results), and there's no easy way to reuse it without mass copy/paste. You should take a look at this answer. You could then write something like scores.uniqued().prefix(3). Simple! – Bensky

Thanks for your comment. The main points of the code above are not only to preserve unique values but also to preserve the original ordering. Using "Set" seems can't preserve the ordering. :) – Norword

Yes it can. Take a look at the answer I can. You use the set for fast answers to the question of "Have I already seen this element?" – Bensky

Answer 50

I've made a simple-as-possible extension for that purpose.

extension Array where Element: Equatable {

    func containsHowMany(_ elem: Element) -> Int {
        return reduce(0) { $1 == elem ? $0 + 1 : $0 }
    }

    func duplicatesRemoved() -> Array {
        return self.filter { self.containsHowMany($0) == 1 }
    }

    mutating func removeDuplicates() {
        self = self.duplicatesRemoved(()
    }
}

You can use duplicatesRemoved() to get a new array, whose duplicate elements are removed, or removeDuplicates() to mutate itself. See:

let arr = [1, 1, 1, 2, 2, 3, 4, 5, 6, 6, 6, 6, 6, 7, 8]

let noDuplicates = arr.duplicatesRemoved()
print(arr) // [1, 1, 1, 2, 2, 3, 4, 5, 6, 6, 6, 6, 6, 7, 8]
print(noDuplicates) // [1, 2, 3, 4, 5, 6, 7, 8]

arr.removeDuplicates()
print(arr) // [1, 2, 3, 4, 5, 6, 7, 8]

Zackzackariah

This is also O(n²). – Bensky

It has 2 downvotes, actually, and idk. Probably because it's needless quadratic – Bensky

Answer 51

This works in Swift 4, if you do not want/need to convert the result to an Array, but can do with a Set. The result is not sorted by default, but you can do that with sorted(), which returns an array, as shown in the print statement.

let array = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]

var result = Set<Int>()
_ = array.map{ result.insert($0) }

print(result.sorted())  // [1, 2, 4, 6, 15, 24, 60]

Deil

This irreversibly loses ordering. Sorting only makes sense if your original ordering was sorted order, which isn't the case your example. Also, don't abuse map where forEach would make more sense. Even still, it could be just let result = Set(array) – Bensky