374

I might have an array that looks like the following:

[1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]

Or, really, any sequence of like-typed portions of data. What I want to do is ensure that there is only one of each identical element. For example, the above array would become:

[1, 4, 2, 6, 24, 15, 60]

Notice that the duplicates of 2, 6, and 15 were removed to ensure that there was only one of each identical element. Does Swift provide a way to do this easily, or will I have to do it myself?

8
  • 14
    The easiest way is to convert the array in an NSSet, NSSet is an unordered collection of objects, if need to keep order NSOrderedSet. Commented Sep 9, 2014 at 7:28
  • You could use the intersection function as you can find in this class with functions for arrays: github.com/pNre/ExSwift/blob/master/ExSwift/Array.swift Commented Sep 9, 2014 at 7:31
  • 3
    Probably the most elegant, smartest and fastest answer is provided by mxcl's answer below. Which also helps maintain order Commented May 10, 2018 at 16:56
  • 1
    Why don't you just use Set from Swift ? You'll be able to provide a list of unordered and unique elements. Commented Mar 11, 2019 at 10:27
  • 1
    Apple finally realized this is a common problem and added their own solution in swift-algorithms. See MH175's answer. Commented Jun 18, 2022 at 21:11

51 Answers 51

681

You can convert to a Set and back to an Array again quite easily:

let unique = Array(Set(originals))

This is not guaranteed to maintain the original order of the array.

Sign up to request clarification or add additional context in comments.

15 Comments

Is there a way to use a set while preserving the original order of the array?
@Crashalot See my answer.
If you need to keep the objects unique by a specific property, than also implement the Hashable and Equatable protocol on that class, instead of just using the Array->Set->Array transformation
Fails if the elements in originals are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array.
I don't understand why this answer has so many upvotes. Seems like maintaining the order of the array is almost certainly a requirement. Otherwise you might as well just use a Set instead of an Array in the first place.
|
252

You can roll your own, e.g. like this:

func unique<S : Sequence, T : Hashable>(source: S) -> [T] where S.Iterator.Element == T {
    var buffer = [T]()
    var added = Set<T>()
    for elem in source {
        if !added.contains(elem) {
            buffer.append(elem)
            added.insert(elem)
        }
    }
    return buffer
}

let vals = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let uniqueVals = uniq(vals) // [1, 4, 2, 6, 24, 15, 60]

And as an extension for Array:

extension Array where Element: Hashable {
    func uniqued() -> Array {
        var buffer = Array()
        var added = Set<Element>()
        for elem in self {
            if !added.contains(elem) {
                buffer.append(elem)
                added.insert(elem)
            }
        }
        return buffer
    }
}

Or more elegantly (Swift 4/5):

extension Sequence where Element: Hashable {
    func uniqued() -> [Element] {
        var set = Set<Element>()
        return filter { set.insert($0).inserted }
    }
}

Which would be used:

[1,2,4,2,1].uniqued()  // => [1,2,4]

17 Comments

You could also implement the body of that function as var addedDict = [T:Bool](); return filter(source) { addedDict(true, forKey: $0) == nil }
@AirspeedVelocity: Did you mean updateValue(true, forKey: $0)... instead of addedDict(true, forKey: $0)...
Oops yes sorry I accidentally the method! Should be return filter(source) { addedDict.updateValue(true, forKey: $0) == nil } as you say.
Just a word of caution: Avoid discussing performance for simple functions like this until you provably depend on their performance, at which point the only thing you should do is benchmark. Too often have I seen unmaintainable code or even less performant code due to making assumptions. :) Also, this is probably easier to grasp: let uniques = Array(Set(vals))
@Blixt Agreed. Once again, here the advantage lies in respecting the order of elements of the original array.
|
133

Swift 4

public extension Array where Element: Hashable {
    func uniqued() -> [Element] {
        var seen = Set<Element>()
        return filter { seen.insert($0).inserted }
    }
}

every attempt to insert will also return a tuple: (inserted: Bool, memberAfterInsert: Set.Element). See documentation.

Using the returned value means we can avoid doing more than one loop, so this is O(n).

5 Comments

After simple profiling, this method is really fast. Its hundreds times faster then using reduce( _: _:), or even reduce(into: _:)
@Kelvin Because all those other algorithm were O(n^2), and nobody noticed.
@Kelvin this answer is identical to Eneko Alonso answer + my comment (Jun 16 '17).
Beware that duplicity is determined by Hash values of two objects/structs and not the == equality in Equatable protocol. If you want to determine duplicity lets say in struct hased only on some properties (not all properties), you need to use custom Hashable implementation with func hash(into hasher: inout Hasher)
We will use Set<Element>(minimumCapacity: count) initialiser in this answer.
94

Use a Set or NSOrderedSet to remove duplicates, then convert back to an Array:

let uniqueUnordered = Array(Set(array))
let uniqueOrdered = Array(NSOrderedSet(array: array))

5 Comments

let uniqueOrderedNames = Array(NSOrderedSet(array: userNames)) as! [String] if u have array of String, not of Any
Fails if the elements in array are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array.
Tested in Swift 5.1b5, given that the elements are Hashable and a desire to retain ordering, the NSOrderedSet(array: array).array is marginally faster than the pure swift func uniqued() using a set with filter. I tested with 5100 strings that resulted in 13 unique values.
Array(NSOrderedSet(array: array)) is not working in Swift 5. Use NSOrderedSet(array: array).array as! [String] instead.
The second one only works for "primitive" types
82

Many answers available here, but I missed this simple extension, suitable for Swift 2 and up:

extension Array where Element:Equatable {
    func removeDuplicates() -> [Element] {
        var result = [Element]()

        for value in self {
            if result.contains(value) == false {
                result.append(value)
            }
        }

        return result
    }
}

Makes it super simple. Can be called like this:

let arrayOfInts = [2, 2, 4, 4]
print(arrayOfInts.removeDuplicates()) // Prints: [2, 4]

Filtering based on properties

To filter an array based on properties, you can use this method:

extension Array {

    func filterDuplicates(@noescape includeElement: (lhs:Element, rhs:Element) -> Bool) -> [Element]{
        var results = [Element]()

        forEach { (element) in
            let existingElements = results.filter {
                return includeElement(lhs: element, rhs: $0)
            }
            if existingElements.count == 0 {
                results.append(element)
            }
        }

        return results
    }
}

Which you can call as followed:

let filteredElements = myElements.filterDuplicates { $0.PropertyOne == $1.PropertyOne && $0.PropertyTwo == $1.PropertyTwo }

5 Comments

@Antoine Thank you for the Filtering based on properties extension. It's really useful. But can you please explain how it works. It's too hard to understand for me. Thank you
Updates for swift 3: func filterDuplicates(_ includeElement: (_ lhs:Element, _ rhs:Element) -> Bool) -> [Element]{
The first part of this answer (extension Array where Element: Equatable) is being superseded by stackoverflow.com/a/36048862/1033581 which offers a more powerful solution (extension Sequence where Iterator.Element: Equatable).
This will have O(n²) time performance, which is really bad for large arrays.
You should use a set to keep track of elements seen so far, to bring this terrible O(n²) complexity back down to O(n)
66

If you put both extensions in your code, the faster Hashable version will be used when possible, and the Equatable version will be used as a fallback.

public extension Sequence where Element: Hashable {
  /// The elements of the sequence, with duplicates removed.
  /// - Note: Has equivalent elements to `Set(self)`.
  var firstUniqueElements: [Element] {
    let getSelf: (Element) -> Element = \.self
    return firstUniqueElements(getSelf)
  }
}

public extension Sequence where Element: Equatable {
  /// The elements of the sequence, with duplicates removed.
  /// - Note: Has equivalent elements to `Set(self)`.
  var firstUniqueElements: [Element] {
    let getSelf: (Element) -> Element = \.self
    return firstUniqueElements(getSelf)
  }
}

public extension Sequence {
  /// The elements of the sequences, with "duplicates" removed
  /// based on a closure.
  func firstUniqueElements<Hashable: Swift.Hashable>(
    _ getHashable: (Element) -> Hashable
  ) -> [Element] {
    var set: Set<Hashable> = []
    return filter { set.insert(getHashable($0)).inserted }
  }

  /// The elements of the sequence, with "duplicates" removed,
  /// based on a closure.
  func firstUniqueElements<Equatable: Swift.Equatable>(
    _ getEquatable: (Element) -> Equatable
  ) -> [Element] {
    reduce(into: []) { uniqueElements, element in
      if zip(
        uniqueElements.lazy.map(getEquatable),
        AnyIterator { [equatable = getEquatable(element)] in equatable }
      ).allSatisfy(!=) {
        uniqueElements.append(element)
      }
    }
  }
}

If order isn't important, then you can always just use this Set initializer.

9 Comments

@DavidSeek like this, uniqueArray = nonUniqueArray.uniqueElements
yeah dont worry. got it working right after. been almost 2 years now :P
This will have O(n²) time performance, which is really bad for large arrays.
The hahsable version will have better performance, but won't preserve the order of elements in the original array. Leo's answer will give both O(n) performance AND preserve object ordering.
@Jessy There are already multiple O(1) answers, but they have way less votes than most of the naive O(n^2) solutions. This one is particularly good for its simplicity: stackoverflow.com/a/46354989/3141234
|
50

edit/update Swift 4 or later

We can also extend RangeReplaceableCollection protocol to allow it to be used with StringProtocol types as well:

extension RangeReplaceableCollection where Element: Hashable {
    var orderedSet: Self {
        var set = Set<Element>()
        return filter { set.insert($0).inserted }
    }
    mutating func removeDuplicates() {
        var set = Set<Element>()
        removeAll { !set.insert($0).inserted }
    }
}

let integers = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let integersOrderedSet = integers.orderedSet // [1, 4, 2, 6, 24, 15, 60]

"abcdefabcghi".orderedSet  // "abcdefghi"
"abcdefabcghi".dropFirst(3).orderedSet // "defabcghi"

Mutating method:

var string = "abcdefabcghi"
string.removeDuplicates() 
string  //  "abcdefghi"

var substring = "abcdefabcdefghi".dropFirst(3)  // "defabcdefghi"
substring.removeDuplicates()
substring   // "defabcghi"

For Swift 3 click here

15 Comments

I like this, it works with an array of dictionarys too!
@Alexander Leo Dabus has replaced the reduce implementation, so now the complexity is different.
The results are interesting. For both 1 million unique items and 8 million, the filter version is faster. However, the filter-based version takes 8.38x longer for 8 million unique items (a hair over O(n) time), where the flatmap-based version takes 7.47x longer for 8 million unique entries than 1 million, suggesting that the flatmap based version scales better. Somehow the flatmap based version does slightly better than O(n) time!
In fact, when I run the test with 64x more items in the array, the flatmap based version is faster.
@Jessy yes it keeps the original order. It is not called sortedSet. Btw it uses the same name Apple uses in NSOrderedSet just dropping the NS
|
34

Swift 4

Guaranteed to keep ordering.

extension Array where Element: Equatable {
    func removingDuplicates() -> Array {
        return reduce(into: []) { result, element in
            if !result.contains(element) {
                result.append(element)
            }
        }
    }
}

8 Comments

I use this now, only changed the method name to removeDuplicates :)
I guess this solution is compact, but I believe that deanWombourne solution posted a year earlier may be slightly more efficient than a reduce: overall, it's just one more line in your whole project to write your function as: var unique: [Iterator.Element] = []; for element in self where !unique.contains(element) { unique.append(element) }; return unique. I admit I haven't tested the relative performances yet.
This will have O(n²) time performance, which is really bad for large arrays.
@NickGaens No it's not, it's O(n²). There's nothing swift about this.
@Cœur reduce or reduce(into:) wouldn't make a critical difference. Rewriting this to not repeatedly call contains would make a MUCH bigger difference.
|
33

No need to write any extensions.

Apple has finally introduced uniqued() method in its Algorithms package.

Points to consider:

  1. Not just Arrays. It can be used on any type conforming to Sequence protocol.
  2. The Element type in Sequence must conform to Hashable protocol

Example:

import Algorithms

let numbers = [1, 2, 3, 3, 2, 3, 3, 2, 2, 2, 1]
print(numbers.uniqued()) // prints [1, 2, 3]

More info https://github.com/apple/swift-algorithms/blob/main/Guides/Unique.md

2 Comments

Algorithms not from Apple i guess.
@UmitKaya It is absolutely from Apple, just published as a separate package to avoid littering Foundation.
32

Inspired by https://www.swiftbysundell.com/posts/the-power-of-key-paths-in-swift, we can declare a more powerful tool that is able to filter for unicity on any keyPath. Thanks to Alexander comments on various answers regarding complexity, the below solutions should be near optimal.

Non-mutating solution

We extend with a function that is able to filter for unicity on any keyPath:

extension RangeReplaceableCollection {
    /// Returns a collection containing, in order, the first instances of
    /// elements of the sequence that compare equally for the keyPath.
    func unique<T: Hashable>(for keyPath: KeyPath<Element, T>) -> Self {
        var unique = Set<T>()
        return filter { unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Note: in the case where your object doesn't conform to RangeReplaceableCollection, but does conform to Sequence, you can have this additional extension, but the return type will always be an Array:

extension Sequence {
    /// Returns an array containing, in order, the first instances of
    /// elements of the sequence that compare equally for the keyPath.
    func unique<T: Hashable>(for keyPath: KeyPath<Element, T>) -> [Element] {
        var unique = Set<T>()
        return filter { unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Usage

If we want unicity for elements themselves, as in the question, we use the keyPath \.self:

let a = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
let b = a.unique(for: \.self)
/* b is [1, 4, 2, 6, 24, 15, 60] */

If we want unicity for something else (like for the id of a collection of objects) then we use the keyPath of our choice:

let a = [CGPoint(x: 1, y: 1), CGPoint(x: 2, y: 1), CGPoint(x: 1, y: 2)]
let b = a.unique(for: \.y)
/* b is [{x 1 y 1}, {x 1 y 2}] */

Mutating solution

We extend with a mutating function that is able to filter for unicity on any keyPath:

extension RangeReplaceableCollection {
    /// Keeps only, in order, the first instances of
    /// elements of the collection that compare equally for the keyPath.
    mutating func uniqueInPlace<T: Hashable>(for keyPath: KeyPath<Element, T>) {
        var unique = Set<T>()
        removeAll { !unique.insert($0[keyPath: keyPath]).inserted }
    }
}

Usage

If we want unicity for elements themselves, as in the question, we use the keyPath \.self:

var a = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
a.uniqueInPlace(for: \.self)
/* a is [1, 4, 2, 6, 24, 15, 60] */

If we want unicity for something else (like for the id of a collection of objects) then we use the keyPath of our choice:

var a = [CGPoint(x: 1, y: 1), CGPoint(x: 2, y: 1), CGPoint(x: 1, y: 2)]
a.uniqueInPlace(for: \.y)
/* a is [{x 1 y 1}, {x 1 y 2}] */

7 Comments

Now that's a good implementation! I only with that key paths were convertible to closures, so that you can use a closure arg to support both arbitrary code (in closures) and mere property look ups (via key paths). Only change I would make is to make keyPath default to \.self, because that's probably the majority of use cases.
@Alexander I tried to default to Self, but then I would need to make Element always Hashable. An alternative to a default value is adding a simple overload without parameters: extension Sequence where Element: Hashable { func unique() { ... } }
Ah yes, makes sense!
Brilliant ... simple, and best of all 'flexible'. Thx.
@Alexander-ReinstateMonica: This looks very similar to your own solution from March 2018: gist.github.com/amomchilov/fbba1e58c91fbd4b5b767bcf8586112b 👏👏👏
|
28

Here's a category on SequenceType which preserves the original order of the array, but uses a Set to do the contains lookups to avoid the O(n) cost on Array's contains(_:) method.

public extension Sequence where Element: Hashable {

    /// Return the sequence with all duplicates removed.
    ///
    /// i.e. `[ 1, 2, 3, 1, 2 ].uniqued() == [ 1, 2, 3 ]`
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234, as 
    ///         per @Alexander's comment.
    func uniqued() -> [Element] {
        var seen = Set<Element>()
        return self.filter { seen.insert($0).inserted }
    }
}

If you aren't Hashable or Equatable, you can pass in a predicate to do the equality check:

extension Sequence {

    /// Return the sequence with all duplicates removed.
    ///
    /// Duplicate, in this case, is defined as returning `true` from `comparator`.
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    func uniqued(comparator: @escaping (Element, Element) throws -> Bool) rethrows -> [Element] {
        var buffer: [Element] = []

        for element in self {
            // If element is already in buffer, skip to the next element
            if try buffer.contains(where: { try comparator(element, $0) }) {
                continue
            }

            buffer.append(element)
        }

        return buffer
    }
}

Now, if you don't have Hashable, but are Equatable, you can use this method:

extension Sequence where Element: Equatable {

    /// Return the sequence with all duplicates removed.
    ///
    /// i.e. `[ 1, 2, 3, 1, 2 ].uniqued() == [ 1, 2, 3 ]`
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    func uniqued() -> [Element] {
        return self.uniqued(comparator: ==)
    }
}

Finally, you can add a key path version of uniqued like this:

extension Sequence {

    /// Returns the sequence with duplicate elements removed, performing the comparison using the property at
    /// the supplied keypath.
    ///
    /// i.e.
    ///
    /// ```
    /// [
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "World")
    ///  ].uniqued(\.value)
    /// ```
    /// would result in
    ///
    /// ```
    /// [
    ///   MyStruct(value: "Hello"),
    ///   MyStruct(value: "World")
    /// ]
    /// ```
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    ///
    func uniqued<T: Equatable>(_ keyPath: KeyPath<Element, T>) -> [Element] {
        self.uniqued { $0[keyPath: keyPath] == $1[keyPath: keyPath] }
    }
}

You can stick both of these into your app, Swift will choose the right one depending on your sequence's Iterator.Element type.


For El Capitan, you can extend this method to include multiple keypaths like this:

    /// Returns the sequence with duplicate elements removed, performing the comparison using the property at
    /// the supplied keypaths.
    ///
    /// i.e.
    ///
    /// ```
    /// [
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Bean"),
    ///   MyStruct(value1: "World", value2: "Sigh")
    ///  ].uniqued(\.value1, \.value2)
    /// ```
    /// would result in
    ///
    /// ```
    /// [
    ///   MyStruct(value1: "Hello", value2: "Paula"),
    ///   MyStruct(value1: "Hello", value2: "Bean"),
    ///   MyStruct(value1: "World", value2: "Sigh")
    /// ]
    /// ```
    ///
    /// - note: Taken from stackoverflow.com/a/46354989/3141234
    ///
    func uniqued<T: Equatable, U: Equatable>(_ keyPath1: KeyPath<Element, T>, _ keyPath2: KeyPath<Element, U>) -> [Element] {
        self.uniqued {
            $0[keyPath: keyPath1] == $1[keyPath: keyPath1] && $0[keyPath: keyPath2] == $1[keyPath: keyPath2]
        }
    }

but (imho) you're probably better off just passing in your own block to self.uniqued.

4 Comments

Heyyy finally someone with an O(n) solution. You can combine the "check" and "insert" set operations into one, by the way. See stackoverflow.com/a/46354989/3141234
Oh, that's clever :)
@deanWombourne How to distinct elements by multiple keypaths?
@EICaptainv2.0 You can just extend the uniqued method to take two generic parameters and check them both for equality - check out the edit I just made. The items are only duplicates if both of the values specified by the keypaths are the same.
17

Think like a functional programmer :)

To filter the list based on whether the element has already occurred, you need the index. You can use enumerated to get the index and map to return to the list of values.

let unique = myArray
    .enumerated()
    .filter{ myArray.firstIndex(of: $0.1) == $0.0 }
    .map{ $0.1 }

This guarantees the order. If you don't mind about the order then the existing answer of Array(Set(myArray)) is simpler and probably more efficient.


UPDATE: Some notes on efficiency and correctness

A few people have commented on the efficiency. I'm definitely in the school of writing correct and simple code first and then figuring out bottlenecks later, though I appreciate it's debatable whether this is clearer than Array(Set(array)).

This method is a lot slower than Array(Set(array)). As noted in comments, it does preserve order and works on elements that aren't Hashable.

However, @Alain T's method also preserves order and is also a lot faster. So unless your element type is not hashable, or you just need a quick one liner, then I'd suggest going with their solution.

Here are a few tests on a MacBook Pro (2014) on Xcode 11.3.1 (Swift 5.1) in Release mode.

The profiler function and two methods to compare:

func printTimeElapsed(title:String, operation:()->()) {
    var totalTime = 0.0
    for _ in (0..<1000) {
        let startTime = CFAbsoluteTimeGetCurrent()
        operation()
        let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
        totalTime += timeElapsed
    }
    let meanTime = totalTime / 1000
    print("Mean time for \(title): \(meanTime) s")
}

func method1<T: Hashable>(_ array: Array<T>) -> Array<T> {
    return Array(Set(array))
}

func method2<T: Equatable>(_ array: Array<T>) -> Array<T>{
    return array
    .enumerated()
    .filter{ array.firstIndex(of: $0.1) == $0.0 }
    .map{ $0.1 }
}

// Alain T.'s answer (adapted)
func method3<T: Hashable>(_ array: Array<T>) -> Array<T> {
    var uniqueKeys = Set<T>()
    return array.filter{uniqueKeys.insert($0).inserted}
}

And a small variety of test inputs:

func randomString(_ length: Int) -> String {
  let letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
  return String((0..<length).map{ _ in letters.randomElement()! })
}

let shortIntList = (0..<100).map{_ in Int.random(in: 0..<100) }
let longIntList = (0..<10000).map{_ in Int.random(in: 0..<10000) }
let longIntListManyRepetitions = (0..<10000).map{_ in Int.random(in: 0..<100) }
let longStringList = (0..<10000).map{_ in randomString(1000)}
let longMegaStringList = (0..<10000).map{_ in randomString(10000)}

Gives as output:

Mean time for method1 on shortIntList: 2.7358531951904296e-06 s
Mean time for method2 on shortIntList: 4.910230636596679e-06 s
Mean time for method3 on shortIntList: 6.417632102966309e-06 s
Mean time for method1 on longIntList: 0.0002518167495727539 s
Mean time for method2 on longIntList: 0.021718120217323302 s
Mean time for method3 on longIntList: 0.0005312927961349487 s
Mean time for method1 on longIntListManyRepetitions: 0.00014377200603485108 s
Mean time for method2 on longIntListManyRepetitions: 0.0007293639183044434 s
Mean time for method3 on longIntListManyRepetitions: 0.0001843773126602173 s
Mean time for method1 on longStringList: 0.007168249964714051 s
Mean time for method2 on longStringList: 0.9114790915250778 s
Mean time for method3 on longStringList: 0.015888616919517515 s
Mean time for method1 on longMegaStringList: 0.0525397013425827 s
Mean time for method2 on longMegaStringList: 1.111266262292862 s
Mean time for method3 on longMegaStringList: 0.11214958941936493 s

12 Comments

unlike Array(Set(myArray)), this works for things that aren't Hashable
... and unlike Array(Set(myArray)) the order of your array is maintained.
It looks like the best answer to me, at least at the present when Swift 5 is already the current version.
@TimMB Oh I misread your post. I saw someone's adaptation that usedthe lastIndex(of:). I totally disagree over the clarity vs optimization point in this case. I don't think this implementation is particularly clear, especially compared to a simple set-based solution. In any case, such code should be extracted out to an extension function. This algorithm becomes basically unusable at even a low input size, like in the thousands to tens of thousands. It's not hard to find such data sets, people can have thousands of songs, files, contacts, etc.
|
13

One more Swift 3.0 solution to remove duplicates from an array. This solution improves on many other solutions already proposed by:

  • Preserving the order of the elements in the input array
  • Linear complexity O(n): single pass filter O(n) + set insertion O(1)

Given the integer array:

let numberArray = [10, 1, 2, 3, 2, 1, 15, 4, 5, 6, 7, 3, 2, 12, 2, 5, 5, 6, 10, 7, 8, 3, 3, 45, 5, 15, 6, 7, 8, 7]

Functional code:

func orderedSet<T: Hashable>(array: Array<T>) -> Array<T> {
    var unique = Set<T>()
    return array.filter { element in
        return unique.insert(element).inserted
    }
}

orderedSet(array: numberArray)  // [10, 1, 2, 3, 15, 4, 5, 6, 7, 12, 8, 45]

Array extension code:

extension Array where Element:Hashable {
    var orderedSet: Array {
        var unique = Set<Element>()
        return filter { element in
            return unique.insert(element).inserted
        }
    }
}

numberArray.orderedSet // [10, 1, 2, 3, 15, 4, 5, 6, 7, 12, 8, 45]

This code takes advantage of the result returned by the insert operation on Set, which executes on O(1), and returns a tuple indicating if the item was inserted or if it already existed in the set.

If the item was in the set, filter will exclude it from the final result.

4 Comments

Not to be picky but you will be performing the insert and the membership test as many times as there are elements so you should count their cost as O(n) as well. This doesn't mean 3xO(n) however because these O's and don't have equal cost with the filter so the addition of O(n)'s is apples to oranges. If we consider set operations to be a O(1) part of the filter cost, the complexity is merely O(n), albeit with a larger "O". Pushing this to the limit, you could also avoid the insertions when the element is already in the set.
You are right, using defer the code would do the set test operation twice, one with contains and one with insert. Further reading Swift documentation, I found that insert returns a tuple indicating if the element was inserted or not, so I've simplified the code removing the contains check.
Nice. Your extension could be optimal by doing it on extension Sequence where Iterator.Element: Hashable { ... }
@AlainT. Nope. Both insert and contains have O(1) complexity. O(1) + O(1) = O(1). These two operations are then done n times (once per call of the closure passed to filter, which is called once per element) I.e. if an operation takes a constant amount of time regardless of input size, then doing it twice still makes it take a constant time that is regardless of input size. The total complexity of this is O(n).
13

An alternate (if not optimal) solution from here using immutable types rather than variables:

func deleteDuplicates<S: ExtensibleCollectionType where S.Generator.Element: Equatable>(seq:S)-> S {
    let s = reduce(seq, S()){
        ac, x in contains(ac,x) ? ac : ac + [x]
    }
    return s
}

Included to contrast Jean-Pillippe's imperative approach with a functional approach.

As a bonus this function works with strings as well as arrays!

Edit: This answer was written in 2014 for Swift 1.0 (before Set was available in Swift). It doesn't require Hashable conformance & runs in quadratic time.

2 Comments

Beware, there are not one, but two ways this runs in quadratic time – both contains and array append run in O(n). Though it does have the benefit of only requiring equatable, not hashable.
this is a really complicated way of writing filter. It's O(n^2) (which is required if you don't want to require Hashable conformance), but you should at least call that out explicitly
13

Swift 5

extension Sequence where Element: Hashable {
    func unique() -> [Element] {
        NSOrderedSet(array: self as! [Any]).array as! [Element]
    }
}

4 Comments

I did some variation so I could select a key to compare. extension Sequence { // Returns distinct elements based on a key value. func distinct<key: Hashable>(by: ((_ el: Iterator.Element) -> key)) -> [Iterator.Element] { var existing = Set<key>() return self.filter { existing.insert(by($0)).inserted } } }
There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See stackoverflow.com/a/55684308/3141234 Please delete this answer.
if your element is Hasable, you can directly use Array(Set(yourElements)
This changes the order of the array.
11

swift 2

with uniq function answer:

func uniq<S: SequenceType, E: Hashable where E==S.Generator.Element>(source: S) -> [E] {
    var seen: [E:Bool] = [:]
    return source.filter({ (v) -> Bool in
        return seen.updateValue(true, forKey: v) == nil
    })
}

use:

var test = [1,2,3,4,5,6,7,8,9,9,9,9,9,9]
print(uniq(test)) //1,2,3,4,5,6,7,8,9

1 Comment

The Bool value obviously is redundant, as your code never reads it. Use a Set instead of a Dictionary and you get my upvote.
10

Swift 4.x:

extension Sequence where Iterator.Element: Hashable {
  func unique() -> [Iterator.Element] {
    return Array(Set<Iterator.Element>(self))
  }

  func uniqueOrdered() -> [Iterator.Element] {
    return reduce([Iterator.Element]()) { $0.contains($1) ? $0 : $0 + [$1] }
  }
}

usage:

["Ljubljana", "London", "Los Angeles", "Ljubljana"].unique()

or

["Ljubljana", "London", "Los Angeles", "Ljubljana"].uniqueOrdered()

1 Comment

This is O(n^2). Don't do this.
10

In Swift 5

 var array: [String] =  ["Aman", "Sumit", "Aman", "Sumit", "Mohan", "Mohan", "Amit"]

 let uniq = Array(Set(array))
 print(uniq)

Output Will be

 ["Sumit", "Mohan", "Amit", "Aman"]

1 Comment

This is a repeat of many of the answers already here, and it doesn't preserve ordering.
8

For arrays where the elements are neither Hashable nor Comparable (e.g. complex objects, dictionaries or structs), this extension provides a generalized way to remove duplicates:

extension Array
{
   func filterDuplicate<T:Hashable>(_ keyValue:(Element)->T) -> [Element]
   {
      var uniqueKeys = Set<T>()
      return filter{uniqueKeys.insert(keyValue($0)).inserted}
   }

   func filterDuplicate<T>(_ keyValue:(Element)->T) -> [Element]
   { 
      return filterDuplicate{"\(keyValue($0))"}
   }
}

// example usage: (for a unique combination of attributes):

peopleArray = peopleArray.filterDuplicate{ ($0.name, $0.age, $0.sex) }

or...

peopleArray = peopleArray.filterDuplicate{ "\(($0.name, $0.age, $0.sex))" }

You don't have to bother with making values Hashable and it allows you to use different combinations of fields for uniqueness.

Note: for a more robust approach, please see the solution proposed by Coeur in the comments below.

stackoverflow.com/a/55684308/1033581

[EDIT] Swift 4 alternative

With Swift 4.2 you can use the Hasher class to build a hash much easier. The above extension could be changed to leverage this :

extension Array
{
    func filterDuplicate(_ keyValue:((AnyHashable...)->AnyHashable,Element)->AnyHashable) -> [Element]
    {
        func makeHash(_ params:AnyHashable ...) -> AnyHashable
        { 
           var hash = Hasher()
           params.forEach{ hash.combine($0) }
           return hash.finalize()
        }  
        var uniqueKeys = Set<AnyHashable>()
        return filter{uniqueKeys.insert(keyValue(makeHash,$0)).inserted}     
    }
}

The calling syntax is a little different because the closure receives an additional parameter containing a function to hash a variable number of values (which must be Hashable individually)

peopleArray = peopleArray.filterDuplicate{ $0($1.name, $1.age, $1.sex) } 

It will also work with a single uniqueness value (using $1 and ignoring $0).

peopleArray = peopleArray.filterDuplicate{ $1.name } 

4 Comments

This could give random results depending on the behavior of "\()", as it may not gives you unique values like conforming to Hashable should. Example, if your elements conform to Printable by all returning the same description, then your filtering fails.
Agreed. Selection of the fields (or formula) that will produce the desired uniqueness pattern will have to take this into account. For many use cases, this provides a simple ad-hoc solution that requires no alteration of the element's class or struct.
@AlainT. Don't do this, really. String's purpose is not to be some ghetto ad-hoc key generation mechanism. Just constrain T to being Hashable.
@Alexander I've applied this idea in a new answer: stackoverflow.com/a/55684308/1033581
8

As was noted at WWDC 2021, Swift has community-developed Algorithms, Collections, and Numerics Packages. The Algorithms package features a uniqued() algorithm.

These are not yet part of the Swift Standard library. You can currently download them from Apple's Github page and/or install them via Swift Package Manager.

WWDC Video:

https://developer.apple.com/videos/play/wwdc2021/10256/

Github page:

https://github.com/apple/swift-algorithms

uniqued() and uniqued(on:) documentation:

https://github.com/apple/swift-algorithms/blob/main/Guides/Unique.md

2 Comments

This should be the top answer
How do I do this in iOS?
5
  1. First add all the elements of an array to NSOrderedSet.
  2. This will remove all the duplicates in your array.
  3. Again convert this orderedset to an array.

Done....

Example

let array = [1,1,1,1,2,2,2,2,4,6,8]

let orderedSet : NSOrderedSet = NSOrderedSet(array: array)

let arrayWithoutDuplicates : NSArray = orderedSet.array as NSArray

output of arrayWithoutDuplicates - [1,2,4,6,8]

Comments

4

You can use directly a set collection to remove duplicate, then cast it back to an array

var myArray = [1, 4, 2, 2, 6, 24, 15, 2, 60, 15, 6]
var mySet = Set<Int>(myArray)

myArray = Array(mySet) // [2, 4, 60, 6, 15, 24, 1]

Then you can order your array as you want

myArray.sort{$0 < $1} // [1, 2, 4, 6, 15, 24, 60]

1 Comment

"Then you can order your array as you want" What if I want the same ordering as from the original array? It's not that easy.
4

In case you need values sorted, this works (Swift 4)

let sortedValues = Array(Set(array)).sorted()

8 Comments

You loosing elements order in this case.
Not at all, that's what the .sorted() at the end is for. Regards.
@MauricioChirino And if your original array was [2, 1, 1]? It would come out [1, 2], that's not ordered :p
@MauricioChirino No, I'm not. If the goal is to remove duplicate values from a sequence, while retaining the order in which elements uniquely appeared, this doesn't do that. The very clear counter example is [2, 1, 1]. The first appearance of unique elements, in order is [2, 1]. That's the correct answer. But using your (incorrect) algorithm, you get [1, 2], which is sorted, but is not in the correct, original, order.
Fails if the elements in array are not Hashable; only Hashable data types can be added to a Set, yet any data type can be added to an array.
|
3

Slightly more succinct syntax version of Daniel Krom's Swift 2 answer, using a trailing closure and shorthand argument name, which appears to be based on Airspeed Velocity's original answer:

func uniq<S: SequenceType, E: Hashable where E == S.Generator.Element>(source: S) -> [E] {
  var seen = [E: Bool]()
  return source.filter { seen.updateValue(true, forKey: $0) == nil }
}

Example of implementing a custom type that can be used with uniq(_:) (which must conform to Hashable, and thus Equatable, because Hashable extends Equatable):

func ==(lhs: SomeCustomType, rhs: SomeCustomType) -> Bool {
  return lhs.id == rhs.id // && lhs.someOtherEquatableProperty == rhs.someOtherEquatableProperty
}

struct SomeCustomType {

  let id: Int

  // ...

}

extension SomeCustomType: Hashable {

  var hashValue: Int {
    return id
  }

}

In the above code...

id, as used in the overload of ==, could be any Equatable type (or method that returns an Equatable type, e.g., someMethodThatReturnsAnEquatableType()). The commented-out code demonstrates extending the check for equality, where someOtherEquatableProperty is another property of an Equatable type (but could also be a method that returns an Equatable type).

id, as used in the hashValue computed property (required to conform to Hashable), could be any Hashable (and thus Equatable) property (or method that returns a Hashable type).

Example of using uniq(_:):

var someCustomTypes = [SomeCustomType(id: 1), SomeCustomType(id: 2), SomeCustomType(id: 3), SomeCustomType(id: 1)]

print(someCustomTypes.count) // 4

someCustomTypes = uniq(someCustomTypes)

print(someCustomTypes.count) // 3

1 Comment

There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See stackoverflow.com/a/55684308/3141234
3

Here is a solution that

  • Uses no legacy NS types
  • Is reasonably fast with O(n)
  • Is concise
  • Preserves element order
extension Array where Element: Hashable {

    var uniqueValues: [Element] {
        var allowed = Set(self)
        return compactMap { allowed.remove($0) }
    }
}

1 Comment

This is nice but would only works for Hashable elements
2

here I've done some O(n) solution for objects. Not few-lines solution, but...

struct DistinctWrapper <T>: Hashable {
    var underlyingObject: T
    var distinctAttribute: String
    var hashValue: Int {
        return distinctAttribute.hashValue
    }
}
func distinct<S : SequenceType, T where S.Generator.Element == T>(source: S,
                                                                distinctAttribute: (T) -> String,
                                                                resolution: (T, T) -> T) -> [T] {
    let wrappers: [DistinctWrapper<T>] = source.map({
        return DistinctWrapper(underlyingObject: $0, distinctAttribute: distinctAttribute($0))
    })
    var added = Set<DistinctWrapper<T>>()
    for wrapper in wrappers {
        if let indexOfExisting = added.indexOf(wrapper) {
            let old = added[indexOfExisting]
            let winner = resolution(old.underlyingObject, wrapper.underlyingObject)
            added.insert(DistinctWrapper(underlyingObject: winner, distinctAttribute: distinctAttribute(winner)))
        } else {
            added.insert(wrapper)
        }
    }
    return Array(added).map( { return $0.underlyingObject } )
}
func == <T>(lhs: DistinctWrapper<T>, rhs: DistinctWrapper<T>) -> Bool {
    return lhs.hashValue == rhs.hashValue
}

// tests
// case : perhaps we want to get distinct addressbook list which may contain duplicated contacts like Irma and Irma Burgess with same phone numbers
// solution : definitely we want to exclude Irma and keep Irma Burgess
class Person {
    var name: String
    var phoneNumber: String
    init(_ name: String, _ phoneNumber: String) {
        self.name = name
        self.phoneNumber = phoneNumber
    }
}

let persons: [Person] = [Person("Irma Burgess", "11-22-33"), Person("Lester Davidson", "44-66-22"), Person("Irma", "11-22-33")]
let distinctPersons = distinct(persons,
    distinctAttribute: { (person: Person) -> String in
        return person.phoneNumber
    },
    resolution:
    { (p1, p2) -> Person in
        return p1.name.characters.count > p2.name.characters.count ? p1 : p2
    }
)
// distinctPersons contains ("Irma Burgess", "11-22-33") and ("Lester Davidson", "44-66-22")

1 Comment

Rather than using a Set with a custom DistinctWrapper, you should use a Dictionary from distinctAttributes to objects. When you follow that logic through, you would eventually end up implementing [Dictionary.init(_:uniquingKeysWith:)]pastebin.com/w90pVe0p(https://developer.apple.com/documentation/…, which is now built into the standard library. Check out how simple this is pastebin.com/w90pVe0p
2

I used @Jean-Philippe Pellet's answer and made an Array extension that does set-like operations on arrays, while maintaining the order of elements.

/// Extensions for performing set-like operations on lists, maintaining order
extension Array where Element: Hashable {
  func unique() -> [Element] {
    var seen: [Element:Bool] = [:]
    return self.filter({ seen.updateValue(true, forKey: $0) == nil })
  }

  func subtract(takeAway: [Element]) -> [Element] {
    let set = Set(takeAway)
    return self.filter({ !set.contains($0) })
  }

  func intersect(with: [Element]) -> [Element] {
    let set = Set(with)
    return self.filter({ set.contains($0) })
  }
}

1 Comment

There no need to use a Bool, when the only value you use is true. You're reaching for a "unit type" (a type with only one possible value). Swift's unit type is Void, whose only value is () (a.k.a. the empty tuple). So you can just use [T: Void]. Though you shouldn't do that, because you've basically just invented Set. Use Set instead. See stackoverflow.com/a/55684308/3141234
2

This is just a very simple and convenient implementation. A computed property in an extension of an Array that has equatable elements.

extension Array where Element: Equatable {
    /// Array containing only _unique_ elements.
    var unique: [Element] {
        var result: [Element] = []
        for element in self {
            if !result.contains(element) {
                result.append(element)
            }
        }

        return result
    }
}

1 Comment

This is also O(n^2).
2
func removeDublicate (ab: [Int]) -> [Int] {
var answer1:[Int] = []
for i in ab {
    if !answer1.contains(i) {
        answer1.append(i)
    }}
return answer1
}

Usage:

let f = removeDublicate(ab: [1,2,2])
print(f)

3 Comments

I think this is the simplest
it keeps the order and gives you an array you want
This is also O(n²).
2

Swift 3/ Swift 4/ Swift 5

Just one line code to omit Array duplicates without effecting order:

let filteredArr = Array(NSOrderedSet(array: yourArray))

1 Comment

Here we are typecasting an array into Orderedset. Definition of "Sets" - sets allow only distinct values (It does not allow duplicates). Hence duplicates will be omitted As we are typecasting with NSOrderedSet hence array order will not be disturbed.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.