98

I'm sure this is an extremely obvious question, and that there's a function that does exactly this, but I can't seem to find it. In PHP, I'd like to know if my array has duplicates in it, as efficiently as possible. I don't want to remove them like array_unique does, and I don't particularly want to run array_unique and compare it to the original array to see if they're the same, as this seems very inefficient. As far as performance is concerned, the "expected condition" is that the array has no duplicates.

I'd just like to be able to do something like

if (no_dupes($array))
    // this deals with arrays without duplicates
else
    // this deals with arrays with duplicates

Is there any obvious function I'm not thinking of?
How to detect duplicate values in PHP array?
has the right title, and is a very similar question, however if you actually read the question, he's looking for array_count_values.

7
  • Do you just want to know if there are any duplicates or the quantity and value of said duplicates etc? Commented Jun 29, 2010 at 23:56
  • 1
    I only need to know if there are any duplicates. Returning a boolean is perfect. Commented Jun 29, 2010 at 23:57
  • 23
    Honestly I think if(count($array) == count(array_unique($array))) is the best you can get. You have to traverse the array this way or another and I think the built-in are optimized for that. array_flip could be considered too. Commented Jun 30, 2010 at 0:02
  • @Felix, you can do better than that. That does three loops, one to create the unique array, one to count it, and one to count the original. Commented Jun 30, 2010 at 0:17
  • @Mike Sherov: Are you sure? I couldn't find anything about it, but I had hoped that PHP arrays have some internal property that keeps track of the length. Do you have an information about this? I would be very interested. Commented Jun 30, 2010 at 7:46

18 Answers 18

263

I know you are not after array_unique(). However, you will not find a magical obvious function nor will writing one be faster than making use of the native functions.

I propose:

function array_has_dupes($array) {
   // streamline per @Felix
   return count($array) !== count(array_unique($array));
}

Adjust the second parameter of array_unique() to meet your comparison needs.

Sign up to request clarification or add additional context in comments.

11 Comments

thanks for the suggestion. My thought at finding a better algorithm is simply that, technically speaking, once you've finished running whatever the builtin array_unique does, you should be able to know if there were any dupes. Thus, anything that does at least as much work as array_unique does more work than necessary. Although yeah, if such a function doesn't exist, I don't particularly feel like writing it.
If you only care about if it has dupes, then that's what I would do. If you care about more than just if it has dupes, then you're right, the above may do more work than it needs. Anything you write is going to be O(n^2). Even if you bail out early. As you said, it's not common that you have dupes. So is it worth your time to make something magical?
Magical? Sure it's a microoptimization, but it's not "magic" to write your own function, and I'm not sure it's that a better solution is that much harder to write than this.
I came here only to find exactly this answer:)
Elegant, but array_unique is somewhat slow. If you know the array to only contain integers and strings, you can replace it with array_flip for much faster results.
|
106

Performance-Optimized Solution

If you care about performance and micro-optimizations, check this one-liner:

function no_dupes(array $input_array) {
    return count($input_array) === count(array_flip($input_array));
}

Description:
Function compares number of array elements in $input_array with array_flip'ed elements. Values become keys and guess what - keys must be unique in associative arrays so not unique values are lost and final number of elements is lower than original.

Warning:
As noted in the manual, array keys can be only type of int or string so this is what you must have in original array values to compare, otherwise PHP will start casting with unexpected results. See https://3v4l.org/7bRXI for an example of this fringe-case failure mode.

Proof for an array with 10 million records:

Test case:

<?php

$elements = array_merge(range(1,10000000),[1]);

$time = microtime(true);
accepted_solution($elements);
echo 'Accepted solution: ', (microtime(true) - $time), 's', PHP_EOL;

$time = microtime(true);
most_voted_solution($elements);
echo 'Most voted solution: ', (microtime(true) - $time), 's', PHP_EOL;

$time = microtime(true);
this_answer_solution($elements);
echo 'This answer solution: ', (microtime(true) - $time), 's', PHP_EOL;

function accepted_solution($array){
 $dupe_array = array();
 foreach($array as $val){
  // sorry, but I had to add below line to remove millions of notices
  if(!isset($dupe_array[$val])){$dupe_array[$val]=0;}
  if(++$dupe_array[$val] > 1){
   return true;
  }
 }
 return false;
}

function most_voted_solution($array) {
   return count($array) !== count(array_unique($array));
}

function this_answer_solution(array $input_array) {
    return count($input_array) === count(array_flip($input_array));
}

Notice that accepted solution might be faster in certain condition when not unique values are near the beginning of huge array.

2 Comments

Can you explain why is this faster? Also this returns the contrary. So to have a fair comparison you should test with : function most_voted_solution($array) { return count($array) === count(array_unique($array)); }
@ErdalG. this is faster because array_flip is native PHP function written in C and flip is pretty simple operation. After flipping not unique values are removed as they could create array key conflict.
41

You can do:

function has_dupes($array) {
    $dupe_array = array();
    foreach ($array as $val) {
        if (++$dupe_array[$val] > 1) {
            return true;
        }
    }
    return false;
}

6 Comments

I like it! Just keep in mind that even with an early return this is an O(n) function. In addition to the overhead of foreach and tracking $dupe_array, I'd love to see some benchmarking. I'd guess for array's with no duplicates, utilizing native functions is faster. Definitely better than O(n^2) though. Nice.
Has a little problem: only works correctly if the values are strings or numbers.
This code gave me an undefined offset error in PHP. Instead, I did: foreach ( $a as $v ) { if ( array_key_exists($v,$dupe) { return true; } else { $dupe[$v] = true; }
How does this even work? Since $dupe_array has not been defined with any values, $dupe_array[$val] should return an undefined index!
what does ++dupe_array[$val] means? and isn't it suppose to be ++dupe_arrays[$key] because simply $val is not $key?
|
23
$hasDuplicates = count($array) > count(array_unique($array)); 

Will be true if duplicates, or false if no duplicates.

2 Comments

This is pretty much a retread of @JasonMcCreary's answer. stackoverflow.com/a/3145647/2943403
But It is throwing duplicate values error even when array has empty values. I have posted answer stackoverflow.com/questions/3145607/… below
6
$duplicate = false;

 if(count(array) != count(array_unique(array))){
   $duplicate = true;
}

1 Comment

This is pretty much a retread of @JasonMcCreary's answer. stackoverflow.com/a/3145647/2943403
5

Here's my take on this… after some benchmarking, I found this to be the fastest method for this.

function has_duplicates( $array ) {
    return count( array_keys( array_flip( $array ) ) ) !== count( $array );
}

…or depending on circumstances this could be marginally faster.

function has_duplicates( $array ) {
    $array = array_count_values( $array );
    rsort( $array );
    return $array[0] > 1;
}

2 Comments

Not sure why you need array_keys() in your answer. array_flip() already condenses your array if the values are the same. Also, != is a sufficient compararor, since the types are inherently the same out of count() (you're the one that mentioned benchmarking). Therefore return count(array_flip($arr)) != count($arr); should be sufficient.
The techniques in this answer have the same vulnerabilities as @s3m3n's function. 3v4l.org/3FlBJ This is an "apples-vs-oranges" comparison, so I'll argue that any benchmark comparisons are inappropriate because the function do not offer identical behavior.
1

Find this useful solution

function get_duplicates( $array ) {
    return array_unique( array_diff_assoc( $array, array_unique( $array ) ) );
}

After that count result if greater than 0 than duplicates else unique.

2 Comments

Despite being a one-liner, this technique appears to be doing more processing than other posted answers. To check if an array is empty without calling count(), just do a falsey check using !: 3v4l.org/O4g3F
maybe this is not the best answer to the exact question, but it is a great answer to the problem of finding the duplicates in an array
1

To remove all the empty values from the comparison you can add array_diff()

if (count(array_unique(array_diff($array,array("")))) < count(array_diff($array,array(""))))

Reference taken from @AndreKR answer from here

Comments

0

Two ways to do it efficiently that I can think of:

  1. inserting all the values into some sort of hashtable and checking whether the value you're inserting is already in it(expected O(n) time and O(n) space)

  2. sorting the array and then checking whether adjacent cells are equal( O(nlogn) time and O(1) or O(n) space depending on the sorting algorithm)

stormdrain's solution would probably be O(n^2), as would any solution which involves scanning the array for each element searching for a duplicate

Comments

0

Keep it simple, silly! ;)

Simple OR logic...

function checkDuplicatesInArray($array){
    $duplicates=FALSE;
    foreach($array as $k=>$i){
        if(!isset($value_{$i})){
            $value_{$i}=TRUE;
        }
        else{
            $duplicates|=TRUE;          
        }
    }
    return ($duplicates);
}

Regards!

2 Comments

#BadCode - The best ways to do this check with functions of PHP itself.
I find variable variables to generally unattractive solution. This technique may fail in certain scenarios. 3v4l.org/kGLWT Moreso, from PHP7.4 and up.
0

I'm using this:

if(count($array)==count(array_count_values($array))){
    echo("all values are unique");
}else{
    echo("there's dupe values");
}

I don't know if it's the fastest but works pretty good so far

1 Comment

Some data types will cause this technique to fail, so this is not a reliable/robust solution. 3v4l.org/FSr7P
0

One more solution from me, this is related to performance improvement

$array_count_values = array_count_values($array);
if(is_array($array_count_values) && count($array_count_values)>0)
{
   foreach ($array_count_values as $key => $value)
   {
      if($value>1)
      {
        // duplicate values found here, write code to handle duplicate values            
      }
   }
}

Comments

0

The shortest one

function no_dupes($array) {
    return $array === array_unique($array);
}

Comments

-1

As you specifically said you didn't want to use array_unique I'm going to ignore the other answers despite the fact they're probably better.

Why don't you use array_count_values() and then check if the resulting array has any value greater than 1?

Comments

-1

Php has an function to count the occurrences in the array http://www.php.net/manual/en/function.array-count-values.php

Comments

-1

You can do it like that way also: This will return true if unique else return false.

$nofollow = (count($modelIdArr) !== count(array_unique($modelIdArr))) ? true : false;

1 Comment

This is pretty much a retread of @JasonMcCreary's answer. stackoverflow.com/a/3145647/2943403 I have voted to delete this post.
-1

The simple solution but quite faster.

$elements = array_merge(range(1,10000000),[1]);

function unique_val_inArray($arr) {
    $count = count($arr);
    foreach ($arr as $i_1 => $value) {
        for($i_2 = $i_1 + 1; $i_2 < $count; $i_2++) {
            if($arr[$i_2] === $arr[$i_1]){
                return false;
            }
        }
    }
    return true;
}

$time = microtime(true);
unique_val_inArray($elements);
echo 'This solution: ', (microtime(true) - $time), 's', PHP_EOL;

Speed - [0.71]!

Comments

-1
function hasDuplicate($array){
  $d = array();
  foreach($array as $elements) {
    if(!isset($d[$elements])){
      $d[$elements] = 1;
    }else{
      return true;
    } 
  } 
  return false;
}

1 Comment

Where is the explanation of what this source code will accomplish?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.