How to avoid duplicate values in an array of arrays?

Question

I know how to avoid duplicates in a one-dimensional array.

However, I have an array of arrays, and two lines of it may hold arrays with different references, but same values. I tried this:

sub unique {
     my %seen;
     grep !$seen{join('',$_)}++, @_ 
}

my @aa = (  ["1","2","3"],["1","2","3"],["1","2","4"] );
my @bb = unique(@aa);
print $_ for (@bb);

It should remove one of the two "123" arrays, but it doesn't. Probably because $_ holds a reference and not an array that can be joined? Of couse, I could loop through the $_ referenced array and concat all values, then use that as key to the %seen hash.

But I suspect there is a very elegant solution in Perl that I don't yet know of...

choroba · Accepted Answer · 2014-09-04 13:46:23Z

5

To fix your naive approach, you should dereference the array references in two places: when serializing and when printing:

# Assumes the elements don't contain the value of $; (0x1C by default)
sub unique {
     my %seen;
     grep ! $seen{ join $;, @$_ }++, @_
}

my @aa = (  ["1","2","3"],["1","2","3"],["1","2","4"] );
my @bb = unique(@aa);
print "@$_\n" for (@bb);

This could still give wrong output, imagine [ "1\x{1C}2", 3 ]. More complex stringification is needed if your data could contain such strings. Fortunately, Perl already has a way to serialize array references: Data::Dumper:

use Data::Dumper;

sub unique {
    my %seen;
    grep ! $seen{ Dumper $_ }++, @_
}

edited Sep 4, 2014 at 13:46

answered Sep 4, 2014 at 12:20

choroba

245k27 gold badges221 silver badges304 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

jackthehipster Over a year ago

Works great. I'll go with the @$_ approach, since my real data structure will not allow such duplicates as in the example. May I ask, what the @$_ actually does? Is @ in front of an array-ref always returning the array itself?

choroba Over a year ago

@jackthehipster: Yes. When dereferencing a more complex structure, you might need curly braces: @{ $hash{key} }.

mpapec Over a year ago

Joining by $; or even single space seems optimal.

jaypal singh Over a year ago

Absolutely not @Borodin. I am not choroba's sock puppet, if thats what you were trying to get at (though I do up vote a lot of his answers but mainly because they are good but I do that for yours mpapec, ikegami, TLP too.. :)). My upvote was due to the fact that I would have solved it the same way and my comment was for the down voter to leave a feedback on what could potentially go wrong with this approach so that I can learn from it.

Borodin Over a year ago

@jaypal: Ah OK. It's just that your comment to go with the +1 was about only the downvote.

|

Collectives™ on Stack Overflow

How to avoid duplicate values in an array of arrays?

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related