Remove pairwise duplicates from 2 numpy arrays using np.unique

Question

I have two arrays I created as followed:

     DotsLat1=np.concatenate((HLat,DotsLatMA,DotsLatMC,DotsLatMB,DotsLatMD,DotsLatMAB,DotsLatMAD,DotsLatMBC,DotsLatMDC), axis=0)
    DotsLon1=np.concatenate((HLon,DotsLonMA,DotsLonMC,DotsLonMB,DotsLonMD,DotsLonMAB,DotsLonMAD,DotsLonMBC,DotsLonMDC), axis=0)#

Which gives the following Latitudes & Longitudes respectively:

    array([51.43584   , 51.47806059, 51.47554269, 51.39361941, 51.39613731,
   51.43584   , 51.44428412, 51.45272824, 51.46117236, 51.46961647,
   51.39361941, 51.40206353, 51.41050764, 51.41895176, 51.42739588,
   51.43584   , 51.45172108, 51.46760215, 51.43584   , 51.41995892,
   51.40407785, 51.47604627, 51.4601652 , 51.46860931, 51.47705343,
   51.41252196, 51.42840304, 51.43684716, 51.44529128, 51.45915804,
   51.44327696, 51.43483284, 51.42638872, 51.39563373, 51.4115148 ,
   51.40307069, 51.39462657])
array([2.59277   , 2.72014661, 2.55890633, 2.46539339, 2.62663367,
   2.59277   , 2.61824532, 2.64372065, 2.66919597, 2.69467129,
   2.46539339, 2.49086871, 2.51634403, 2.54181935, 2.56729468,
   2.59277   , 2.57922453, 2.56567906, 2.59277   , 2.60631547,
   2.61986094, 2.59115438, 2.60469985, 2.63017518, 2.6556505 ,
   2.64533626, 2.63179079, 2.65726611, 2.68274144, 2.54020374,
   2.55374921, 2.52827389, 2.50279856, 2.59438562, 2.58084015,
   2.55536482, 2.5298895 ])

There are some points with the same longitude AND the same latitude (like the first ones for example). I want to delete those points in both arrays if BOTH latitude and longitude have duplicates (so if the points would be plotted over each other in a map). It is thus important that the right order is maintained.

When I use

DotsLat2=np.unique(DotsLat1)
DotsLon2=np.unique(DotsLon1)

the order is no longer correct and my points are scattered.

When I use

DotsLat2=list(set([DotsLat1]))
DotsLon2=list(set([DotsLon1]))

the error is

unhashable type: 'numpy.ndarray'

Any idea how to get rid of the error and create my unique points?

To get a list out of a numpy ndarray use the tolist() method: DotsLat1.tolist() — Sergio
– Sergio, Commented Mar 13, 2018 at 11:28
I would put the lats and long in the same array adding an extra dimension, instead of two separate arrays, and then you could use np.unique — J_P
– J_P, Commented Mar 13, 2018 at 11:28

jpp · Accepted Answer · 2018-03-13 11:35:34Z

import numpy as np
lat=np.array([51.43584   , 51.47806059, 51.47554269, 51.39361941, 51.39613731,
   51.43584   , 51.44428412, 51.45272824, 51.46117236, 51.46961647,
   51.39361941, 51.40206353, 51.41050764, 51.41895176, 51.42739588,
   51.43584   , 51.45172108, 51.46760215, 51.43584   , 51.41995892,
   51.40407785, 51.47604627, 51.4601652 , 51.46860931, 51.47705343,
   51.41252196, 51.42840304, 51.43684716, 51.44529128, 51.45915804,
   51.44327696, 51.43483284, 51.42638872, 51.39563373, 51.4115148 ,
   51.40307069, 51.39462657])
long=np.array([2.59277   , 2.72014661, 2.55890633, 2.46539339, 2.62663367,
   2.59277   , 2.61824532, 2.64372065, 2.66919597, 2.69467129,
   2.46539339, 2.49086871, 2.51634403, 2.54181935, 2.56729468,
   2.59277   , 2.57922453, 2.56567906, 2.59277   , 2.60631547,
   2.61986094, 2.59115438, 2.60469985, 2.63017518, 2.6556505 ,
   2.64533626, 2.63179079, 2.65726611, 2.68274144, 2.54020374,
   2.55374921, 2.52827389, 2.50279856, 2.59438562, 2.58084015,
   2.55536482, 2.5298895 ])

setl = np.column_stack((lat, long))
print(setl)
print(setl.shape)

setl2 = np.unique(setl, axis=0)
print( setl2 )
print(setl2.shape)

Then simply unpack to get your components again:

lat_unique, long_unique = setl2[:, 0], setl2[:, 1]

Rao Sahab · Accepted Answer · 2018-03-13 11:32:16Z

0

Use the following example on yours

tp = np.array([1, 2, 3, 1, 5, 6, 7, 6, 9, 10])

fp = np.array([3, 2, 13, 3, 15, 16, 17, 16, 19, 20])

combined = np.vstack((tp, fp)).T

x = np.random.rand(combined.shape[1])
y = combined.dot(x)
unique, index = np.unique(y, return_index=True)

combined[index]

tp[index]

answered Mar 13, 2018 at 11:32

Rao Sahab

1,3111 gold badge11 silver badges13 bronze badges

Collectives™ on Stack Overflow

Remove pairwise duplicates from 2 numpy arrays using np.unique

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related