I have a 2D NumPy array. I want to slice out unequal length subsets of columns and put them in a single array with the rest of the values being filled by nan. That is to say in:
data = np.random.normal(size=(100,4))
I want to index from [75, 33, 42, 54] to end. That is to say row index from 75 to the end in column 0, row 33 to the end in column 1 and so on.
I tried data[[slice(75,100),slice(33,100)],:] but it didn't work.
You can do it by creating a mask, with True for the indices you want to be np.nan and False otherwise:
import numpy as np
data = np.random.normal(size=(5,4))
b = np.array([0, 1, 2, 3])
mask = np.arange(len(data))[:, None] < b
data[mask] = np.nan
data
Output:
array([[-0.53306108, nan, nan, nan],
[ 1.32282687, 0.83204007, nan, nan],
[-1.07143908, 0.12972517, -0.4783274 , nan],
[ 0.39686727, -1.20532247, -2.17043218, 0.74859079],
[ 1.82548696, 0.98669461, -1.17961517, -0.7813723 ]])
To get repeatable results, I seeded the random generator:
np.random.seed(0)
and then created the source array:
data = np.random.normal(size=(100,4))
The list of starting indices I defined as:
ind = [75, 33, 42, 54]
An initial step is to compute the max length of the output column:
ml = max([ 100 - i for i in ind ])
Then you can generate the output array as:
result = np.array([ np.pad(data[rInd:, i[0]], (0, ml - 100 + rInd),
constant_values = np.nan) for i, rInd in np.ndenumerate(ind) ]).T
This code:
The result, for my source data, is:
array([[-1.30652685, 0.03183056, 0.92085882, -0.04225715],
[ 0.66638308, -0.20829876, -1.03424284, 0.48148147],
[ 0.69377315, 0.4393917 , -0.4555325 , 0.23218104],
[-1.12682581, 0.94447949, -0.6436184 , -0.49331988],
[-0.04217145, -0.4615846 , -1.10438334, 0.7811981 ],
[-0.71960439, -0.82643854, -1.29285691, 0.67690804],
[-1.15735526, -1.07993151, 0.52327666, -0.29779088],
[-0.70470028, 1.92953205, 2.16323595, 1.07961859],
[ 0.77325298, 0.84436298, 1.0996596 , -0.57578797],
[-1.75589058, 0.31694261, -0.02432612, 0.69474914],
[ 1.0685094 , -0.65102559, 0.91017891, 0.61037938],
[-0.44092263, -0.68954978, -0.94444626, -0.0525673 ],
[ 0.5785215 , -1.37495129, 2.25930895, 0.08842209],
[ 1.36453185, -1.60205766, -0.46359597, -2.77259276],
[-1.84306955, 1.5430146 , 0.15650654, -0.39095338],
[ 0.69845715, -1.1680935 , -1.42406091, 2.06449286],
[-0.01568211, 0.82350415, -1.15618243, 1.53637705],
[-0.26773354, -0.23937918, 0.42625873, 1.21114529],
[ 0.84163126, -1.61695604, -0.13288058, -0.48102712],
[ 0.64331447, -0.09815039, 1.15233156, 1.13689136],
[-1.69810582, -0.4664191 , 0.52106488, 0.37005589],
[ 0.03863055, 0.37915174, 0.69153875, -0.6801782 ],
[ 1.64813493, -0.34598178, -1.5829384 , -1.34671751],
[-0.35343175, 0.06326199, -0.59631404, 1.07774381],
[ 0.85792392, -0.23792173, 0.52389102, 0.09435159],
[ nan, 0.41605005, 0.39904635, -0.10730528],
[ nan, -2.06998503, -0.65240858, -0.89091508],
[ nan, -0.39727181, -2.03068447, 2.2567235 ],
[ nan, -1.67600381, -0.69204985, -1.18894496],
[ nan, -1.46642433, -1.04525337, 0.60631952],
[ nan, -0.31932842, -0.62808756, 1.6595508 ],
[ nan, -1.38336396, -0.1359497 , -1.2140774 ],
[ nan, -0.50681635, -0.39944903, 0.15670386],
[ nan, 0.1887786 , -0.11816405, -1.43779147],
[ nan, 0.09740017, -1.33425847, -0.52118931],
[ nan, 0.39009332, -0.13370156, 0.6203583 ],
[ nan, -0.11610394, -0.38487981, 0.33996498],
[ nan, 1.02017271, -0.0616264 , -0.39484951],
[ nan, 0.60884383, 0.27451636, -0.99312361],
[ nan, 1.30184623, -0.15766702, 0.49383678],
[ nan, -1.06001582, 0.74718833, 0.88017891],
[ nan, 0.58295368, -2.65917224, -1.02250684],
[ nan, 1.65813068, -0.6840109 , -1.47183501],
[ nan, -0.46071979, -0.68783761, -0.2226751 ],
[ nan, -0.15957344, -0.36469354, -0.76149221],
[ nan, -0.73067775, -0.76414392, 0.85255194],
[ nan, -0.28688719, -0.6522936 , nan],
[ nan, -0.81299299, -0.47965581, nan],
[ nan, -0.31229225, 0.93184837, nan],
[ nan, 0.94326072, -0.19065349, nan],
[ nan, -1.18388064, 0.28044171, nan],
[ nan, 0.45093446, 0.04949498, nan],
[ nan, -0.4533858 , -0.20690368, nan],
[ nan, -0.2803555 , -2.25556423, nan],
[ nan, 0.34965446, -0.98551074, nan],
[ nan, -0.68944918, 0.56729028, nan],
[ nan, -0.477974 , -0.29183736, nan],
[ nan, 0.00377089, 1.46657872, nan],
[ nan, 0.16092817, nan, nan],
[ nan, -1.12801133, nan, nan],
[ nan, -0.24945858, nan, nan],
[ nan, -1.57062341, nan, nan],
[ nan, 0.38728048, nan, nan],
[ nan, -1.6567151 , nan, nan],
[ nan, 0.16422776, nan, nan],
[ nan, -1.61647419, nan, nan],
[ nan, 1.14110187, nan, nan]])
Note that the above code contains i[0], because np.ndenumerate returns as the first result a tuple of indices.
Since data is a 1-D array, we are interested in the first index only, so after i I put [0].
This one works for me:
data = np.random.normal(size=(100,4))
slices_values = [75, 33, 42, 54] # You name your slices here
slices = [] # In this list you will keep the slices
for i in slices_values:
x = slice(i, 100)
slices.append(data[x])
Now you can confirm the shape of each slices:
slices[0].shape # (25, 4)
slices[1].shape # (67, 4)
slices[2].shape # (58, 4)
slices[3].shape # (46, 4)
It is difficult to find out what is the expected result, But if IIUC, one way is to create an array and fill it using looping:
data = np.random.normal(size=(5, 4))
ids = np.array([2, 1, 2, 3])
def test(data, ids):
arr = np.empty_like(data)
for i, j in enumerate(ids):
arr[:j, i] = data[:j, i]
arr[j:, i] = np.nan
return arr
res = test(data, ids)
# [[ 0.1768507210788626 2.3777541249700573 0.998732857053734 -1.3101507969798436 ]
# [ 0.18018992116935298 nan -1.443125868756967 -1.3992855573400653 ]
# [ nan nan nan -0.2319322879433409 ]
# [ nan nan nan nan]
# [ nan nan nan nan]]
or:
def test(data, ids):
arr = np.empty_like(data)
for i, j in enumerate(ids):
arr[:j, i] = np.nan
arr[j:, i] = data[j:, i]
return arr
# [[ nan nan nan nan]
# [ nan -1.7647540193678475 nan nan]
# [ 0.8203539992532282 1.2952993197746814 0.9421974218807785 nan]
# [-0.6313979666045816 -0.6421770233773478 -0.3816716009896775 -1.7634440039930654]
# [ 1.611668212682313 -0.878108388861928 -0.4985770669099582 0.9072434022928676]]