Split numpy array into chunks

Question

I have an array x of len 30. I want to separate it out into chunks of 8 samples each in 2 different ways:

First, I want to separate it avoiding any overlap so that I end up with 3 arrays of length 8 and the final array will be only 6 (due to some samples being missing).

Secondly, I want to separate it so that the final array will be the last 2 samples of the previous array plus the final 6.

I want to do this efficiently on large arrays.

I have tried using np.array_split as follows

x = np.array([1 ,1, 2 ,1 ,1 ,2 ,1, 0 ,3, 1, 2 ,2, 1, 2, 1, 1,50,1 ,1, 1, 1, 4, 1, 11, 15, 0, 0, 1, 1,0])

y = np.array_split(x, np.ceil(len(x)/8))

However, that results in:

y = [array([1, 1, 2, 1, 1, 2, 1, 0]),
     array([3, 1, 2, 2, 1, 2, 1, 1]),
     array([50,  1,  1,  1,  1,  4,  1]),
     array([11, 15,  0,  0,  1,  1,  0])]

But it gives 2x8 length arrays and 2x7 length arrays, not what I want.

How can I achieve this ?

if instead of a list of arrays you end up with a single 2D array, would that work for you? If so, then you can just reshape your initial array. — user20396240
– user20396240, Commented Nov 24, 2022 at 12:31
@SembeiNorimaki a 2D array is fine for what I need later on... not quite sure how that would work though with the inequalities in rows of the 2D array for reshaping, could you please explain further? — Jack
– Jack, Commented Nov 24, 2022 at 12:36
I've Updated 2 solutions @Jack . Pls, Try them & Let me know if any issues. — Bhargav
– Bhargav, Commented Nov 24, 2022 at 13:36
@Bhargav sorry I've been pulled into something else, I will go back and test your solutions asap, thank you — Jack
– Jack, Commented Nov 24, 2022 at 14:24

chrslg · Accepted Answer · 2022-11-24 13:22:45Z

1

import numpy as np

x = np.array([1 ,1, 2 ,1 ,1 ,2 ,1, 0 ,3, 1, 2 ,2, 1, 2, 1, 1,50 ,1 ,1, 1, 1, 4, 1, 11, 15, 0, 0, 1, 1,0])

def split_reminder(x, chunk_size, axis=0):
    indices = np.arange(chunk_size, x.shape[axis], chunk_size)
    return np.array_split(x, indices, axis)

split_reminder(x, 8)

Checkout the below link for reference: Similar answer

edited Nov 24, 2022 at 13:22

chrslg

15.2k11 gold badges26 silver badges42 bronze badges

answered Nov 24, 2022 at 12:46

harry_09

262 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Tat · Accepted Answer · 2022-11-24 12:43:33Z

0

"""for the first you can use range"""
x = np.array([1 ,1, 2 ,1 ,1 ,2 ,1, 0 ,3, 1, 2 ,2, 1, 2, 1, 1,50,1 ,1, 1, 1, 4, 1, 11, 15, 0, 0, 1, 1,0])
res = [x[i:i+8] for i in range(0, len(x), 8)]
"""for the second you could just pop the first item"""
res.pop(0)
print(res)

edited Nov 24, 2022 at 12:43

answered Nov 24, 2022 at 12:36

Tat

514 bronze badges

1 Comment

Jack Over a year ago

hi @Tat thank you, I will try that. It does use a for loop so I would prefer a solution without them if you know of one

user19077881 · Accepted Answer · 2022-11-24 12:53:42Z

0

You could split just the part of the array will produces your chunk size then add back on an array of the final 8 values

num = int(len(x)/8)
y = np.array_split(x[:num*8], num)
y += [x[-9:-1]]

answered Nov 24, 2022 at 12:53

user19077881

5,5792 gold badges8 silver badges22 bronze badges

Comments

Bhargav · Accepted Answer · 2022-11-24 14:05:19Z

Use utilpsace

from utilspie import iterutils

x = np.array([1 ,1, 2 ,1 ,1 ,2 ,1, 0 ,3, 1, 2 ,2, 1, 2, 1, 1,50,1 ,1, 1, 1, 4, 1, 11, 15, 0, 0, 1, 1,0])

print(list(iterutils.get_chunks(x, 8)))

Gives

[array([1, 1, 2, 1, 1, 2, 1, 0]),  #Length 8
 array([3, 1, 2, 2, 1, 2, 1, 1]),  #Length 8
 array([50,  1,  1,  1,  1,  4,  1, 11]), #Length 8
 array([15,  0,  0,  1,  1,  0])]  #Length 6

Solution 2

Fill uneven array lengths with above array elements using bottleNeck

Complete code. ##

import numpy as np
from utilspie import iterutils
import itertools
from bottleneck import push

x = np.array([1 ,1, 2 ,1 ,1 ,2 ,1, 0 ,3, 1, 2 ,2, 1, 2, 1, 1,50,1 ,1, 1, 1, 4, 1, 11, 15, 0, 0, 1, 1,0])

x =(list(iterutils.get_chunks(x, 8)))

x_new=np.array(list(itertools.zip_longest(*x, fillvalue=np.nan))).T

x_new=push(x_new, axis=0)
print(x_new)

Gives #

[[ 1.  1.  2.  1.  1.  2.  1.  0.]  #Length 8
 [ 3.  1.  2.  2.  1.  2.  1.  1.]  #Length 8
 [50.  1.  1.  1.  1.  4.  1. 11.]  #Length 8
 [15.  0.  0.  1.  1.  0.  1. 11.]] #Length 8

Collectives™ on Stack Overflow

Split numpy array into chunks

4 Answers 4

Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related