Why is numpy shuffle not working in a loop?

Question

Thanks very much in advance. I have a numpy array called X_test X_test.shape
(11829, 16, 9) I am trying to shuffle the 2nd column of each element of the array with the code below. However, X_test_s does not appear to be shuffled and I can't work out why.

X_test_s=X_test.copy()
for i in range(len(X_test)):
    tempmat=X_test[i]
    temp2=tempmat[:,1]
    np.random.shuffle(temp2)
    tempmat[:,1]=temp2
    X_test_s[i]=tempmat

X_test[3][:,1] array([0.42449842, 0.15417107, 0.42766631, 0.48785639, 0.67159451, 0.56494192, 0.44772967, 0.64836325, 0.2312566 , 0.13093981, 0.46462513, 0.7001056 , 0.51953537, 0.68637804, 0.61034847, 0.57338965])

X_test_s[3][:,1] array([0.42449842, 0.15417107, 0.42766631, 0.48785639, 0.67159451, 0.56494192, 0.44772967, 0.64836325, 0.2312566 , 0.13093981, 0.46462513, 0.7001056 , 0.51953537, 0.68637804, 0.61034847, 0.57338965])

Hamid Rasti · Accepted Answer · 2024-06-19 10:00:57Z

0

You can

X_test_s = X_test.copy()
for i in range(len(X_test)):
    tempmat = X_test_s[i]
    temp2 = tempmat[:, 1].copy()  # Explicitly copy to avoid any reference issues
    np.random.shuffle(temp2)
    tempmat[:, 1] = temp2
    # X_test_s[i] = tempmat  # This is actually redundant

answered Jun 19, 2024 at 10:00

Hamid Rasti

9951 gold badge8 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

ken · Accepted Answer · 2024-06-19 11:45:37Z

They are not unshuffled, but both have been shuffled.

import numpy as np

np.random.seed(1)
X_test = np.arange(64).reshape(4, 4, 4)

print(f"Before shuffle: {X_test[3][:, 1]=}")

X_test_s = X_test.copy()
for i in range(len(X_test)):
    tempmat = X_test[i]       # <-- tempmat is a view of X_test.
    temp2 = tempmat[:, 1]     # <-- temp2 is also a view of X_test.
    np.random.shuffle(temp2)  # <-- This updates X_test through the view.
    tempmat[:, 1] = temp2
    X_test_s[i] = tempmat

print(f"After shuffle: {X_test[3][:, 1]=}")
print(f"After shuffle: {X_test_s[3][:, 1]=}")

Before shuffle: X_test[3][:, 1]=array([49, 53, 57, 61])
After shuffle: X_test[3][:, 1]=array([57, 61, 49, 53])   # Shuffled.
After shuffle: X_test_s[3][:, 1]=array([57, 61, 49, 53]) # Also shuffled.

This is because you are creating a view of X_test (as commented in the code above).

Here is how to fix:

import numpy as np

np.random.seed(1)
X_test = np.arange(64).reshape(4, 4, 4)

print(f"Before shuffle: {X_test[3][:, 1]=}")

X_test_s = X_test.copy()
for i in range(len(X_test_s)):
    tempmat = X_test_s[i]     # <-- tempmat is now a view of X_test_s.
    temp2 = tempmat[:, 1]
    np.random.shuffle(temp2)  # <-- This updates X_test_s through the view.
    # tempmat[:, 1] = temp2   # <-- This line is redundant.
    # X_test_s[i] = tempmat   # <-- This line is also redundant.

print(f"After shuffle: {X_test[3][:, 1]=}")
print(f"After shuffle: {X_test_s[3][:, 1]=}")

Before shuffle: X_test[3][:, 1]=array([49, 53, 57, 61])
After shuffle: X_test[3][:, 1]=array([49, 53, 57, 61])   # Not shuffled.
After shuffle: X_test_s[3][:, 1]=array([57, 61, 49, 53]) # Shuffled.

Collectives™ on Stack Overflow

Why is numpy shuffle not working in a loop?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related