How to merge one numpy array onto multiple dataframes

Question

I have a bunch of data frames. They all have the same columns but different amounts of rows. They look like this:

I have them all stored in a list.

Then, I have a numpy array where each item maps to a row in each individual df. The numpy array looks like this:

[3 1 1 2 4 0 6 7 2 1 3 2 5 5 5]

If I were to pd.concat my list of dataframes, then I could merge the np array onto the concatenated df. However, I want to preserve the individual df structure, so it should look like this:

Shubham Sharma · Accepted Answer · 2020-05-14 17:44:00Z

1

Considering the given dataframes & array as,

df1 = pd.DataFrame([1,0,0,1,1,0])
df2 = pd.DataFrame([1,0,0,1])
df3 = pd.DataFrame([1,0,0,1,1])

arr = np.array([3, 1, 1, 2, 4, 0, 6, 7, 2, 1, 3, 2, 5, 5, 5])

You can use numpy.split to split an array into multiple sub-arrays according to the given dataframes. Then you can append those arrays as columns to their respective dataframes.

Use:

dfs = [df1, df2, df3]

def get_indices(dfs):
    """
    Returns the split indices inside the array.
    """
    indices = [0]
    for df in dfs:
        indices.append(len(df) + indices[-1])
    return indices[1:-1]

# split the given arr into multiple sections.
sections = np.split(arr, get_indices(dfs))
for df, s in zip(dfs, sections):
    df[1] = s # append the section of array to dataframe
    print(df)

This results:

edited May 14, 2020 at 17:44

answered May 14, 2020 at 17:13

Shubham Sharma

71.8k6 gold badges26 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

V. Ayrat Over a year ago

This also can be rewritten like split_at = [df_1.shape[0], df_1.shape[0] + df_2.shape[0]]; df_1[1], df_2[1], df_3[1] = np.split(a, split_at)

Collectives™ on Stack Overflow

How to merge one numpy array onto multiple dataframes

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related