1

I am having trouble adding several dataframes in a list of dataframes. My goal is to add dataframes from a list of dataframes based on the criteria from another list.

Example: Suppose we have a list of 10 Dataframes, DfList and another list called OrderList.

Suppose OrderList = [3, 2, 1, 4].

Then I would like to obtain a new list of 4 Dataframes in the form [DfList(0) + DfList(1) + DfList(2), DfList(3) + DfList(4), DfList(5), DfList(6) + DfList(7) + DfList(8) + DfList(9)]

I have tried a few ways to do this creating functions using DataFrame.add. Initially, my hope was that I could use the form sum(DfList(0), DfList(1), DfList(2)) to do this but quickly learned that sum() doesn't seem to be supported with DataFrames.

I was hoping to use something like sum(DfList[0:2]) and making OrderList cumulative so I could just use sum(DfList[OrderList[i]:OrderList[i+1]]) but keep getting unsupported operand type errors.

Is there an easy way to do this that I am not considering or is there a different approach entirely that you would suggest?

EDIT: The output I am looking for is another list of DataFrames containing four summed DataFrames based on OrderList (across all columns.) Three DataFrames added together for the first, two for the second, one for the third, and four for the fourth.

Improve this question
3
  • Hello there, and welcome to StackOverflow! I got a little confused with the question. First, you have a list of DataFrames, right? Do you want to create a new list of DataFrames or create a new DataFrame that is the sum of others? Second, what role does OrderList have in this? Commented May 20, 2019 at 23:19
  • I apologize; I should have worded it better. I am looking for new list of DataFrames containing the sums of smaller lists defined by the list, OrderList (I should have used a better name.) So for the first DataFrame, I am looking for a sum of all the columns of the first three, second DataFrame the sum of all the columns of the next two, third DataFrame the sum of all the columns of the next one, and a fourth DataFrame with the sum of all the columns of the last four. Commented May 21, 2019 at 0:40
  • There are two things you are asking here... First, how to use OrderList=[3, 2, ...] to sum the first three elements, then the next two, etc. Next, you are asking how to add different numbers of DataFrames. The second question the guy below already answered :) For the first, you should start with something simpler, like summing [1, 2, 3, 4, 5] with order [2, 3] to give [3, 12]. This way you don't have to worry about data structures whatsoever, just the language. Sorry I don't have the time to come with an answer for you, but it shouldn't be hard, just do your best :) Commented May 22, 2019 at 0:30

1 Answer 1

Reset to default
1

If you have a list of DataFrames as you said, you can use the operation sum(DfList[0:2]), but you need to be careful with the order of the columns in each DataFrame in your list because the order provided is used when adding the DataFrames. The addition does not occur accordingly to the names of the columns. If you need, the order of the columns can be changed as showed in this other question.

This example illustrates the issue:

import pandas as pd

df1 = pd.DataFrame({1:[1,23,4], 2:['x','y','z']})
df2 = pd.DataFrame({2:['x','y','z'], 1:[1,23,4]})

try:
    df1 + df2
except TypeError:
    print("Error")

df1 = pd.DataFrame({1:[1,23,4], 2:['x','y','z']})
df2 = pd.DataFrame({1:[1,23,4], 2:['x','y','z']})

#works fine
df1 + df2

Also, the logic that you used for the cumulative sum in sum(DfList[OrderList[i]:OrderList[i+1]])is not correct. For this to be the case, the OrderList would also need to be cumulative and have one extra element to start from zero, so instead of OrderList = [3, 2, 1, 4], you would have OrderList = [0, 3, 5, 6, 10].

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

I apologize, I should have explained the issue better. I had added the 0 to the list in my attempt as well as used np.cumsum but the issue I was having was more due to the addition of DataFrames themselves. Spelling out df1+df2+df3+df4 is the problem since the addition itself is to be done on different numbers of DataFrames. df1 + df2 + df3 is working for me but the number of dfs being added need to be changed based on OrderList. That was the reason I was erring towards the side of sum(DfList[OrderList[i]:OrderList[i+1]]). While df1 + df2 +df3 is working for me, sum(df1, df2, df3) isn't.
In order for it to work with the sum function, you just need to wrap the DataFrames in a list, so instead of sum(df1, df2, df3), you should use sum([df1, df2, df3]).
I still seem to be getting the unsupported operand type(s) for +: 'int' and 'list' error. The snippet I am using thats giving me this is for i in range(len(OrderList)): sum([DfList[OrderList[i]:OrderList[i+1]]]). I also tried for i in OrderList: sum([DfList[i:i+1]]). Note that I have the added 0 in the front of np.cumsum(OrderList).
Sorry for the late follow up but to provide some closure, I had ended up rewriting a large part of what I was working on and then Pedro's suggestion to wrap it in a list before using the sum ended up working smoothly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.