Python: List handling in functions

Question

Sorry I could not describe the question name better - I am really at a loss what the problem could be, hence I cannot be very specific.

In Python, I have the following code:

# input
list_environments = ["tree", "bush"]
list_fruit = ["banana", "kiwi"]
list_properties = [
                   ["_size", [["width"], ["height"], ["depth"]]],
                   ["_colour", [["rgb_r"], ["rgb_g"], ["rgb_b"]]],
                   ["_shape", [["is_round"], ["is_square"], ["is_flat"]]]
                   ]

# create list of all combinations: [environment, fruit, properties set]
list_combinations = []
for env in list_environments:
    for fruit in list_fruit:
        for prop in list_properties:
            list_combinations.append([env, fruit, prop])

# define function
def process_entry(inp_environment, inp_fruit, inp_property):

    # print data at start of function
    print(".")
    print("Dataset before processing:", inp_environment, inp_fruit, inp_property)

    # - create temporary list for property value pairs -
    tmp_list = inp_property[1] # takes only the sublist from property list

    # add info for size
    if inp_property[0] == "_size":
        tmp_list[0].append(1) # width is 1 
        tmp_list[1].append(2) # height is 2
        tmp_list[2].append(3) # depth is 3
    # add info for colour
    elif inp_property[0] == "_colour":
        tmp_list[0].append(100) # r-value is 100
        tmp_list[1].append(101) # g-value is 101
        tmp_list[2].append(102) # b-value is 102
    # add info for shape 
    elif inp_property[0] == "_shape":
        tmp_list[0].append(0) # is not round
        tmp_list[1].append(0) # is not square
        tmp_list[2].append(1) # is flat

    print("Dataset now:", inp_environment, inp_fruit, tmp_list)    

# call processing function for every entry in combination list
for entry in list_combinations:
    process_entry(entry[0], entry[1], entry[2])

What I would expect now is that the function will assign the (static) values to every single combination of the for-loop at the end. Instead, the assigned values are -kept- with the properties list and the output always gets extended by another value set. The output looks like this:

.
Dataset before processing: tree banana ['_size', [['width'], ['height'],['depth']]]
Dataset now: tree banana [['width', 1], ['height', 2], ['depth', 3]]
.
Dataset before processing: tree banana ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]
Dataset now: tree banana [['rgb_r', 100], ['rgb_g', 101], ['rgb_b', 102]]
.
Dataset before processing: tree banana ['_shape', [['is_round'], ['is_square'], ['is_flat']]]
Dataset now: tree banana [['is_round', 0], ['is_square', 0], ['is_flat', 1]]

-> All good after the first loop through the properties so far.

Now...:

Dataset before processing: tree kiwi ['_size', [['width', 1], ['height', 2], ['depth', 3]]]
Dataset now: tree kiwi [['width', 1, 1], ['height', 2, 2], ['depth', 3, 3]]
.
Dataset before processing: tree kiwi ['_colour', [['rgb_r', 100], ['rgb_g', 101], ['rgb_b', 102]]]
Dataset now: tree kiwi [['rgb_r', 100, 100], ['rgb_g', 101, 101], ['rgb_b', 102, 102]]
.
Dataset before processing: tree kiwi ['_shape', [['is_round', 0], ['is_square', 0], ['is_flat', 1]]]
Dataset now: tree kiwi [['is_round', 0, 0], ['is_square', 0, 0], ['is_flat', 1, 1]]

Now things go wrong: The values from the first property-loop are still there and the properties sublist gets extended. This is really strange because I think that due to my combination list that was created earlier, I am giving the function a "fresh" dataset to work with. Does anyone see where my mistake is?

Thanks heaps in advance!

Alan · Accepted Answer · 2015-05-05 11:13:04Z

2

The problem is this line: tmp_list = inp_property[1]. That gives simply gives a new name to an existing list, which you then proceed to modify, thereby modifying the original list. And actually the problem goes a level deeper, because you then append to sublists. To fix this, just import the copy module and change this to tmp_list = copy.deepcopy(inp_property[1]).

answered May 5, 2015 at 11:13

Alan

9,65020 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

fraggykrueger Over a year ago

Thank you - from your and Bruno's answer I was already able to fix it! :-) I applied the copying before the function calls - as mhawke also pointed out and suggested later.

mhawke · Accepted Answer · 2015-05-05 11:58:36Z

Other answers have highlighted the problem that tmp_list is a reference to the same list, not a copy and that you can use tmp_list = copy.deepcopy(inp_property[1]) within function process_entry() to work on a separate copy.

But it's worth mentioning that after the initialisation of list_combinations, all of the properties lists are in fact the same list - for the same reason that a reference to list_properties is being used, not a copy. Consider the value of list_combinations immediately after initialisation:

>>> from pprint import pprint
>>> pprint(list_combinations)
[['tree', 'banana', ['_size', [['width'], ['height'], ['depth']]]],
 ['tree', 'banana', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['tree', 'banana', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]],
 ['tree', 'kiwi', ['_size', [['width'], ['height'], ['depth']]]],
 ['tree', 'kiwi', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['tree', 'kiwi', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]],
 ['bush', 'banana', ['_size', [['width'], ['height'], ['depth']]]],
 ['bush', 'banana', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['bush', 'banana', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]],
 ['bush', 'kiwi', ['_size', [['width'], ['height'], ['depth']]]],
 ['bush', 'kiwi', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['bush', 'kiwi', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]]]

If one of the properties sublists is changed, that change affects all entries in list_combinations:

>>> list_combinations[0][2][1][0].append('hi there')
>>> pprint(list_combinations)
[['tree', 'banana', ['_size', [['width', 'hi there'], ['height'], ['depth']]]],
 ['tree', 'banana', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['tree', 'banana', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]],
 ['tree', 'kiwi', ['_size', [['width', 'hi there'], ['height'], ['depth']]]],
 ['tree', 'kiwi', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['tree', 'kiwi', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]],
 ['bush', 'banana', ['_size', [['width', 'hi there'], ['height'], ['depth']]]],
 ['bush', 'banana', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['bush', 'banana', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]],
 ['bush', 'kiwi', ['_size', [['width', 'hi there'], ['height'], ['depth']]]],
 ['bush', 'kiwi', ['_colour', [['rgb_r'], ['rgb_g'], ['rgb_b']]]],
 ['bush', 'kiwi', ['_shape', [['is_round'], ['is_square'], ['is_flat']]]]]

So another way to fix this is to make copies of the lists contained in list_properties at initialisation time:

from copy import deepcopy

list_combinations = []
for env in list_environments:
    for fruit in list_fruit:
        for prop in list_properties:
            list_combinations.append([env, fruit, deepcopy(prop)])

Thanks, very helpful to see how the change affects all the lists!

bruno desthuilliers · Accepted Answer · 2015-05-05 11:16:02Z

0

This:

tmp_list = inp_property[1]

doesn't copy inp_property[1] into tmp_list, it just makes tmp_list points to the same object as inp_property[1]. So modifying tmp_list is the same as modifying inp_property[1].

Same thing in when you pass entry[2] to process_entry() - process_entry() doesn't get a copy of entry[2], it gets a reference to the same object under the name inp_property, so modifying inp_property[1] is the same as modifying entry[2][1] - which is the same as modifying list_combinations[i][2][1] (for 'i' = the current iteration in the for loop).

answered May 5, 2015 at 11:16

bruno desthuilliers

78.3k6 gold badges102 silver badges129 bronze badges

1 Comment

fraggykrueger Over a year ago

Thank you, Bruno - your explanations with the various modification options (which are all modifying the same thing in the end) are very good :-)

Collectives™ on Stack Overflow

Python: List handling in functions

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related