538

Suppose I have a dictionary of lists:

d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}

Now I want to remove key-value pairs where the values are empty lists. I tried this code:

for i in d:
    if not d[i]:
        d.pop(i)

but this gives an error:

RuntimeError: dictionary changed size during iteration

I understand that entries can't be added or removed from a dictionary while iterating through it. How can I work around this limitation in order to solve the problem?

See Modifying a Python dict while iterating over it for citations that this can cause problems, and why.

Improve this question
3
  • In addition to the answers given below, you may consider checking for the empty list at the places where you remove element(s) from the list. You could write a helper function: def remove_helper(d, k, elem): d[k].remove(elem); if not d[k]: del d[k]. Commented Oct 7, 2022 at 17:43
  • There are, practically speaking, perhaps three meaningfully distinct ways to do this, and in my assessment every meaningful minor variation on those ways can be covered by considering at most 6 of the answers here. It's somewhat worrying that there are 8 more undeleted, and another 5 that were previously deleted. Commented Jul 27, 2023 at 23:38
  • @joseville to be clear: you propose updating the dict when lists become empty, rather than iterating to remove the empty lists? That could be a practical approach to the broader problem in context, depending on the exact original requirements. But I agree it doesn't justify a separate answer. Commented Jul 27, 2023 at 23:40

15 Answers 15

Reset to default
830

In Python 3.x and 2.x you can use use list to force a copy of the keys to be made:

for i in list(d):

In Python 2.x calling .keys made a copy of the keys that you could iterate over while modifying the dict:

for i in d.keys():

but on Python 3.x, .keys returns a view object instead, so it won't fix your error.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

I believe you meant 'calling keys makes a copy of the keys that you can iterate over' aka the plural keys right? Otherwise how can one iterate over a single key? I'm not nit picking by the way, am genuinely interested to know if that is indeed key or keys
Or tuple instead of list as it is faster.
To clarify the behavior of python 3.x, d.keys() returns an iterable (not an iterator) which means that it's a view on the dictionary's keys directly. Using for i in d.keys() does actually work in python 3.x in general, but because it is iterating over an iterable view of the dictionary's keys, calling d.pop() during the loop leads to the same error as you found. for i in list(d) emulates the slightly inefficient python 2 behavior of copying the keys to a list before iterating, for special circumstances like yours.
In python3.x, list(d.keys()) creates the same output as list(d), calling list on a dict returns the keys. The keys call (though not that expensive) is unnecessary.
@DanielChin: That documentation is for the wrong version of Python. See docs.python.org/3/library/stdtypes.html#dict.items.
|
145

You only need to use copy:

This way you iterate over the original dictionary fields and on the fly can change the desired dict d. It works on each Python version, so it's more clear.

In [1]: d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}

In [2]: for i in d.copy():
   ...:     if not d[i]:
   ...:         d.pop(i)
   ...:         

In [3]: d
Out[3]: {'a': [1], 'b': [1, 2]}

(BTW - Generally to iterate over copy of your data structure, instead of using .copy for dictionaries or slicing [:] for lists, you can use import copy -> copy.copy (for shallow copy which is equivalent to copy that is supported by dictionaries or slicing [:] that is supported by lists) or copy.deepcopy on your data structure.)

Improve this answer

2 Comments

for key, value in dic.copy().items():
Use of the .copy() method should be the accepted anwser. Thanks.
74

Just use dictionary comprehension to copy the relevant items into a new dict:

>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.items() if v}
>>> d
{'a': [1], 'b': [1, 2]}

For this in Python 2:

>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.iteritems() if v}
>>> d
{'a': [1], 'b': [1, 2]}
Improve this answer

3 Comments

d.iteritems() gave me an error. I used d.items() instead - using python3
This works for the problem posed in OP's question. However anyone who came here after hitting this RuntimeError in multi-threaded code, be aware that CPython's GIL can get released in the middle of the list comprehension as well and you have to fix it differently.
Is dictionary comprehension the same as list comprehension? Dictionary comprehension does not seem to be very popular (or in the official documentation). There is PEP 274, but was it implemented?
46

This worked for me:

d = {1: 'a', 2: '', 3: 'b', 4: '', 5: '', 6: 'c'}
for key, value in list(d.items()):
    if value == '':
        del d[key]
print(d)
# {1: 'a', 3: 'b', 6: 'c'}

Casting the dictionary items to list creates a list of its items, so you can iterate over it and avoid the RuntimeError.

Improve this answer

Comments

17

To avoid "dictionary changed size during iteration error".

For example: "when you try to delete some key",

Just use 'list' with '.items()'. Here is a simple example:

my_dict = {
    'k1':1,
    'k2':2,
    'k3':3,
    'k4':4
 
    }
    
print(my_dict)

for key, val in list(my_dict.items()):
    if val == 2 or val == 4:
        my_dict.pop(key)

print(my_dict)

Output:

{'k1': 1, 'k2': 2, 'k3': 3, 'k4': 4}

{'k1': 1, 'k3': 3}

This is just an example. Change it based on your case/requirements.

Improve this answer

1 Comment

This approach is already covered by singrium's answer from 2020, and is a trivial variant of other approaches seen in the original answers from 2012.
15

I would try to avoid inserting empty lists in the first place, but, would generally use:

d = {k: v for k,v in d.iteritems() if v} # re-bind to non-empty

If prior to 2.7:

d = dict( (k, v) for k,v in d.iteritems() if v )

or just:

empty_key_vals = list(k for k in k,v in d.iteritems() if v)
for k in empty_key_vals:
    del[k]
Improve this answer

5 Comments

+1: last option is interesting because it only copies the keys of those items that need deleting. This may give better performance if only a small number of items need deleting relative to the size of the dict.
@MarkByers yup - and if a large number do, then re-binding the dict to a new one that's filtered is a better option. It's always the expectation of how the structure should work
One danger with rebinding is if somewhere in the program there was an object that held a reference to the old dict it wouldn't see the changes. If you're certain that's not the case, then sure... that's a reasonable approach, but it's important to understand that it's not quite the same as modifying the original dict.
@MarkByers extremely good point - You and I know that (and countless others), but it's not obvious to all. And I'll put money on the table it hasn't also bitten you in the rear :)
The point of avoiding to insert the empty entries is a very good one.
13

For Python 3:

{k:v for k,v in d.items() if v}
Improve this answer

2 Comments

Nice and concise. Worked for me in Python 2.7 also.
This approach was covered by the original answers.
8

You cannot iterate through a dictionary while it’s changing during a for loop. Make a casting to list and iterate over that list. It works for me.

    for key in list(d):
        if not d[key]:
            d.pop(key)
Improve this answer

1 Comment

This approach was covered by the original answers.
3

Use a list to collect the keys that should be removed; then use the pop dictionary method to remove the identified keys while iterating through the list (a separate object, so the error will not occur).

d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}
pop_list = []

for i in d:
    if not d[i]:
        pop_list.append(i)

for x in pop_list:
    d.pop(x)

print(d)
Improve this answer

1 Comment

This technique is mentioned in Jon Clements' answer from 2012, but using a comprehension for the first loop.
2

Python 3 does not allow deletion while iterating (using the for loop above) a dictionary. There are various alternatives to do it; one simple way is to change the line

for i in x.keys():

with

for i in list(x)
Improve this answer

1 Comment

This approach was covered by the original answers.
1

  • The Python "RuntimeError: dictionary changed size during iteration" occurs when we change the size of a dictionary when iterating over it.

  • To solve the error, use the copy() method to create a shallow copy of the dictionary that you can iterate over, e.g., my_dict.copy().

    my_dict = {'a': 1, 'b': 2, 'c': 3}

for key in my_dict.copy():
    print(key)
    if key == 'b':
        del my_dict[key]

print(my_dict) # 👉️ {'a': 1, 'c': 3}

  • You can also convert the keys of the dictionary to a list and iterate over the list of keys.

    my_dict = {'a': 1, 'b': 2, 'c': 3}

for key in list(my_dict.keys()):
    print(key)
    if key == 'b':
        del my_dict[key]

print(my_dict)  # 👉️ {'a': 1, 'c': 3}
Improve this answer

1 Comment

This is completely redundant with the original answers.
0

For situations like this, I like to make a deep copy and loop through that copy while modifying the original dict.

If the lookup field is within a list, you can enumerate in the for loop of the list and then specify the position as the index to access the field in the original dict.

Improve this answer

Comments

0 
d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}

d = {k:v for k,v in d.items() if type(v) == list and len(v)>0}

print(d)

Output:

{'a': [1], 'b': [1, 2]}
Improve this answer

Comments

-1

Nested null values

Let's say we have a dictionary with nested keys, some of which are null values:

dicti = {
"k0_l0":{
    "k0_l1": {
        "k0_l2": {
                "k0_0":None,
                "k1_1":1,
                "k2_2":2.2
                }
        },
        "k1_l1":None,
        "k2_l1":"not none",
        "k3_l1":[]
    },
    "k1_l0":"l0"
}

Then we can remove the null values using this function:

def pop_nested_nulls(dicti):
    for k in list(dicti):
        if isinstance(dicti[k], dict):
            dicti[k] = pop_nested_nulls(dicti[k])
        elif not dicti[k]:
            dicti.pop(k)
    return dicti

Output for pop_nested_nulls(dicti)

{'k0_l0': {'k0_l1': {'k0_l2': {'k1_1': 1,
                               'k2_2': 2.2}},
           'k2_l1': 'not '
                    'none'},
 'k1_l0': 'l0'}
Improve this answer

1 Comment

This is essentially inventing a new problem to solve. The issue in the OP isn't at all specific to "removing nulls"; that's just one of countless examples of processing that would run into the same category of problem. And of course a nested structure can be processed using recursion; that's completely irrelevant to the topic.
-1

This approach can be used if the values in the dictionary are also unique:

keyToBeDeleted = None
for k, v in mydict.items():
    if(v == match):
        keyToBeDeleted = k
        break
mydict.pop(keyToBeDeleted, None)
Improve this answer

1 Comment

There's a useful idea here, but the implementation is specific to a problem that differs from OP's example. It would be better to give a general technique (e.g., show the code with some placeholder for deciding whether an item should be removed, and show code that can remove multiple items) and explain how it solves the problem. However, Rohit's answer from 2018 already demonstrates exactly that.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.