Check that a key from json output exists

Question

I keep getting the following error when trying to parse some json:

Traceback (most recent call last):
  File "/Users/batch/projects/kl-api/api/helpers.py", line 37, in collect_youtube_data
    keywords = channel_info_response_data['items'][0]['brandingSettings']['channel']['keywords']
KeyError: 'brandingSettings'

How do I make sure that I check my JSON output for a key before assigning it to a variable? If a key isn’t found, then I just want to assign a default value. Code below:

try:
    channel_id = channel_id_response_data['items'][0]['id']
    channel_info_url = YOUTUBE_URL + '/channels/?key=' + YOUTUBE_API_KEY + '&id=' + channel_id + '&part=snippet,contentDetails,statistics,brandingSettings'
    print('Querying:', channel_info_url)
    channel_info_response = requests.get(channel_info_url)
    channel_info_response_data = json.loads(channel_info_response.content)
    no_of_videos = int(channel_info_response_data['items'][0]['statistics']['videoCount'])
    no_of_subscribers = int(channel_info_response_data['items'][0]['statistics']['subscriberCount'])
    no_of_views = int(channel_info_response_data['items'][0]['statistics']['viewCount'])
    avg_views = round(no_of_views / no_of_videos, 0)
    photo = channel_info_response_data['items'][0]['snippet']['thumbnails']['high']['url']
    description = channel_info_response_data['items'][0]['snippet']['description']
    start_date = channel_info_response_data['items'][0]['snippet']['publishedAt']
    title = channel_info_response_data['items'][0]['snippet']['title']
    keywords = channel_info_response_data['items'][0]['brandingSettings']['channel']['keywords']
except Exception as e:
    raise Exception(e)

except Exception as e: raise Exception(e) This does nothing FYI except masking your error — FHTMitchell
– FHTMitchell, Commented Aug 29, 2018 at 13:30
In this scenario I guess it wouldn’t hurt to just ignore the assignment of the variable instead of crashing out — methuselah
– methuselah, Commented Aug 29, 2018 at 13:32
You just want to do if key in Whatever nested object: Do sth else: Create key with default value — E.Serra
– E.Serra, Commented Aug 29, 2018 at 13:40

Vasyl Moskalov · Accepted Answer · 2018-08-29 13:35:17Z

2

You can either wrap all your assignment in something like

try:
    keywords = channel_info_response_data['items'][0]['brandingSettings']['channel']['keywords']
except KeyError as ignore:
    keywords = "default value"

or, let say, use .has_key(...). IMHO In your case first solution is preferable

answered Aug 29, 2018 at 13:35

Vasyl Moskalov

4,6403 gold badges24 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

FHTMitchell Over a year ago

or dict.get(key, default) though in a long chain like this that will be easier when we have None coalescing operators python.org/dev/peps/pep-0505

pfctgeorge · Accepted Answer · 2018-08-29 13:42:53Z

2

suppose you have a dict, you have two options to handle the key-not-exist situation:

1) get the key with default value, like

d = {}
val = d.get('k', 10)

val will be 10 since there is not a key named k

2) try-except

d = {}
try:
  val = d['k']
except KeyError:
  val = 10

This way is far more flexible since you can do anything in the except block, even ignore the error with a pass statement if you really don't care about it.

edited Aug 29, 2018 at 13:42

answered Aug 29, 2018 at 13:37

pfctgeorge

7283 silver badges9 bronze badges

Comments

bruno desthuilliers · Accepted Answer · 2018-08-29 14:07:02Z

How do I make sure that I check my JSON output

At this point your "JSON output" is just a plain native Python dict

for a key before assigning it to a variable? If a key isn’t found, then I just want to assign a default value

Now you know you have a dict, browsing the official documention for dict methods should answer the question:

https://docs.python.org/3/library/stdtypes.html#dict.get

get(key[, default])

Return the value for key if key is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a KeyError.

so the general case is:

var = data.get(key, default)

Now if you have deeply nested dicts/lists where any key or index could be missing, catching KeyErrors and IndexErrors can be simpler:

try:
   var = data[key1][index1][key2][index2][keyN]
except (KeyError, IndexError):
   var = default

As a side note: your code snippet is filled with repeated channel_info_response_data['items'][0]['statistics'] and channel_info_response_data['items'][0]['snippet'] expressions. Using intermediate variables will make your code more readable, easier to maintain, AND a bit faster too:

# always set a timeout if you don't want the program to hang forever
channel_info_response = requests.get(channel_info_url, timeout=30)
# always check the response status - having a response doesn't
# mean you got what you expected. Here we use the `raise_for_status()`
# shortcut which will raise an exception if we have anything else than
# a 200 OK.
channel_info_response.raise_for_status()

# requests knows how to deal with json:
channel_info_response_data = channel_info_response.json()

# we assume that the response MUST have `['items'][0]`,
# and that this item MUST have "statistics" and "snippets"
item = channel_info_response_data['items'][0]
stats = item["statistics"] 
snippet = item["snippet"]

no_of_videos = int(stats.get('videoCount', 0))
no_of_subscribers = int(stats.get('subscriberCount', 0))
no_of_views = int(stats.get('viewCount', 0))
avg_views = round(no_of_views / no_of_videos, 0)

try:
   photo = snippet['thumbnails']['high']['url']
except KeyError:
   photo = None

description = snippet.get('description', "")
start_date = snippet.get('publishedAt', None)
title = snippet.get('title', "")
try:
   keywords = item['brandingSettings']['channel']['keywords']
except KeyError
   keywords = ""

You may also want to learn about string formatting (contatenating strings is quite error prone and barely readable), and how to pass arguments to requests.get()

Collectives™ on Stack Overflow

Check that a key from json output exists

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related