Using mongo with FLASK and python

Question

I am trying to learn python, mongodb and flask and am using the VERY EXCELLENT blog from Miguel Grinberg who provides a great set of tutorials at blog.miguelgrinberg.com

I got a small RESTful server working fine but now want to pull stuff from mongo not mysql

I can pull a mongo record out using the code below but am struggling to get it to render.

I have used arrows in the code below to show where I am struggling, lack of experience I think. Any thoughts would be appreciated.

#!flask/bin/python
from flask import Flask, jsonify, abort, make_response, url_for
from pymongo import MongoClient

# connect to mongo database hosted on AWS
# the script expects the host name to be in  /etc/hosts file

'''
Set up global variables here
'''
mongo_server = "mongo_api"
mongo_port = "27017"
mongo_user = "admin"
mongo_passwd = ":mysecretpassword@"
connect_string = "mongodb://"+ mongo_user 
                             + mongo_passwd 
                             + mongo_server 
                             + ":" 
                             + mongo_port

app = Flask(__name__)

@app.errorhandler(404)
def not_found(error):
    return make_response(jsonify( { 'error': 'Notfound' } ), 404)


def make_public_page(page):
    new_page = {}
    for field in page:
        if field == 'id':
            new_page['uri'] = url_for('get_page', page_id = page['id'], _external = True)
        else:
            new_page[field] = page[field]
    return new_page



@app.route('/api/v1.0/pages/<int:page_id>',methods = ['GET'])
def get_page(page_id):
    '''
    Can connect otherwise exit with message
    '''
    try:
        connection = MongoClient(connect_string)    # equal to > show dbs
    except:
        exit("Error: Unable to connect to the database") # exit with an error
    '''
    connect to database and pull back collections
    '''
    db = connection.test_database # equal to > use test_database                
    pages = db.pages
    page = pages.find_one({"id": int(page_id)})   <------ this pulls back a document
    if page == None:  <---- if a null set comes back then this works great
        abort(404)
    return jsonify( { 'page' : make_public_page(page[0])} ) <- error says its not json

if __name__ == '__main__':
    app.run(debug = True)

Any help appreciated, page[0] is the code that's just not working I get a

TypeError: ObjectId('527e17c538320915e9893f17') is not JSON serializable

Thanks in advance

BTW Can't recommend Miguel's mega tutorial enough as a place to start to build stuff

This is late but you can have a look at github.com/Bleezworld/flask_skeleton, it is a mongodb/Flask website skeleton with some examples of use (user login etc) — Thomas
– Thomas, Commented Jan 26, 2016 at 4:37

zero323 · Accepted Answer · 2013-11-09 19:49:26Z

12

First of all find_one will return single dictionary or None if there is no matching element in collection. So I think that page[0] is equivalent to getting value of page dictionary for key 0

If returned documents contains ObjectId as _id you cannot simply use jsonify because, like ObjectId is not JSON serializable. You can use something like this:

jsonify({ 'page': make_public_page({k:v for k, v in page.items() if k != '_id'}))

or you can simply remove _id by calling page.pop('_id')

You can also use bson.json_util. It contains tools for conversion between BSON and JSON.

from flask import Response 
from bson import json_util

And then replace jsonify with something similar to this:

return Response(
    json_util.dumps({'page' : make_public_page(page)}),
    mimetype='application/json'
)

Edit

If you want short and dirty way of dealing with the problem you can do it like this:

from bson import json_util, ObjectId
import json

#Lets create some dummy document to prove it will work
page = {'foo': ObjectId(), 'bar': [ObjectId(), ObjectId()]}

#Dump loaded BSON to valid JSON string and reload it as dict
page_sanitized = json.loads(json_util.dumps(page))

edited Nov 9, 2013 at 19:49

answered Nov 9, 2013 at 16:09

zero323

331k108 gold badges981 silver badges958 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Richard Over a year ago

Thanks sero -using your page.pop('_id') looks like it will help when combined with Miguels explanation. Thanks for taking the time to reply appreciated. I dont have enough rep to promote this answer but will when i do

zero323 Over a year ago

I've posted an edit with alternative way of dealing with the problem. It is not very elegant but effective.

enthus1ast · Accepted Answer · 2014-07-06 15:20:29Z

3

you could set the default encoder:

import json
from bson.objectid import ObjectId

def newEncoder(o):
    if type(o) == ObjectId:
        return str(o)
    return o.__str__

....

return json.dumps(list(somecollection.find(expr)) ,  default=newEncoder )

or you could subclass json.encoder.JSONEncoder

answered Jul 6, 2014 at 15:20

enthus1ast

2,10916 silver badges23 bronze badges

Comments

Miguel Grinberg · Accepted Answer · 2013-11-09 19:03:01Z

1

From what I see in your code it appears you are not using Mongo's own IDs (which are stored in key _id), but instead you are generating your own integer IDs, which you store in key id. Is this correct?

The problem with your code is that the Mongo _id key is in the dict objects that you send to make_public_page(), and those cannot be serialized to JSON.

You can address this problem by skipping key _id:

def make_public_page(page):
    new_page = {}
    for field in page:
        if field == 'id':
            new_page['uri'] = url_for('get_page', page_id = page['id'], _external = True)
        elif field != '_id':
            new_page[field] = page[field]
    return new_page

As a side note, I think it is better to not invent your own IDs and just use the ones from Mongo.

answered Nov 9, 2013 at 19:03

Miguel Grinberg

68.1k14 gold badges143 silver badges160 bronze badges

1 Comment

Richard Over a year ago

Yes you are correct, I am carrying my own ID, very new to mongo and nosql so still on the steep part of the learning curve. good advice on using native _id and would never have got that it was unable to convert the mongo value. Many thanks I will push on tomorrow.

Collectives™ on Stack Overflow

Using mongo with FLASK and python

3 Answers 3

2 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related