11

I am pickling Python Objects in Django and saving it in MySQL db. So far i have followed these simple rules:

  1. cPickle.dumps(object) #to convert python object to pickled object

  2. cPickle.loads(pickled_object) # to load back the python object from pickled object

  3. My Django Model Field is Text Field

  4. MySQL db field Type is longblob Attributes binary

  5. MySQL db encoding is utf8_unicode_ci

Unfortunately i am getting following error while loading back python object.

Type Error: ('an integer is required', <type 'datetime.date'>, ('x07xb6x0bx06',))

It seems to me by looking on error value x07xb6x0bx06 this is an encoding problem. Did i miss some important step?? Can any one help me to solve this problem??

Improve this question
3
  • 1
    Do you have a specific reason to use pickle? It's a binaryish format and only usable by python. If you have a choice, would JSON not be a better option? Commented Nov 16, 2011 at 10:28
  • JSON does not server my purpose, I try to use json.dumps but got this error some_object is not JSON serializable. And the object is pure Pythonic. Commented Nov 16, 2011 at 11:55
  • 2
    You should try to make that object serialisable too. It's a big help to have human readable stuff in your db when you're trying to find a bug. Commented Nov 16, 2011 at 12:40

4 Answers 4

Reset to default
7

If you are trying to store the output of cPickle.dumps in a VARCHAR column, then your issue is that you are trying to store a byte-string in a character column. The fix in that case is to encode your object as unicode(base64.encode(cPickle.dumps(myobject))) and then store it.

Alternatively: 

object2varchar = lambda obj: unicode(base64.encode(cPickle.dumps(obj)))
store(object2varchar([1, 'foo']))
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1

one more rule: connect to mysql with option charset=utf8?

UPD1: Sometimes it is a good idea to look at the complete SQL query, I usually do it that way:

>>> conn = MySQLdb.connect(**db_params)
>>> "INSERT INTO tbl VALUES (%s)" % conn.literal((your_pickled_item, ))
Improve this answer

5 Comments

@aamir-adnan, have you tried to save and read a new item instead of reading already saved?
Good question, yes i checked it, it works fine when i convert and restore python object within code, it is problem with db while reading or not storing properly. What should i do? please help.
@aamir-adhan, I meant that the previous items might have been saved in a wrong encoding and noew can now be properly read.
Instead of using Django Model (confused with behavior of Django Models Text field with MySQL blob field), i directly makes a connection to db and store the pickled object directly in db. But again same error :-(
yes i always update the fields and then read the new value. I am not reading old saved values.
0

Newtover's answer is probably the correct one, but have a look at

https://github.com/bradjasper/django-pickledfield

It really saved me some time, and can store pretty much anything you want.

Improve this answer

Comments

0

You can try this, now! django-picklefield https://pypi.org/project/django-picklefield/

To use, just define a field in your model:

>>> from picklefield.fields import PickledObjectField
... class SomeObject(models.Model):
...     args = PickledObjectField()

and assign whatever you like (as long as it's picklable) to the field:

>>> obj = SomeObject()
>>> obj.args = ['fancy', {'objects': 'inside'}]
>>> obj.save()
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.