Object Persistence in Python with many files open

Question

I'm trying to save the state of my program using pickle, so that I can jump to different states with objects created on a different run.

The issue is that virtually all of these objects (there are quite a few) all have logger objects, so they are all modifying files, and this screws pickling up.

Is there any way to just close all currently open file, so that I can just pickle them?

When you unpickle to "jump" to a state, what state do you expect or want the loggers to be in? — martineau
– martineau, Commented Aug 22, 2014 at 19:55
If your issue is that the logger instances don't serialize with pickle... maybe you could use a better pickler? I know that the dill pickler can serialize logger instances. — Mike McKerns
– Mike McKerns, Commented Aug 22, 2014 at 20:45

Martijn Pieters · Accepted Answer · 2014-08-22 18:45:14Z

2

You can limit what is pickled by including a __getstate__ method:

def __getstate__(self):
    state = vars(self).copy()
    del state['logger']  # remove logger object
    return state

You may need to include a __setstate__ as well in that case, to recreate the logger object.

answered Aug 22, 2014 at 18:45

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

H X Over a year ago

All ready tried this. Issue is that logger is also present implicitly in many objects.

Martijn Pieters Over a year ago

@HX: it cannot be present implicitly; it may be present inside other attributes though, like in a list or dictionary. It is still up to your application code to either not reference these on instances or to produce a state that doesn't include them.

Collectives™ on Stack Overflow

Object Persistence in Python with many files open

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related