6

I have some piece of python code which generates a MemoryError after a while. I know that it consumes a lot of memory. So, I decided to put the code within a try/except block so that the skeleton looks like the following:

while True:

      while True:

            try:
            #---- do some stuff

            except MemoryError as err:
                   print(err)
                   break

So, my idea is to break out of the first while-loop if a MemoryError occurs and since I have an outer while-loop, it will start the program again.

It seems that it works for the moment but I am not sure. After a while, it stops again and I need to restart the program again. Does somebody know a better solution so that the program can run after the MemoryError again?

Improve this question
6
  • Just to be clear, you ask what to do when MemoryError occurs? Commented Feb 5, 2017 at 21:36
  • yes. Or better: How to continue the try-block after MemoryError occurs? Commented Feb 5, 2017 at 21:37
  • Possibly related: How can I explicitly free memory in Python? Commented Feb 5, 2017 at 21:45
  • What kinds of data structures are you using? Just standard lists and dictionaries, or are you using some third party package like numpy or pandas. Memory error when trying to create and use very large numpy arrays comes up periodically on SO. Commented Feb 5, 2017 at 21:53
  • In another post you say you are learning Python. A beginner shouldn't be getting memory errors unless they are doing something very wrong. Don't try to except out of the error. Figure out what is using memory uncontrollably. Commented Feb 5, 2017 at 21:56

3 Answers 3

Reset to default
6

Note that Python only throws the MemoryError when it realizes it will overuse the memory beforehand. If it happens by accident (or "unnoticed" by Python) then you're out of luck. The documentation already mentions this:

MemoryError

Raised when an operation runs out of memory but the situation may still be rescued (by deleting some objects). The associated value is a string indicating what kind of (internal) operation ran out of memory. Note that because of the underlying memory management architecture (C’s malloc() function), the interpreter may not always be able to completely recover from this situation; it nevertheless raises an exception so that a stack traceback can be printed, in case a run-away program was the cause.

So if there is nothing to rescue or if the interpreter can't recover there is no MemoryError.

A good approach would require knowing what you're doing and how. In a majority of cases generators (see for example PEP 289 on generator expressions) or map-reduce approaches can save you a lot of memory. These might be applicable here as well.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

the code in the try-block, opens a file, writes some stuff into it & delete it. And this happens in a endless while-loop.
and? How does that create a MemoryError? What stuff? Why delete the files? Do you close file handles?
That only answers parts of the questions. Maybe you could edit your question to show a bit more of your actual code?
Thanks for this answer. I had noticed that python is very inconsistent in throwing MemoryError. Sometimes it does throw and error and many other times it does not.
2

It is hard to assess what to do without knowing what do you do inside this try but I will try.

Frstrly, regarding continuing the try-except block. I am afraid you cannot do this.

So short answer is: you cannot go back to try block to place where exception occured, you can go to first line of try

What you can do:

I usually handle my exceptions like the following. Create while True loop as such:

while True:
    try:
        #some code here
    except SomeException:
        continue

Now you can continue to try after exception occured.

Second way (but not reccomended) is to embedd your code using some function and recursively execute it.

def foo():
    try:
        while True:
            foo2()
    except StopIteration:
        #end code

def foo2():
    while True:
        try:
            #some code here
        except MemoryError:
            foo2()
            raise StopIteration()

However this is very DANGEROUS however if your memory is being exceeded not many times (<1000) this can be okay way to go if you need to do something before while True however you need to watch solution closely.

Improve this answer

2 Comments

Thanks, for your answer. In the try-block, the program opens a file, write some data to it and deletes it after a while.
@aminakoy I do not see any other solution to your problem. How about posting your code where you do file operations in another question and maybe we can help you create program so it will not raise MemoryException?
0

Your asking a very ambiguous question, because you're not plotting the scenario that fills the memory.

What do you do when memory fills? You empty it! The good practice is that you have to be aware of how much memory your script consumes.

  • If you can make your program break down to more steps and consume less memory, then do that! If you

  • If you can empty some lists/arrays and continue execution, then do it!

  • If you believe your program will consume so much memory, then if a MemoryError occures, and there's no programmatic way around it, then the system of the user of your program does not meet the requirements of using your program, and crashing with a memory error is OK!

Improve this answer

3 Comments

The scenario is the following: The program opens files, write some data into it, and the deletes it.
@aminakoy Why would you delete a file after opening it? Anyway my answer should still put things in perspective.
it deletes not all files. Only that files which fullfils a certain criteria.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.