What is the Python "with" statement designed for?

Question

I came across the Python with statement for the first time today. I've been using Python lightly for several months and didn't even know of its existence! Given its somewhat obscure status, I thought it would be worth asking:

What is the Python with statement designed to be used for?
What do you use it for?
Are there any gotchas I need to be aware of, or common anti-patterns associated with its use? Any cases where it is better use try..finally than with?
Why isn't it used more widely?
Which standard library classes are compatible with it?

Just for the record, here is with in Python 3 documentation. — Alexey
– Alexey, Commented Mar 18, 2017 at 17:50
coming from a Java background, it helps me to remember it as the corresponding "try with resources" in Java, even if that may not be entirely correct. — vefthym
– vefthym, Commented May 13, 2019 at 18:00
Just for the record, here is PEP-0343: python.org/dev/peps/pep-0343 — Priyanka
– Priyanka, Commented Sep 9, 2020 at 10:37

Tamás · Accepted Answer · 2010-12-29 13:58:22Z

464

I believe this has already been answered by other users before me, so I only add it for the sake of completeness: the with statement simplifies exception handling by encapsulating common preparation and cleanup tasks in so-called context managers. More details can be found in PEP 343. For instance, the open statement is a context manager in itself, which lets you open a file, keep it open as long as the execution is in the context of the with statement where you used it, and close it as soon as you leave the context, no matter whether you have left it because of an exception or during regular control flow. The with statement can thus be used in ways similar to the RAII pattern in C++: some resource is acquired by the with statement and released when you leave the with context.

Some examples are: opening files using with open(filename) as fp:, acquiring locks using with lock: (where lock is an instance of threading.Lock). You can also construct your own context managers using the contextmanager decorator from contextlib. For instance, I often use this when I have to change the current directory temporarily and then return to where I was:

from contextlib import contextmanager
import os

@contextmanager
def working_directory(path):
    current_dir = os.getcwd()
    os.chdir(path)
    try:
        yield
    finally:
        os.chdir(current_dir)

with working_directory("data/stuff"):
    # do something within data/stuff
# here I am back again in the original working directory

Here's another example that temporarily redirects sys.stdin, sys.stdout and sys.stderr to some other file handle and restores them later:

from contextlib import contextmanager
import sys

@contextmanager
def redirected(**kwds):
    stream_names = ["stdin", "stdout", "stderr"]
    old_streams = {}
    try:
        for sname in stream_names:
            stream = kwds.get(sname, None)
            if stream is not None and stream != getattr(sys, sname):
                old_streams[sname] = getattr(sys, sname)
                setattr(sys, sname, stream)
        yield
    finally:
        for sname, stream in old_streams.iteritems():
            setattr(sys, sname, stream)

with redirected(stdout=open("/tmp/log.txt", "w")):
     # these print statements will go to /tmp/log.txt
     print "Test entry 1"
     print "Test entry 2"
# back to the normal stdout
print "Back to normal stdout again"

And finally, another example that creates a temporary folder and cleans it up when leaving the context:

from tempfile import mkdtemp
from shutil import rmtree

@contextmanager
def temporary_dir(*args, **kwds):
    name = mkdtemp(*args, **kwds)
    try:
        yield name
    finally:
        shutil.rmtree(name)

with temporary_dir() as dirname:
    # do whatever you want

edited Dec 29, 2010 at 13:58

answered Jun 10, 2010 at 8:51

Tamás

48.2k12 gold badges107 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user1526546 Over a year ago

Thanks for adding the comparison to RAII. As a C++ programmer that told me everything I needed to know.

Musixauce3000 Over a year ago

Okay so let me get this clear. You're saying that the with statement is designed to fill a variable with data until the instructions under it are complete, and then free the variable?

Musixauce3000 Over a year ago

Because I used it to open a py script. with open('myScript.py', 'r') as f: pass. I expected to be able to call the variable f to see the text content of the document, as this is what would appear if the document were assigned to f via a regular open statement: f = open('myScript.py').read(). But instead I got the following: <_io.TextIOWrapper name='myScript.py' mode='r' encoding='cp1252'>. What does it mean?

Tony Suffolk 66 Over a year ago

@Musixauce3000 - using with doesn't remove the need to read the actual file. The with calls open - it doesn't know what you need to do with it - you might want to do a seek for instance.

Bob Steinke Over a year ago

@Musixauce3000 The with statement can fill a variable with data or make some other change to the environment until the instructions under it are complete, and then does any kind of cleanup that is needed. The kinds of cleanup that can be done are things like closing an open file, or as @Tamas has in this example, changing directories back to where you were before, etc. Since Python has garbage collection, freeing a variable isn't an important use case. with is generally used for other kinds of cleanup.

legoscia · Accepted Answer · 2021-06-21 16:56:58Z

104

I would suggest two interesting lectures:

PEP 343 The "with" Statement
Effbot Understanding Python's "with" statement

1. The with statement is used to wrap the execution of a block with methods defined by a context manager. This allows common try...except...finally usage patterns to be encapsulated for convenient reuse.

2. You could do something like:

with open("foo.txt") as foo_file:
    data = foo_file.read()

OR

from contextlib import nested
with nested(A(), B(), C()) as (X, Y, Z):
   do_something()

OR (Python 3.1)

with open('data') as input_file, open('result', 'w') as output_file:
   for line in input_file:
     output_file.write(parse(line))

OR

lock = threading.Lock()
with lock:
    # Critical section of code

3. I don't see any Antipattern here.
Quoting Dive into Python:

try..finally is good. with is better.

4. I guess it's related to programmers's habit to use try..catch..finally statement from other languages.

edited Jun 21, 2021 at 16:56

legoscia

42.1k23 gold badges123 silver badges179 bronze badges

answered Jun 10, 2010 at 7:49

systempuntoout

74.5k47 gold badges176 silver badges247 bronze badges

4 Comments

detly Over a year ago

It really comes into its own when you're dealing with threading synchronisation objects. Relatively rare in Python, but when you need them, you really need with.

snuggles Over a year ago

diveintopython.org is down (permanently?). Mirrored at diveintopython.net

Angry 84 Over a year ago

Example of a good answer, open file is a prime example that shows behind the scenes of opening,io,closing the file operations are hidden cleanly with a custom reference name

Priyanka Over a year ago

Another small example for using with keyword when dealing with file objects is given in the docs: docs.python.org/3/tutorial/…

Peter Mortensen · Accepted Answer · 2025-07-28 22:57:48Z

The Python with statement is built-in language support of the resource acquisition is initialization (RAII) idiom commonly used in C++. It is intended to allow safe acquisition and release of operating system resources.

The with statement creates resources within a scope/block. You write your code using the resources within the block. When the block exits, the resources are cleanly released, regardless of the outcome of the code in the block (that is whether the block exits normally or because of an exception).

Many resources in the Python library that obey the protocol required by the with statement and so can used with it out-of-the-box. However, anyone can make resources that can be used in a with statement by implementing the well-documented protocol: PEP 0343

Use it whenever you acquire resources in your application that must be explicitly relinquished, such as files, network connections, locks and the like.

John La Rooy · Accepted Answer · 2010-06-10 10:25:19Z

29

An example of an antipattern might be to use the with inside a loop when it would be more efficient to have the with outside the loop

for example

for row in lines:
    with open("outfile","a") as f:
        f.write(row)

vs

with open("outfile","a") as f:
    for row in lines:
        f.write(row)

The first way is opening and closing the file for each row which may cause performance problems compared to the second way with opens and closes the file just once.

answered Jun 10, 2010 at 10:25

John La Rooy

306k54 gold badges378 silver badges513 bronze badges

Comments

Peter Mortensen · Accepted Answer · 2025-07-28 22:59:06Z

Again for completeness, I'll add my most useful use case for with statements.

I do a lot of scientific computing and for some activities I need the Decimal library for arbitrary precision calculations. Some part of my code I need high precision and for most other parts I need less precision.

I set my default precision to a low number and then use with to get a more precise answer for some sections:

from decimal import localcontext

with localcontext() as ctx:
    ctx.prec = 42   # Perform a high precision calculation
    s = calculate_something()
s = +s  # Round the final result back to the default precision

I use this a lot with the Hypergeometric Test which requires the division of large numbers resulting form factorials. When you do genomic scale calculations you have to be careful of round-off and overflow errors.

stefanB · Accepted Answer · 2010-06-10 07:45:21Z

11

See PEP 343 - The 'with' statement, there is an example section at the end.

... new statement "with" to the Python language to make it possible to factor out standard uses of try/finally statements.

answered Jun 10, 2010 at 7:45

stefanB

80.4k28 gold badges119 silver badges143 bronze badges

1 Comment

uuu777 Over a year ago

If I just have one TextIO handle would it close automatically when going out of scope?

Peter Mortensen · Accepted Answer · 2025-07-28 22:51:40Z

5

Points 1, 2, and 3 being reasonably well covered:

4: it is relatively new, only available in Python 2.6+ (or Python 2.5 using from __future__ import with_statement)

edited Jul 28 at 22:51

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Jun 10, 2010 at 8:34

cobbal

70.9k20 gold badges146 silver badges159 bronze badges

Comments

bgse · Accepted Answer · 2024-04-24 14:19:28Z

4

Another example for out-of-the-box support, and one that might be a bit baffling at first when you are used to the way built-in open() behaves, are connection objects of popular database modules such as:

The connection objects are context managers and as such can be used out-of-the-box in a with-statement, however when using the above note that:

When the with-block is finished, either with an exception or without, the connection is not closed. In case the with-block finishes with an exception, the transaction is rolled back, otherwise the transaction is commited.

This means that the programmer has to take care to close the connection themselves, but allows to acquire a connection, and use it in multiple with-statements, as shown in the psycopg2 docs:

conn = psycopg2.connect(DSN)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL1)

with conn:
    with conn.cursor() as curs:
        curs.execute(SQL2)

conn.close()

In the example above, you'll note that the cursor objects of psycopg2 also are context managers. From the relevant documentation on the behavior:

When a cursor exits the with-block it is closed, releasing any resource eventually associated with it. The state of the transaction is not affected.

edited Apr 24, 2024 at 14:19

answered Aug 28, 2017 at 21:28

bgse

8,6572 gold badges43 silver badges43 bronze badges

2 Comments

Wolf Over a year ago

I don't understand the balancing of conn: why do we need the explicit close at the end? I tried to find an explanation in the initd.org link you gave, but it seems now dead.

bgse Over a year ago

@Wolf The reason is that leaving the with-block does not close the connection, only any pending transaction from within the block is commited in case of normal termination, or rolled back in case of an expection. Sorry about the link rot, I've fixed it.

Peter Mortensen · Accepted Answer · 2025-07-28 22:50:24Z

4

The with statement works with so-called context managers.

The idea is to simplify exception handling by doing the necessary cleanup after leaving the 'with' block. Some of the Python built-ins already work as context managers.

edited Jul 28 at 22:50

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Jun 10, 2010 at 7:45

zefciu

2,0762 gold badges17 silver badges40 bronze badges

Comments

Peter Mortensen · Accepted Answer · 2025-07-28 23:01:11Z

3

In Python, generally a with statement is used to open a file, process the data present in the file, and also to close the file without calling a close() method. A with statement makes the exception handling simpler by providing cleanup activities.

General form of with:

with open(“file name”, “mode”) as file_var:
    processing statements

Note: There isn't any need to close the file by calling close() upon file_var.close()

edited Jul 28 at 23:01

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Feb 13, 2018 at 16:15

Tushar Patil

1,4991 gold badge18 silver badges21 bronze badges

1 Comment

Peter Mortensen Jul 28 at 23:02

This will not work: "SyntaxError: invalid character in identifier". What web page was it plagiarised from?

Peter Mortensen · Accepted Answer · 2025-07-28 23:05:15Z

2

The answers here are great, but just to add a simple one that helped me:

with open("foo.txt") as file:
    data = file.read()

open returns a file
Since 2.6 Python added the methods __enter__ and __exit__ to file.
with is like a for loop that calls __enter__, runs the loop once, and then calls __exit__
with works with any instance that has __enter__ and __exit__

A file is locked and not reusable by other processes until it's closed, and __exit__ closes it.

Source: Understanding Python's "with" statement

edited Jul 28 at 23:05

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Jan 3, 2022 at 11:32

Timo Huovinen

56k34 gold badges163 silver badges150 bronze badges

Collectives™ on Stack Overflow

What is the Python "with" statement designed for?

11 Answers 11

5 Comments

4 Comments

Comments

Comments

Comments

1 Comment

Comments

2 Comments

Comments

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

11 Answers 11

5 Comments

4 Comments

Comments

Comments

Comments

1 Comment

Comments

2 Comments

Comments

1 Comment

Comments

Linked

Related