How to properly redirect Python script output to file?

Question

I am trying to run a script remotely on a server and I intend to use something along the following lines: nohup ./script.py > runtime.out 2> runtime.err & and monitor the script's progress with tail -f runtiime.out. The problem I am having is that the redirect doesn't seem to work as expected. For the purposes of my problem my problem can be reproduced as described below:

script.py:

#!/usr/bin/env python3

import time
if __name__=='__main__':
    for i in range(1000):
        print("hi")
        time.sleep(1)

Then in shell run ./print.py > a.out &. This will give the PID of the proccess and will exit as expected. However a.out is empty despite the program running. Also if i do ./print.py > a.out without the '&' the a.out file remains empty until I Ctrl-C the command. Then it displays all expected output until the termination of the script.

I thought the ">" redirected continuously the stdout and stderr and not only at command completion.

@JervenClark Why is that? There's no need to open the file inside of Python. stdout can be flushed or be made unbuffered as two of the answers show. — John Kugelman
– John Kugelman, Commented Dec 27, 2021 at 2:24
@JohnKugelman Thank you for this. I didn't know earlier you could flush using print. I only know you could do in a file. — Jerven Clark
– Jerven Clark, Commented Dec 27, 2021 at 2:31

John Kugelman · Accepted Answer · 2021-12-27 02:23:04Z

9

The simplest way to do that is just by using -u flag of the python command. It should look like that:

nohup python3 -u script.py > runtime.out 2> runtime.err &

According to the python3 --help:

-u : force the stdout and stderr streams to be unbuffered; this option has no effect on stdin; also PYTHONUNBUFFERED=x

edited Dec 27, 2021 at 2:23

John Kugelman

364k70 gold badges555 silver badges600 bronze badges

answered Dec 27, 2021 at 2:18

Sylogista

5853 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

shriakhilc · Accepted Answer · 2021-12-27 02:08:45Z

3

Using print("hi", flush=True) will keep forcing the stream to flush contents, so it will continuously update the output file. I don't have enough information about your program to suggest alternatives, but I would look for a better method if possible.

answered Dec 27, 2021 at 2:08

shriakhilc

3,0102 gold badges14 silver badges17 bronze badges

Collectives™ on Stack Overflow

How to properly redirect Python script output to file?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related