Why is redirecting script's output to a file giving an incorrect output?

Question

This is the python script sum_columns.py:

#!/usr/bin/env python2.7

import sys

num_nodes = 0
num_edges = 0

for line in sys.stdin:
   line = line.strip()
   col_1, col_2 = line.split(',', 1)

   col_1 = int(col_1)
   col_2 = int(col_2)

   num_nodes += col_1
   num_edges += col_2

print '%s, %s' % (num_nodes, num_edges)

There's a file called outputs.dat with 1 line:

10, 10

I am trying to redirect the output of the python script into the input file outputs.dat, but when I do this, it writes out 0, 0, when I am expecting 10, 10. When I redirect to another file not named outputs.dat I get 10, 10. Why is this happening?

Here are the commands I am executing.

[local_serv]$ cat outputs.dat
10, 10
[local_serv]$ cat outputs.dat | ./sum_columns.py 
10, 10
[local_serv]$ cat outputs.dat | ./sum_columns.py > test.dat; cat test.dat
10, 10
[local_serv]$ cat outputs.dat | ./sum_columns.py > outputs.dat; cat outputs.dat
0, 0
[local_serv]$

My understanding is that > rewrites outputs.dat (if it already exists), why would it rewrite it as 0, 0 when the python script prints 10, 10?

Jörg W Mittag · Accepted Answer · 2019-01-27 21:06:35Z

1

My understanding is that > rewrites outputs.dat (if it already exists),

Yes, exactly, it truncates the file.

why would it rewrite it as 0, 0 when the python script prints 10, 10?

It doesn't print that. The Python script reads the contents of the file to perform some calculations based on its contents, but > truncates the file, so when Python reads its contents, it is empty.

answered Jan 27, 2019 at 21:06

Jörg W Mittag

371k79 gold badges457 silver badges666 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

24n8 Over a year ago

It seems that I missed the order of operations here. Are you saying the python script is executed after the > is executed?

Jörg W Mittag Over a year ago

Yes. > means that the script is started with its standard output stream connected to outputs.dat. Before that, the file will be truncated.

24n8 Over a year ago

ahh I wasn't aware of that. I thought the order of operations in linux terminal is always from left to right.

Jörg W Mittag Over a year ago

How can the output of the script end up in the file if the file is not connected to the output before the script is started? What if the script prints something as the very first thing it does? What if the script tests whether the standard output is a file or a terminal and then does different things based on that? The only possible way how this can work is if the standard output gets redirected before the command is started.

24n8 Over a year ago

OHH. I totally missed that. In the Python script, I put the print command at the very end of the program, so I was thinking it the shell would open the output file for writing when it sees the first print command.

Collectives™ on Stack Overflow

Why is redirecting script's output to a file giving an incorrect output?

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related