3

as an example I have two scripts, say script1.py

f = open("output1.txt", "w")
count = 1
for i in range(100):                  
         f.write(str(count) + "\n")
         print(str(count))
         count +=1
f.close

This script prints numbers from 1 to 100 to a file and to standard output.

Then I have a second script, say script2.py

import sys
import time
stdin = sys.stdin
f1 = open("output2.txt", "w")
for line in stdin:
    if len(line)>0:
        print(line.strip())
        time.sleep(0.05)
        f1.write(line.strip() + "\n")

which reads data from standard input and prints them to a file. I added a time.sleep command to ensure the second script consumes data at a far lower rate than they are produced by the first one.

I run the scripts from the command line as

python3 script1.py | python3 script2.py

so redirecting the standard output of the first (so the print() command) to the standard input of the second one.

It works as somehow expected, two files are generated containing numbers from 1 to 100.

I am nevertheless wondering how the data transfer part works, from first to second script.

  1. the first script generates data at a faster rate. Where are these data stored, waiting for the second script to access them?
  2. Is there some sort of buffer that is put in place between the two process? Or what else?
  3. Is Python responsible for this, or the OS?
  4. Is the buffer limited in size? Can it be programmed (e.g. accessed to direct data to another target as well)?

Thanks a bunch

0

1 Answer 1

1

it is because of the pipe "|", more info here: https://ss64.com/nt/syntax-redirection.html

commandA  |  commandB     Pipe the output from commandA into commandB

so the prints from your script1 are sent to your script2.


My guess on how it works is that every print is saved in memory as a big string and then sent back (as text) to the second that s why sys.stdin works

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.