How to transfer live data from one Python script to another?

Question

Currently I am writing on a program with 2 separate file Main.py and App.py, Main.py takes in readings such as distance and temperature and writes it in a file named data.txt and App.py then reads from the text file.

#main.py

def scan():
  global object_temp, close_enough, real_distance #close enough writes true is someone is near

while True:
  f=open("data.txt","a+")
  a=str(scan())+"\n"
  f.write(a):
  log.log(a)
  f.close()



#data.txt 
#imagine this but with almost 60000 line each time I run it as it's printing every second
[26.03, 30.91, 126.5, False]
[25.97, 30.89, 125.69, False]
[25.97, 30.89, 124.74, False] 
.
.
etc



#app.py

def getValues():
    global prevValues
    f=open("data.txt","r")
    latestValue = str(f.read())
    f.close()
    #log.log(latestValue,"LATEST VALUES")
    latestValue = latestValue.split("\n")[-2].strip('][').split(', ')
    log.log(latestValue,"LATEST VALUES")
    if latestValue=="":
        return(prevValues)
    else:
        return(latestValue)
        prevValues=latestValue

The problem now is that the text file gets flooded with readings and over time will slow down the program and I do know it is not the most efficient way to go about doing this but I am just getting into python, so is there anyway to transfer the data directly from Main.py to App.py or a way to delete the text file reading after reaching a certain number of lines? Example after 50 lines it starts deleting/overwriting the lines?

Does this answer your question? Passing data between separately running Python scripts — metatoaster
– metatoaster, Commented Nov 18, 2020 at 4:27
You may want to look at asyncio (docs.python.org/3/library/asyncio.html#module-asyncio) you will see that this is designed to do just what you are looking for. — Paul Brennan
– Paul Brennan, Commented Nov 18, 2020 at 4:28

Huzefa Sadikot · Accepted Answer · 2020-11-18 05:20:32Z

3

You can use the pythons multiprocessing module and then implement a pipe between two modules. You may also use a queue but pipe improves the performance of your programme as Queue is built on top of Pipe. But pipe has its disadvantages too:

A Pipe() can only have two endpoints.
A Queue() can have multiple producers and consumers.

Refer these links to better understand the topic:

Regarding how to delete a block of code or script automatically please refer this stack overflow question:

How to make scripts auto-delete at the end of execution?

Since you need to update the file without automatically deleting it here are some answers which might help:

Is it possible for a running python program to overwrite itself?

edited Nov 18, 2020 at 5:20

answered Nov 18, 2020 at 4:45

Huzefa Sadikot

5811 gold badge7 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

yong jie Over a year ago

Yes the pipe function looks exactly like what i need, thank you for your time!

yong jie Over a year ago

regarding the deletion of script, I don't want to delete the script entirely but i want to overwrite the lines in the script keeping the number of lines the same, example the text has 10 lines when a new line is added into the script it, the script will delete/overwite line 1 keeping the lines at 10.

Collectives™ on Stack Overflow

How to transfer live data from one Python script to another?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related