writing into .csv-file inside of "while"-loop

Question

By using

result = open("data/"+ name + "_" + timestamp + ".csv", "w")
result.write("time; data1; data2; data3 \n")`

I open a file and fill it with the column identifiers.

Using

while True:
    timestamp = time.strftime("%H:%M:%S", time.localtime())
    data1,data2,data3 = device.fetchData()

    result.write(timestamp +";"+ str(data1) +";"+ str(data1) +";"+ str(data3) +"\n")
    time.sleep(seconds)

the .csv-file should be filled with measuring data. The problem now is, that if I check the file after exiting the script, it's completely empty, not even the column identifiers are present. However, if I use a for-loop, it works like it should.

Very strange for my understanding.

You haven't shown the version with the for loop so it's hard to guess. Also it's unclear what fetchData does. Try to create a minimal example that shows the issue, without dependencies to other parts of your project. — BartoszKP
– BartoszKP, Commented Apr 29, 2015 at 16:58

Nathan Hoover · Accepted Answer · 2015-04-29 18:32:08Z

1

I assume you want to leave this program running indefinitely to collect data from some kind of sensor, and so I suspect the issue is the default buffering from your open() call.

Firstly, you should almost always be using a "with" block like @Spirine suggests, but in your case a slight modification is in order:

with open("data/"+ name + "_" + timestamp + ".csv", "w", 1) as result:

The , 1 at the end indicates line buffering, meaning that Python will write the file to disk at the end of each line. Also, consider using str.format() to make it a bit more polished:

log_line_template = "{ts:%H:%M:%S};{d1};{d2};{d3}\n"
filename = "data/{n}_{ts:%H_%M_%S}.csv".format(n=name, ts=datetime.now())

with open(filename, "w", 1) as result:
    result.write("time; data1; data2; data3 \n")`

    while True:
        data1,data2,data3 = device.fetchData()
        result.write(log_line_template.format(
            ts=datetime.now(), data1, data2, data3
        ))
        time.sleep(seconds)

answered Apr 29, 2015 at 18:32

Nathan Hoover

6363 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Kreatronik Over a year ago

Ok, that sounded good until i tried it out and: Same result as before:( The file is empty. I didnt't use the formatting stuff, as i don't understand it yet, but this shouldn't stop it from working. I run the script inside the eclipse environment and stop it with the stop-button.

Kreatronik Over a year ago

A colleague gave me solution:

Kreatronik Over a year ago

A colleague gave me the solution: Inside the loop there has to be a result.flush() to write the data to the disk. Can you explain, why the ",1" has no effect?

Nathan Hoover Over a year ago

Weird weird weird. I didn't really try this before posting the answer, but I did some experiments after the fact. Here's what I found: a simplified version of the code I showed you does not write to the file, as you saw. This is with Python 2.7.6 on Ubuntu 14.04. Changing the buffering value to a number > 1 does work, no flush() needed. If I separate out the "\n" and do a write("\n") by itself, the 1 value works as expected. I think this is a Python bug.

Spirine · Accepted Answer · 2015-04-29 16:59:20Z

0

If your file isn't correctly written it's because you're program is incorrectly stopped: to escape of the while loop, what do you do ? If you don't want to modify your code too much, you could just use the open context manager:

with open("data/"+ name + "_" + timestamp + ".csv", "w") as result:
    result.write("time; data1; data2; data3 \n")`

    while True:
        timestamp = time.strftime("%H:%M:%S", time.localtime())
        data1,data2,data3 = device.fetchData()

        result.write(timestamp +";"+ str(data1) +";"+ str(data1) +";"+ str(data3) +"\n")
        time.sleep(seconds)

As it, no matter what happens, your file will be correctly closed in the end of your program.

answered Apr 29, 2015 at 16:59

Spirine

1,8772 gold badges16 silver badges28 bronze badges

Collectives™ on Stack Overflow

writing into .csv-file inside of "while"-loop

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related