Adding header to new CSV file

Question

I'm creating a temperature logger program that records date, time, and temperatures. The program should create a new .csv file that includes data and column headers. However I'm finding it difficult to add headers to the new file. I tried using two while loops, one before the other and the file didn't seem to append the headers, but recorded the data OK. Then I tried to nest in an if statement and it wrote the headers but also added them to the end of every new line of data (see picture below).

My code is as follows:

datetime1 = str(datetime.now())

#create new file with datestamp
extension = ".csv"
file_name = datetime1 + extension
file1 = open(file_name, 'a')

#add header to .csv
header = str("Date,Time,Minutes,InletTemp,ControlTemp,BedTemp,OutletTemp")

#x = 1
#while x >= 1:
#    file1 = open(file_name, 'a')
#    file1.write(header)
#    x -= 1

#start button function
def start():
    while(1):
        x = 1
        if x >= 1:
            file1 = open(file_name, 'a')
            file1.write(header)
      
            
#gather the data from each thermocouple
        t1 = THERMO.getTEMP(0,1)
        t2 = THERMO.getTEMP(0,2)
        t3 = THERMO.getTEMP(0,3)
        t4 = THERMO.getTEMP(0,4)

#themoplate LED cycle
        THERMO.toggleLED(0)
        
#write sensor data to a file
        datetime2 = datetime.now()
        data = str(datetime2) + ',' + ' ' + str(t1) + ',' + str(t2) + ',' + str(t3) + ',' + str(t4)
        file1 = open(file_name, 'a') 
        file1.write('\n')
        file1.write(data)
        
#write sensor data to the console
        print (str(datetime2))
        print ('Temperature on Channel 1:',t1)
        print ('Temperature on Channel 2:',t2)
        print ('Temperature on Channel 3:',t3)
        print ('Temperature on Channel 4:',t4)
        time.sleep(1)

while loop with nested if statement:

You only want to write the header to a file once. If you're opening a new file, you don't want "append" mode. Just use "w". And remember it is pointless to call the str() function on something that's already a string. — Tim Roberts
– Tim Roberts, Commented Oct 28, 2021 at 16:32
Thanks Tim, yes in one of my many iterations of this code I was getting a tuple error for some reason and that's why I tired to cast as a string. But yes, you're correct I changed it and its still a string. — pequalsmv
– pequalsmv, Commented Oct 28, 2021 at 18:19

Tim Roberts · Accepted Answer · 2021-10-28 16:36:08Z

Something like this is closer to what you want. This leaves the file open continuously. If you don't like that, do file1.close() after printing the header, and file1=open(file_name, 'a') before writing the data and file1.close() after.

datetime1 = str(datetime.now())

#create new file with datestamp
extension = ".csv"
file_name = datetime1 + extension

#add header to .csv
header = "Date,Time,Minutes,InletTemp,ControlTemp,BedTemp,OutletTemp"

#start button function
def start():
    file1 = open(file_name, 'w')
    print(header, file=file1)

    while(1):     
            
#gather the data from each thermocouple
        t1 = THERMO.getTEMP(0,1)
        t2 = THERMO.getTEMP(0,2)
        t3 = THERMO.getTEMP(0,3)
        t4 = THERMO.getTEMP(0,4)

#themoplate LED cycle
        THERMO.toggleLED(0)
        
#write sensor data to a file
        datetime2 = datetime.now()
        data = ','.join( (str(datetime2), str(t1), str(t2), str(t3), str(t4) )
        print( data, file=file1)
        
#write sensor data to the console
        print (str(datetime2))
        print ('Temperature on Channel 1:',t1)
        print ('Temperature on Channel 2:',t2)
        print ('Temperature on Channel 3:',t3)
        print ('Temperature on Channel 4:',t4)
        time.sleep(1)

Philippe Miron · Accepted Answer · 2021-10-28 16:42:46Z

1

You set x=1 at the beginning of the while(True) loop and it is never updated, that's why it gets added every line.

I think you should, first open the file in write mode, something like:

with open(file_name, 'w') as file1:
    file1.write(header)

which will erase previous data and write your header. And then you open the file once before the while(True) in append mode as you were doing (file1 = open(file_name, 'a')).

answered Oct 28, 2021 at 16:42

Philippe Miron

1781 silver badge10 bronze badges

Comments

dsillman2000 · Accepted Answer · 2021-10-28 16:39:31Z

1

It looks to me like you're re-opening and re-headering your file after every line, meaning the header gets concatenated to the end of every line. To fix this, fix your while-loop, like so:

def start():
    x = 1
    while 1:
        if x == 1:
            file1 = open(file_name, 'w')
            file1.write(header)
            file1.close()
    # ...etc

That way, you only write the header exactly once when the loop starts, and not every time a reading is recorded. I also recommend calling close() every time you finish writing to the file.

edited Oct 28, 2021 at 16:39

answered Oct 28, 2021 at 16:36

dsillman2000

1,0471 gold badge9 silver badges20 bronze badges

2 Comments

Tim Roberts Over a year ago

Or you could do that outside the loop, and you don't really want 'a' mode, and he is still mismanaging the newlines.

dsillman2000 Over a year ago

@TimRoberts You're right, fixing it now.

Mark Tolonen · Accepted Answer · 2021-10-28 16:50:23Z

1

A cleaner solution is to use the csv module to write the data. Open the file once at the beginning to write the header, and then append new data as collected:

from datetime import datetime
import csv
import time


now = datetime.now()
filename = f'sensors_{now:%Y%m%d%H%M%S}.csv'

# Write mode to start a new file
with open(filename, 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow('Date Time Minutes InletTemp ControlTemp BedTemp OutletTemp'.split())

def start():
    while True:
        # Simulate data
        t1,t2,t3,t4 = 45.1,45.2,45.3,45.4

        # collect column data in a list
        now = datetime.now()
        row = [now.strftime('%Y-%m-%d'),now.strftime('%H:%M:%S.%f'),'',t1,t2,t3,t4]

        # append new data
        with open(filename, 'a', newline='') as f:
            writer = csv.writer(f)
            writer.writerow(row)

        # Write sensor data to the console
        print(now)
        print('Temperature on Channel 1:',t1)
        print('Temperature on Channel 2:',t2)
        print('Temperature on Channel 3:',t3)
        print('Temperature on Channel 4:',t4)
        time.sleep(1)

start()

Example output:

sensors_20211028094726.csv:

Date,Time,Minutes,InletTemp,ControlTemp,BedTemp,OutletTemp
2021-10-28,09:47:26.710602,,45.1,45.2,45.3,45.4
2021-10-28,09:47:27.718613,,45.1,45.2,45.3,45.4
2021-10-28,09:47:28.734657,,45.1,45.2,45.3,45.4
2021-10-28,09:47:29.750682,,45.1,45.2,45.3,45.4

answered Oct 28, 2021 at 16:50

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

2 Comments

pequalsmv Over a year ago

Mark, This looks much cleaner, and I wasn't aware of the csv module. Thanks for pointing that out, I will test it in my next version.

Mark Tolonen Over a year ago

@pequalsmv The csv module handles proper quoting and escaping of fields that contain string quoting characters or delimiters such as comma as well. It's a much better option that writing the csv lines manually.

AlbertCory · Accepted Answer · 2021-10-28 16:32:07Z

0

import csv # at the top of your program

Then use a DictWriter to build the CSV. That will put in the headers for you.

answered Oct 28, 2021 at 16:32

AlbertCory

12 bronze badges

2 Comments

Tim Roberts Over a year ago

Assuming he had a set of dictionaries, which he doesn't right now.

AlbertCory Over a year ago

I don't know what that means. You don't need "a set of dictionaries" to use DictWriter.

Collectives™ on Stack Overflow

Adding header to new CSV file

5 Answers 5

Comments

Comments

2 Comments

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

2 Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related