25

So I want to convert a simple tab delimited text file into a csv file. If I convert the txt file into a string using string.split('\n') I get a list with each list item as a string with '\t' between each column. I was thinking I could just replace the '\t' with a comma but it won't treat the string within the list like string and allow me to use string.replace. Here is start of my code that still needs a way to parse the tab "\t". 

import csv
import sys

txt_file = r"mytxt.txt"
csv_file = r"mycsv.csv"

in_txt = open(txt_file, "r")
out_csv = csv.writer(open(csv_file, 'wb'))

file_string = in_txt.read()

file_list = file_string.split('\n')

for row in ec_file_list:       
    out_csv.writerow(row)
Improve this question

3 Answers 3

Reset to default
48

csv supports tab delimited files. Supply the delimiter argument to reader:

import csv

txt_file = r"mytxt.txt"
csv_file = r"mycsv.csv"

# use 'with' if the program isn't going to immediately terminate
# so you don't leave files open
# the 'b' is necessary on Windows
# it prevents \x1a, Ctrl-z, from ending the stream prematurely
# and also stops Python converting to / from different line terminators
# On other platforms, it has no effect
in_txt = csv.reader(open(txt_file, "rb"), delimiter = '\t')
out_csv = csv.writer(open(csv_file, 'wb'))

out_csv.writerows(in_txt)
Improve this answer
Sign up to request clarification or add additional context in comments.

19 Comments

-1 You are presuming that the OP is on Python 2.x; in that case the input file should be opened with 'rb' mode. Also not ensuring that at least the output file is closed, preferably both files.
bikeshedding. Both files are closed as soon as the script terminates. Which is.. immediately. +1.
@JohnMachin I didn't presume anything. I changed as little as possible to show how to convert a file. with isn't necessary if the program is going to terminate immediately -- the file will be closed. I added a comment to indicate care should be taken if it is a long running program.
It's nothing to do preserving line endings. Read the docs for csv.reader for both Python 2.7 and 3.2. See also stackoverflow.com/questions/5180555/python-2-and-3-csv-reader
"The only difference between the two modes is how newlines are handled on Windows.": Wrong; see my answer. What is "the encoding of the file is ASCII-compatible" supposed to mean? In any case, the sample data file in my answer is encoded in ASCII.
|
1

Why you should always use 'rb' mode when reading files with the csv module:

Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

What's in the sample file: any old rubbish, including control characters obtained by extracting blobs or whatever from a database, or injudicious use of the CHAR function in Excel formulas, or ...

>>> open('demo.txt', 'rb').read()
'h1\t"h2a\nh2b"\th3\r\nx1\t"x2a\r\nx2b"\tx3\r\ny1\ty2a\x1ay2b\ty3\r\n'

Python follows CP/M, MS-DOS, and Windows when it reads files in text mode: \r\n is recognised as the line separator and is served up as \n, and \x1a aka Ctrl-Z is recognised as an END-OF-FILE marker.

>>> open('demo.txt', 'r').read()
'h1\t"h2a\nh2b"\th3\nx1\t"x2a\nx2b"\tx3\ny1\ty2a' # WHOOPS

csv with a file opened with 'rb' works as expected:

>>> import csv
>>> list(csv.reader(open('demo.txt', 'rb'), delimiter='\t'))
[['h1', 'h2a\nh2b', 'h3'], ['x1', 'x2a\r\nx2b', 'x3'], ['y1', 'y2a\x1ay2b', 'y3']]

but text mode doesn't:

>>> list(csv.reader(open('demo.txt', 'r'), delimiter='\t'))
[['h1', 'h2a\nh2b', 'h3'], ['x1', 'x2a\nx2b', 'x3'], ['y1', 'y2a']]
>>>
Improve this answer

2 Comments

Do you have a python.org reference for the Ctrl-z behavior? I don't see any mention of it.
@agf: No. It's a consequence of CPython 2.X delegating responsibility for deciding what to do to the C stdio library of the target compiler.
1

This is how i Do it

import csv

with open(txtfile, 'r') as infile, open(csvfile, 'w') as outfile:
     stripped = (line.strip() for line in infile)
     lines = (line.split(",") for line in stripped if line)
     writer = csv.writer(outfile)
     writer.writerows(lines)
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.