1

So I've created Python code that scrapes several different websites and then does some things with the lists created. I know how to turn a list into a string and how to then write a csv file from the string. My question is, can I write several strings into the same csv file and put each in its own row with a unique header? The code below will get all the data into the csv file, but not in its own rows.

str1 = "\n".join(data1)
str2 = "\n".join(data2)
str3 = "\n".join(matches)
now = datetime.datetime.now()
now_str = now.strftime("%Y-%m-%d-%H-%M")
outfilename = 'data-{}.csv'.format(now_str)
outpath = 'data/'
outFile = open(outfilename, 'write')
outFile.write(str1)
outfile.write(str2)
outfile.write(str3)
outFile.close()
Improve this question
6
  • Is one CSV file the best format for this? Commented Nov 3, 2013 at 1:37
  • For someone to answer your question, they will need to know what data1, data2, and matches are, or what output you are now getting. Commented Nov 3, 2013 at 1:37
  • Thanks, @Paul. data1 and data2 are lists of binomial scientific names. Matches is a list of binomial scientific names that are common to both data1 and data2. This is a daily scraping report that I'd like to be able to store in a single csv file each day. Commented Nov 3, 2013 at 1:41
  • Python has a builtin module, conveniently named csv, which is designed to make reading and writing csv files straightforward. Commented Nov 3, 2013 at 1:43
  • SoI've heard (re: Python csv module). Guess I need to read up. Thanks. Commented Nov 3, 2013 at 1:44

1 Answer 1

Reset to default
2

The simple answer to your question is no, you should probably not create a CSV with multiple sections, each with independent headers.

There is no gold standard that defines the CSV file format, though RFC 4180 might come close. So while you are technically free to create your own extensions and call it a CSV file (such as multiple independent subfiles with their own headers, as you propose) this is extremely uncommon among CSVs. I predict it would give undesirable results in most software if you attempted to read such a file.

Best practice would be to create one CSV per dataset if the datasets have different schema.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you very much for that. As you may have intuited, I'm a bit of a noob, but I'm learning. So you're saying to go ahead and write a separate csv for each scraped dataset each day? The datasets are all the same schema (if I'm interpreting schema) correctly. They are all binomial scientific names.
When you said "unique header", I took this to mean each header was different. That means the column names or types or number of columns are different and, thus, it's a different schema. If all the data really do share the same column headers then go ahead and write a single CSV file and just stick a single header on the first line. That would be a well-constructed CSV file. Good luck. If my answer satisfies you, please mark it as "answered", above. Thanks.
Thanks, and I'm sorry for any confusion. I do need to break the data sets out into separate columns representing the different websites from which the data was scraped, so I may just go ahead and do as you suggested and have each dataset be its own csv file.
Sounds like the right thing to do. The nice thing about constructing valid CSV files is even if you end up with a large collection of them, it's a file format any spreadsheet and many other programs can read and write so you can always read them or merge them later with common or free tools. Good luck.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.