How can I create a loop with pandas read_csv?
I need to create a data loop to list and save to the database.
How can I do this loop with the data from a csv?
thank you all for your attention
produtos = pd.read_csv('tabela.csv', delimiter=';')
for produto in produtos:
print(produto['NOME'])
To iterate in the DataFrame resulted by calling pandas read_csv you should use the command iterrows() for iteration, as in the below example:
for produto in produtos.iterrows():
print(produto['NOME'])
If you have files that you need to save, I recommend this
import os
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('sqlite://', echo=False)
path = "C:/path/to/directory"
# list all files in the directory, assuming this directory
# contains only files csv files that you need to save
for file in os.listdir(path):
df = pd.read_csv(path+file)
# some other data cleaning/manipulation
# write dataframe to database
df.to_sql("table_name", con=engine)
Alternative, you can create a list with all files locations and iterate through that one instead. More info on to_sql() and check out this answer
If you can create loop with Pandas by column name:
produtos = pd.read_csv('tabela.csv', delimiter=';')
for i, produto in produtos.iterrows():
print(produto['NOME'])
But if you want to insert directly on your database use sqlalchemy and function to_sql like this:
from sqlalchemy import create_engine
import pandas as pd
...
engine = create_engine("mysql://user:pwd@localhost/database")
produtos = pd.read_csv('tabela.csv', delimiter=';')
if_exists_do = 'append'
produtos.to_sql('table_name', con=engine, if_exists=if_exists_do)
Then it will be inserted on database. The var 'if_exists_do' can receive value 'replace' if you want this.
You can iterate using a reader, which is obtained by specifying a chunksize in the call to read_csv().
Here I use CSV data in memory and read two rows at once.
Replace StringIO(s) by your file, and use a chunsize of 1 if you want to read a single row at once:
import pandas as pd
from numpy import random as npr, round
from io import StringIO
# Some CSV data to read, in a string
s = pd.DataFrame(round(npr.randn(10, 4), 2)).to_csv()
# Use the reader in a with/as structure
with pd.read_csv(StringIO(s),index_col=0, chunksize=2) as reader:
for i, chunk in enumerate(reader):
print(f'\nChunk {i:}')
print(chunk)
Chunk 0
0 1 2 3
0 -1.50 -1.78 -0.53 1.09
1 -0.35 -0.79 0.20 1.08
Chunk 1
0 1 2 3
2 -1.44 -1.21 -0.79 1.09
3 0.23 2.13 0.94 -0.04
Chunk 2
[...]
