Importing netCDF data to pandas DataFrame

Question

I am trying to convert a netCDF file to csv. I have succeeded to do that. However, as illustrated in the attached image, i have two problems in the exported csv. dataframe. First, i need to split datetime into two separate columns (Only Date, Only time). Second, when i run the code i get the same values for the 3 different variables (par,temp, and evap). Here is the code:

import netCDF4
from netCDF4 import Dataset
from netCDF4 import num2date
import pandas as pd 
import numpy as np 
import os

# Reading netCDF file
data = Dataset(r'G:\trial\Jan.nc', 'r')

# Defining parameters and dimensions 
lat = data.variables['latitude'][:]
long = data.variables['longitude'][:]
time = data.variables['time']
times = num2date(time[:], time.units)
par = data.variables['par'][:]
temp = data.variables['t2m'][:]
evap = data.variables['e'][:]

# Importing netCDF data from a single point 
idku_lat = 31.249464
idku_long = 30.210806
lat_diff = (lat - idku_lat)**2
lon_diff = (long - idku_long)**2
min_lat = lat_diff.argmin()
min_long = lon_diff.argmin()

# Creating a pandas DataFrame and import values from  the netCDF file

df = pd.DataFrame(0, columns = ['par','temp','evap'], index = times)

dt = np.arange(0, data.variables['time'].size)

for time_index in dt:
    df.iloc[time_index]= par[time_index,min_lat,min_long]


# Exporting the file to csv.
df.to_csv('my_data.csv')

dataframe

Have a look at how to ask a good question. Provide expected output and actual output. — Han-Kwang Nienhuys
– Han-Kwang Nienhuys, Commented Jun 14, 2020 at 17:43
Looks like you liked the answer by Dani56. If so, you can mark it as the solution! — AstroFloyd
– AstroFloyd, Commented Oct 1, 2023 at 9:28

Dani56 · Accepted Answer · 2020-06-22 17:35:14Z

4

I'm not sure what your expected ouput is but if you simply want to convert your dataset into a DataFrame then save it as a csv file, you could do this.

import xarray as xr

ds = xr.open_dataset('G:\trial\Jan.nc')
df = ds.to_dataframe()
df = df.reset_index()

date = df['time'].dt.strftime('%m/%d/%Y')
time =  df['time'].dt.strftime('%H:%M:%S')

df['Date'] = date 
df['time'] = time



df.to_csv('myfile.csv',index=True, header=True)

edited Jun 22, 2020 at 17:35

answered Jun 20, 2020 at 20:03

Dani56

3821 gold badge2 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Amr Talaat Over a year ago

Thank you so much, it works perfectly . But, if i need to separate Datetime into two columns (for time and date separately), How can i do that ?

Dani56 Over a year ago

The answer above should answer your second question. The only way I could think of is to format your existing time dimension into separate Date and Time format before adding it your DataFrame and saving it as a csv file.

Collectives™ on Stack Overflow

Importing netCDF data to pandas DataFrame

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related