0

Here is my pandas Dataframe

import datetime as dt
d1=dt.date(2021,1,1)
d2=dt.date(2021,1,13)
df = pd.DataFrame({'date':[d1,d2],'location':['Paris','New York'],'Number':[2,3]})

Here is my problem

df = df.set_index(['date'])
df = df.reset_index()
print(df.loc[df.date == d1]) # Ok !
df = df.set_index(['location','date'])  # order has not importance
df = df.reset_index()
print(df.loc[df.date == d1]) # Not OK ! it returns an Empty DataFrame 

It seems that when I set_index with two columns and reset_index the type of date is lost.

I don't understand why it is working in the first case and not in the second, where I need to do pd.to_datetime(df['date']).dt.date to work again ?

2 Answers 2

1

Since it is change the column type to datetime64[ns] and the output for single value is Timestamp we need add date convert

df.date[0]
Out[175]: Timestamp('2021-01-01 00:00:00')

df.loc[df.date.dt.date == d1]
Out[177]: 
  location       date  Number
0    Paris 2021-01-01       2
Sign up to request clarification or add additional context in comments.

Comments

0

To explain the difference, we can inspect the data type of the elements and see the difference.

Applying the type function on the dataframe at different stages:

df.applymap(type)

You can see from the following that the data type at the last stage is different. Hence, no match at the last stage.

Output:

import datetime as dt
d1=dt.date(2021,1,1)
d2=dt.date(2021,1,13)
df = pd.DataFrame({'date':[d1,d2],'location':['Paris','New York'],'Number':[2,3]})

df.applymap(type)

                      date       location         Number
0  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date
1  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date

df = df.set_index(['date'])
df = df.reset_index()
print(df.loc[df.date == d1]) # Ok !

df.applymap(type)

                      date       location         Number
0  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date
1  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date


df = df.set_index(['location','date'])  # order has not importance
df = df.reset_index()
print(df.loc[df.date == d1]) # Not OK ! it returns an Empty DataFrame 

df.applymap(type)

        location                                                date         Number
0  <class 'str'>  <class 'pandas._libs.tslibs.timestamps.Timestamp'>  <class 'int'>         # <=== data type changed to pandas._libs.tslibs.timestamps.Timestamp 
1  <class 'str'>  <class 'pandas._libs.tslibs.timestamps.Timestamp'>  <class 'int'>         # <=== data type changed to pandas._libs.tslibs.timestamps.Timestamp 

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.