loop through python datetimes in specified range TypeError

Question

I want to create a scatter plot for every 10 minutes in the time period specified by t_list. I get the error TypeError: cannot compare a dtyped [datetime64[ns]] array with a scalar of type [bool] in the line df_t = df[(df['datetime']>=t & df['datetime']<t_end)] but the type for t and t_endare both datetime. non of the variables are type bool.

    import pandas as pd
    import matplotlib.pyplot as plt
    from datetime import datetime, timedelta

    df_data = pd.read_csv('C:\SCADA.csv')#import data

    #format Timestamp as datetime
    df_data['datetime'] = pd.to_datetime(df_data['TimeStamp'] )

    #create df of time period
    df = df_data[(df_data['datetime']>= datetime(2017, 12, 23, 06,00, 00)) &
             (df_data['datetime']< datetime(2017, 12, 23, 07, 00, 00))]

    #time period I want to create 10 min plots for        
    t_list = [datetime(2017, 12, 23, 06, 00, 00), datetime(2017, 12, 23, 07, 00, 00)] 



    for t in t_list:
        t_end = t + timedelta(minutes = 10)

        #breaks here with 
        TypeError: cannot compare a dtyped [datetime64[ns]] array with a 
        scalar of type [bool]

        df_t = df[(df['datetime']>=t & df['datetime']<t_end)]
        #code continues with plotting scatter plots within the loop

Can you please post the output of type(df['datetime'].iloc[0]), type(t) and type(t_end)? — alec_djinn
– alec_djinn, Commented Oct 17, 2019 at 17:11
pandas.tslib.Timestamp, datetime.datetime and datetime.datetime respectively — squar_o
– squar_o, Commented Oct 18, 2019 at 10:30

fsl · Accepted Answer · 2019-10-17 19:49:28Z

2

When boolean indexing with multiple conditions, you should wrap each single condition in brackets.

From the docs:

Another common operation is the use of boolean vectors to filter the data. The operators are: | for or, & for and, and ~ for not. These must be grouped by using parentheses, since by default Python will evaluate an expression such as df.A > 2 & df.B < 3 as df.A > (2 & df.B) < 3, while the desired evaluation order is (df.A > 2) & (df.B < 3).

Thus, adding the brackets to your last line should work:

df_t = df[(df['datetime']>=t) & (df['datetime']<t_end)]

edited Oct 17, 2019 at 19:49

answered Oct 17, 2019 at 19:05

fsl

3,2801 gold badge12 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

squar_o Over a year ago

It's always something simple that I've missed. Thank you for the info

Collectives™ on Stack Overflow

loop through python datetimes in specified range TypeError

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related