0

I have a dataframe. I am performing forward and backward substraction. Later, perform a comparison and produce boolean outputs. Next, I want to perform logical and on these results and produce one result.

Code:

xdf = pd.DataFrame({'data':range(0,6)},index=pd.date_range('2022-06-03 00:00:00', '2022-06-03 00:00:25', freq='5s'))

# perform 1 row backward substraction
bs = xdf['data'].diff(1).abs().le(1)

# perform 1 row forward substraction
fs= xdf['data'].diff(-1).abs().le(1)


bs = 
2022-06-03 00:00:00    False
2022-06-03 00:00:05     True
2022-06-03 00:00:10     True
2022-06-03 00:00:15     True
2022-06-03 00:00:20     True
2022-06-03 00:00:25     True
Freq: 5S, Name: data, dtype: bool

fs = 

2022-06-03 00:00:00     True
2022-06-03 00:00:05     True
2022-06-03 00:00:10     True
2022-06-03 00:00:15     True
2022-06-03 00:00:20     True
2022-06-03 00:00:25    False

Present and expected output:

xdf['validation'] = np.logical_and(sa,sb)
2022-06-03 00:00:00    False
2022-06-03 00:00:05     True
2022-06-03 00:00:10     True
2022-06-03 00:00:15     True
2022-06-03 00:00:20     True
2022-06-03 00:00:25    False
Freq: 5S, Name: data, dtype: bool

The output is correct and this is what I am expecting. My question, is there a way I can compute all the above (forward substraction and backward substraction) in a single code of line?

Improve this question
2
  • 1
    You are testing whether the abs(a-b)<1 It really does not matter whether it is b-a or a-b since you are doing the abs. Hence just calculate the back difference and use that, while changing the first and last values to False since we should not have those values Commented Jun 3, 2022 at 18:18
  • @onyambu I posted a new question stackoverflow.com/questions/72494309/… Commented Jun 3, 2022 at 19:24

2 Answers 2

Reset to default
1

Maybe you can try loop the [1,-1] and use np.logical_and.reduce

xdf['validation'] = np.logical_and.reduce([xdf['data'].diff(x).abs().le(1) for x in [1,-1]])

print(xdf)

                     data  validation
2022-06-03 00:00:00     0       False
2022-06-03 00:00:05     1        True
2022-06-03 00:00:10     2        True
2022-06-03 00:00:15     3        True
2022-06-03 00:00:20     4        True
2022-06-03 00:00:25     5       False
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1

IIUC, you can use a rolling max, then check whether the max is ≤ your target:

xdf['validation'] = xdf['data'].diff(-1).abs().rolling(2).max().le(1)

output:

                     data  validation
2022-06-03 00:00:00     0       False
2022-06-03 00:00:05     1        True
2022-06-03 00:00:10     2        True
2022-06-03 00:00:15     3        True
2022-06-03 00:00:20     4        True
2022-06-03 00:00:25     5       False
Improve this answer

4 Comments

But it is only doing forward substraction. correct? or is it covering both sides? Can you you bit explanation. This seems to be simple and straight forward. I missing the logic here.
as pointed out by @onyambu, abs(a-b) is the same as abs(b-a) so the order doesn't matter, what matters is the shift. So in this case it is strictly equivalent. It wouldn't be with a non symmetrical operator. Similarly, using the max is specific because we use le later. For a ge we would need to use min ;)
In my code, I am basically checking if the data changed within a value in a given time, say 10s, not more than that. I am checking this before and after. In your code, you are doing the rolling(2). Is it doing the same thing? or am I just checking just instantaneous value before 1 min and now and not in between value? I want to check all the samples in the last 1 min and see if all values less than the defined.
Hi I posted a new question stackoverflow.com/questions/72494309/…

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.