323 questions
3
votes
1
answer
75
views
How to pass argument to func in `pandas.resampler.agg()` when using dict input?
I am trying to resample a pandas dataframe, and for some columns I would like to sum on. additionally, I want to get None/nan as result when there is no rows in a resampling period. For aggregation on ...
- 97
1
vote
1
answer
116
views
How do I make Pandas resample align on the day for timeframes that don't divide into 24 hours?
The resample function doesn't work with 5h or 10h intervals. It starts okay and then it stops aligning on the day.
d = {'Open': 'first', 'High': 'max', 'Low': 'min', 'Close': 'last'}
df = read....
- 15
0
votes
1
answer
75
views
Computing the min() and max() of a dataframe for a resampled period but also retrieve the exact dates the min and max occurred?
I have a dataframe that has a datetime index and a single column containing the price of the asset. For example,
date asset_value
2023-05-30 136.57000000
2023-05-31 133....
2
votes
1
answer
123
views
How to resample a dataframe to stretch from start to enddate in intervals (containing 0 for not available values)
I have the following setup:
I have sparse information about queries hitting my endpoint at certain timepoints in a csv file. I parse this csv file with dates according to date_format='ISO8601' in the ...
- 5,356
2
votes
1
answer
39
views
Converting data wide to long, collapsing to 1 minute intervals
Looking for python help for converting data from wide to long (?)
My data looks something like this:
channelId,utc,scet,val1,val2
A-0001,2024-061T22:00:05.02064,0.03,3,
A-0002,2024-061T22:00:06.02064,...
1
vote
1
answer
88
views
Pandas Resample 2M and 3M for every month
I have this code to calculate Returns:
import yfinance as yf
import numpy as np
import pandas as pd
df = yf.download('SPY', '2023-01-01')
df = df[['Close']]
df['d_returns'] = np.log(df.div(df.shift(1)...
-1
votes
1
answer
177
views
What is the rule of pandas resample with rolling 30 minutes?
I am trying to resample a kbars dataframe from 1 minute to 30 minutes. And I need a rolling window. However, it seems that's a fixed time frame.
For example, it's now 9:28 pm. The latest row of the ...
1
vote
1
answer
1k
views
Pandas 2.2: FutureWarning: Resampling with a PeriodIndex is deprecated
The pandas version 2.2 raises a warning when using this code:
import pandas as pd
df = pd.DataFrame.from_dict({"something": {pd.Period("2022", "Y-DEC"): 2.5}})
# ...
- 3,618
0
votes
1
answer
101
views
Sample 30day increments and create a new data frame from existing daily frame
I am storing daily data into a dataframe and wish to create a new data frame which has only data for last day of the month ( or last 30days or say 60days) and plot those periods returns for the given ...
0
votes
0
answers
20
views
Why not all files are included in the resampled dataframe?
I want to resample one min stock data into 1 hour.
Here is my code(my orginal coded is in chinese):
import pandas as pd
import glob
path = 'C:/Users/Desktop/fut_data_1min_2023'
all_files =...
- 1
0
votes
1
answer
67
views
Alternating averaging times every row
This is probably a silly question that was already answered, but I'm struggling to find the answer I need. I'm looking to average a large dataset with 1-second times, switching between averaging every ...
- 27
0
votes
1
answer
73
views
How can I keep string data with aggregation when resampling data? [duplicate]
I'd like to average my minute data to 10-min averages of those 10 timepoints, however, I have important data in columns.
This is the first thing I tried df1.resample('10min', on='Date_Time').mean(d)....
- 15
2
votes
1
answer
80
views
Using Pandas to resample and combine data series with different timestamp indices
I'm attempting to use pandas to manipulate some data and not seeming to find a built-in way to do resample my data to merge datasets with differing time indices.
It's not hard to do what I want using ...
2
votes
2
answers
117
views
Resampling data from 6 min to 5min with nan
I have a linear interpolation problem with nans in my data. I have instantaneous measurements that I want to resample from 6 min intervals to 5 min intervals.
df = pd.DataFrame(zip(['10:00','10:06','...
- 640
2
votes
1
answer
154
views
Pandas resample method does not work properly
I have been analyzing the Seoul Bike Sharing Demand dataset, which is available at Seoul Bike Sharing Demand
. During my analysis, I found the need to use a resampling method. To accomplish this, I ...
- 145
0
votes
0
answers
136
views
why pandas infer freq returns Daily when it is Hourly data
I have hourly data, when I infer it, it returns Daily data. I am surprized. My actual goal is converting this hourly data to daily mean.
df = pd.read_csv('/kaggle/input/hourly-energy-consumption/...
- 4,702
0
votes
1
answer
54
views
Check Dataframe Exactly at 10 min interval starting from 9.15
I am stuck on code , where I want to check dataframe every 10 min starting from 09:15.
If I use minute%10=0 then It checks at 9:20 and 9.30 which is not correct.
It should always check at 9.25 and 9....
- 5
0
votes
1
answer
697
views
Resampling of a DataFrame by 1 Hour in pandas gives unexpexted NaN values
Resampling of a DataFrame by 1 Hour in pandas gives unexpected NaN values
I have a dataframe having 3 columns. 1st Column contains date ( like 2020-07-01,2020-07-01...); 2nd column contains time ( ...
- 31
0
votes
0
answers
48
views
how to split data hourly in Pandas
i have a data resembling this
EndTime Duration PartsProduced
StartTime
2020-09-03 00:14:51 2020-09-03 00:46:56 ...
- 1
0
votes
1
answer
429
views
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index', but my index is a DateTimeIndex
I have some time series data that is measured hourly, I want to average the values over different resolutions by resampling. I keep getting the following error:
TypeError: Only valid with ...
- 115
2
votes
1
answer
525
views
Pandas.resample with min and max of each time period
I have the following dataframe (top 30 lines shown, its weather station data at 5 minute intervals, 2 years of data) and need to resample it into days with mean values for each column, I also need to ...
- 101
1
vote
0
answers
25
views
Accessing values in Multi-Index dataframe with datetimes
I have a problem trying to access values in a custom dataframe:
My original dataframe is:
print(df)
Data1 Data2 Data3
IMX01 2022-08-12 22:00:00 400.00 ...
- 43
0
votes
2
answers
51
views
Python resample to only keep every 5th day by group
I have a dataframe, consisting of daily stock observations, date and PERMNO (Identifier). I want to resample the dataframe to only consist of observations for every 5th trading day for every stock. ...
- 27
1
vote
1
answer
136
views
Pandas resample aggregation, intra month periods?
I'm trying to aggregate some data across periods using resampling and so far have something like this:
Data:
val1 val2 val3 ...
date
2022-01-29 0.01 0.08 0.03
2022-01-30 0.04 -0.07 ...
- 103
2
votes
1
answer
87
views
How can I make NaN values sum to NaN rather than 0 when using df.resample?
I have the following example dataframe:
>>> import pandas as pd
>>> import numpy as np
>>> d = {'date': pd.date_range(start='2022-12-09 00:00:00',
...
- 14.6k