Python panda datastructure

Question

I had a datastructure as a numpy array. Python accessing a data structure

Instead, I see I can get it as a panda dataframe. To write readable code, I'd like to address the data in the values column (time-data and a price) using the key EnergyDataPd["en":"Import Balance"][whole column of times][whole column of prices] and then do math on the data - for instance multiply the prices with a different column of time/price data.

The column values holds an array of time and money data: [[1546297200000, -10.903], [1546300800000, etc, and is what I'd like to do operations on.

So, how do I access the data?

     key                                                color           chartType   chartTitle  xAxisFormat xAxisLabel  y1AxisLabel y1AxisDecimalPlaces y2AxisLabel y2AxisDecimalPlaces datasource  date    type    yAxis   values  disabled
0   [{'en': 'Import Balance', 'de': 'Import Saldo'...   rgb(125,25,125) multiChart  [{'en': 'Electricity production and spot price...   unixTime    [{'en': 'Date', 'de': 'Datum', 'fr': 'Date', '...   [{'en': 'Power (GW)', 'de': 'Leistung (GW)', '...   2.0 [{'en': 'Price (Euro / MWh, Euro / t CO2)', 'd...   2.0 EPEX SPOT, 50 Hertz, Amprion, Tennet, TransnetBW    2020-05-04 13:07:46 area    1   [[1546297200000, -10.903], [1546300800000, -10...   NaN
1   [{'en': 'Conventional > 100 MW', 'de': 'Konven...   rgb(158,152,148)    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT area    1   [[1546297200000, 21.848], [1546300800000, 19.2...   NaN
2   [{'en': 'Wind', 'de': 'Wind', 'fr': 'Éolien', ...   rgb(175,196,165)    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT area    1   [[1546297200000, 23.642], [1546300800000, 25.4...   NaN
3   [{'en': 'Solar', 'de': 'Solar', 'fr': 'Solaire...   rgb(255,205,100)    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT area    1   [[1546297200000, 0], [1546300800000, 0], [1546...   NaN
4   [{'en': 'Load', 'de': 'Last', 'fr': 'Charge', ...   rgb(50,50,50)   NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    1   [[1546297200000, 42.563], [1546300800000, 41.4...   NaN
5   [{'en': 'Day Ahead Auction', 'de': 'Day Ahead ...   rgb(255,0,0)    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    2   [[1546297200000, 28.32], [1546300800000, 10.07...   NaN
6   [{'en': 'Intraday Continuous Index Price', 'de...   rgb(66,146,198) NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    2   [[1546297200000, 26.12], [1546300800000, 12.02...   NaN
7   [{'en': 'Intraday Continuous Average Price', '...   rgb(8,81,156)   NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    2   [[1546297200000, 26.12], [1546300800000, 12.02...   true
8   [{'en': 'Intraday Continuous Low Price', 'de':...   rgb(33,113,181) NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    2   [[1546297200000, 8.4], [1546300800000, -200], ...   true
9   [{'en': 'Intraday Continuous High Price', 'de'...   rgb(107,174,214)    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    2   [[1546297200000, 60.7], [1546300800000, 35], [...   true
10  [{'en': 'Intraday Continuous ID3-Price', 'de':...   rgb(116,196,118)    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaT line    2   [[1546297200000, 22.6], [1546300800000, 15.1],...   true

pandas documentation pandas.pydata.org/pandas-docs/stable/reference/api/… could help you here, I just guess you are after df.to_numpy() which returns you well known structure, that can be accessed like described for example here: stackoverflow.com/questions/25129195/… — Kube Kubow
– Kube Kubow, Commented Jan 6, 2022 at 11:27

newton · Accepted Answer · 2022-01-06 11:32:29Z

0

You can create 2 new columns using from your 'values' column by operating on it.

Firstly define how you want to extract/process the Values column's value to get your data:

def getTimeData(row):
    return row[0] #1st item in **Values** column's value

def getMoneyData(row):
    return row[1] #2nd item in **Values** column's value

Then you can create the two columns as needed:

df['MoneyData'] = df.apply(getMoneyData, axis=1)
df['TimeData'] = df.apply(getTimeData, axis=1)

answered Jan 6, 2022 at 11:32

newton

13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

havarsak Over a year ago

I can of course re-format the data into another table structure. But, I'm wondering if I can use the existing data structure as it is, and use the dict in "key" to access the data to extract columns and data points. In concise code.

newton Over a year ago

yes, you could do that. but each time you access it you will need to use list operation.

Collectives™ on Stack Overflow

Python panda datastructure

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related