Python Correlation index

Question

Given a data frame "df", I need to obtain the correlation index between mean price and total volume for Region = "California".

Given Dataframe:

Correlation index between California mean price and total volume:

cali_mean = df.groupby('Region').get_group('California')['AveragePrice'].mean()
max_volume = (df.groupby('Region')['TotalVolume'].sum()).max() #Output: 1028981653.17

# Correlation index between California mean price and total volume
df[cali_mean].corr(df['max_volume'])

When I tried determining the correlation index between California's mean price and total volume, I got the following error message. Is there a way to fix this?

Error message

KeyError                                  Traceback (most recent call last)
~/opt/miniconda3/lib/python3.9/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3620             try:
-> 3621                 return self._engine.get_loc(casted_key)
   3622             except KeyError as err:

~/opt/miniconda3/lib/python3.9/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

~/opt/miniconda3/lib/python3.9/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 1.3939644970414187

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
/var/folders/wv/42dn23fd1cb0czpvqdnb6zw00000gn/T/ipykernel_18660/3247367876.py in <module>
      1 # Correlation index between California mean price and total volume
----> 2 df[cali_mean].corr(df['max_volume'])

~/opt/miniconda3/lib/python3.9/site-packages/pandas/core/frame.py in __getitem__(self, key)
   3503             if self.columns.nlevels > 1:
   3504                 return self._getitem_multilevel(key)
-> 3505             indexer = self.columns.get_loc(key)
   3506             if is_integer(indexer):
   3507                 indexer = [indexer]

~/opt/miniconda3/lib/python3.9/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3621                 return self._engine.get_loc(casted_key)
   3622             except KeyError as err:
-> 3623                 raise KeyError(key) from err
   3624             except TypeError:
   3625                 # If we have a listlike key, _check_indexing_error will raise

KeyError: 1.3939644970414187

cali_mean is a mean value not a column, which error implies. — keramat
– keramat, Commented Mar 24, 2022 at 6:18
Also note that there is not any correlation between two numbers. — keramat
– keramat, Commented Mar 24, 2022 at 6:18
@keramat How do I determine the correlation index then if mean value is not a column? — c200402
– c200402, Commented Mar 24, 2022 at 6:22

keramat · Accepted Answer · 2022-03-24 06:56:41Z

1

Note that the correlation is a measure of two vectors. So you can use:

df = pd.read_csv('avocado.csv')
temp = df[df['Region']=='California']
temp['AveragePrice'].corr(temp['TotalVolume'])

Output:

-0.7913852550045145

edited Mar 24, 2022 at 6:56

answered Mar 24, 2022 at 6:31

keramat

4,6138 gold badges29 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

c200402 Over a year ago

What do the "tv" and "av" represent here? Meaning how do I fill in the values for "tv" and "av" so that it reflects mean price and total volume of California region?

c200402 Over a year ago

How do you know which values to fill the vector "av" and "tv" with?

c200402 Over a year ago

When I filled the vector with my data, I got "nan" output. Is this expected as you mentioned?

keramat Over a year ago

Can you send a sample of your data in a format that I can copy?

c200402 Over a year ago

My data is in a form of a CSV file. Is there a way to send you that?

|

Collectives™ on Stack Overflow

Python Correlation index

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related