for loop to create a matrix in python

Question

I am trying to study the probability of having a zero value in my data and I have developed a code that outputs the value of a column of data when the other is zero which is what I need. But having to do that for each column vs all other 28 of my 577by29 dataframe is difficult so I decided to create a for loop that does that for me where I have this:

import numpy as np
import pandas as pd
allchan = pd.read_csv('allchan.csv',delimiter = ' ')
allchanarray = np.array(allchan)
dfallchan = pd.DataFrame(allchanarray,range(1,578),dtype=float)
y = pd.DataFrame()
x = pd.DataFrame()
for n in range(0,29):
    x[n] = dfallchan[(dfallchan[0]>0) & (dfallchan[n]==0)][0]
    y[n] = x[n].count()
x.to_excel('n.xlsx', index=False, sheet_name='ValForOtherZero')
y.to_excel('v.xlsx', index=False, sheet_name='CountOfZeroVlas')

The problem that is that the loop for some reason goes properly through the lines:

 x[n] = dfallchan[(dfallchan[0]>0) & (dfallchan[n]==0)][0]
 y[n] = x[n].count()

but it repeats the value of n=6 for the second condition:

(dfallchan[n]==0)

the output of the code should return different values of the first channel as the zeros are randomly distributed in my input file, but my output is correct for the data until the the 6th column -as my columns(0-5) should be empty- where it repeats the output for all other columns! output: output 1

you can see that the code loops correctly as the output data frame has n=29 columns but not for the condition specified above.

Please help, Thanks!

That isn't an error, it's a warning. See more about it here: stackoverflow.com/questions/20625582/… — Alex S
– Alex S, Commented Aug 8, 2017 at 16:41
I am have read the warning and it seams that the type of the variable x was inappropriate.... — Faisal
– Faisal, Commented Aug 8, 2017 at 22:06
I am now running into another issue and have edited the question! — Faisal
– Faisal, Commented Aug 8, 2017 at 22:07

Faisal · Accepted Answer · 2017-08-10 21:46:33Z

0

Finally Got it!

This code does exactly what I want!

# In[9]:

import numpy as np
import pandas as pd


# In[10]:

allchan = pd.read_csv('allchan.csv',delimiter = ' ')


# In[11]:

allchanarray = np.array(allchan)


# In[12]:

dfallchan = pd.DataFrame(allchanarray,range(1,578),dtype=float)


# In[13]:

v = pd.DataFrame(columns=range(0,29))
y = pd.DataFrame()
k = pd.DataFrame(columns=range(0,29))


# In[14]:

for n in range(0,29):
    x = dfallchan[(dfallchan[0]>0) & (dfallchan[n]==0)][0]
    y = y.append(x)
    v = y.transpose()
    k = v.count()


# In[15]:

v.columns=range(0,29)
k = k.values.reshape(1,29)


# In[16]:

v.to_excel("Chan1-OthersZeroVals.xlsx", index=False)
pd.DataFrame(k).to_excel("Chan1-OtherZeroCount.xlsx", index=False)

answered Aug 10, 2017 at 21:46

Faisal

111 silver badge2 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

TrigonaMinima Over a year ago

Appending to dataframes is inefficient. Instead make the whole data first then make a dataframe from the final data.

TrigonaMinima Over a year ago

look at my answer above. Ask if you face any issue.

TrigonaMinima · Accepted Answer · 2017-08-13 12:46:48Z

0

This will more efficient.

all_values = []
for n in range(0,29):
    condition = (dfallchan[0]>0) & (dfallchan[n]==0)
    count = condition.sum()
    vals = dfallchan[condition][0].values
    all_values.append(vals)

all_values_df = pd.DataFrame(all_values).transpose()

Here, I am first creating a list of lists and appending all the values to it. Then at the end I am creating the dataframe and transposing it.

answered Aug 13, 2017 at 12:46

TrigonaMinima

1,9981 gold badge26 silver badges42 bronze badges

Collectives™ on Stack Overflow

for loop to create a matrix in python

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related