How to Run script for multiple text files with a common string?

Question

In my research work I have multiple text files has a common string "Max", and Max has different values in range of 0.10 to 2.00 with step 0.10 as follow:

 A_100Hz_Rate20Hz_5tot_0.10Max_1_ 
 A_100Hz_Rate20Hz_5tot_0.10Max_2_
 A_100Hz_Rate20Hz_5tot_0.10Max_3_
 .
 .
 .
 A_100Hz_Rate20Hz_5tot_2.00Max_1_ 
 A_100Hz_Rate20Hz_5tot_2.00Max_2_
 A_100Hz_Rate20Hz_5tot_2.00Max_3_

I need to import all files depending on the value of Max (ex: 0.10Max) to get the average of files with the same Max values separately to get:

 Ave_A_100Hz_Rate20Hz_5tot_0.10Max_3_
 .
 .
 .
 Ave_A_100Hz_Rate20Hz_5tot_2.00Max_3_

I’ve tried a manual glob module, and it's working good for one value of "Max" but for the full range it doeson't work. This is my code:

import numpy as np
import glob
import pandas as pd

h = np.linspace(0.10,2.00,20)  
for x in h: 
     x1 = ("%.2f" % x)
     glob_path = 'input/*_{}Vbr_*.txt'.format(x1)
     import_files = glob.glob(glob_path)
     print(x,import_files )
     for index, file_name in enumerate(import_files ):
          merged_data = pd.read_csv(file_name, header=None, delimiter="\t").values
          if index==0:    
               summation = merged_data
          else:
               summation = summation + merged_data
          averaging = summation/len(import_files)         
          np.savetxt('output/Ave_'+file_name[10:], averaging, delimiter="\t" )

I need to write a general script. But, in my case now I used the script with just two values x = 1.50 and x = 2.0 to make it simple. I tried print(import_files) and expected the output to be:

['input\\A_100Hz_Rate20Hz_5tot_1.50Max_1_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_1.50Max_2_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_1.50Max_3_.txt']


['input\\A_100Hz_Rate20Hz_5tot_2.00Max_1_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_2.00Max_2_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_2.00Max_3_.txt']

But the actual output is (in short):

0.1 []
0.2 []
1.5 ['input\\A_100Hz_Rate20Hz_5tot_1.50Max_1_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_1.50Max_2_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_1.50Max_3_.txt']
1.6 []
1.7 []
2.0['input\\A_100Hz_Rate20Hz_5tot_2.00Max_1_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_2.00Max_2_.txt', 
'input\\A_100Hz_Rate20Hz_5tot_2.00Max_3_.txt']

and it caused an error in the kernel

      np.savetxt('output/Ave_'+file_name[10:], averaging, delimiter="\t" )

 NameError: name 'file_name' is not defined

Please, Any suggestions?

As far as the actual output is concerned, you will get empty list ([ ]) for cases where there is no match with glob pattern. For NameError: name 'file_name' is not defined, I am guessing it is some indentation issue with respect to inner for loop. How did the code go pass "pd.read_csv(file_name..." line which also uses same variable (file_name) — ranjith
– ranjith, Commented Mar 25, 2019 at 16:26
Just to elaborate on ranjith's comment, you should replace the print(import_files ) with print(x, import_files) to make more evident for which x value you get empty lists. — Serge Ballesta
– Serge Ballesta, Commented Mar 26, 2019 at 8:54
@ranjith the code is working good without my manual glob for the full range h = np.linspace(0.10,2.00,20) for x in h: x1 = ("%.2f" % x) glob_path = 'input/*{}Vbr*.txt'.format(x1) — H. H
– H. H, Commented Mar 26, 2019 at 9:15
@SergeBallesta - print(x, import_files) gives me: 0.1 [ ] , 0.2 [ ] and so on... — H. H
– H. H, Commented Mar 26, 2019 at 9:33

Serge Ballesta · Accepted Answer · 2019-03-26 10:38:26Z

2

I think that you have just to test whether import_file is empty:

for x in h: 
     x1 = ("%.2f" % x)
     glob_path = 'input/*_{}Vbr_*.txt'.format(x1)
     import_files = glob.glob(glob_path)
     print(x,import_files )
     if len(import_files) != 0:
         for index, file_name in enumerate(import_files ):
              merged_data = pd.read_csv(file_name, header=None, delimiter="\t").values
              if index==0:    
                   summation = merged_data
              else:
                   summation = summation + merged_data
          averaging = summation/len(import_files)         
          np.savetxt('output/Ave_'+file_name[10:], averaging, delimiter="\t" )

edited Mar 26, 2019 at 10:38

answered Mar 26, 2019 at 10:23

Serge Ballesta

150k13 gold badges137 silver badges267 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

H. H Over a year ago

it gave me error if (len(import_files != 0): ^ SyntaxError: invalid syntax

Collectives™ on Stack Overflow

How to Run script for multiple text files with a common string?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related