Regex Search Replace for Multiple Files in a Directory using Python

Question

Where am I going wrong ??

import os, os.path, re

path = "D:\python-test"
myfiles = os.listdir(path)

REGEXES = [(re.compile(r'dog'), 'cat'),
           (re.compile(r'123'), '789')]
for f in myfiles:

    file_name, file_extension = os.path.splitext(f)

    if file_extension in ('.txt', '.doc', '.odt', '.htm', '.html', '.java'):

        input_file = os.path.join(path, f)

        with open(input_file, "w") as fi:
            for line in fi:
                for search, replace in REGEXES:
                    line = search.sub(replace, line)
                fi.write(line)

Somehow its not working. I want to make the replacements in the current file and not a new file.

Update: How about creating A_reg.java from A.java. Moving A.java to a separate local folder and then renaming A_reg.java back to A.java . Possible ? If yes, please help me out with the code.

As soon as the file is opened in 'w' mode, it is truncated. use r+ or a+ modes. — Ashwini Chaudhary
– Ashwini Chaudhary, Commented Jan 21, 2013 at 17:25

fge · Accepted Answer · 2013-01-21 17:30:57Z

2

This is perfectly normal: you overwrite the files themselves. Write in new files, then rename.

Also, opening the way you do truncates files:

$ cat t.txt 
foo
$ python
>>> f = open("t.txt", "w")
>>> f.close()
>>> exit()
$ cat t.txt
# file is empty!!
$

edited Jan 21, 2013 at 17:30

answered Jan 21, 2013 at 17:23

fge

122k35 gold badges266 silver badges340 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

Debajyoti Das Over a year ago

I want to make the replacements in the current file and not a new file. Please help.

fge Over a year ago

You cannot do that. No program ever does that. Imagine that you write a line which is longer than the original: what do you think will happen?

mgilson Over a year ago

@fge -- You certainly can do that. Of course, as you point out, to get reasonable results, the replacement field needs to be the same size as what it is replacing. (And this is significantly harder to accomplish than simply writing a new file and renaming)

fge Over a year ago

@mgilson sure you can, if you are ready to corrupt your inputs. I am not, however ;)

fge Over a year ago

Well, you don't have to move the original file. Just write to the replacement file, ensure that it is written, close it, then rename. Renaming is an atomic operation.

|

Debajyoti Das · Accepted Answer · 2013-01-21 20:07:37Z

Based on the inputs from Fge. I could make it work by using

from shutil import move

move(output_file, input_file)

Thus the working code will be

import os, os.path, re
from shutil import move

path = "D:\python-test"
myfiles = os.listdir(path)

REGEXES = [(re.compile(r'dog'), 'cat'),
           (re.compile(r'123'), '789')]
for f in myfiles:

file_name, file_extension = os.path.splitext(f)
generated_output_file = file_name + "_regex" + file_extension

if file_extension in ('.txt', '.doc', '.odt', '.htm', '.html', '.java'):

    input_file = os.path.join(path, f)
    output_file = os.path.join(path, generated_output_file)

    with open(input_file, "r") as fi, open(output_file, "w") as fo:
        for line in fi:
            for search, replace in REGEXES:
                line = search.sub(replace, line)
            fo.write(line)

move(output_file, input_file)

Collectives™ on Stack Overflow

Regex Search Replace for Multiple Files in a Directory using Python

2 Answers 2

14 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

14 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related