0

I have a log file having the following output and I have shortened it as it goes to thousands of lines:

Time = 1

smoothSolver:  Solving for Ux, Initial residual = 0.230812, Final residual = 0.0134171, No Iterations 2
smoothSolver:  Solving for Uy, Initial residual = 0.283614, Final residual = 0.0158797, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.190444, Final residual = 0.016567, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.0850116, Final residual = 0.00375608, No Iterations 3
time step continuity errors : sum local = 0.00999678, global = 0.00142109, cumulative = 0.00142109
smoothSolver:  Solving for omega, Initial residual = 0.00267604, Final residual = 0.000166675, No Iterations 3
bounding omega, min: -26.6597 max: 18468.7 average: 219.43
smoothSolver:  Solving for k, Initial residual = 1, Final residual = 0.0862096, No Iterations 2
ExecutionTime = 4.84 s  ClockTime = 5 s

Time = 2

smoothSolver:  Solving for Ux, Initial residual = 0.0299872, Final residual = 0.00230507, No Iterations 2
smoothSolver:  Solving for Uy, Initial residual = 0.145767, Final residual = 0.00882969, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.0863129, Final residual = 0.00858536, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.394189, Final residual = 0.0175138, No Iterations 3
time step continuity errors : sum local = 0.00862823, global = 0.00212477, cumulative = 0.00354587
smoothSolver:  Solving for omega, Initial residual = 0.00258475, Final residual = 0.000222705, No Iterations 3
smoothSolver:  Solving for k, Initial residual = 0.112805, Final residual = 0.0054572, No Iterations 3
ExecutionTime = 5.9 s  ClockTime = 6 s

Time = 3

smoothSolver:  Solving for Ux, Initial residual = 0.128298, Final residual = 0.0070293, No Iterations 2
smoothSolver:  Solving for Uy, Initial residual = 0.138825, Final residual = 0.0116437, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.0798979, Final residual = 0.00491246, No Iterations 3
GAMG:  Solving for p, Initial residual = 0.108748, Final residual = 0.00429273, No Iterations 2
time step continuity errors : sum local = 0.0073211, global = -0.00187909, cumulative = 0.00166678
smoothSolver:  Solving for omega, Initial residual = 0.00238456, Final residual = 0.000224435, No Iterations 3
smoothSolver:  Solving for k, Initial residual = 0.0529661, Final residual = 0.00280851, No Iterations 3
ExecutionTime = 6.92 s  ClockTime = 7 s

I need to extract Time = 1, 2, 3 and corresponding cumulative values using Python's regular expression. More precisely, I need to extract only the values 1, 2, 3 and 0.00142109, 0.00354587, 0.00166678 that corresponds to cumulative at Time = 1,2 and 3 and write to an another file.

Currently, this is what I have:

contCumulative_0_out = open('contCumulative_0', 'w+')

with open(logFile, 'r') as logfile_read:
for line in logfile_read:
    line = line.rstrip()
    iteration_time = re.findall(r'^Time = ([0-9]+)', line)
    print iteration_time
    contCumulative_0 = re.search(r'cumulative = ((\d|.)+)', line)
    if contCumulative_0:        
        cumvalue = contCumulative_0.groups(1)
        contCumulative_0_out.write('\n'.join(cumvalue))

The variable iteration_time grabs the Time value, however this is not available within the next subsequent if loop and therefore I am unable to combine Time and cumulative to give me 1 0.00142109 in the output file.

Improve this question
3
  • Do the 'Time'/'time step' linesoccur only once in the logfile, or multiple times? Please update the example with those, like @Andy did. Commented Jan 19, 2015 at 3:13
  • @smci My log file goes into many thousands of lines and is identical to what Andy has put up. I will update my original post. Commented Jan 19, 2015 at 3:36
  • Yes please just post enough snippet needed for this question Commented Jan 19, 2015 at 3:40

3 Answers 3

Reset to default
1

When there is no 'Time' or 'cumulative' in this line, there is no need to overwrite that variable. You can do this:

...
with open(logFile, 'r') as logfile_read:
for line in logfile_read:
    line = line.rstrip()
    if 'Time' in line:
        iteration_time = re.findall(r'^Time = ([0-9]+)', line)
        print iteration_time
    if 'cumulative' in line:
        contCumulative_0 = re.search(r'cumulative = ((\d|.)+)', line)
        if contCumulative_0:
            cumvalue = contCumulative_0.groups(1)
            contCumulative_0_out.write('\n'.join(cumvalue))
...
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thanks every one. @m170897017 you solution worked quite well. I now have tuples and need to convert to string in order to get the right format.
Yes @m170897017 I have adopted your solution and many thanks for it.
1

Your code is writing over iteration_time in every iteration of the for loop. That is the problem. You will need to stop populating this variable after it has been successfully populated for the first find.

To do this, in the for loop do a test for iteration_time and only if it is non- existent or None do the regex search for Time. You can do soemthing like this:

contCumulative_0_out = open('contCumulative_0', 'w+')

with open(logFile, 'r') as logfile_read:
    iteration_time = None
    for line in logfile_read:
        line = line.rstrip()
        time_match = re.findall(r'^Time = ([0-9]+)', line)
        if time_match:
            iteration_time = time_match
            print iteration_time
        else:  # Because if there is time_match, there is no 'cumulative = ...'
            contCumulative_0 = re.search(r'cumulative = ((\d|.)+)', line)
            if contCumulative_0:        
                cumvalue = contCumulative_0.groups(1)
                # You can check and use iteration_time here
                contCumulative_0_out.write('\n'.join(cumvalue))

Hope this helps.

Improve this answer

Comments

1

You can do this with a regex, assuming that your log format is the same for all of your entries. The explanation of what is going on is below:

import re

s = """Time = 1

smoothSolver:  Solving for Ux, Initial residual = 0.230812, Final residual = 0.0134171, No Iterations 2
smoothSolver:  Solving for Uy, Initial residual = 0.283614, Final residual = 0.0158797, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.190444, Final residual = 0.016567, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.0850116, Final residual = 0.00375608, No Iterations 3
time step continuity errors : sum local = 0.00999678, global = 0.00142109, cumulative = 0.00142109
smoothSolver:  Solving for omega, Initial residual = 0.00267604, Final residual = 0.000166675, No Iterations 3
bounding omega, min: -26.6597 max: 18468.7 average: 219.43
smoothSolver:  Solving for k, Initial residual = 1, Final residual = 0.0862096, No Iterations 2
ExecutionTime = 4.84 s  ClockTime = 5 s

Time = 2

smoothSolver:  Solving for Ux, Initial residual = 0.230812, Final residual = 0.0134171, No Iterations 2
smoothSolver:  Solving for Uy, Initial residual = 0.283614, Final residual = 0.0158797, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.190444, Final residual = 0.016567, No Iterations 2
GAMG:  Solving for p, Initial residual = 0.0850116, Final residual = 0.00375608, No Iterations 3
time step continuity errors : sum local = 0.00999678, global = 0.00142109, cumulative = 0.00123456
smoothSolver:  Solving for omega, Initial residual = 0.00267604, Final residual = 0.000166675, No Iterations 3
bounding omega, min: -26.6597 max: 18468.7 average: 219.43
smoothSolver:  Solving for k, Initial residual = 1, Final residual = 0.0862096, No Iterations 2
ExecutionTime = 4.84 s  ClockTime = 5 s
"""

regex = re.compile("^Time = (\d+?).*?cumulative = (\d{0,10}\.\d{0,10})",re.DOTALL|re.MULTILINE)

for x in re.findall(regex,s):
    print "{} => {}".format(x[0], x[1])

This outputs two results (because I've added two log entries, instead of just the one you provided):

1 => 0.00142109
2 => 0.00123456

What is happening?

The RegEx being utilized is this:

^Time = (\d+?).*?cumulative = (\d{0,10}\.\d{0,10})

This Regex is looking for your Time = string at the beginning of the line, and matching the digit that follows. Then it does a non-greedy match to the string cumulative = and captures the digits that follow that. The non-greedy is important, otherwise you'd only get one result in your entire log because it'd match the first instance of Time = and the last instance of cumulative =.

It then prints each result. Each captured result contains the time value and the cumulative value. This portion of the code can be modified to print to a file if required.

This regex works across multiple lines because it utilizes two flags: DOTALL and MULTILINE

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.