Extracting a pattern from python string using regular expression

Question

I have a string from one of the log files as below.

pf_string = "2018-02-01 00:54:49,285 [210.67.123.00]  [ABC,CDE,sfv4_ABC.,dbPool5,11689563,fp2871,en_US]  UNKNOWN-UNKNOWN EVENT-UNKNOWN-UNKNOWN-pc4bcf46t-20180201005446-663570 2994 770 3199 168 26 [Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; ABC-IE11; rv:11.0) like Gecko]     3677610951-0 PERFORMANCE PM_REVIEW FORM_DETAIL [[95211KB 480ms 460ms 20ms 212KB 0KB 118KB 57KB 0 0 ]] 74 139 - - - -   "

Now I want to extract a pattern like below:

Module_id -> PERFORMANCE 
Page Name -> PM_REVIEW 
Page Qualifier -> FORM_DETAIL

Here is a regular expression like below:

perfLogPatternPage = re.compile('(?P<module_id>\w+)\s(?P<page_name>\w+)\s(?P<page_qualifier>\w+)\s\[\[')


print perfLogPatternPage.match(pf_string).group('module_id')
print perfLogPatternPage.match(pf_string).group('page_name')
print perfLogPatternPage.match(pf_string).group('page_qualifier')

But this doesn't seem to work and give the right result.

Can someone suggest what's wrong?

RomanPerekhrest · Accepted Answer · 2018-02-16 17:42:53Z

2

It would be enough to apply re.search() function at once:

import re

pf_string = "2018-02-01 00:54:49,285 [210.67.123.00]  [ABC,CDE,sfv4_ABC.,dbPool5,11689563,fp2871,en_US]  UNKNOWN-UNKNOWN EVENT-UNKNOWN-UNKNOWN-pc4bcf46t-20180201005446-663570 2994 770 3199 168 26 [Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; ABC-IE11; rv:11.0) like Gecko]     3677610951-0 PERFORMANCE PM_REVIEW FORM_DETAIL [[95211KB 480ms 460ms 20ms 212KB 0KB 118KB 57KB 0 0 ]] 74 139 - - - -   "

m = re.search(r'(?P<module_id>\w+)\s+(?P<page_name>\w+)\s+(?P<page_qualifier>\w+)\s(?=\[\[.)', pf_string)
module_id, page_name, page_qualifier = m.groups()

answered Feb 16, 2018 at 17:42

RomanPerekhrest

93.1k4 gold badges75 silver badges112 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ajax1234 · Accepted Answer · 2018-02-16 18:18:38Z

1

You can try this:

import re
pf_string = "2018-02-01 00:54:49,285 [210.67.123.00]  [ABC,CDE,sfv4_ABC.,dbPool5,11689563,fp2871,en_US]  UNKNOWN-UNKNOWN EVENT-UNKNOWN-UNKNOWN-pc4bcf46t-20180201005446-663570 2994 770 3199 168 26 [Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; ABC-IE11; rv:11.0) like Gecko]     3677610951-0 PERFORMANCE PM_REVIEW FORM_DETAIL [[95211KB 480ms 460ms 20ms 212KB 0KB 118KB 57KB 0 0 ]] 74 139 - - - -"
results = dict(zip(['Module_id', 'Page Name', 'Page Qualifier'], re.findall('(?<=\-\d)[a-zA-Z\s_]+(?=\[\[\d)', pf_string)[0].split()))

Output:

{'Module_id': 'PERFORMANCE', 'Page Qualifier': 'FORM_DETAIL', 'Page Name': 'PM_REVIEW'}

edited Feb 16, 2018 at 18:18

answered Feb 16, 2018 at 17:59

Ajax1234

71.7k9 gold badges67 silver badges110 bronze badges

2 Comments

Baktaawar Over a year ago

pf_string is the variable to store the string.

Baktaawar Over a year ago

Thanks. It seems to work. Let me try on few other test strings.

heemayl · Accepted Answer · 2018-02-16 17:38:31Z

You can do:

\d+-\d+\s+(?P<module_id>[A-Z_]+)\s+(?P<page_name>[A-Z_]+)\s+(?P<page_qualifier>[A-Z_]+)

\d+-\d+\s+ matches one or more digits, followed by -, followed by one or more digits, then one or more spaces
The each named captured group matches one or more uppercased-alphabetic characters or underscore
The \s+ in between the captured groups matches one or more spaces

Example:

In [12]: rcomp = re.compile(r'\d+-\d+\s+(?P<module_id>[A-Z_]+)\s+(?P<page_name>[A-Z_]+)\s+(?P<page_qualifier>[A-Z_]+)')

In [13]: out = rcomp.search(pf_string)

In [14]: out.group('module_id')
Out[14]: 'PERFORMANCE'

In [15]: out.group('page_name')
Out[15]: 'PM_REVIEW'

In [16]: out.group('page_qualifier')
Out[16]: 'FORM_DETAIL'

Valdi_Bo · Accepted Answer · 2018-02-16 18:02:35Z

0

Your regex requires a few corrections:

Start from the start of the string (^).
"Consume" three times:
- A sequence of chars other than [.
- [ char.
- A sequence of chars other than ].
- ] char.
"Consume" a sequence of spaces (actually white chars), a sequence of digits or - chars and another sequence of spaces.
Then put your 3 named capturing groups, separated with a sequence of spaces.

So the whole regex can look like below:

^(?:[^\[]+\[[^\]]+\]){3}\s+[-\d]+\s+(?P<module_id>\w+)\s+(?P<page_name>\w+)\s+(?P<page_qualifier>\w+)

For a working example see https://regex101.com/r/e048Q3/1

answered Feb 16, 2018 at 18:02

Valdi_Bo

31.1k4 gold badges29 silver badges45 bronze badges

Collectives™ on Stack Overflow

Extracting a pattern from python string using regular expression

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related