fetch data using regex

Question

I am trying to fetch data using regex

logdata='146.204.224.152 - feest6811 [21/Jun/2019:15:45:24 -0700] "POST /incentivize HTTP/1.1" 302 4622\n197.109.77.178 - kertzmann3129 [21/Jun/2019:15:45:25 -0700] "DELETE /virtual/solutions/target/web+services HTTP/2.0" 203 26554'
pattern="""(?P<host>.*)( - \ )(?P<user_name>\w*)"""
for item in re.finditer(pattern,logdata,re.VERBOSE):
    print(item.groupdict())

The output looks like

{'host': '146.204.224.152 ', 'user_name': 'feest6811'}
{'host': '197.109.77.178 ', 'user_name': 'kertzmann3129'}

But I want

{'host': '146.204.224.152', 'user_name': 'feest6811', 'time': '21/Jun/2019:15:45:24 -0700', 'request': 'POST /incentivize HTTP/1.1'}
{'host': '197.109.77.178', 'user_name': 'kertzmann3129', 'time': '21/Jun/2019:15:45:25 -0700', 'request': 'DELETE /virtual/solutions/target/web+services HTTP/2.0'}

shouldn't it be easier to change the log format for better parsing ? or create a seperate log file for that. Also you should NOT!!! post public IP Adresses, you can get sued from the perons because you violate the DSGVO. — Marc
– Marc, Commented Sep 8, 2020 at 12:56
Regex101 is invaluable for problems like yours: regex101.com/r/xM2aYZ/1 — Code Different
– Code Different, Commented Sep 8, 2020 at 13:00

Wiktor Stribiżew · Accepted Answer · 2020-09-08 13:39:13Z

2

You can use

r'(?P<host>[\d.]+)\ -\ (?P<user_name>\w+)\ \[(?P<time>[^][]+)]\ "(?P<request>[^"]+)"'

See the regex demo

Details

(?P<host>[\d.]+) - Group "host": one or more digits / dots
\ -\ - a string
(?P<user_name>\w+) - Group "user_name": one or more word chars
\ \[ - space + [
(?P<time>[^][]+) - Group "time": one or more chars other than ] and [
]\ " - ] " substring
(?P<request>[^"]+) - Group "request": one or more chars other than a "
" - a " char.

edited Sep 8, 2020 at 13:39

answered Sep 8, 2020 at 13:18

Wiktor Stribiżew

631k41 gold badges502 silver badges632 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Kanika Singhal Over a year ago

r'(?P<host>[\d.]+)\ -\ (?P<user_name>\w+)\ [(?P<time>[^][]+)]\ "(?P<request>[^"]+)"'

Wiktor Stribiżew Over a year ago

@KanikaSinghal Ok, you might not need the start of string check.

Collectives™ on Stack Overflow

fetch data using regex

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related