python, regex, matching strings with repeating characters

Question

I am trying to search Apache log files for specific entries related to specific vulnerability scans. I need to match strings from a separate file against the URI content in the weblogs. Some of the strings I am trying to find contain repeating special characters like '?'.

For example, I need to be able to match an attack that contains just the string '????????' but I don't want to be alerted on the string '??????????????????' because each attack is tied to a specific attack ID number. Therefore, using:

if attack_string in log_file_line:
    alert_me()

...will not work. Because of this, I decided to put the string into a regex:

if re.findall(r'\%s' % re.escape(attack_string),log_file_line):
    alert_me()

...which did not work either because any log file line containing the string '????????' is matched even if there are more than 8 '?' in the log file line.

I then tried adding boundaries to the regex:

if re.findall(r'\\B\%s\\B' % re.escape(attack_string),log_file_line):
    alert_me()

...which stopped matching in both cases. I need to be able to dynamically assign the string I am looking for but I don't want to match on just any line that contains the string. How can I accomplish this?

If you use raw string, you shouldn't double up the \. Check whether this is the problem. — nhahtdh
– nhahtdh, Commented Oct 4, 2012 at 1:36
r'\?\?\?\?\?\?\?\?(?!\?)' Not quite sure what you're asking. — Joel Cornett
– Joel Cornett, Commented Oct 4, 2012 at 2:17
If there is whitespace after the successfully found string, this regex would cut it re.findall(r'\%s\s' % re.escape(attack_string),log_file_line) — Tadgh
– Tadgh, Commented Oct 4, 2012 at 2:24
@Mozoby and Tadgh - There isn't always going to be a whitespace character after the string. Also, sometimes there may be more than one string that needs to match on a particular log file line before alerting. Each signature contains one or more string(s) that must be matched in order. On each iteration, a new string from the signature stack is placed in attack_string and compared to log_file_line. Since "GET /vulnerable.page????????" and "GET /vulnerable.page/????????/gotya.exploit" and "GET /vulnerable.page" are all different signatures, I can't necessarily count on whitespaces. — user1718725
– user1718725, Commented Oct 4, 2012 at 5:55

Toto · Accepted Answer · 2012-10-04 08:12:14Z

How about:

(?:[^?]|^)\?{8}(?:[^?]|$)

Explanation:

(?-imsx:(?:[^?]|^)\?{8}(?:[^?]|$))

matches as follows:

NODE                     EXPLANATION
----------------------------------------------------------------------
(?-imsx:                 group, but do not capture (case-sensitive)
                         (with ^ and $ matching normally) (with . not
                         matching \n) (matching whitespace and #
                         normally):
----------------------------------------------------------------------
  (?:                      group, but do not capture:
----------------------------------------------------------------------
    [^?]                     any character except: '?'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    ^                        the beginning of the string
----------------------------------------------------------------------
  )                        end of grouping
----------------------------------------------------------------------
  \?{8}                    '?' (8 times)
----------------------------------------------------------------------
  (?:                      group, but do not capture:
----------------------------------------------------------------------
    [^?]                     any character except: '?'
----------------------------------------------------------------------
   |                        OR
----------------------------------------------------------------------
    $                        before an optional \n, and the end of
                             the string
----------------------------------------------------------------------
  )                        end of grouping
----------------------------------------------------------------------
)                        end of grouping
----------------------------------------------------------------------

Collectives™ on Stack Overflow

python, regex, matching strings with repeating characters

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related