Python: HTML regex not matching

Question

I have this code:

reg = re.search('<div class="col result_name">(.*)</div>', html)
print 'Value is', reg.group()

Where 'html' contains something like this:

        <div class="col result_name">
            <h4>Blah</h4>
            <p>
                blah
            </p>
        </div>

But it's not returning anything.

Value is
Traceback (most recent call last):
  File "run.py", line 37, in <module>
    print 'Value is', reg.group()

... and this is why you should NOT 'parse' HTML with regex. — user225312
– user225312, Commented Jan 10, 2011 at 18:40
@A A: No, that is why you should not 'parse' anything with regex without reading the re docs. — John Machin
– John Machin, Commented Jan 10, 2011 at 20:46

Community · Accepted Answer · 2017-05-23 12:18:41Z

6

Don't use regex to parse html. Use a html parser

import lxml.html
doc = lxml.html.fromstring(your_html)
result = doc.xpath("//div[@class='col result_name']")
print result

Obligatory link:

RegEx match open tags except XHTML self-contained tags

edited May 23, 2017 at 12:18

CommunityBot

11 silver badge

answered Jan 10, 2011 at 18:41

nosklo

224k58 gold badges300 silver badges299 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Zeno Over a year ago

I'm getting results like this: [<Element div at 0xb72edc8c>, <Element div at 0xb72edcbc>

nosklo Over a year ago

@Zeno: Yeah, those are all the divs lxml found in your html. The elements. You can print them, or do further parsing with them. For example, try this: for onediv in result: print lxml.html.tostring(onediv, pretty_print=True)

Zeno Over a year ago

Does xpath support regex? I want to do something like (col|row) in there.

Ulrich Schwarz · Accepted Answer · 2011-01-10 18:39:42Z

3

The dot does not neccessarily match newlines in REs, you need the DOTALL flag (?s) for that.

answered Jan 10, 2011 at 18:39

Ulrich Schwarz

7,7351 gold badge39 silver badges49 bronze badges

Comments

Thom Wiggers · Accepted Answer · 2011-01-10 18:40:09Z

2

http://docs.python.org/library/re.html :

The special characters are:

'.' (Dot.) In the default mode, this matches any character except a newline. If the DOTALL flag has been specified, this matches any character including a newline.

answered Jan 10, 2011 at 18:40

Thom Wiggers

7,0822 gold badges41 silver badges66 bronze badges

Collectives™ on Stack Overflow

Python: HTML regex not matching

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related