Regular expression for extracting tag attributes

Question

I'm trying to extract the attributes of a anchor tag (<a>). So far I have this expression:

(?<name>\b\w+\b)\s*=\s*("(?<value>[^"]*)"|'(?<value>[^']*)'|(?<value>[^"'<> \s]+)\s*)+

which works for strings like

<a href="test.html" class="xyz">

and (single quotes)

<a href='test.html' class="xyz">

but not for string without quotes:

<a href=test.html class=xyz>

How can I modify my regex making it work with attributes without quotes? Or is there a better way to do that?

Thanks!

Update: Thanks for all the good comments and advices so far. There is one thing I didn't mention: I sadly have to patch/modify code not written by myself. And there is no time/money to rewrite this stuff from bottom up.

Answer 1

I suggest that you use HTML Tidy to convert the HTML to XHTML, and then use a suitable XPath expression to extract the attributes.

Answer 2

If you want to be general, you have to look at the precise specification of the a tag, like here. But even with that, if you do your perfect regexp, what if you have malformed html?

I would suggest to go for a library to parse html, depending on the language you work with: e.g. like python's Beautiful Soup.

Answer 3

Token Mantra response: you should not tweak/modify/harvest/or otherwise produce html/xml using regular expression.

there are too may corner case conditionals such as \' and \" which must be accounted for. You are much better off using a proper DOM Parser, XML Parser, or one of the many other dozens of tried and tested tools for this job instead of inventing your own.

I don't really care which one you use, as long as its recognized, tested, and you use one.

my $foo  = Someclass->parse( $xmlstring ); 
my @links = $foo->getChildrenByTagName("a"); 
my @srcs = map { $_->getAttribute("src") } @links; 
# @srcs now contains an array of src attributes extracted from the page.

Answer 4

If youre in .NET I recommend the HTML agility pack, very robust even with malformed HTML.

Then you can use XPath.

Answer 5

If you have an element like

<name attribute=value attribute="value" attribute='value'>

this regex could be used to find successively each attribute name and value

(\S+)=["']?((?:.(?!["']?\s+(?:\S+)=|[>"']))+.)["']?

Applied on:

<a href=test.html class=xyz>
<a href="test.html" class="xyz">
<a href='test.html' class="xyz">

it would yield:

'href' => 'test.html'
'class' => 'xyz'

Answer 6

I'd reconsider the strategy to use only a single regular expression. Sure it's a nice game to come up with one single regular expression that does it all. But in terms of maintainabilty you are about to shoot yourself in both feet.

Answer 7

Just to agree with everyone else: don't parse HTML using regexp.

It isn't possible to create an expression that will pick out attributes for even a correct piece of HTML, never mind all the possible malformed variants. Your regexp is already pretty much unreadable even without trying to cope with the invalid lack of quotes; chase further into the horror of real-world HTML and you will drive yourself crazy with an unmaintainable blob of unreliable expressions.

There are existing libraries to either read broken HTML, or correct it into valid XHTML which you can then easily devour with an XML parser. Use them.

Answer 8

Although the advice not to parse HTML via regexp is valid, here's a expression that does pretty much what you asked:

/
   \G                     # start where the last match left off
   (?>                    # begin non-backtracking expression
       .*?                # *anything* until...
       <[Aa]\b            # an anchor tag
    )??                   # but look ahead to see that the rest of the expression
                          #    does not match.
    \s+                   # at least one space
    ( \p{Alpha}           # Our first capture, starting with one alpha
      \p{Alnum}*          # followed by any number of alphanumeric characters
    )                     # end capture #1
    (?: \s* = \s*         # a group starting with a '=', possibly surrounded by spaces.
        (?: (['"])        # capture a single quote character
            (.*?)         # anything else
             \2           # which ever quote character we captured before
        |   ( [^>\s'"]+ ) # any number of non-( '>', space, quote ) chars
        )                 # end group
     )?                   # attribute value was optional
/msx;

"But wait," you might say. "What about *comments?!?!" Okay, then you can replace the . in the non-backtracking section with: (It also handles CDATA sections.)

(?:[^<]|<[^!]|<![^-\[]|<!\[(?!CDATA)|<!\[CDATA\[.*?\]\]>|<!--(?:[^-]|-[^-])*-->)

• Also if you wanted to run a substitution under Perl 5.10 (and I think PCRE), you can put \K right before the attribute name and not have to worry about capturing all the stuff you want to skip over.

Answer 9

a great resource for regular expressions is http://regexlib.com

Answer 10

You cannot use the same name for multiple captures. Thus you cannot use a quantifier on expressions with named captures.

So either don’t use named captures:

(?:(\b\w+\b)\s*=\s*("[^"]*"|'[^']*'|[^"'<>\s]+)\s+)+

Or don’t use the quantifier on this expression:

(?<name>\b\w+\b)\s*=\s*(?<value>"[^"]*"|'[^']*'|[^"'<>\s]+)

This does also allow attribute values like bar=' baz='quux:

foo="bar=' baz='quux"

Well the drawback will be that you have to strip the leading and trailing quotes afterwards.

Answer 11

something like this might be helpful

'(\S+)\s*?=\s*([\'"])(.*?|)\2