Why does preg_match_all poop out after so many characters?

Question

I'm having a problem with my preg_match_all statement. It's been working perfectly as I've been typing out an article but all of a sudden after it passed a certain length is stopped working all together. Is this a known issue with the function that it just doesn't do anything after so many characters?

$number = preg_match_all("/(<!-- ([\w]+):start -->)\n?(.*?)\n?(<!-- \\2:stop -->)/s", $data, $matches, PREG_SET_ORDER);

It's been working fine all this time and works fine for other pages, but once that article passed a certain length, poof, it stopped working for that article. Is there another solution I can use to make it work for longer blocks of text? The article that is being processed is about 33,000 characters in length (including spaces).

I asked a question like this before but got only one answer which I never actually tested. The previous time I had just found another way to get around it for that particular scenario, but this time there is no way to get around it because it's all one article. I tried changing the pcre.backtrack_limit and pcre.recursion_limit up to even 500,000 with absolutely no effect. Are there any other ideas on why this is occurring and what I can do to get it to continue working even for these massive blocks of text? A 30,000 character limit seems to be a bit low, that's only 5,000-6,000 words (this one is about 5,700). Breaking it apart isn't really an option here because it won't find the start and stop if they are in two separate blocks of text.

Answer 1

I bumped into this one once, and the only way I could solve it back then, was by splitting the string. You could explode() or preg_split().

Quoting literally from my source code:

    // regexps have failed miserably on very large tables...
    $parts = explode("<table",$html);

But this was two years ago.

Answer 2

It looks like you're working with HTML. You might want to consider working with one of the various parsers. For example, DOM has a specific class for comments, so we know it can work with them. Unfortunately the DOM is a bit awkward to work with.

Another option might be to use XMLReader, which reads XML as a stream and processes it as tokens along the way. It seems to understand what comments are. I've never used it myself, so I can't tell you how well it works. (You can use DOM's loadHTML and saveXML methods to convert your HTML into XML, assuming it's not too horribly formed.)

Finally, you might consider writing a tokenizer or parser for your custom comments. It shouldn't be too difficult, and may well be faster for you to hack together than learning either of the XML solutions I've pointed out.