Ruby MatchData class is repeating captures, instead of including additional captures as it "should"

Question

Ruby 1.9.1, OSX 10.5.8

I'm trying to write a simple app that parses through of bunch of java based html template files to replace a period (.) with an underscore if it's contained within a specific tag. I use ruby all the time for these types of utility apps, and thought it would be no problem to whip up something using ruby's regex support. So, I create a Regexp.new... object, open a file, read it in line by line, then match each line against the pattern, if I get a match, I create a new string using replaceString = currentMatch.gsub(/./, '_'), then create another replacement as whole string by newReplaceRegex = Regexp.escape(currentMatch) and finally replace back into the current line with line.gsub(newReplaceRegex, replaceString) Code below, of course, but first...

The problem I'm having is that when accessing the indexes within the returned MatchData object, I'm getting the first result twice, and it's missing the second sub string it should otherwise be finding. More strange, is that when testing this same pattern and same test text using rubular.com, it works as expected. See results here

My pattern:

(<(?:WEBOBJECT|webobject) (?:NAME|name)=(?:[a-zA-Z0-9]+.)+(?:[a-zA-Z0-9]+)(?:>))

Text text:

<WEBOBJECT NAME=admin.normalMode.someOtherPatternWeDontWant.moreThatWeDontWant>moreNonMatchingText<WEBOBJECT NAME=admin.SecondLineMatch>AndEvenMoreNonMatchingText

Here's the relevant code:

tagRegex = Regexp.new('(<(?:WEBOBJECT|webobject) (?:NAME|name)=(?:[a-zA-Z0-9]+\.)+(?:[a-zA-Z0-9]+)(?:>))+')  

testFile = File.open('RegexTestingCompFix.txt', "r+")
lineCount=0
testFile.each{|htmlLine|
lineCount += 1
puts ("Current line: #{htmlLine} at line num: #{lineCount}")
tagMatch = tagRegex.match(htmlLine)
if(tagMatch)

  matchesArray = tagMatch.to_a  
  firstMatch = matchesArray[0]  
  secondMatch = matchesArray[1]  
  puts "First match: #{firstMatch} and second match #{secondMatch}"  
  tagMatch.captures.each {|lineMatchCapture|  
    puts "Current capture for tagMatches: #{lineMatchCapture} of total match count #{matchesArray.size}"  
    #create a new regex using the match results; make sure to use auto escape method  
    originalPatternString = Regexp.escape(lineMatchCapture)  
    replacementRegex = Regexp.new(originalPatternString)  
    #replace any periods with underscores in a copy of lineMatchCapture  
    periodToUnderscoreCorrection = lineMatchCapture.gsub(/\./, '_')  
    #replace original match with underscore replaced copy within line  
    htmlLine.gsub!(replacementRegex, periodToUnderscoreCorrection)  
    puts "The modified htmlLine is now: #{htmlLine}"    
    }  
end  

}

I would think that I should get the first tag in matchData[0] then the second tag in matchData1, or, what I'm really doing because I don't know how many matches I'll get within any given line is matchData.to_a.each. And in this case, matchData has two captures, but they're both the first tag match

which is: <WEBOBJECT NAME=admin.normalMode.someOtherPatternWeDontWant.moreThatWeDontWant>

So, what the heck am I doing wrong, why does rubular test give me the expected results?

Answer 1

You want to use the on String#scan instead of the Regexp#match:

tag_regex = /<(?:WEBOBJECT|webobject) (?:NAME|name)=(?:[a-zA-Z0-9]+\.)+(?:[a-zA-Z0-9]+)(?:>)/

lines = "<WEBOBJECT NAME=admin.normalMode.someOtherPatternWeDontWant.moreThatWeDontWant>moreNonMatchingText\
     <WEBOBJECT NAME=admin.SecondLineMatch>AndEvenMoreNonMatchingText"

lines.scan(tag_regex)
# => ["<WEBOBJECT NAME=admin.normalMode.someOtherPatternWeDontWant.moreThatWeDontWant>", "<WEBOBJECT NAME=admin.SecondLineMatch>"]

A few recommendations for next ruby questions:

• newlines and spaces are your friends, you don't loose points for using more lines on your code ;-)
• use do-end on blocks instead of {}, improves readability a lot
• declare variables in snake case (hello_world) instead of camel case (helloWorld)

Hope this helps

Answer 2

Small bits: This regexp helps you get "normalMode" .. But not "secondLineMatch":

<webobject name=\w+\.((?:\w+)).+> (with option 'i', for "case insensitive")

This regexp helps you get "secondLineMatch" ... But not "normalMode":

<webobject name=\w+\.((?:\w+))> (with option 'i', for "case insensitive").

I'm not really good at regexpt but I'll keep toiling at it.. :)

And I don't know if this helps you at all, but here's a way to get both:

<webobject name=admin.(\w+) (with option 'i').

Answer 3

I ended up using the String.scan approach, the only tricky point there was figuring out that this returns an array of arrays, not a MatchData object, so there was some initial confusion on my part, mostly due to my ruby green-ness, but it's working as expected now. Also, I trimmed the regex per Trevoke's suggestion. But snake case? Never...;-) Anyway, here goes:

tagRegex = /(<(?:webobject) (?:name)=(?:\w+\.)+(?:\w+)(?:>))/i  
testFile = File.open('RegexTestingCompFix.txt', "r+")  
lineCount=0  
testFile.each do |htmlLine|  
  lineCount += 1  
  puts ("Current line: #{htmlLine} at line num: #{lineCount}")  
    oldMatches = htmlLine.scan(tagRegex) #oldMatches thusly named due to not explicitly using Regexp or MatchData, as in "the old way..."  
    if(oldMatches.size > 0) 
      oldMatches.each_index do |index|   
        arrayMatch = oldMatches[index]  
        aMatch = arrayMatch[0]  
        #create a new regex using the match results; make sure to use auto escape method  
        replacementRegex = Regexp.new(Regexp.escape(aMatch))  
        #replace any periods with underscores in a copy of lineMatchCapture  
        periodToUnderscoreCorrection = aMatch.gsub(/\./, '_')  
        #replace original match with underscore replaced copy within line, matching against the new escaped literal regex  
        htmlLine.gsub!(replacementRegex, periodToUnderscoreCorrection)  
        puts "The modified htmlLine is now: #{htmlLine}"         
      end # I kind of still prefer the brackets...;-)  
    end  
  end

Now, why does MatchData work the way it does? It seems like it's behavior is a bug really, and certainly not very useful in general if you can't get it provide a simple means of accessing all the matches. Just my $.02