tags:

views:

93

answers:

3

I'm parsing some text from an XML file which has sentences like "Subtract line 4 from line 1.", "Enter the amount from line 5" i want to replace all occurrences of line with line_ eg. Subtract line 4 from line 1 --> Subtract line_4 from line_1

Also, there are sentences like "Are the amounts on lines 4 and 8 the same?" and "Skip lines 9 through 12; go to line 13." I want to process these sentences to become "Are the amounts on line_4 and line_8 the same?" and "Skip line_9 through line_12; go to line_13."

A: 

This works for the specific examples including the ones in the OP comments. As is often the case when using regex to do parsing, it becomes a hodge-podge of additional cases and tests to handle ever-increasing known inputs. This handles the lists of line numbers using a while loop with a non-greedy match. As written, it is simply processing an input line-by-line. To get series of line numbers across line boundaries, it would need to be changed to process it as one chunk with matching across lines.

open( ARGV[0], "r" ) do |file|
  while ( line = file.gets )
    # replace both "line ddd" and "lines ddd" with line_ddd 
    line.gsub!( /(lines?\s)(\d+)/, 'line_\2' )
    # Now replace the known sequences with a non-greedy match
    while line.gsub!( /(line_\d+[a-z]?,?)(\sand\s|\sthrough\s|,\s)(\d+)/, '\1\2line_\3' )
    end
    puts line
  end
end

Sample Data: For this input:

Subtract line 4 from line 1.
Enter the amount from line 5
on lines 4 and 8 the same?
Skip lines 9 through 12; go to line 13.
... on line 10 Form 1040A, lines 7, 8a, 9a, 10, 11b, 12b, and 13
Add lines 2, 3, and 4

It produces this output:

Subtract line_4 from line_1.
Enter the amount from line_5
on line_4 and line_8 the same?
Skip line_9 through line_12; go to line_13.
... on line_10 Form 1040A, line_7, line_8a, line_9a, line_10, line_11b, line_12b, and line_13
Add line_2, line_3, and line_4
Mark Wilkins
+2  A: 

Here's a working implementation with rspec test. You call it like this: output = LineIdentifier[input]. To test, spec file.rb after installing rspec gem.

require 'spec'

class LineIdentifier
  def self.[](input)
    output = input.gsub /line (\d+)/, 'line_\1'
    output.gsub /lines (\d+) (and|from|through) (line )?(\d+)/, 'line_\1 \2 line_\4'
  end
end

describe "LineIdentifier" do
  it "should identify line mentions" do
    examples = { 
      #Input                                         Output
     'Subtract line 4 from line 1.'               => 'Subtract line_4 from line_1.',
     'Enter the amount from line 5'               => 'Enter the amount from line_5',
     'Subtract line 4 from line 1'                => 'Subtract line_4 from line_1',
    }
    examples.each do |input, output|
      LineIdentifier[input].should == output
    end
  end
  it "should identify line ranges" do
    examples = { 
      #Input                                         Output
     'Are the amounts on lines 4 and 8 the same?' => 'Are the amounts on line_4 and line_8 the same?',
     'Skip lines 9 through 12; go to line 13.'    => 'Skip line_9 through line_12; go to line_13.',
    }
    examples.each do |input, output|
      LineIdentifier[input].should == output
    end
  end
end
Mark Thomas
Explanation for the downvote?
Mark Thomas
Hi Mark, i've come across a few lines in my xmls as "Enter the total of the amounts from Form 1040A, lines 7, 8a, 9a, 10, 11b, 12b, and 13"and"Add lines 2, 3, and 4"How should i deal with these?
charudatta
i have come as far as using (\d+\w?) for the line numbers
charudatta
A: 

sed is your friend:

lines.sed:

#!/bin/sed -rf
s/lines? ([0-9]+)/line_\1/g
s/\b([0-9]+[a-z]?)\b/line_\1/g

lines.txt:

Subtract line 4 from line 1.
Enter the amount from line 5
Are the amounts on lines 4 and 8 the same?
Skip lines 9 through 12; go to line 13.
Enter the total of the amounts from Form 1040A, lines 7, 8a, 9a, 10, 11b, 12b, and 13
Add lines 2, 3, and 4

demo:

$ cat lines.txt | ./lines.sed
Subtract line_4 from line_1.
Enter the amount from line_5
Are the amounts on line_4 and line_8 the same?
Skip line_9 through line_12; go to line_13.
Enter the total of the amounts from Form 1040A, line_7, line_8a, line_9a, line_10, line_11b, line_12b, and line_13
Add line_2, line_3, and line_4

You can also make this into a sed one-liner if you prefer, although the file is more maintainable.

bukzor