Parsing HTML "Visually"

Question

OKay I am at loss how to name this question. I have some HTML files, probably written by lord Lucifier himself, that I need to parse. It consists of many segments like this, among other html tags

<p>HeadingNumber</p>
<p style="text-indent:number;margin-top:neg_num ">Heading Text</p>
<p>Body</p>

Notice that the heading number and text are in seperate p tags, aligned in a horizontal line by css. the css may be whatever Lucifier fancies, a mixture of indents, paddings, margins and positions.

However that line is a single object in my business model and should be kept as such. So How do I detect whether two p elements are visually in a single line and process them accordingly. I believe the HTML files are well formed if it helps.

Answer 1

You didn't specify how you were parsing, but this is possible in jQuery since you can determine the offset position of any element from the window origin. Check out the example here.

The code:

$(function() {

function sameHorizon( obj1, obj2, tolerance ) {

    var tolerance = tolerance || 0;

    var obj1top = obj1.offset().top;
    var obj2top = obj2.offset().top;

    return (Math.abs(obj1top - obj2top) <= tolerance);

}

$('p').each(function(i,obj) {
    if ($(obj).css('margin-top').replace('px','') < 0) {
        var p1 = $(obj).prev('p');
        var p2 = $(obj);
        var pTol = 4; // pixel tolerance within which elements considered aligned

        if (sameHorizon(p1, p2, pTol)) {
            // put what you want to do with these objects here
            // I just highlighted them for example
            p1.css('background','#cc0');
            p2.css('background','#c0c');

            // but you can manipulate their contents
            console.log(p1.html(), p2.html());
        }
    }
});

​});

This code is based on the assumption that if a <p> has a negative margin-top then it is attempting to be aligned with the previous <p>, but if you know jQuery it should be apparent how to alter it to meet different criteria.

If you can't use jQuery for your problem, then hopefully this is useful for someone else who is or that you can set something up in jQuery to parse this and output new markup.

Answer 2

You may run irobotsoft web scraper and have a test:

• Open the page in its browser window
• Select and mark the line
• Use menu: Design -> Practice HTQL and see if it can extract the line.

Answer 3

I don't have a ton of experience using it, but if the HTML is well formed and depending on what format you need your parsed data in, you may be able to treat it as an XML doc and use XQuery to parse out your data.

Also open up the HTML in Firefox and see if you can figure out what CSS styles are being applied using Firebug. It may give you a better clue as to how the HTML is being lined up...although it looks like its being done using the 'margin-top:negative_number'...if that's the case I think XQuery should be able to find the elements with that particular style applied.