ansaurus

Question

Answer 1

+2 A:

I've heard that Beautiful Soup is very good at parsing html. It will probably be able to help you get correct html out.

recursive 2008-12-11 16:58:58

Answer 2

A:

I was going to mention the base HTMLParser that's built in Python, since I'm not sure what the end-result your trying to get to is, it may or may not get you there, you'll work with the handlers primarily

curtisk 2008-12-11 17:07:16

Answer 3

+3 A:

Take a look at the truncate_html_words function in django.utils.text. Even if you aren't using Django, the code there does exactly what you want.

Carl Meyer 2008-12-11 18:03:44

Answer 4

A:

You can use a mix of regex, BeautifulSoup or Tidy (I prefer BeautifulSoup). The idea is simple - strip all the HTML tags first. Find the nth word (n=7 here), find the number of times the nth word appears in the string till n words - coz u are looking only for the last occurrence to be used for truncation.

Here is a piece of code, though a bit messy but works

import re
from BeautifulSoup import BeautifulSoup
import tidy

def remove_html_tags(data):
    p = re.compile(r'<.*?>')
    return p.sub('', data)

input_string='<p>This is some text with a <a href="http://www.example.com/" '\
    'title="Example link">bit of linked text in it</a></p>'

s=remove_html_tags(input_string).split(' ')[:7]

###required to ensure that only the last occurrence of the nth word is                                                                                      
#  taken into account for truncating.                                                                                                                       
#  coz if the nth word could be 'a'/'and'/'is'....etc                                                                                                       
#  which may occur multiple times within n words                                                                                                            
temp=input_string
k=s.count(s[-1])
i=1
j=0
while i<=k:
    j+=temp.find(s[-1])
    temp=temp[j+len(s[-1]):]
    i+=1
####                                                                                                                                                        
output_string=input_string[:j+len(s[-1])]

print "\nBeautifulSoup\n", BeautifulSoup(output_string)
print "\nTidy\n", tidy.parseString(output_string)

The output is what u want

BeautifulSoup
<p>This is some text with a <a href="http://www.example.com/" title="Example link">bit</a></p>

Tidy
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<meta name="generator" content=
"HTML Tidy for Linux/x86 (vers 6 November 2007), see www.w3.org">
<title></title>
</head>
<body>
<p>This is some text with a <a href="http://www.example.com/"
title="Example link">bit</a></p>
</body>
</html>

Hope this helps

Edit: A better regex

`p = re.compile(r'<[^<]*?>')`

JV 2008-12-11 18:24:11

ansaurus

tags:

views:

answers:

Split HTML after N words in python

related questions