i am screenscraping using watir and i download a xls file. when i open this file in notepad, i find that its just a bunch of html tables. is there a any function or gem that will convert this page into a bunch of arrays. any ideas is appreciated.
+1
A:
In general it is a simple exercise to walk through a HTML file with a table and extract rows and columns as long as they don't use colspan or rowspan attributes. Those mess up the logical flow requiring some sensing of the gaps they cause, and a need to fill in the gaps with the repeated value from the *spans. http://stackoverflow.com/questions/2062051/ruby-nokogiri-parsing-html-table-ii might help.
From looking at XLS files on my desktop I don't think they're XML or HTML. I'm not sure what you downloaded. I did a quick search and roo (http://roo.rubyforge.org/) appears to be a good starting point.
Greg
2010-10-16 17:08:40
+1
A:
- Narrow it down to ...
- Clear out the whitespace
- Replace the tabs with "
- Replace tags with ",
- Replace the & & tags with nothing
- Replace the tags with |
- Split the rows with |
- Split the fields with ,
You can simplify it a little bit more, but that's the gist of it.
Dave McNulla
2010-10-16 17:21:40
+1
A:
XLS is a binary format. If you are seeing HTML tables in the file contents it means you probably did not download the file correctly.
How is the XLS file being downloaded through Watir? Are you having to automate the File Download window, or did you just follow a link to the XLS file and write the contents to a file?
JEH
2010-10-22 15:08:38