web scraping/parsing of college course site

Question

Trying to parse/scrape the course site for memphis. The site is "https://spectrumssb2.memphis.edu/pls/PROD/bwckgens.p_proc_term_date". It appears to be some sort of javascript issue, or dynamic generation of the text. I can see the underlying DOM structure using livehttpdheaders/Firefox, but not when I simply view the underlying source/text of the page..

Thoughts/Comments/Pointers would be appreciated...

Answer 1

Well this modern days the site may be assembled in few steps. First the main structure is pulled in and then, often based on identity of the user additional AJAX calls are executed. Your best bet is to sniff HTTP to see what kind of requests are issued between the site is initially requested and when it's fully built

Since you are using firebug you can get HttpFox add-on which gives you what you need