Scraping Flash: accessing background files, perhaps in Mechanize?

Question

Hi there

I'm scraping a website in Flash, writing in Python. I can see in Firebug that the page loads its Flash file and then some background data in an .asmx file.

The background data is what I'm interested in - so how can I get hold of the .asmx file? I already know what it's called.

I can't get at the .asmx file directly, but can I grab it using Mechanize?

--- UPDATE ----

The page I'm scraping is http://www.citroen.co.uk/new-cars/car-range/#/configurator/1C58AF/pop/pre-configuration/

The .asmx file is https://sfg-bpf.servicesgp.mpsa.com/uk/services/ServicePSAGF_Dealer.asmx - I can view it in Firebug.

Answer 1

can I grab it using Mechanize?

I don't believe so. The .asmx extension says that the resource you are accessing is a (SOAP-based) .NET Web service, written in a language such as C# or VB.NET. Normally the .asmx code would return a SOAP response, perhaps to be parsed by the Flash application. But it's hard to see what's going on without a little more detail - for example, whether the .asmx request is a separate Ajax request.

Update: The link to the Flash page doesn't work for me now; it worked once, then subsequent requests get redirected to an error page.

The .asmx page you linked to just shows the entry point to the Web service; you'd have to make a request to a specific entry point with appropriate parameters to get the actual XML data (assuming of course that you're authorised).