Use powershell to automate file download from website (dont know the url to file)

Question

I want to automate a daily download of a report from our credit card provider's website.

The file is dynamically generated, so I wont know the actual URL for the file itself.

I'm able to use powershell to navigate and login to the web page, and to click the "Download Report" button. But then the "File Download" dialog box appears, and I cant figure out how to press the "Save" button on that dialog.

It seems like my options are:

• Somehow find that window, and try sending keystrokes

• Somehow find the URL of the file when the dialog appears, and then download directly.

• Find a way to disable "File Download" box in ie. (putting it in trusted sites didnt work)

I'm using:

$ie = New-Object -com "InternetExplorer.Application"

What would you suggest is the best approach?

Answer 1

For this purpose the best option is to ask the provider (if possible) to publish some API.

For this purpose sending keystrokes is very fragile. What after clicking Save? A dialog is opened and you should navigate to a folder and click OK. It means that you need to find the window again. And what if there is some similar window opened? Will your script be able to find the right window? (for other scenarios where you need to hide/show/find windows and send keystrokes I would recommend Autohotkey.)

Other possibilities depend on what site it is. It seems that the file is generated as a response to the POST command. That means that there is no direct link that you could download. And even if there was a link, it is likely that it couldn't be downloaded via Net.WebClient because you wouldn't send any authentication info (session cookies or session id in query string). You would need to parse e.g. the cookies from $ie.Document.cookie and add them to the request.

However, in some cases you could use [System.Net.WebRequest]$webRequest = [System.Net.WebRequest]::Create($url); and specify the POST method and data and wait for response (which would be the file). That would mimic clicking on the submit button. But again, without credentials, it will probably fail.

Without access to the site, it is hard to say something more.

Answer 2

If you absolutely NEED to programmatically interact with an application I'd suggest using something like http://www.codeplex.com/white

This library provides a wrapper to some automation functions that are provided by .NET 3.5

As stated above the best approach is to use Net.WebClient instead of an instance of IE. If you need to reconstruct the GET or POST request you may be able to use Firebug to inspect the page provided by your vendor.