Programmatically automating a web login

Question

I am trying to create a C# Winforms application that will automatically log me into a site and download data. Specifically, I want to have my application automatically log into my online banking site, log me in, and download my transaction history. I can do this manually by logging in through a web browser and downloading it. I am trying to automate this. I know I probably need to use HttpWebRequest and HttpWebResponse. Does anyone have an example of this or a framework of the steps I need to take to accomplish this? Keep in mind it will be secure site (https) and I will somehow have to collect session information and retain the session information for the duration of the session. Any thoughts?

Answer 1

using System;

using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Net;
using System.IO;

namespace testSSL
{
    public partial class FormDownload : Form
    {
        private bool success;
        private const string filename = "file.txt";
        private const string url_string = "https://some.url.com";
        private Uri url;
    public FormDownload()
    {
        InitializeComponent();
        success = false;
        url = new Uri(url_string);
    }

    public bool StartDownload()
    {
        this.ShowDialog();
        return success;
    }

    private void Form1_Load(object sender, EventArgs e)
    {
        this.Activate();

        progressBar1.Maximum = 100;
        label1.Text = "Working";

        WebClient client = new WebClient();
        client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(client_DownloadProgressChanged);
        client.DownloadFileCompleted += new AsyncCompletedEventHandler(client_DownloadFileCompleted);

        //possible fix for running on w2k
        ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;

        string user="user", pass="pass";
        client.Credentials = new NetworkCredential(user, pass);
        try
        {
            client.DownloadFileAsync(url, filename);
        }
        catch (Exception ue)
        {
            writeException(ue.Message);
        }

    }

    void client_DownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
    {
        if (e.Error != null)
        {
            writeException(e.Error.Message);
            success = false;
        }
        else
        {
            label1.Text = "Done";
            System.Threading.Thread.Sleep(100);
            success = true;
        }
        this.Close();
    }

    void client_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
    {
        progressBar1.Value = e.ProgressPercentage;
    }

    private void writeException(string ex)
    {
        ex = "Date: " + DateTime.Now.ToString() + " Exception: " + ex + "\r\n";
        File.AppendAllText("downloadLog.txt", ex);
        MessageBox.Show("An error has occurred; it has been logged");
        this.Close();
    }
}

}

Answer 2

Have a look at Selenium, with that you can automate a sequence of interactions between user and browser.

You might be lucky in just being able to use web request and response to login, though many banks are making the move to javascript based login forms to obfuscate passwords in order to prevent trojans. See Citibank (AU) and Westpac (AU). It might be difficult enough to circumvent that you may have to resort to logging in manually and having a GreaseMonkey script automate the downloading.

For interests sake, it's also worth doing some research on banking trojans and how they handle the automated actions on behalf of a user. See Zeus Banking Trojan.

Answer 3

Browser automation (see link below) may be useful.

But remember... a login page is really a complex client application, capable of forming a complex, even encrypted web request. So, by circumventing what you may perceive to be just an interface, you're actually circumventing an entire client app (which could formulate a simple HTTP POST or could perform some complex JavaScript manipulation followed by connection to a flash player and then a direct connection to a login server, but you get the point). The login interface (really potentially a small app nowadays), could be updated drastically at any time, invalidating your automated login software.

So... you may want to use something that can automate it at a high level, working with the available interface (rather than circumventing it and attempting to formulate your own HTTP requests), and something like http://seleniumhq.org/ may help.

Just be careful writing software that blindly sends your credentials to a web page. You're not there to monitor it when there's a hijacking of the page or the page suddenly isn't encrypted for some reason. Just a thought.

Answer 4

Look into Open Financial Exchange specification. That is how Quicken/Money etc download transactions from your financial institutions.