Automatically download sales reports from iTunes Connect

Question

I had a nice and hacky Perl script to automatically scrape and download sales report files from iTunes Connect. As of today, Apple overhauled the sales report site. It looks a lot nicer, but it uses a lot of JavaScript and simple scraping isn't going to work any more.

So, does anybody know of a way to scrape this new site effectively?

Some previous questions point to various scripts and online services. I presume they are all broken now as well. If you know of one that is still functional, please let me know.

Answer 1

Try the free iMacros Firefox addon. It has extensive web scraping support and since it works in the browser, it can handle Javascript. You can start it also via the command line.

Edit: This does indeed work. Here's a macro for downloading the past 3 days of sales reports. I haven't yet tried integrating with command line tools, but it should work.

VERSION BUILD=6650406 RECORDER=FX
TAB T=1
URL GOTO=https://itunesconnect.apple.com/
TAG POS=1 TYPE=INPUT:IMAGE FORM=NAME:appleConnectForm ATTR=NAME:1.Continue&&SRC:https://itunesconnect.apple.com/AppleConnect/US-EN/labelconnect/btn_signin.png
TAG POS=1 TYPE=B ATTR=TXT:Sales<SP>and<SP>Trends
TAG POS=1 TYPE=A ATTR=ID:theForm:saletestid
TAG POS=1 TYPE=SELECT FORM=NAME:theForm ATTR=ID:theForm:datePickerSourceSelectElementSales CONTENT=1
TAG POS=1 TYPE=A ATTR=ID:theForm:downloadLabel2
ONDOWNLOAD FOLDER=~/Downloads/iTCSales/ FILE=Daily-{{!NOW:yyyymmdd}}-1.txt.gz WAIT=YES
TAG POS=1 TYPE=SELECT FORM=NAME:theForm ATTR=ID:theForm:datePickerSourceSelectElementSales CONTENT=2
TAG POS=1 TYPE=A ATTR=ID:theForm:downloadLabel2
ONDOWNLOAD FOLDER=~/Downloads/iTCSales FILE=Daily-{{!NOW:yyyymmdd}}-2.txt.gz WAIT=YES
TAG POS=1 TYPE=SELECT FORM=NAME:theForm ATTR=ID:theForm:datePickerSourceSelectElementSales CONTENT=3
TAG POS=1 TYPE=A ATTR=ID:theForm:downloadLabel2
ONDOWNLOAD FOLDER=~/Downloads/iTCSales FILE=Daily-{{!NOW:yyyymmdd}}-3.txt.gz WAIT=YES
TAG POS=1 TYPE=DIV ATTR=TXT:Done
TAG POS=1 TYPE=DIV ATTR=TXT:Done
TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:signOutForm ATTR=VALUE:Sign<SP>Out

Answer 2

let us know if it actually works in the new format apple has brought up

Answer 3

http://twitter.com/viva/status/24133713255

just need to sift through the ajax cruft. if it's even possible. very annoyed with apple at the moment.

Answer 4

http://code.google.com/p/appdailysales/ was just updated to support the Sept 2010 iTunes connect changes.