I was wondering if it is possible to drive Flash application programatically to provide some sort of automation for web Flash-based applications that demand user interactions? The naive approach would require taking screenshots and recognizing interactive areas based on some predefined patterns. Then we can emulate mouse clicks/keyboards by sending windows messages directly to the browser.
I'm wondering whether there is a simpler/more subtler way of achieving this.
Are you trying to play a game or run an application on someone else's website which is why you can't use an autohotkey program? In that case, you will have to take some type of screenshot and do pixel analysis to figure out what is happening on screen and what you want to do with that data.
Unfortunatley you won't be able to pull data out of the Flash app that you don't control unless you reverse engineer it and that's pretty unlikely and possibly illegal. If it is your app, then you can simply pass commands to a service that interacts with the Flash app backend, but I'm guessing not.
You aren't thinking of Adobe Flex or Open Laszlo, are you? Would you be thinking of how to develop a programmatically/algorithmically controlled Flash application?
They are both open source products.
A free open source Flash SDK was first offered by Laszlo. And then followed by Adobe themselves open sourcing their SDK.
You should try your hands on either SDKs.
With Flex or Laszlo SDK, you could develop an application having a constant client-server relationship, where the user could be required to provide user inputs like mouse or keyboard clicks on buttons or menus and send requests to the server. The server would be able to respond by sending more data, images, movie clips to the async client.
Alternatively, the client application could be made into a stand-alone (serverless) Flash application - operated in the same user input driven way as the client-server mode. Such a standalone Flash app could be launched from an HTML page or from a Flash viewer.
One of the joys of developing Flash apps is the presence of async sockets. However, with the advent of HTML 5, providing similar capabilities as Flash, Flash technology seems to be on the obsolete end.