How can a program control another program?

Question

Yesterday while playing videos games I started to think about player made bots that would play for people and I wondered to myself how they work? Do they tell the video game that a key was pressed or mouse was clicked? If not is there a way to have your program tell another program a key was pressed? I know this is a rather dangerous topic so you can say as much as you feel comfortable with, even if its just an overview of how a bot works. I would like to make a project out of this though, where I try to make a program to beat the first level of some game. So any resources or examples you are willing to tell other people would be appreciated.

Update: So it seems clear that one of main ways to control another program is to emulate keystrokes, so what are some methods to do this (in any language)?

Answer 1

There are a couple of kinds of bots. From what I remember when I used to play CS (a long, long time ago)

Some bots use the libraries of the application itself to inspect the environment (the map, where players are etc...) The bots also use the library to control movement. I doubt there are many bot implementations that invoke mouse messages to tell the game how to move etc...

Some bots emulate the application level protocol (if it's multi-player). So in other words, the game doesn't even need to run. A bot can run in a virtual game.

I'm going to run counter to what one other poster has suggested: writing a game bot is probably a more valuable exercise than actually playing the game, and being curious about how they work is a good sign. :)

Answer 2

A bot like you describe it will simply emulate key presses and mouse movements/clicks, so the game cannot tell if a human is playing or another program. It would not have to interface with the game at all.

Answer 3

Some programs (such as ones that grind for you in an MMORPG) are simply emulating keystrokes. A tool to do this is AutoIt.

As for bots that actually play the games themselves, I've not really done anything with them, but I'm assuming they would use some sort of predefined set of actions, maybe a heuristic, but not likely.

Answer 4

Bots will be acting as clients to the game (server). Then, they can send appropriate data just as a user may be sending manually. The bot(client) will be analysing the data from the server ("someone is about to attack from the left"), et cetera. The client should be then able to compute the best move and send the appropriate data to the server to execute it.

I am not sure if this will work with all games.

Answer 5

I've written a bunch of bots at one time or another (from Pogo games to Yohoho Puzzle Pirates). For windows, you're usually going to either be sending Win32 events to simulate mouse movements, or spoof the actually low-level messages sent between windows when the mouse is actually clicked. A lot of it really depends on how the program reacts (by accepting the message with the coordinates, or, in Java's case, immediately reading the mouse coordinates). The "automation" part usually involves reading the screen and writing heuristics or algorithms for determining the state, but can also be as nice as packet sniffing (a lot of information there in poor poker implementations) or as hacky as reading memory locations directly. Pretty big "field", and poorly documented as it's pretty profitable and not hard to get into.

Sending Input

C/C++ (in Windows)

For keys, try CodeProject:

http://www.codeproject.com/KB/cpp/sendkeys_cpp_Article.aspx

And messages:

http://www.codeproject.com/KB/threads/sendmsg.aspx

Your best bet is to learn to send messages using the Win32 API, then use something like Spy++ or its derivatives to "reverse engineer" how KeyPresses and mouse movements are sent to the window.

Java

Java has an amazingly portable Robot class that is able to:

1. Read Pixels from the screen.
2. Control the mouse.
3. Send keys.

I'd give that a shot if you're looking for quick and easy.

Basic Logic

This is described elsewhere on the internet in depth, but most bots follow a simple state-machine program flow. You read the screen (or packets, or memory), find out what "state" you're in based on your readings and past data, do calculations, and send the result back out to the program.

Reading the screen can be difficult, but can be made easier if you consider that a lot of times, there are a few "lucky" pixels relative to the window that will give you an idea of what state the program is in. The process of finding these pixels can be automated.

Answer 6

You may find interesting this blog post: How I Built a Working Poker Bot, Part 1.

Also, a good book about making bots: Exploiting Online Games: Cheating Massively Distributed.Systems.

Answer 7

The biggest flaw in this concept is not the input, but the game output. Your bot can't just randomly press keys, it needs to know what is "happening". Very few games allow you to query this data which would leave your bot to solve a very tricky problem - Converting 60 frames per second of 2D visual data into a virtual space.

There are actuially games out there that not only allow bot players, but encourage them. Open RTS is one of those but there are also simpler ones like GUN-TACTYX and crobots which are good starting points.

Answer 8

It very much depends on the game. For example I made a bot for Travian (which is an online game) that used the internet explorer activeX to automate moves. I worte the application in c# and to get it to do stuff took about 30 minute. It's simple to call methods on the activeX an make it do things like fill in text fields or click on buttons. For login for example i used the folowing :

var inputs = web.Document.GetElementsByTagName("input").
                        Cast<HtmlElement>();
var nume = inputs.First(h => h.GetAttribute("type") == "text");
var pass = inputs.First(h => h.GetAttribute("type") == "password");
var login = inputs.First(h => h.GetAttribute("type") == "image");
var form = web.Document.GetElementsByTagName("form")[0];

nume.SetAttribute("value", "user");
pass.SetAttribute("value", "pass");
login.InvokeMember("Click");

For those games that are desktop based things get more complicated. Windows allows for one aplication to send messages to another application so i guess you could use that to send messages to a game (although wheather this works for games that use directX for input i don't know)

Answer 9

When you say "control a program" think about how programs are controlled. Most accept keyboard/mouse input.

If you write a program to simulate keyboard/mouse input, there you go.

A corollary to this would be that if a program accepted arguments passed through an input box, write a program to send arguments as if they were sent through an input box.

This is how all bots work. Hope it sheds some light for you!

Edit: This is how the vast majority of bots work :-)

Answer 10

You can emulate mouse movements/clicks and keyboard input in Java with the Robot class. It also lets you take screen shots.

Answer 11

I think AutoIT is a great language for doing this sort of software. Even though I haven't used it to build bots, sending clicks and keys was very easy.

Answer 12

My understanding is that most high-end gaming bots use system calls to map themselves into the game client's memory space, reverse engineer its layout, and directly manipulate the client program's internal variables. That's way more work than I'd be up for just for fun.

Answer 13

For java, the class "Robot" works well for regular desktop application. But it can't can't control the directX game, like "Need for Speed Most Wanted"--a racing game. In "Need For Speed Underground", "Up, Left, Right" works well except for "Down". I searched the keyboard/mouse events in directx programming. And I found that it accepts the events via "asking" keyboard/mouse diretly! And I want to ask you guys how to simulate the keyboard/mouse events in this situation?