Controlling application running on a virtual desktop (linux)

Question

Hi guys,

I need to run an application on a virtual Xorg desktop (let say desktop #2) and control it via another app running on the root desktop (desktop #1). That would include screen capture and mouse movements. So basically I capture the application window from desktop #2, scrape it and then click on buttons via my app on desktop #1. Anyway you got the idea.

Now my big question is: is that doable ? and if so what's the best language to do it ?

btw I'm using Linux

Thanks in advance

Luke

Answer 1

It is certainly doable. If you use VirtualBox over linux, then you can use the Remote Desktop Protocol to run and control the virtual machine remotely.

VirtualBox supports the standard Remote Desktop Protocol where a virtual machine can act as an RDP server, allowing you to "run" the virtual machine remotely on some thin client that merely displays the RDP data.

Hope this helps.

Answer 2

From this post someone mentions recording X11 events, this may get you halfway.

Answer 3

If you're talking about text-mode consoles, there's this: http://ace-host.stuart.id.au/russell/files/conspy/

Answer 4

Most of the window managers implement desktop management by hiding the applications that should not be shown.

You can send xevents to push buttons on the application. I doubt you'll be able to capture the screen contents of the app, since it's not mapped.

You might be able to run your application under Xnest, and you then could capture the screen contents. However, in I my experience, Xnest does not render applications correctly in some instances.

Answer 5

You could start x11vnc on the X server you want to control, and then control it via VNC.

Or I think X11 lets simulate events directly. And you can certainly grab the contents of a window. Look at the code for xwd for an example of getting a bitmap image of a window.