Is it possible to build a WinForm app (or another type of .NET app) which allows me to interact with other windows outside the applicaiton itself?

Question

I'm learning Chinese at the moment and I have gotten my hand on a Chinese dictionary definition.

Now I would like to make an interface.

All I really want the application to do is when I point my mouse pointer over any text on the screen (in any window), it would identify the text I am pointing at and then display a small form over it, which would the chinese transaction.

Is that possible to do? Can WinForms apps interact with windows outside of it's own application?

Answer 1

In C# you can get text under mouse cursor by P/Invoking

• GetCursorPos
• GetClassName

• SendMessage

  • WM_GETTEXT
  • WM_GETTEXTLENGTH

• WindowFromPoint

Like mentioned here

here is another example in C++

Answer 2

A WinForms application can interact with the Windows of other applications. Window handles exist in a global namespace so if you can get the handle of another application's window you can send it messages. You will have to use pinvoke to do some of this, have a look at WindowFromPoint

However, there is no standardized way to display text in a window; there are dozens of APIs for displaying text. So when you point at text with a mouse, you can only get the pixels, but not necessarily the text.

Some window classes will allow you to send class-specific messages query for the text at a specific location, but many will not. Your best bet is probably to use the same methods that screen readers for the blind use http://en.wikipedia.org/wiki/Screen_reader