How can I make a T9 style on-screen keyboard for Windows?

Question

Sometimes at night, I like to watch movies in bed, or TV shows online. This is convenient since my computer is right beside my desk, so I just spin one of my monitors around, disable my other screen and pull my mouse over. My keyboard doesn't quite reach without re-routing the cable in a way that doesn't work when I move back to my desk the next day. Sometimes while I'm watching movies, my friends try to talk to me, and I would like to be able to talk back without jumping up, spinning the monitor around, moving the mouse back and sitting in the chair again.

What I would like to do is make an on-screen keyboard to be used with the mouse -- but in a T9 phone-keypad style to (hopefully) minimize the number of clicks and amount of moving the mouse around, missing targets. I'd like to do this in Python since I'm already familiar with the language, but I'm not sure where to start.

One thing I'm not sure of, is how to click the on-screen keyboard without stealing focus from the chat window. Can this be accomplished? Or can the application remember the last focused control in the last focused window and send keystrokes to it?

Also, would I need an external library to do any of this window management and keystroke sending?

Help is greatly appreciated, and if such a thing already exists (in any language), pointing me towards it would also be greatly appreciated.

I'll definitely open source it and post a link to the project here if and when I develop it, incase anyone else would find this sort of thing useful :)

Answer 1

About 12 years ago, I wrote a program for Windows that sat in the tray and would send keystrokes to certain windows when they gained focus. I no longer have the code, and I've forgotten all the details.

Still, the process will work something like this.

For your GUI, if using Python, you probably want to use PyQT or wxPython. Both libraries make it easy to write GUI apps (though you can use the Windows APIs directly).

If it were me, though, I would defer the GUI programming and use PythonWin. Use its GUI tools (lots of examples in the source) to create a simple dialog (also a Window) to do the event handling.

There are probably a couple of approaches for your application to select a target window. The virtual keyboard window will probably have to steal focus (to receive mouse events), but it will then need to know to which window to send the keystrokes.

• You can have a drop-down control in the dialog that allows you to select a target window (you can easily grab the title of each window for target selection), or
• When your window gains focus (there's an event you can trap, something like WM_FOCUS), you can either query for the last window that had focus or you can keep tabs on which windows have focus and use the last one you noticed.

In either case, once you have a handle to the target window, you can use SendMessage to send keystrokes to the target window. I suggest at first just relaying regular keystrokes, and worry later about capturing mouse clicks.

Edit I was able to cobble this together for sending keystrokes to another window.

import win32ui
import win32con
import time
from ctypes import *

PUL = POINTER(c_ulong)
class KeyBdInput(Structure):
    _fields_ = [("wVk", c_ushort),
                ("wScan", c_ushort),
                ("dwFlags", c_ulong),
                ("time", c_ulong),
                ("dwExtraInfo", PUL)]

class HardwareInput(Structure):
    _fields_ = [("uMsg", c_ulong),
                ("wParamL", c_short),
                ("wParamH", c_ushort)]

class MouseInput(Structure):
    _fields_ = [("dx", c_long),
                ("dy", c_long),
                ("mouseData", c_ulong),
                ("dwFlags", c_ulong),
                ("time",c_ulong),
                ("dwExtraInfo", PUL)]

class Input_I(Union):
    _fields_ = [("ki", KeyBdInput),
                 ("mi", MouseInput),
                 ("hi", HardwareInput)]

class Input(Structure):
    _fields_ = [("type", c_ulong),
                ("ii", Input_I)]

def send_char(char):
    FInputs = Input * 1
    extra = c_ulong(0)
    ii_ = Input_I()
    KEYEVENTF_UNICODE = 0x4
    ii_.ki = KeyBdInput( 0, ord(char), KEYEVENTF_UNICODE, 0, pointer(extra) )
    x = FInputs( ( 1, ii_ ) )
    windll.user32.SendInput(1, pointer(x), sizeof(x[0]))

if __name__ == '__main__':
    wnd = win32ui.FindWindow(None, '* Untitled - Notepad2 (Administrator)')
    type_this = 'jaraco'
    wnd.SetFocus()
    wnd.SetForegroundWindow()
    for char in type_this:
        send_char(char)

I found that the PostMessage technique did not work very well (I could not get it to work at all for me).

I also found this article on identifying the last active window.