Simulate mouse click/Detect color under cursor in Python

Question

Hi, I am very new to python. I am trying to write a program that will click the mouse at (x, y), move it to (a, b), and then wait until the color under the mouse is a certain color, lets say #fff. When it is that color, it clicks again and then repeats.

I cannot find a good API for mouse related stuff for python.

Answer 1

The API for simulating mouse events depends on your platform. I don't know any cross-platform solution.

On Windows, you can access the Win32 API thanks to ctypes. see mouse_event on MSDN. You may also be interested by pywinauto

For getting the color under the mouse, you need the mouse position. See GetCursorPos on MSDN. Then if your app has an API for getting the color at this position you can use it. If not, you can try to grab a small portion of the screen around the cursor and to use PIL for getting the colors of every pixel in this area. I think that PIL screen capture is only working on Windows paltform but I am not sure.

I am using the following function for a similar need:

def grab_main_color(self, rect, max_colors=256):
    """returns a tuple with the RGB value of the most present color in the given rect"""
    img=ImageGrab.grab(rect)
    colors = img.getcolors(max_colors)
    max_occurence, most_present = 0, 0
    try:
        for c in colors:
            if c[0] > max_occurence:
                (max_occurence, most_present) = c
        return most_present
    except TypeError:
        raise Exception("Too many colors in the given rect")

Answer 2

if you're on Windows, then, for this kind of thing, you really want to try autohotkey. It's not python, but it is extremely powerful for doing this kind of thing on a Windows machine. The user community is extremely helpful, also. Check out their "ask for help" forum.