Multiple python scripts sending messages to a single central script

Question

I have a number of scripts written in Python 2.6 that can be run arbitrarily. I would like to have a single central script that collects the output and displays it in a single log.

Ideally it would satisfy these requirements:

• Every script sends its messages to the same "receiver" for display.
• If the receiver is not running when the first script tries to send a message, it is started.
• The receiver can also be launched and ended manually. (Though if ended, it will restart if another script tries to send a message.)
• The scripts can be run in any order, even simultaneously.
• Runs on Windows. Multiplatform is better, but at least it needs to work on Windows.

I've come across some hints:

• os.pipe()
• multiprocess
• Occupying a port
• mutex
• logging.handlers.SocketHandler (thanks @Dan Head)

From those pieces, I think I could cobble something together. Just wondering if there is an obviously 'right' way of doing this, or if I could learn from anyone's mistakes.

Answer 1

I built a server to use a Windows named pipe, using the following key code:

    def run( self ):
        # This is the main server loop for the Win32 platform
        import win32pipe
        import win32file
        self.pipeHandle = win32pipe.CreateNamedPipe(
            '\\\\.\\pipe\\myapp_requests',
            win32pipe.PIPE_ACCESS_DUPLEX,
            win32pipe.PIPE_TYPE_BYTE |
            win32pipe.PIPE_READMODE_BYTE |
            win32pipe.PIPE_WAIT,
            1,
            4096,
            4096,
            10000,
            None)
        if self.pipeHandle == win32file.INVALID_HANDLE_VALUE:
            print 'Failed to create named pipe %s!' % self.pipeName
            print 'Exiting...'
            sys.exit(1)
        while True:
            # Open file connection
            win32pipe.ConnectNamedPipe( self.pipeHandle )

            # Run the main message loop until it exits, usually because
            # of a loss of communication on the pipe
            try:
                self.messageLoop()
            except ServerKillSignal:
                break

            # Return the pipes to their disconnected condition and try again
            try: win32pipe.DisconnectNamedPipe( self.pipeHandle )
            except: pass
        win32file.CloseHandle( self.pipeHandle )
        print "Exiting server"

The method messageLoop() reads data from the pipe, using win32file.ReadFile(), until win32file.error is thrown. Then it exits, allowing run() to restart it.

In my implementation, the users were not likely to have administrator access, so this could not be started as a system service. Instead, I coded the client to check for the existence of the pipe at '\.\pipe\pyccf_requests'. If it doesn't exist, then the client starts a new server process.

Answer 2

I'd consider using logging.handlers.SocketHandler for the message passing parts of this, it sounds like you have a logging type use case in mind already.

The standard libraries logging facilities are very flexible and configuration driven so you should be able to adapt them to your requirements.

This doesn't handle the automatic restarting part of your question. For UNIX you'd probably just use pid files and os.kill(pid, 0) to check if it's running, but I don't know what their equivalents in the Windows world would be.

Answer 3

Dan Head's answer is exactly what you want.

Item #2, "If the receiver is not running when the first script tries to send a message, it is started", probably won't work. Something has to be running to receive a message. I suggest writing a demon process which starts on bootup, and tell Windows to restart it if it dies.