Sharing queues between processes using nested classes (Python)

Question

Hello everyone, I have a question about sharing queues between processes in Python. Below, I have three queues, one main process, and three inner processes. Each inner process will be adding and getting values from the various queues (they need easy access to the queues).

I think it works as it is right now, but this code is the foundation of a big project I'm going to work on and I want to make sure that there is not a better way of doing this that I don't know about. I just sort of came up with this idea. It appears from some other posts that nested classes is not very Python-like.

Any advice? Is this code easy or hard to read? Abandon nested classes or leave it as it is?

Thanks all.

class MainController(Process):
    def __init__(self):
        self.queue_stream   = Queue()
        self.queue_language = Queue()
        self.queue_expander = Queue()

        self.control_stream   = self.StreamController(self).start()
        self.control_language = self.LanguageController(self).start()
        self.control_expander = self.ExpanderController(self).start()

        print 'Launching Main Controller'

    class StreamController(Process):
        def __init__(self, main):
            Process.__init__(self)
            self.main = main
            print 'Launching Stream Controller'

        def run(self):
            while True:
                self.main.queue_stream.put('hello, stream')

    class LanguageController(Process):
        def __init__(self, main):
            Process.__init__(self)
            self.main = main
            print 'Launching Language Controller'

        def run(self):
            while True:
                print self.main.queue_stream.get()
                self.main.queue_language.put('hello, language')

    class ExpanderController(Process):
        def __init__(self, main):
            Process.__init__(self)
            self.main = main
            print 'Launching Expander Controller'

        def run(self):
            while True:
                print self.main.queue_language.get()
                self.main.queue_expander.put('hello, expander')

def main():
    # Launch all queues for the system
    control_main = MainController()

if __name__ == '__main__':
    print 'Launching System...'
    main()

Answer 1

I recommend you to use the threading module instead of process. I suggest you to use nested classes only if the child classes are extending on functionality the parent class.

class   WorkerThread(threading.Thread):

Another recommendation is to use a shared lock between your child threads in order to prevent race conditions on your shared Queue.

            tasks_lock.acquire()
            ret = tasks_queue.get()
            tasks_lock.release()

Take a look on this Example

Answer 2

The child now have to know about the implementation of the father. I discourage that

def infinite_producer(queue):
    while True:
        queue.put('hello, stream')

class MainController(Process):
    def __init__(self):
        self.queue_stream   = Queue()
        self.queue_language = Queue()
        self.queue_expander = Queue()
        self.self.control_stream = Process(target=infinite_producer,self.queue_stream)

    def run(self):
        self.control_stream.start()

 #... etc you get the idea.

if __name__ == '__main__':
    print 'Launching System...'
    control_main = MainController()
    control_main.start()