tags:

views:

435

answers:

3

I want to pipe [edit: real-time text] the output of several subprocesses (sometimes chained, sometimes parallel) to a single terminal/tty window that is not the active python shell (be it an IDE, command-line, or a running script using tkinter). IPython is not an option. I need something that comes with the standard install. Prefer OS-agnostic solution, but needs to work on XP/Vista.

I'll post what I've tried already if you want it, but it’s embarrassing.

A: 

You say "pipe" so I assume you're dealing with text output from the subprocesses. A simple solution may be to just write output to files?

e.g. in the subprocess:

  1. Redirect output %TEMP%\output.txt
  2. On exit, copy output.txt to a directory your main process is watching.

In the main process:

  1. Every second, examine directory for new files.
  2. When files found, process and remove them.

You could encode the subprocess name in the output filename so you know how to process it.

John Fouhy
I should have mentioned that I plan to pipe stdout real-time for each process. The file solution is workable, but my users are accustomed to a 30-60hz response time. I'm sure a HDD can handle it, but it seems unnecessary. Also, I'm still unclear on how to spawn a shell-like window that only displays output text (separate from the current IDE / script-stdout) be it from a file or piped output.
bpowah
+2  A: 

A good solution in Unix would be named pipes. I know you asked about Windows, but there might be a similar approach in Windows, or this might be helpful for someone else.

on terminal 1:

mkfifo /tmp/display_data
myapp >> /tmp/display_data

on terminal 2 (bash):

tail -f /tmp/display_data

Edit: changed terminal 2 command to use "tail -f" instead of infinite loop.

redmoskito
http://en.wikipedia.org/wiki/Tail_(Unix)#File_monitoring
alex vasi
@alex Thank for the tip, I've updated my example.
redmoskito
A: 

You could make a producer-customer system, where lines are inserted over a socket (nothing fancy here). The customer would be a multithreaded socket server listening to connections and putting all lines into a Queue. In the separate thread it would get items from the queue and print it on the console. The program can be run from the cmd console or from the eclipse console as an external tool without much trouble.

From your point of view, it should be realtime. As a bonus, You can place producers and customers on separate boxes. Producers can even form a network.

Some Examples of socket programming with python can be found here. Look here for an tcp echoserver example and here for a tcp "hello world" socket client.

There also is an extension for windows that enables usage of named pipes.

On linux (possibly cygwin?) You could just tail -f named-fifo.

Good luck!

Reef
+1: This sounds very much like what I need. I might eventually distribute these processes over a network in the future.-1: No references/examples. (Google of "producer-customer system socket" is returning links to prosthetic manufacturing solutions ...and variations thereof are not very fruitful)Thanks for the suggestion!
bpowah
It was `google python socket`, first result, then scroll down to the "examples" section. It seemed obvious to me, sorry about that. I have added some links to examples that might help You.
Reef
my search results return:1st:http://code.google.com/p/couchdb-python/issues/detail?id=682nd:http://groups.google.com/group/project-voldemort/browse_thread/thread/2fc477c4c1e3fee93rd:http://pleac.sourceforge.net/pleac_python/sockets.htmlIs the third the one you are referring to? It seems the most useful, however none of them have an 'examples' section. When you don't fully understand what you are looking for, it is not obvious when you have found it, and it is especially hard to determine if you have found a high-quality source.
bpowah
http://docs.python.org/library/socket.html#example - I blame google. I have diffrent results than You.
Reef