pipes

Using linux "cut" with stdin

I'm trying to pipe data into "cut" to, say, cut away the first column of text. This works $ cat test.txt | cut -d\ -f2- Reading from stdin also works: $ cut -d\ -f2- - ? doc/html/analysis.html ? doc/html/classxytree-members.html <CTRL+D> However, as soon as a pipe is involved, it doesn't accept my <CTRL+D> anymore, and I can't si...

Problem with temporary unnamed pipe in bash script

Hi, I have the following question: When I execute the following script directly in a terminal window, the commands behave as expected. $ diff <(echo tmp) <(echo tmp1) 1c1 < tmp --- > tmp1 However when I write the same command in a shell script #! /bin/bash diff <(echo tmp) <(echo tmp1) I get the following error message: $ sh tes...

subprocess.Popen(..).communicate(..) throw away data at random when used with graphviz!

Hey all! I am using graphviz's dot to generate some svg graphs for a web application. I call dot using Popen: p = subprocess.Popen(u'/usr/bin/dot -Kfdp -Tsvg', shell=True,\ stdin=subprocess.PIPE, stdout=subprocess.PIPE) str = u'long-unicode-string-i-want-to-convert' (stdout,stderr) = p.communicate(str) What happends is...

GetLastError returning ERROR_SUCCESS after calling ConnectNamedPipe

When I call ConnectNamedPipe with an OVERLAPPED structure and check GetLastError afterwards, I often see GetLastError return ERROR_SUCCESS. The way I'm reading the documentation, that should never happen. My understanding is that if a client has already connected, ERROR_PIPE_CONNECTED should be set, not ERROR_SUCCESS. Has anyone else se...

pipe bash output to two different streams

I have a bashscript that spawns processes on two different machines over ssh and then cat's the output of one into a text file. How can I have the output ALSO displayed in the terminal as it's running? ...

How to output to a file with information from a pipe in C?

I'm confused about what I'm doing wrong when I'm attempting to output to a file after I've execed a second program. Say I have input file that has the following names: Marty B. Beach 7 8 zachary b. Whitaker 12 23 Ivan sanchez 02 15 Jim Toolonganame 9 03 After my programs finish, it will convert the student's names to their...

Create a pipe that writes to multiple files (tee).

I would like to create a pipe in a ksh script (using exec) that pipe's to a tee, and sends the output to a pipe. Current: #Redirect EVERYTHING exec 3>&1 #Save STDOUT as 3 exec 4>&2 #Save STDERR as 4 exec 1>${Log} #Redirect STDOUT to a log exec 2>&1 #Redirect STDERR to STDOUT What'd I'd like to do (but I don't have the syntax correct)...

output from linux pipe in two inpdendent files

How do I pipe standard output from linux pipe in two inpdendent files? I use a tool called openRTSP and which to standard output in two independent files i.e. openRTSP > /tmp/file1 > /tmp/file2 ...

Pipes and sockets - "message passing" or "shared memory"?

Ok, this is a stupid question... Are pipes, FIFOs, and sockets shared memory or message passing..? At first I thought they were shared memory because pipes use read() and write(), but now I'm just totally confused. Technically the "messages" are stored in the kernal's address space, so is it a message or stored memory? I have an exam ...

shell-scripting: Use a pipe as an input for tar

Hi there! I'm trying to figure out a way to use tar+pipes on a Ubuntu Server LTS. I've got a postgresql command (pg_dump) that outputs lots of sql on the standard output: pg_dump -U myUser myDB I know how to redirect that to a file: pg_dump -U myUser myDB > myDB.sql In order to save some disk space, I would rather have it compres...

Can someone explain pipe buffer limitations?

Python documentation to Popen states: Warning Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. Now, I'm trying to figure out how this deadlock can occur and why. My mental model: subproccess produces someth...

Using named pipe to communicate between unicode and non-unicode processes on windows.

If a process with unicode enabled creates a named pipe, it must pass a LPCTSTR for the pipe name, in this case a LPCWSTR. Will a second process wihtout unicode be able to open that pipe by passing a LPCSTR for the pipe name? Also, can I call CreateNamedPipeW or CreateNamedPipeA and ignore whether unicode is enabled, or do I have to call...

Tricking a Unix Commandline Program into Accepting a File Stream

Hypothetical situation. I have a command line program in *nix (linux, BSD, etc.). It was written so that you pass it a text file as an argument $ program file.txt Run the program, it looks at the text in file.txt. Is it possible to "trick" this program into accepting input from a file stream rather than reading a file via disk? ...

How to view some data in gvim without saving it first?

For example suppose I do a certain man or maybe a grep. I want to see the results using gvim without the normal procedure of saving them first and opening the saved file [because I don't need the results once I view them] I tried two ways, both the methods fail: a) man gcc | gvim //opens a blank gvim window b) man gcc > gvim //sa...

Recommended way to communicate between processes running on different computers on the same network

I'm working on what should (hopefully) be a small feature for a .NET software solution that will allow one application to report on the status of multiple instances of another application running on more than one computer on the same network. I took a look at this MSDN article on named pipes, and set up a simple client/server project th...

What are the blocking issues when sending large amounts of data down a pipe using proc_open

Hi, I'm using proc_open to run a filter on some output from a database, potentially a large number of lines. Do I need to handle the potential for blocking specially, i.e. is it likely that sending a large chunk of data to the filter and not reading from it's stdout will cause it to block it's stdin, preventing my script from writing un...

Serialization over Pipes

I wrote several simulation programs in C++ and want to connect their outputs/inputs with pipes (best solution would probably be to use the C++ streams). For this I would like to serialize some objects (for example the simulations output/input are tensors and matrices). How should I handle this problem? I searched around for some time fo...

Trouble with piping through sed

I am having trouble piping through sed. Once I have piped output to sed, I cannot pipe the output of sed elsewhere. wget -r -nv http://127.0.0.1:3000/test.html Outputs: 2010-03-12 04:41:48 URL:http://127.0.0.1:3000/test.html [99/99] -> "127.0.0.1:3000/test.html" [1] 2010-03-12 04:41:48 URL:http://127.0.0.1:3000/robots.txt [83/83] -> ...

socat usage for FIFO speed vs socket speed on localhost

Hello, As per a suggestion on stackoverflow, to compare IPC on a single machine using a) sockets (TCP) on localhost to localhost b) using FIFOs (between Java and C) To answer (a), I used netcat to gauge transfer speed (91 MBytes/sec)[1] (b) Q: How can I test FIFO write speed using socat? My approach(where /tmp/gus is created using...

Piping EOF problems with stdio and C++/Python

I got some problems with EOF and stdio in a communication pipeline between a python process and a C++ program. I have no idea what I am doing wrong. When I see an EOF in my program I clear the stdin and next round I try to read in a new line. The problem is: for some reason the getline function immediatly (from the second run always, the...