Create a pipe that writes to multiple files (tee).

Question

I would like to create a pipe in a ksh script (using exec) that pipe's to a tee, and sends the output to a pipe.

Current:

#Redirect EVERYTHING
exec 3>&1 #Save STDOUT as 3
exec 4>&2 #Save STDERR as 4
exec 1>${Log} #Redirect STDOUT to a log
exec 2>&1 #Redirect STDERR to STDOUT

What'd I'd like to do (but I don't have the syntax correct):

#Redirect EVERYTHING
exec 3>&1 #Save STDOUT as 3
exec 4>&2 #Save STDERR as 4
exec 1>tee -a ${Log} >&3  #Redirect STDOUT to a log
exec 2>&1 #Redirect STDERR to STDOUT

How can I create this pipe?

Answer 1

I know bash not ksh, but there's a lot of overlap, so maybe this will work there too.

process1 N> >(process2)

Creates a subshell running process2. That subshell receives as its stdin the data from process1's file descriptor N. So in particular, you could do:

process1 1> >(tee -a mylog >&3)

I don't know whether this would also work if process1 is replaced with exec, but you could give it a try.

Answer 2

There are |& and >&p in ksh, but I couldn't get them to do what you're looking for. Maybe you can.

Answer 3

Instead of:

exec 1>tee -a ${Log} >&3

do simply:

tee -a ${Log} >&3 &

tee will fork into the background, and will consume the calling process' (i.e. your script's) STDIN as it was at the time that tee forked.

Cheers, V.

Answer 4

I worked out a solution using named pipes.

#!/bin/ksh

LOG=~/testLog.log
PIPE=~/logPipe
mkfifo ${PIPE}
exec 3>&1 #Save STDOUT as 3
exec 4>&2 #Save STDERR as 4
tee -a ${LOG} <${PIPE} >&3 & #Start tee off the logpipe in the background
exec 1>${PIPE} #Redirect stdout to the pipe
exec 2>&1 #Redirect STDERR to STDOUT

echo "TEST"
echo Test 2

ls | grep -i "test"

rm -f ${PIPE} #Remove the pipe

Answer 5

Here's a solution I use. It works under ksh on my Mac. It's nicely encapsulated into start_logging() and stop_logging() functions to make life easy.

The code looks like this in practice:

# Optional:
#   Set the name and location of the log file.
#   OUTPUT_LOG=output.log    # default
#   Set the name and location of the named pipe used.
#   OUTPUT_PIPE=output.pipe  # default

start_logging
# Default is to append to an existing log file.
# start_logging delete_existing_logfile
echo "This is on standard out"
echo "This is on standard err" >&2
stop_logging

Here is the whole file. The start and stop functions along with the example above are all at the bottom of the file. To make it easier to use, just put the start and stop functions in their own file and source them in the scripts where you need the logging.

#!/bin/sh

# Author: Harvey Chapman <hchapman _AT_ 3gfp.com>
# Description: POSIX shell functions that can be used with tee to simultaneously put
#              stderr and stdout to both a file and stdout
#
# Based on:
#    Re: How to redirect stderr and stdout to a file plus display at the same time
#    http://www.travishartwell.net/blog/2006/08/19_2220

#
# Original example function from Travis Hartwell's blog.
# Note: I've made minor changes to it.
example()
{
  OUTPUT_LOG=output.log
  OUTPUT_PIPE=output.pipe

  # This should really be -p to test that it's a pipe.
  if [ ! -e $OUTPUT_PIPE ]; then
      mkfifo $OUTPUT_PIPE
  fi

  # This should really be -f to test that it's a regular file.
  if [ -e $OUTPUT_LOG ]; then
      rm $OUTPUT_LOG
  fi

  exec 3>&1 4>&2
  tee $OUTPUT_LOG < $OUTPUT_PIPE >&3 &
  tpid=$!
  exec > $OUTPUT_PIPE 2>&1

  echo "This is on standard out"
  echo "This is on standard err" >&2

  exec 1>&3 3>&- 2>&4 4>&-
  wait $tpid

  rm $OUTPUT_PIPE
}

# A slightly reduced version of example()
example2()
{
  OUTPUT_LOG=output.log
  OUTPUT_PIPE=output.pipe

  rm -f $OUTPUT_PIPE
  mkfifo $OUTPUT_PIPE
  rm -f $OUTPUT_LOG

  tee $OUTPUT_LOG < $OUTPUT_PIPE &
  tpid=$!

  exec 3>&1 4>&2 >$OUTPUT_PIPE 2>&1

  echo "This is on standard out"
  echo "This is on standard err" >&2

  exec 1>&3 3>&- 2>&4 4>&-
  wait $tpid
  rm -f $OUTPUT_PIPE
}

#
# Logging methods based on above. See the example below for how to use them.
#

# Usage: start_logging [delete_existing_logfile]
start_logging()
{
  # Check to see if OUTPUT_LOG and OUTPUT_PIPE need to be defined.
  if [ -z "$OUTPUT_LOG" ]; then
    OUTPUT_LOG=output.log
  fi
  if [ -z "$OUTPUT_PIPE" ]; then
    OUTPUT_PIPE=output.pipe
  fi
  # Make sure that we're not already logging.
  if [ -n "$OUTPUT_PID" ]; then
    echo "Logging already started!"
    return 1
  fi

  # Always remove the log and pipe first.
  rm -f $OUTPUT_PIPE
  # Delete the logfile first if told to.
  if [ "$1" = delete_existing_logfile ]; then
    rm -f $OUTPUT_LOG
  fi

  mkfifo $OUTPUT_PIPE
  tee -a $OUTPUT_LOG < $OUTPUT_PIPE &
  OUTPUT_PID=$!

  exec 3>&1 4>&2 >$OUTPUT_PIPE 2>&1
}

stop_logging()
{
  # Make sure that we're currently logging.
  if [ -z "$OUTPUT_PID" ]; then
    echo "Logging not yet started!"
    return 1
  fi
  exec 1>&3 3>&- 2>&4 4>&-
  wait $OUTPUT_PID
  rm -f $OUTPUT_PIPE
  unset OUTPUT_PID
}

example3()
{
  start_logging
  #start_logging delete_existing_logfile
  echo "This is on standard out"
  echo "This is on standard err" >&2
  stop_logging
}

#example
#example2
example3