submitting multiple files from shell script through standard input

Question

I have a program which reads from standard input when invoked from the command line. Normally the user would enter some information, press C-d on a new line to signal end of file, and the program would process the file. Then the user would again be prompted, would enter some more text, and again press C-d to signal the end of the second file.

I'm looking to write a shell script which will invoke this program and automatically submit the two files. I would like to do something like this:

$ myprogram < file1.txt < file2.txt

OR

$ myprogam <file1.txt file2.txt

Any suggestions? I'm using the bash shell under Linux.

Answer 1

A likely thing to do would be to use expect.

Answer 2

Obviously you can use cat to send two files into a programs standard input, but this is probably not what you want:

$ cat file1.txt file2.txt | myprogram

In the above case myprogram wouldn't be able to tell where file1.txt ends and file2.txt begins, unless they had specific formatting and it was watching the stream for certain formatting that signified the end or beginning of a file.

There is no standard way for multiple streams to be sent into standard input. Essentially there's just a single stream coming into standard input.

You would probably structure your script so it accepted multiple arguments on the command line and internally opened and processed them one at a time. That is, you would call it like so:

$ myprogram file1.txt file2.txt

Answer 3

Jonathan Leffler says in his comment, that my solution is incorrect. Maybe it is. I actually haven't checked it thoroughly. But let it be here just for info that it's a wrong answer, at least.

---

Bash can echo a control character with its echo command. EOF is C-d, as you've already noted, so let's use a subshell:

$ ( cat file1.txt ; echo -e '\cD' ; cat file2.txt ) | myprogram

This will execute commands within the parentheses and pipe the output of them all as if it were a separate program.

If you are uncertain that it pipes correctly, try this:

$ (echo "Hello" ; echo -e '\cD' ; echo "bash") | tee

Answer 4

Does it have to be 1 invocation? If not:

for file in file1 file2 file3; do myprogram < $file; done

Answer 5

I've written a simple Python program to simulate the behavior of the program you describe. At a bare minimum, this puts us all in the position of talking about the same thing instead of assuming how your program works. (Of course, if your program doesn't actually work the way I'm describing it here, please correct me.)

#!/usr/bin/python

def read_a_file():
    print('Type the contents of a file now.')
    while True:
        try:
            s = raw_input('>')
        except EOFError:
            break

read_a_file()
read_a_file()
print('We have now read two files. Quitting.')

You can run this script, type the contents of the first file, hit CTRL-D, type the contents of the second file, hit CTRL-D, and the program exits. This appears to be the behavior you're describing.

My first thought was to use expect, as chaos suggested. However, I can't find any support in expect for piping the contents of a file to a process. (I assume the contents of the files are different every time. If they're the same, then expect would work, just by putting the contents of the files in the expect script.)

I should note that I'm by no means an expect expert; there may be a way to do this that isn't obvious to me.

My second thought was to write a Python script that launched your program, and fed it the contents of the files and the EOF characters. However, Jonathan Leffler's comments made me think that echoing EOF characters would not work.

My third thought is pretty damn kludgy. You could write a script that creates an expect script by plugging the contents of the files into the expect script, and then executing the expect script. I suspect that would work.