How can I run the first process from a list of processes stored in a file and immediately delete the first line as if the file was a queue and I called "pop"?

Question

I'd like to call the first command listed in a simple text file with \n as the separator in a pop-like fashion:

---

Figure 1:

cmdqueue.lst :

proc_C1
proc_C2
proc_C3
.
.

---

Figure 2:

Pop the first command via popcmd:

proc_A | proc_B | popcmd cmdqueue.lst | proc_D

---

Figure 3:

cmdqueue.lst :

proc_C2
proc_C3
proc_C4
.
.

Answer 1

I think you would need to rewrite the file - e.g. run a command to list all lines but the first, write that to a temporary file and rename it to the original. That could be done using tail or awk or perl depending on the commands you have available.

Answer 2

pop-cmd.py:

#!/usr/bin/env python
import os, shlex, sys
from subprocess import call
filename = sys.argv[1]
lines = open(filename).readlines()
if lines:
    command = lines[0].rstrip()
    open(filename, "w").writelines(lines[1:])
    if command:
        sys.exit(call(shlex.split(command) + sys.argv[2:]))

Example:

proc_A | proc_B | python pop-cmd.py cmdstack.lst | proc_D

Answer 3

If you want to treat a file like a stack, then a better approach would be to have the top of the stack at the end of the file.

Thus you can easily cut off the file at the beginning of the last line (= pop), and simply append to the file as you push.

Answer 4

Ooh, that's an amusing one-liner.

Okay, here's the deal. What you want is a program that, when called, prints the first line of the file to stdout, then delete that line from the file. Sounds like a job for sed(1).

Try

proc_A | proc_B | `(head -1 cmdstack.lst; sed -i -e '1d' cmdstack.lst)` | proc_D

I'm sure that someone who had already had their coffee could change the sed program to not need the head(1) call, but that works, and shows off using a subshell ("( foo )" runs in a sub-process.)

Answer 5

You can't write to the beginning of a file, so cutting out line 1 would be a lot of work (rewrite the rest of the file (which isn't actually that much work for the programmer (it's what every other answer post has written for you :) ) ) ).

I'd recommend keeping the whole thing in memory and using a classic stack rather than a file.

Answer 6

You can use a little bash script; name it "popcmd":

#!/bin/bash
cmd=`head -n 1 $1`
tail -n +2 $1 > ~tmp~
mv -f ~tmp~ $1
$cmd

edit: Using sed for the middle two lines, like Charlie Martin showed, is much more elegant, of course:

#!/bin/bash
cmd=`head -n 1 $1`
sed -i -e '1d' $1
$cmd

edit: You can use this exactly as in your example usage code:

proc_A | proc_B | popcmd cmdstack.lst | proc_D

Answer 7

I assume that you are constantly appending to the file also, so rewriting the file puts you in danger of overwriting data. For this type of task I think you would be better using individual files for each queue entry, using date/time to determine order, and then as you process each file you could append the data to a log file and then delete the trigger file.

Really need more information in order to suggest a good solution. It's important to know how the file is getting updated. Is it a lot of separate processes, just one process, etc.