views:

259

answers:

4

To process a bunch of data and get it ready to be inserted into our database, we generate a bunch of shell scripts. Each of them has about 15 lines, one for each table that the data is going. One a recent import batch, some of the import files failed going into one particular table. So, I have a bunch of shell scripts (about 600) where I need to comment out the first 7 lines, then rerun the file. There are about 6000 shell scripts in this folder, and nothing about a particular file can tell me if it needs the edit. I've got a list of which files that I pulled from the database output.

So how do I write a bash script (or anything else that would work better) to take this list of file names and for each of them, comment out the first 7 lines, and run the script?

EDIT:

#!/usr/bin/env sh

cmd1
cmd2
cmd3
cmd4
cmd5
cmd6
cmd7
cmd8

Not sure how readable that is. Basically, the first 7 lines (not counting the first line) need to have a # added to the beginning of them. Note: the files have been edited to make each line shorter and partially cut off copying out of VIM. But in the main part of each file, there is a line starting with echo, then a line starting with sqlldr

+3  A: 

Using sed, you can specify a line number range in the file to be changed.

#!/bin/bash

while read line
do
    # add a comment to beginning of lines 1 - 7 and rename the script
    sed '3,9 s/^/#/' $line > $line.new
    exec $line.new
done < "filelist.txt"

You may wish to test this before running it on all of those scripts...

EDIT: changed the lines numbers to reflect comments.

Buggabill
?s: 1) ^ is the special character for beginning of line?2) Is 1 the first line, or 0 the first line of a file?3) How can this be set up to take as input the list of files to change? IE how do I tell the while read line to pull from the file list? Is it as simple as putting the list of files in a file and redirecting input from that file?
David Oneill
^ is the character that represents the beginning of a line. 1 is the first line. This will read a file "filelist.txt" line by line and change each file in that list. This will only change lines 1-7 as requested - including the first line. The numbers at the beginning can be adjusted to accommodate what you are looking for.
Buggabill
Are you opening "filelist.tx" or is it being piped or redirected into the script?
David Oneill
This opens the file "filelist.txt" and reads it line by line.
Buggabill
I think you want to put that "exec $line.new" in a subshell.
Dean Povey
ok, somebody fill me in on what filelist.txt is?? if OP is going to modify 600 shell scripts, then just sed '1,7 s/^/#/' *.sh will do. There is no need for the while loop. am i missing something?
ghostdog74
You want to do lines 3-9 so you don't comment out the shebang and the blank line.
Dennis Williamson
@ghostdog: there are 600 shell scripts i need to change, but 6000 or so in the folder. So I can't do do that.@Dennis Yeah, I'll be adjusting the line numbers to skip those.
David Oneill
Yes, I will be testing this quite a bit before I hit go :) Thanks for your help!
David Oneill
This is how I understand it... The while loop is a statement in scripting. This takes the contents of that file and redirects them into the while loop where they are read line by line into the variable $line. Someone else could explain this better than me, I think...
Buggabill
+1  A: 

Ultimately you're going to want to use the linux command sed. Whatever logic you need to place in the script, you know. But your script will ultimately call sed. http://lowfatlinux.com/linux-sed.html

lilott8
+2  A: 

Roughly speaking:

#!/bin/sh
for file in "$@"
do
    out=/tmp/$file.$$
    sed '2,8s/^/#/' < $file > $out
    $SHELL $out
    rm -f $out
done

Assuming you don't care about checking for race conditions etc.

Dean Povey
What could cause race conditions in this situation? Would that only occur if multiple instances of this script were running at the same time?
David Oneill
Multiple instances of _this_ script should be okay (that's what the $$ is for, it adds the process id to files). The problem is a malicious user who creates a bunch of readonly /tmp/foo.[range of guesses at your pid] files with stuff they want to execute using your privileges.
Dean Povey
Ah, ok. Thanks for the tip
David Oneill
A: 

ex seems made for what you want to do.

For instance, for editing one file, with a here document:

#!/bin/sh

ex test.txt << END
1,12s/^/#/
wq
END

That'll comment out the first 12 lines in "test.txt". For your example you could try "$FILE" or similar (including quotes!).

Then run them the usual way, i.e. ./"$FILE"

edit: $SHELL "$FILE" is probably a better approach to run them (from one of the above commenters).

wds