Search and replace hundreds of strings in tens of thousands of files?

Question

I am looking into changing the file name of hundreds of files in a (C/C++) project that I work on. The problem is our software has tens of thousands of files that including (i.e. #include) these hundreds of files that will get changed. This looks like a maintenance nightmare. If I do this I will be stuck in Ultra-Edit for weeks, rolling hundreds of regex's by hand like so:

^\#include.*["<\\/]stupid_name.*$

with

#include <dir/new_name.h>

Such drudgery would be worse than peeling hundreds of potatoes in a sunken submarine in the antarctic with a spoon. I think it would rather be ideal to put the inputs and outputs into a table like so:

stupid_name.h <-> <dir/new_name.h>
stupid_nameb.h <-> <dir/new_nameb.h>
stupid_namec.h <-> <dir/new_namec.h>

and feed this into a regular expression engine / tool / app / etc...

My Ultimate Question: Is there a tool that will do that?

Bonus Question: Is it multi-threaded?

I looked at quite a few search and replace topics here on this website, and found lots of standard queries that asked a variant of the following question:

standard question: Replace one term in N files.

as opposed to:

my question: Replace N terms in N files.

Thanks in advance for any replies.

Answer 1

I think your idea of putting the old/new names into a single location is a good one. It would certainly reduce the difficulty of maintaining and verifying the changes. It seems like this is the obvious answer, but I think that using any of the popular scripting languages such as ruby, python, perl, etc. would make this task fairly straightforward. The script could read in the file that has the old/new replacement information, construct the appropriate regular expressions from that, and then process the files that need the replacements.

The script could be written as a multi-threaded utility, although it doesn't seem like there would be a lot of benefit in this type of situation. If I understand the question, this should be basically a one-time usage so high performance does not seem like the top priority.

Answer 2

Will this (Wingrep) do the trick?

Answer 3

in *nix, (or GNU win32) , you can use GNU find and sed together... eg

find /path -type f -name "*.c" -exec  sed -i.bak 's/^\#include.*["<\\/]stupid_name.*$/#include <dir\/new_name.h>/' "{}" +;

explanation,

the find command starts finding files (-type f) starting from /path. -name "*.c" searches for all .c files, then for each one found, do a sed to change the string to the new string. -i.bak asks sed to save the original file as backup before doing inplace editing. "{}" means the file passed to sed

Answer 4

I would use awk, a command line tool similar to sed.

mv file.x file.x.bak;
awk '{
  gsub( "#include \"bad_one.h\"" , "#include \"good_one.h\"" );
  gsub( "#include \"bad_two.h\"" , "#include \"good_two.h\"" );
}' file.x.bak > file.x;

Once you are at a terminal, use man awk to see more details.

Answer 5

Make a series of perl one-liners to edit the files in place, like so:

perl -i.bak -p -e 's/stupid_old_name/cool_new_name/' *.c

This has the added bonus of saving the originals of any changed files with a .bak extension.

I'd make a bunch of these, if I didn't know perl that well. I'd even put all the one-liners into a shell script, but then I'm not trying to impress any of the unix graybeards out there.

This website explains edit in place with perl very well: http://www.rice.edu/web/perl-edit.html

PS - Since I do know perl fairly well, I'd just write the was/is table in a "real" perl script and use it to open and parse all the files.

Answer 6

PowerGREP can do that. It can search for multiple search strings (literal text or regular expressions) in any combination of files, and is multithreaded (starting with PowerGREP 4, the current version).

You can save your searches for later re-use, too.