Apply regular expression substitution globally to many files with a script

Question

I want to apply a certain regular expression substitution globally to about 40 Javascript files in and under a directory. I'm a vim user, but doing this by hand can be tedious and error-prone, so I'd like to automate it with a script.

I tried sed, but handling more than one line at a time is awkward, especially if there is no limit to how many lines the pattern might match.

I also tried this script (on a single file, for testing):

ex $1 <<EOF
gs/,\(\_\s*[\]})]\)/\1/
EOF

The pattern will eliminate a trailing comma in any Perl/Ruby-style list, so that "[a, b, c,]" will come out as "[a, b, c]" in order to satisfy Internet Explorer, which alone among browsers, chokes on such lists.

The pattern works beautifully in vim but does nothing if I run it in ex, as per the above script.

Can anyone see what I might be missing?

Answer 1

If you are on windows, Notepad++ allows you to run simple regexes on all opened files.

Search for ,\s*\] and replace with ]

should work for the type of lists you describe.

Answer 2

1) Open all the files with vim:

bash$ vim $(find . -name '*.js')

2) Apply substitute command to all files:

:bufdo %s/,\(\_\s*[\]})]\)/\1/ge

3) Save all the files and quit:

:wall
:q

I think you'll need to recheck your search pattern, it doesn't look right. I think where you have \_\s* you should have \_s* instead.

Edit: You should also use the /ge options for the :s... command (I've added these above).

Answer 3

you can use a combination of the find command and sed

find /path -type f -iname "*.js" -exec sed -i.bak 's/,[ \t]*]/]/' "{}" +;

Answer 4

You can automate the actions of both vi and ex by passing the argument +'command' from the command line, which enables them to be used as text filters.

In your situation, the following command should work fine:

find /path/to/dir -name '*.js' | xargs ex +'%s/,\(\_\s*[\]})]\)/\1/g' +'wq!'

Answer 5

You asked for a script, but you mentioned that you are vim user. I tend to do project-wide find and replace inside of vim, like so:

:args **/*.js | argdo %s/,\(\_\s*[\]})]\)/\1/ge | update

This is very similar to the :bufdo solution mentioned by another commenter, but it will use your args list rather than your buflist (and thus doesn't require a brand new vim session nor for you to be careful about closing buffers you don't want touched).

• :args **/*.js - sets your arglist to contain all .js files in this directory and subdirectories
• | - pipe is vim's command separator, letting us have multiple commands on one line
• :argdo - run the following command(s) on all arguments. it will "swallow" subsequent pipes
  • % - a range representing the whole file
  • :s - substitute command, which you already know about
  • :s_flags, ge - global (substitute as many times per line as possible) and suppress errors (i.e. "No match")
  • | - this pipe is "swallowed" by the :argdo, so the following command also operates once per argument
  • :update - like :write but only when the buffer has been modified

This pattern will obviously work for any vim command which you want to run on multiple files, so it's a handy one to keep in mind. For example, I like to use it to remove trailing whitespace (%s/\s\+$//), set uniform line-endings (set ff=unix) or file encoding (set filencoding=utf8), and retab my files.