Windows batch script to copy and modify files containing redirect characters

Question

We are trying to convert a bash shell script into a Windows batch script. We ship these scripts with our software product, and we cannot assume that the customer will have, or will be able to download, sed/awk/cygwin or any other non-standard tools. Therefore, the script must work with whatever tools come out-of-the-box with Windows. The minimum target platform is Windows XP SP2. This script is called from another batch script.

One part of the script needs to search through a file for a particular string and modify that string. In bash, this is easy. The approach we are taking in the Windows batch script is to walk through the file, line by line. If a line does not contain the target string, we echo it to a temp file as-is. If a line contains the target string, we echo a hardcoded string to the temp file.

The problem is that some lines of the input file contain the less-than and greater-than signs. If we put quotation marks around the lines, they are written intact to the temp file, but quoted.

"-- this is a comment in the output .sql file"
"SELECT foo FROM bar WHERE something < 10;"
"--<SpecialInternalTagComment>"

If we unquote them using %~1, then CMD.EXE attempts to treat the < and > as redirection symbols, and produces errors when the script encounters one of those lines, because the resulting command would be something like this:

echo --<SpecialInternalTagComment> >> temp.file

Here's what we have right now. It produces the file contents with every line enclosed in quotation marks.

:clean_install
    echo New installation >> %INSTALL_LOG%

    echo > %INSTALL_FOLDER%"\sql\oracle\table_maintenance_tmp.sql
    for /f "usebackq tokens=* delims=" %%A in (table_maintenance.sql) do (
       call :replace_width_and_write_line "%%A"
    )
    goto done

:replace_width_and_write_line
    echo %1 | find "&&TARGET_STRING" > NUL
    if %ERRORLEVEL% == 0 (
        echo    TABLE_PARTITION_WIDTH CONSTANT NUMBER := 7 ; >> %INSTALL_FOLDER%"\sql\oracle\table_maintenance_tmp.sql 
        goto :eof
    )
    echo %1 >> %INSTALL_FOLDER%"\sql\oracle\table_maintenance_tmp.sql
    goto :eof

The question is, how can we echo the contents of a string to a file, unquoted, if the string contains redirect symbols? If that can't be done, then is there another way to accomplish our original goal, which is to take as input a file that may or may not contain a certain target string, and produce as output a file that is identical to the original file except that if that target string existed, it is replaced by a new string?

I've searched here on stackoverflow, and the two most useful questions are these:
* How do you strip quotes out of an ECHO’ed string in a Windows batch file?
* Dealing with quotes in Windows batch scripts

Note: Please don't suggest Cygwin, sed, awk, Perl, Python, Ruby, unx, or any other technology that does not ship with Windows XP out of the box. Those answers will be voted down.

Thanks,
shoover

Answer 1

I'd suggest using VBScript here to process the file. It's included by default as well and string handling is much less error-prone.

There is a possibility though, to escape the special characters before printing the line. It's tedious and slow, though.

First you need to have your line in an environment variable. Use quotes to avoid any special characters from messing everything up:

set "tmp=%~1"

Careful to put the quotes around the complete argument, not only the part after the =, since otherwise you'll end up with the quotes in the variable as well.

Then you can start escaping the special characters, one by one:

set "tmp=%tmp:<=^<%"
set "tmp=%tmp:>=^>%"
set "tmp=%tmp:&=^&%"
set "tmp=%tmp:|=^|%"

etc.

This should work, but manipulating environment variables is pretty slow, so in large files this won't be fun.

Following a short transcript of my playing around with those replacements. Embedded quotes in uneven numbers cause this to fail, though, so careful:

> set "a=foo <bar> baz"

> set a
a=foo <bar> baz

> echo %a%
The system cannot find the file specified.

> set "a=%a:<=^<%"

> set a
a=foo ^<bar> baz

> set "a=%a:>=^>%"

> set a
a=foo ^<bar^> baz

> echo %a%
foo <bar> baz

In general, I'd suggest using VBScript, though, as a readily-available alternative to batch files and much more powerful, or at least, less surprising in dealing with such things.

Answer 2

You can use edlin.

From its Wikipedia article:

"Its persistence can probably be explained by the fact that it can be invoked to automatically perform small modifications on text files, by piping a script of commands to it through standard input."

Note however that it is not available on 64 bit versions of Windows and line length is limited to 253 characters.

Answer 3

cscript is a standard component installed for Windows. http://en.wikipedia.org/wiki/Windows%5FScript%5FHost

Below is a script that would clean out lines of text that would match on a regular expression. The script could be tweaked to add command line parameters for the search rgx and replacement text.

/*
One part of the script needs to search through a file for a particular string and modify that string.
In bash, this is easy. The approach we are taking in the Windows batch script is to walk through the
file, line by line. If a line does not contain the target string, we echo it to a temp file as-is.
If a line contains the target string, we echo a hardcoded string to the temp file.
*/

/**
 * Usage: type <in.file> | cscript //Nologo test.js > <out.file>
 */
function main() {
    var rgxScrubMatch = /<SpecialInternalTagComment>/;
    var replacement = "safe text";
    scrubLines(rgxScrubMatch, replacement, WScript.StdIn, WScript.StdOut);
}
main();

/**
 * Will echo back line 
 *
 * @param rgxScrubMatch uses replacement on match
 * @param replacement string to replace entire line that matches
 * @param input input stream
 * @param output output stream
 * @return undefined
 */
function scrubLines(rgxScrubMatch, replacement, input, output) {
    while(!input.AtEndOfStream) {
     var line = input.ReadLine();
     if(rgxScrubMatch.test(line)) {
      output.Write(replacement + "\r\n");
     } else {
      output.Write(line + "\r\n");
     }
    }
}