How do you "debug" a regular expression with sed?

Question

I'm trying to use a regexp using sed. I've tested my regex with kiki, a gnome application to test regexpd, and it works in kiki.

date: 2010-10-29 14:46:33 -0200;  author: 00000000000;  state: Exp;  lines: +5 -2;  commitid: bvEcb00aPyqal6Uu;

I want to replace author: 00000000000; with nothing. So, I created the regexp, that works when I test it in kiki:

author:\s[0-9]{11};

But doesn't work when I test it in sed.

sed -i "s/author:\s[0-9]{11};//g" /tmp/test_regex.txt

I know regex have different implementations, and this could be the issue. My question is: how do I at least try do "debug" what's happening with sed? Why is it not working?

Answer 1

The fact that you are substituting author: 00000000000 is already said in sed when you add the s before the first /.

Answer 2

My version of sed doesn't like the {11} bit. Processing the line with:

sed 's/author: [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9];//g'

works fine.

And the way I debug it is exactly what I did here. I just constructed a command:

echo 'X author: 00000000000; X' | sed ...

and removed the more advanced regex things one at a time:

• used <space> instead of \s, didn't fix it.
• replaced [0-9]{11} with 11 copies of [0-9], that worked.

It pretty much had to be one of those since I've used every other feature of your regex before with sed successfully.

But, in fact, this will actually work without the hideous 11 copies of [0-9], you just have to escape the braces [0-9]\{11\}. I have to admit I didn't get around to trying that since it worked okay with the multiples and I generally don't concern myself too much with brevity in sed since I tend to use it more for quick'n'dirty jobs :-)

But the brace method is a lot more concise and adaptable and it's good to know how to do it.

Answer 3

You are using the -i flag incorrectly. You need to put give it a string to put on the temporary file. You also need to escape your curly braces.

sed -ibak -e "s/author:\s[0-9]\{11\};//g" /tmp/test_regex.txt

I usually debug my statement by starting with a regex I know will work (like 's/author//g' in this case). When that works I know that I have the right arguments. Then I expand the regex incrementally.

Answer 4

In sed you need to escape the curly braces. "s/author:\s[0-9]\{11\};//g" should work.

Sed has no debug capability. To test you simplify at the command line iteratively until you get something to work and then build back up.

$ echo 'xx a: 00123 b: 5432' | sed -e 's/a:\s[0-9]\{5\}//
xx  b: 5432

Answer 5

That looks more like a perl regex than it does a sed regex. Perhaps you would prefer using

perl -pi.orig -e 's/author:\s[0-9]{11};//g' file1 file2 file3

At least that way you could always add -Mre=debug to debug the regex.

Answer 6

There is a Python script called sedsed by Aurelio Jargas which will show the stepwise execution of a sed script. A debugger like this isn't going to help much in the case of characters being taken literally (e.g. {) versus having special meaning (e.g. \{), especially for a simple substitution, but it will help when a more complex script is being debugged.

The latest SVN version.
The most recent stable release.
^{_{Disclaimer: I am a minor contributor to sedsed.}}

Another sed debugger, sd by Brian Hiles, written as a Bourne shell script (I haven't used this one).