Perl regex replacing at word boundary. Detecting "/" as a word boundary

Question

Hi everyone, I am running into a strange regex issue.... I have a document where I am doing a replace... as an example I want to replace "DEXX" with "DEXX/AREX" and then with the next substitution replace... "AREX" with "AREX/CUBE"

DEXX and AREX are stored in a hash like so.... "DEXX" => "AREX", "AREX" => "CUBE"

The regex I have is this.....

foreach (keys %hashstore){
    $doc=~s!\b($_)\b!$1/$hashstore{$_}!ig;
}

What's happening is that "DEXX" is being replaced with "DEXX/AREX" ok but when "DEXX/AREX" is encountered the regex is replacing "DEXX/AREX" with "DEXX/AREX/CUBE" when it should only be replacing "AREX" when it finds it as a standalone word not as part of another combination like "DEXX/AREX"

It seems to detect "/" as a word boundary. Has anyone encountered this or know of a fix around it? Many thanks! Amy

Answer 1

But / is a word boundary. From perldoc perlreref:

\b Match word boundary (between \w and \W).

In light of your comment below, you should avoid the loop:

#!/usr/bin/perl

use strict; use warnings;
use Regex::PreSuf;

my %lookup = (
    "DEXX" => "AREX",
    "AREX" => "CUBE",
);

my $doc = 'DEXX AREX AREX DEXX AREX DEXX DEXX DEXX AREX';
my $re = presuf keys %lookup;

$doc =~ s{($re)}{$1/$lookup{$1}}g;

print $doc, "\n";

Output:

DEXX/AREX AREX/CUBE AREX/CUBE DEXX/AREX AREX/CUBE DEXX/AREX DEXX/AREX DEXX/AREX
AREX/CUBE

Of course, you don't have to use Regex::PreSuf if you only have two keys:

s{(AREX|DEXX)}{$1/$lookup{$1}}g;

will also do. But, for a longer list of keys, I find Regex::PreSuf to be very handy.

Update: Of course, if the keys can occur in any case in the text, you can use uc to transform when looking up the replacement:

So, either

$doc =~ s{($re)}{join '/', uc($1), $lookup{uc $1}}eig;

or

$doc =~ s{($re)}{join '/', $1, $lookup{uc $1}}eig;

depending on what you need.

Also, ysth points out in the comments "With 5.10 and later, Regex::PreSuf generates a poorer regex than the naive alternation in most cases." So,

my $re = join '|', map quotemeta, sort { length($b) <=> length($a) } keys %lookup; 

might be better. The sort is needed if some keys might be initial substrings of other keys.

Answer 2

The word boundary is any time there's a transition between \w and \W, which is to say [a-zA-Z0-9_] and [^a-zA-Z0-9_] if you're dealing with ASCII.

You should be able to get around this problem by using a negative lookbehind:

foreach (keys %hashstore){
    $doc=~s!(?<!/)\b($_)\b!$1/$hashstore{$_}!ig;
}

Answer 3

\b is equivalent to (though more efficient than) (?:(?<!\w)(?=\w)|(?<=\w)(?!\w)). If you want a different set of word characters than the default, just use that but with \w replaced by an appropriate character class.

Answer 4

First off I am indebted to Sinan (who isn't in regards to Perl on SO? I know I have been lurking a long time....) and ysth. Thanks to these two I have a better grasp of regexes. My solution however was the following...

my $pat = join '|', keys(%hashstore);
$doc =~ s!\b($pat)\b!$1/$hashstore{uc($1)}!ig;

Problem that I had was I replacing my replacements! Normally I really try to hash these things out but this was such a tight deadline and Sinan and ysth, you both rock severely! Amy