Matching duplicate whitespace with preg_replace

Question

I'm writing a WordPress plugin, and one of the features is removing duplicate whitespace.

My code looks like this:

return preg_replace('/\s\s+/u', ' ', $text, -1, $count);

• I don't understand why I need the u modifier. I've seen other plugins that use preg_replace and don't need to modify it for Unicode. I believe I have a default installation of WordPress .

• Without the modifier, the code replaces all the spaces with Unicode replacement glyphs instead of spaces.

• With the u modifier, I don't get the glyphs, and it doesn't replace all the whitespace.

Each space below has from 1-10 spaces. The regex only removes on space from each group.

Before:

This sentence  has extra space.  This doesn’t.  Extra  space, Lots          of extra space.

After:

This sentence has extra space. This doesn’t. Extra space, Lots         of extra space.

$count = 9

How can I make the regex replace the whole match with the one space?

---

Update: If I try this with regular php, it works fine

$new_text = preg_replace('/\s\s+/', ' ', $text, -1, $count);

It only breaks when I use it within the wordpress plugin. I'm using this function in a filter:

function jje_test( $text ) {
    $new_text = preg_replace('/\s\s+/', ' ', $text, -1, $count);
    echo "Count: $count";
    return $new_text;
}

add_filter('the_content', 'jje_test');

I have tried:

• Removing all other filters on the_content
  remove_all_filters('the_content');
• Changing the priority of the filter added to the_content, earlier or later
• All kinds of permutations of \s+, \s\s+, [ ]+ etc.
• Even replacing all single spaces with an empty string, will not replace the spaces

Answer 1

The u modifier simply puts it into UTF-8 mode, which is useful if you need to do anything specific with characters that have a code point above 0x7f. You can still work on UTF-8 encoded strings without using that modifier, you just won't be able to specifically match or transform such characters easily.

There are some whitespace characters in Unicode that are above 0x7f. It's pretty rare to encounter them in most data. But you may see, for example, a non-breaking space character, which is unicode \uA0, or some rarer characters.

I don't know why using it would cause Unicode "replacement" glyphs to be output. I'd say it would be a problem elsewhere... what character encoding are you outputting your script as?

Answer 2

Don't know about any modifiers, but this did the trick:

<?php
$text = ' Hi,   my name is    Andrés.  ';
echo preg_replace(array('/^\s+/', '/\s+$/', '/\s{2,}/'), ' ', $text);
/*
Hi, my name is Andrés.
*/
?>

Answer 3

preg_replace('!\s+!', ' ', 'This sentence  has extra space.  This doesn’t.  Extra  space, Lots          of extra space.');

Answer 4

This will replace all sequences of two or more spaces, tabs, and/or line breaks with a single space:

return preg_replace('/[\p{Z}\s]{2,}/u', ' ', $text);

You need the /u flag if $text holds text encoded as UTF-8. Even if there are no Unicode characters in your regex, PCRE has to interpret $text correctly.

I added \p{Z} to the character class because PCRE only matches ASCII characters when using shorthands such as \s, even when using /u. Adding \p{Z} makes sure all Unicode whitespace is matched. There might be other spaces such as non-breaking spaces in your string.

I'm not sure if using echo in a WordPress filter is a good idea.

Answer 5

To answer jjeaton's follow-up question in the comments to my first reply, the following replaces each sequence of spaces, tabs, and/or line breaks with the first character in that sequence. Effectively, this deletes the second and following whitespace characters in each sequence of two or more whitespace characters. A run of spaces is replaced with a single space, a run of tabs is replaced with a single tab, etc. A run of a space and a tab (in that order) is replaced with a space, and a run of a tab and a space is replaced with a tab, etc.

return preg_replace('/([\p{Z}\s])[\p{Z}\s]+/u', '$1', $text);

This regex works by first matching one space and capturing it with a capturing group, followed by one or more spaces. The replacement text is simply reinserts the text matched byt the first (and only) capturing group.