views:

229

answers:

2

I need a regular expression that matches UTF-8 letters and digits, the dash sign (-) but doesn't match underscores (_), I tried these silly attempts without success:

  • ([\w-^_])+
  • ([\w^_]-?)+
  • (\w[^_]-?)+

The \w is shorthand for [A-Za-z0-9_], but it also matches UTF-8 chars if I have the u modifier set.

Can anyone help me out with this one?

+5  A: 

Try this:

(?:[\w\-](?<!_))+

It does a simple match on anything that is encoded as a \w (or a dash) and then has a zero-width lookbehind that ensures that the character that was just matched is not a underbar.

Otherwise you could pick this one:

(?:[^_\W]|-)+

which is a more set-based approach (note the uppercase P)

OK, I had a lot of fun with unicode in php's flavor of PCREs :D Peekaboo says there is a simple solution available:

[\p{L}\p{N}\-]+

\p{L} matches anything unicode that qualifies as a Letter (note: not a word character, thus no underbars), while \p{N} matches anything that looks like a number (including roman numerals and more exotic things).
\- is just an escaped dash. Although not strictly necessary, I tend to make it a point to escape dashes in character classes... Note, that there are dozens of different dashes in unicode, thus giving rise to the following version:

[\p{L}\p{N}\p{Pd}]+

Where "Pd" is Punctuation Dash, including, but not limited to our minus-dash-thingy. (Note, again no underbar here).

dionadar
Good one, thanks!
Alix Axel
will negating \W not include hypen ?
codaddict
@dionadar - this doesn't match accented characters for me.
meder
@codadict As far as I know, the hyphen is not included in \w - and even if it was, it would not hurt to state it like this ;)
dionadar
@meder OP states: "The \w [...] also matches UTF-8 chars if I have the u modifier set."
dionadar
@meder: Both the regexes work for me, maybe it's the PHP version you're testing with?
Alix Axel
@Alix, dionadar - perhaps I just missed something.. output http://medero.org/dump/i18n.php code http://medero.org/dump/code.txt are the accented characters matching for you guys?
meder
@meder: Yes, see http://www.rubular.com/regexes/12922, http://www.rubular.com/regexes/12923 and http://www.rubular.com/regexes/12924. I get the exact same matches PHP 5.3.0 / Windows 7.
Alix Axel
@meder: My guess is that your `i18n.php` file is not encoded with UTF-8 no BOM?
Alix Axel
@meder, Alix: Substituting the \w with \p{L} should reduce incompatibility problems, will update answer asap
dionadar
@dionadar: The problem with `\p{L}` is that it matches only letters - I also need numbers, and I have no idea what the difference between `\p{Nd}`, `\p{Nl}` and `\p{No}` is. If you do, please let me know.
Alix Axel
\p{N} includes all kinds of numbers - afaik Nd does the 0-9 dance, while Nl includes roman literals (in unicode a roman 1 is not the letter I, but rather something that looks like it) and No is pretty much everything they could not find in the other two, but still is a number.
dionadar
@dionadar: Thanks for the explanation, I had already posted a related question: http://stackoverflow.com/questions/2062521/regex-unicode-properties-reference-and-examples, hopefully I'll be able to understand `\p` classes a little better.
Alix Axel
Oh, on a sidenote: I tested the last version with an English-US php 5.something on a ubuntu vm against german umlauts (my spanish does not even go far enough to spell movie quotes) - and it works (w/o the /u switch i might add) even if the file is not even unicode encoded.
dionadar
+1  A: 

I am not sure which language you use, but in PERL you can simply write: [[:alnum:]-]+ when the correct locale is set.

Jiri Klouda
That's nice to know, but I'm using PHP (PCRE engine).
Alix Axel
Tried it in PHP and Rubular (Ruby), see http://www.rubular.com/regexes/12922 and http://www.rubular.com/regexes/12923.
Alix Axel
I've corrected a small mistake there.
Jiri Klouda
[:alnum:] could be replaced with \p{IsAlnum} in PCRE you could try \p{L}\p{N}
Jiri Klouda