How to detect a C++ identifier string?

Question

E.g:

isValidCppIdentifier("_foo") // returns true
isValidCppIdentifier("9bar") // returns false
isValidCppIdentifier("var'") // returns false

I wrote some quick code but it fails: my regex is "[a-zA-Z_$][a-zA-Z0-9_$]*" and I simply do regex.IsMatch(inputString).

Thanks..

Answer 1

It should work with some added anchoring:

"^[a-zA-Z_][a-zA-Z0-9_]*$"

If you really need to support ludicrous identifiers using Unicode, feel free to read one of the various versions of the standard and add all the ranges into your regexp (for example, pages 713 and 714 of http://www-d0.fnal.gov/~dladams/cxx_standard.pdf)

Answer 2

Matti's answer will work to sanitize identifiers before inserting into C++ code, but won't handle C++ code as input very well. It will be annoying to separate things like L"wchar_t string", where L is not an identifier. And there's Unicode.

Clang, Apple's compiler which is built on a philosophy of modularity, provides a set of tokenizer functions. It looks like you would want clang_createTranslationUnitFromSourceFile and clang_tokenize.

I didn't check to see if it handles \Uxxxx or anything. Can't make any kind of gurarantees. Last time I used LLVM was five years ago and it wasn't the greatest experience… but not the worst either.

On the other hand, GCC certainly has it, although you have to figure out how to use cpp_lex_direct.