Indeed, it is almost always worth avoiding this. The main valid time it might be useful is when dealing with an external library (from another language) that has a class / etc that happens to be a C# keyword, but I'll admit I've never had this problem in reality. It helps that C# allows you to rename method argument names when overriding / implementing interfaces, so that avoids one common set of cases.
Another (arguably lazy) use is in code generation; if you always prefix fields / variables / etc with @theName, then you don't have to worry about special cases. Personally, I simply cross-check against a list of C# keywords / contextual keywords, and add something like underscore.
The @ symbol's other use (verbatim string literals) is @"much more useful".