Considering this question of SO, where whole C# in-memory compiler is being called. When only lexical and syntactic analyzing is required: parse text as a stream of lexemes, check them and exit.
Is it possible in current version of System.CodeDom.Compiler, if not - will it be?
If you can use Mono, I believe it has a C# parser/lexer you may be able to use.
Here's a link to look into. As for what the MS C# team is planning to do, there is some talk of at some point making the C# compiler into a "service" - but it's unclear what that means or when that will happen.
While it might look like the code is compiled in-memory (CompilerParameters.GenerateInMemory), that's not what actually happens. The same compiler as the one used in Visual Studio is used to compile the code (csc.exe). It gets started by CreateProcess (much like Process.Start) and runs out-of-process to compile the code to an assembly on disk in a temporary folder. The GenerateInMemory option invokes Assembly.LoadFrom() to load the assembly.
You'll get the equivalent of a syntax check simply by setting GenerateInMemory to false and delete the OutputAssembly after it is done.
While this might sound kinda backwards, the huge benefit it has is that this won't put any memory pressure on your process. This will hold you over until C# 5.0 ships.