It's to denote that the header isn't system-wide.
This is a convention, not a requirement.
By the way, those aren't inverted commas, they're quotation marks. There is a difference in the field of typography.
At least for C, it makes no difference nowadays. The ISO standard states that the location of the files is implementation defined in both cases.
The usual way is to use <> for system headers (things under /usr/include for example) and "" for your own headers, but it's not required.
The relevant bits of C99 are from 6.10.2, "Source file inclusion", quoted below.
A preprocessing directive of the form
# include <h-char-sequence> new-line
searches a sequence of implementation-defined places for a header identified uniquely by
the specified sequence between the < and > delimiters, and causes the replacement of that
directive by the entire contents of the header. How the places are specified or the header
identified is implementation-defined.
A preprocessing directive of the form
# include "q-char-sequence" new-line
causes the replacement of that directive by the entire contents of the source file identified
by the specified sequence between the " delimiters. The named source file is searched
for in an implementation-defined manner. If this search is not supported, or if the search
fails, the directive is reprocessed as if it read
# include <h-char-sequence> new-line
with the identical contained sequence (including > characters, if any) from the original
directive.
Yeah, from what I've heard, angle brackets (<'s) are used to denote that the header was provided with the compiler, OR that the compiler has been told about a directory in which the header file can be found (-I). Quotes ("'s) are usually used for header files within the source tree. But like others have mentioned, it's not a requirement.