The name "C sharp" was inspired from musical notation where a sharp indicates that the written note should be made a half-step higher in pitch.[6] This is similar to the language name of C++, where "++" indicates that a variable should be incremented by 1. The sharp symbol also resembles a ligature of four "+" symbols (in a two-by-two grid), further implying that the language is an increment of C++.
Due to technical limitations of display (standard fonts, browsers, etc.) and the fact that the sharp symbol (♯, U+266F, MUSIC SHARP SIGN) is not present on the standard keyboard, the number sign (#, U+0023, NUMBER SIGN) was chosen to represent the sharp symbol in the written name of the programming language.[7] This convention is reflected in the ECMA-334 C# Language Specification.3 However, when it is practical to do so (for example, in advertising or in box art[8]), Microsoft uses the intended musical symbol.
From Wikipedia.
Also, in this interview, Anders Hejlseberg says:
"We wanted to have a reference to the language’s C heritage in the name and finally settled on C#. Some other candidates I recall were e-C, Safe C, C-square, C-cube, C-prime, C-star, and Cesium… Looking and those now I’m pretty happy with our choice."
Finally, Naomi Hamilton asked Anders Hejlseberg the question directly:
[NH] Why was the language originally named Cool, and what promoted the change to C#?
[AH] The code name was Cool, which stood for ‘C like Object Oriented Language’. We kind of liked that name: all of our files were called .cool and that was kind of cool! We looked seriously at keeping the name for the final product but it was just not feasible from a trademark perspective, as there were way too many cool things out there.
So the naming committee had to get to work and we sort of liked the notion of having an inherent reference to C in there, and a little word play on C++, as you can sort of view the sharp sign as four pluses, so it’s C++++. And the musical aspect was interesting too. So C# it was, and I’ve actually been really happy with that name. It’s served us well.