If you are serious about learning COM, Don Box's "Essential COM" is definitely an absolute "must read". COM can be confusing and in my humble opinion Don Box is one of the few people who actually "got it".
The example code in "Essential COM" is in C++. You won't find many books that support COM in C. You can write COM in C, but it will be very, very painful. COM is optimized for C++ development.
This book is not perfect nor "complete". There are some (granted, a bit esoteric) areas that the book skims over. For example, the book has like 1 1/2 pages on "monikers" (I have never seen a treatment of monikers that satisfies me). I consider this book to be THE fundamental book.
Second, in real life you are likely to want to use a supporting library such as ATL, rather than writing all the COM glue directly. There are too many ways to make subtle mistakes in COM even in the basic set up. ATL will give you good patterns and implement the boring code for you. In learning, you are better off using plain C++.
There are many books about ATL and several are quite good. I understand that ATL has changed quite a bit since the old days of VC++6, but I don't have first hand knowledge there: sadly, most of the COM code I work with is forever locked to the flavor of C++ in VC6.
Make sure whatever book you get is written for the version of Visual Studio and/or ATL you are planing on using.
Some background on COM books:
Note that there are a lot of books out there that misunderstand COM, or focus on the wrong things. The older books are worse in this respect. Some of the first few books treated COM as little more than a plumbing detail needed to make OLE work ("Object Linking and Embedding", that's what allows you to drag-and-drop a spreadsheet range into a Word document). Because of that, a lot of the material out there is very confusing. It took a while before people realized that OLE wasn't that important and that COM really was.
By the time Don Box published "Essential COM", the cracks on the foundation of COM had started to become evident. There isn't anything terribly flawed with COM, but the needs of the development community had evolved and outgrown what COM could do without serious revamping.
.NET was born out of that effort to address the limitations in COM, especially in the area of "type information". Just a few years after "Effective COM" was published, the attention of the community shifted away to .NET. Because of that, good COM training material is now and will likely remain forever limited.
So, COM is not broken, and it works great for the things it's used for (that's why Explorer still uses it). It's just not the best solution anymore for many of the problems that need solving today.
In summary:
I recommend "Essential COM" for the basics. Then, any of many good ATL books available (no strong preferences there), and then use other resources like MSDN or -- of course -- Stack Overflow, to cover areas that are of particular interest to you.
If you'd rather avoid relying on resources of the dead-tree variety, go ahead and learn ATL from the web. But some books are worth reading the old fashioned way -- and "Essential COM" is one of them.
Good luck.