As quoted above, "If you don't test your private methods, how do you know they won't break?"
This is a major issue. One of the big points of unit tests is to know where, when, and how something broke ASAP. Thus decreasing a significant amount of development & QA effort. If all that is tested is the public, then you don't have honest coverage and delineation of the internals of the class.
I've found one of the best ways to do this is simply add the test reference to the project and put the tests in a class parallel to the private methods. Put in the appropriate build logic so that the tests don't build into the final project.
Then you have all the benefits of having these methods tested and you can find problems in seconds versus minutes or hours.
So in summary, yes, unit test your private methods.