I agree with your concern. The challenge is to maximize the amount of time you can spend thinking through and visualling your problem, and minimizing the amount of time you spend figuring out and micro- (or macro-) adjusting the tool. I always find myself spending far too much time thinking about the tool.
Visio is popular for the reason that there are a lot of preconstructed pieces, but it can get to be overwhelming pretty quickly, particularly if you try to use the semantic pieces (i.e. are more than just a connectible element); you end up needing to understand the implications and restrictions of the semantic model you are (perhaps unintentionally) building.
(In particular, I wouldn't use something with the name "UML" on it unless I intentionally wanted to develop UML diagrams, with all that implies.)
Same thing for database diagrams. If you understand ERD and the relational model and your goal is a picture of a coherent database design, then those tools are the right answer. But they quickly get beyond the point where non-database people have a clue.
I'd start with paper diagrams that you think are good representations of something you want to explain. Then take the candidate tools and see how quickly and easily you can duplicate what's on paper, accurately, completely, and pleasingly. (Doing it this way might also make some of your bad examples, including e.g. visio, more palatable.)
The other suggestion would be to take some of the examples you describe and compare your work to, and try to discover how they were created.