I know I'm coming to this thread rather late, but I actually have run usability tests comparing a WYSIWYG editor (iWeb) to a non-WYSIWYG editor based mostly on markdown.
Here's what I've found that users struggle with when using markdown:
- Tags that require character-level precision. For example, a user's inclination is to put a space between the square brackets and parentheses when making a link--but that doesn't work. Likewise, lists only work when there is a space after the asterisk or dash.
- They get paragraphs fine, but users are often inclined to use a single linebreak (when formatting an address, for example) and these are ignored if you don't add two spaces to the line before. Not intuitive, and those two spaces are invisible.
- More complicated tags, like for links and images, slow them down more than simple ones (like for strong and em)--but they can eventually get it if there's a helpful guide.
- Users are often not confident that X will work or unsure of exactly what it will do.
- Users don't always understand paths, directories, files, file extensions, URLs, etc. This makes making links and images difficult.
Here's what I've found helps:
- Provide a clear and explicit guide to markdown
- Tell them they are using a markup language
- Style the text field so that it doesn't look like a WYSIWYG field (eg: use a fixed width font)
- Consider syntax highlighting
- Be clear about any situations that are counter-intuitive or require attention to detail
- Provide a preview option (it doesn't have to be real-time, but it must be easy and unobtrusive)
- Depending on the application, consider tweaking it slightly (like allowing a URL like www.example.com and adding http:// automatically)
- At the very least, provide buttons for the more complicated tags like images and links. It's trivially easy for the programmer to make a little dialog box that asks for the URL and the link text and inserts the code automatically. Consider making the language easy to understand. Instead of asking for the URL, ask "What web address would you like to link to?" Take a look at how Javascript WYSIWYG editors use non-technical language here.
- Give them an idea of why this is important. Explain that semantics and presentation are separated in web design, and that computers generate bad code--so they are helping the program understand the semantic structure of the document.
I was surprised to find that people were mostly getting it, with no guidance from me and just a short guide to the markup language. In a study with 22 total users, the average satisfaction was slightly lower for iWeb than with my application.
If you're talking about an entire WYSIWYG web design program, not just a little widget for text entry--they're not as usable as you'd think. There are so many little details, and so much can go wrong, even when the interaction is supposed to be intuitive.
Caveat: my participants were all college students of varying technical skill levels, but they were all more tech-savvy than the average user. They were also being compensated for their time, which may account for their interest in the task.
Edit Almost all of the above issues have been improved by the stuff in the second list. The only one I'm still really struggling with is the single linebreak problem.