The reason you are having problems is because the_permalink() and the_title() are echoing functions. Instead use get_permalink() and $post->post_title. Remember get_permalink() requires the post id ($post->ID) as a parameter.
This explains why the second example works in your initial question. If you call an echoing function from within a string, the echo will output before the end of the string.
So this:
echo 'static content '.echoing_function().' more static content';
will not work as expected, instead, it will output this:
*echoing_function output* static content more static content
In PHP you can use both single and double quotes. When I am building strings with HTML I generally begin the string with a single quote, that way, I can use HTML-compatible double quotes within the string without escaping.
So to round it up, it would look something like:
echo '<li><a href="'.get_permalink($post->ID).'">'.$post->post_title.'</a></li>';
Or as you originally requested, to simply escape them, put a backslash before the quote. Like so (the single quotes have been removed)
echo "<li><a href=\"".get_permalink($post->ID)."\">".$post->post_title."</a></li>";
This is of course assuming you are calling this from within the loop, otherwise a bit more than this would be required to get the desired output.