The difference between single and double quotes in PHP is that double quotes are "intelligent" in that they will parse for variables when being read, while single quotes are "dumb" and will not try to parse any character in the string.
These result in some minor differences in what characters you can use; basically, the only character you need to escape when using single quotes is a single quote itself:
'\''
While if you use double quotes you have to escape other characters:
"\$"
But it also allows for some nifty things like adding a new-line to the end:
"my string\n"
With single quotes you would have to do a concatenation:
'my string' . chr(10)
'my string' . "\n"
Generally, single quotes are faster because they are "dumb".
However, normally one should not really worry about these issues, that is called Premature optimization, and should be avoided.
A couple of words about optimization: generally one should first write the program the way it should work, and then find the biggest bottlenecks and fix those particular ones. If string speed really is an issue for you in PHP, you might want to consider switching to another language.
Regarding speed: you probably want to focus more on memory usage than on CPU time. In these cases the CPU time could be considered pretty constant. CPU time is more relevant when writing algorithms that will iterate many times.
Regarding concatenations: the more you concatenate strings using the dot-operator, the more memory you will be using.
Consider this:
$str1 = 'asdf';
$str2 = 'qwer';
// this will result in more memory being allocated for temporary storage
echo $str1 . $str2;
// this will not allocate as much memory as the previous example
echo $str1;
echo $str2;