php isn't really my thing, but it seems like you need to clear the response stream before writing out your content. Perhaps something else is writing html out to the stream, before reaching this function? Like a template or master page or something of that nature?
The HTML content looks like a typical header/nav bar. if there something else that's automatically including that content that you need to disable for this route?
You've got a control flow problem somewhere - it seems the html head part and navigation is included by default in any page, including in the one that generates the CSV. One solution would be to check for a CSV request and if that's the case, don't include the html code, another one would be to use output buffering and then discard all the previous output right before outputting the CSV data.
I suggest initializing the csv_output variable to the empty string at the start of your method. All of your operations in the method are concatenations, so you are likely bringing along some old data.
My guess is that you've got some sort of template that generates the same HTML header and footer regardless of the page that is requested. Sometime before the exportCSV function is called, the header is generated.
You don't show the bottom of the output, but I'll bet the footer is there too, since I suspect the flow control will continue on to that code after the function exits.
Please consider that you are showing potentially sensitive data to the entire world. You've got a dozen people who now have their email and street addresses published.