You know, if you give the address to the Google geolocation service it returns the latitude and longitud, so you can mark the place in the map or go there in the street view thing.
Anyway, even when the place showed in street view is kinda precise, it doesn't show the orientation properly. I mean, it's not "looking at" the house what matches the address, it's just... standing in front of the house and watching the street, or the sky, or whatever you want, but not the house.
Adjusting orientation is possible using the right parameters: This should be looking straight into a car in Helsinki, Finland.
However, I don't think it is possible to get the correct "looking at" perspective for arbitrary addresses in street view - I assume that information is simply too difficult to collect and would require man-years of manual correction.
I don't think there is any way except to manually correct the addresses you want to show.
I solved this calculating the position of the house I wanted to show, then the position of the next and the previous, so I got 3 points and I can calculate the perpendicular direction.