I work for a "consulting" company, in effect, a team of us (usually the same group of people) get tasked to build something for someone requiring our services. You know, basically they pay our consulting company, and the company takes a portion, and pays us a percentage of it (fixed percentage for each employee of the firm in our contracts).
Pretty much we all work together and understand how we work. We have a cohesiveness about us that we know the best hours for eachother, trust each other enough to know if someone is leaving they are done with a portion, or will continue it at a later time. As long as your not missing deadlines you should be fine. There are days when I will cut out at 5, and be completely done with it for the day (until I start thinking about something and realize that I should get it in) at which time I VPN in and write it up. I get paid for 40 hours of work a week. This means I will work 40 hours a week (or more if I need to in order to make deadlines). It really is situational.
No one here can honestly say each and every hour they are getting paid they are gainfully employed with their tasks. It just doesn't happen. You're in a bad mood. You get distracted. You have to make pee-pee. Get coffee, smoke break, whatever. BUT, so long as you are meeting deadlines, no one should notice. Personally, IMO, if whoever you work for allocates 8 hours of work/day average to you, you really should have no reason not to get it done minus specific circumstances. Now if your getting allocated more work than your getting paid to do, you need to buckle down and get it done (if your gunning for a promotion) or talk to your employer about it and get your workload down (if you know they won't promote you, won't pay you the OT, etc.) Life is too short, spend time with family. If you have no wife, go find a girl somewhere (or boy). You'll get burnt out if you don't get a life outside of programming.
If your on a team, you should be having this discussion with them, or at least the supervisor. If your a lone wolf, talk with whoever signs your paychecks and writes your eval reports.
Sometimes I have a family even and need to cut out at 3pm one day, but, I'll make it up sometime in that week. I get paid when the job gets done though. So the sooner we get the job done, the sooner we can get started on the next contract, thus making more money. As with any career, its all a balancing act. You have to balance personal time with professional time.
However, your question is entirely circumstantial. Look at what everyone else is doing, the overall picture, the overall work environment, etc.