Theres several factors that you need to look at.
Making sure that there's only 1 session per ip address is going to be the best way to do this, the only down side to this is if the user does not log out you may have issues.
What I would do is
Store logged in users in a table and track there last activity, if your site is a site where the user may not change page for some time then you should set up a javascript heartbeat witch just ping's your server from the browser so that PHP is always up to date.
If user A is logged in and another attempt from a different IP tries to log in, then you should deactivate user A's logged in status and allow user B to start a new session.
you should also track the IP Addresses a user uses to log in, so that you can have an incremented table like
user_id ip hits
--------------------------------------
1 xxx.yyy.zzz.244 6
1 xxx.yyy.zzz.224 4
1 xxx.yyy.zzz.264 1
1 xxx.yyy.zzz.124 1
1 xxx.yyy.zzz.174 1
2 aaa.bbb.zzz.678 3
.....
This way if a user A is logged in and User B attempts, you can run a check to see if this IP has been used before, if it has then you just run a direct logout, otherwise you can ask a secret question provided upon registration to help with the security side of things