In order to create a lan messenger in c# I need to detect the host computers online obviously using the same lan messenger software. I tried searching for a starting point to do so, but in vain. I need to know how to start the work.
Can you use Windows Connection Foundation? If so, you could use WCF to implement the WS-Discovery protocol. Here's a brief howto.
You could have your application listen on a predefined port, and when somebody connects to that port send back some kind of acknowledgement like the application name and IP.
So when a new client comes online, it would send out a broadcast a UDP packet advertising it's IP. Other clients would listen for these "announcements" on the relevant port, and send back to the sender their own IP (via some other port). So now the initial sender will be contacted by all running clients on the LAN with their IPs. The other clients could also record the IP of the sender of the "announcement" - this will let them keep an up to date list of other clients on the network too.
To keep the lists current, you could have all the clients broadcast their existance on the network at semi-random intervals (eg every minute + rand(10) seconds).
All of the above is assuming you're talking about clients on the same network. If you're doing this on the internet, you'll need some central point that will keep track of the ips of logged in clients.
There are two ways I've done this in the past: having all the clients connect to predefined host (easy, but requires some client configuration) and having the host (or client) broadcast their existence via the the 'broadcast' address (eg, 10.0.255.255) (hard, firewalls/NATs can make life painful, clients require no configuration).
But, yes -- if WCF implements a discovery protocol, go with that. Provided it does what you want, it's probably better than anything you (or most people, for that matter) could write.
Depending on if you plan on relying on a centralized server or not, you have options:
1) No Server: When a client comes online, he broadcasts its identity, asking for other clients to send theirs.
2) Centralized server: A new client connects, he registers to the server and downloads the list of clients. Each client is then notified (either through polling, a duplex contract if using WCF, or through basic socket connection) of the new client.
The first version would be based on UDP sockets. Notice that this does not work out of the local network as I believe those packets will not pass through routers. It is also probably a bad design because a large number of clients will just swamp the network with packets. But hey, it might just be enough for you.
Also WCF has a peer to peer support, it might be interesting for you. Here's is an article about it.
I think it is possible to ping the broadcast ip and get an answer from everyone. Or something like that. Could start with that, and then check if the clients are listening on some port or something.
Alternative is to let the clients tell the server that "we are still here!" every so often.
You might like using ZeroConf aka Bonjour, Rendevous, or sometimes Avahi.