Christopher's answer is partially correct.
Programs A and B actually have a handle to a socket descriptor stored in the underlying OS's socket implementation. Packets are delivered to this underlying socket, and then any process which has a handle to that socket resource can read or write it.
For example, say you are writing a simple server on a Unix like OS such as Linux or Mac OSX.
Your server accepts a connection, at which point a connection consisting of
( src IP, src Port, dest IP, dest Port )
comes in to existence in the underlying OS socket layer. You then fork a process to handle the connection - at this point you now have two processes with handles to the socket both of which can read / write it.
Typically ( always ) the original server will close it's handle to the socket and let the forked process handle it. There are many reasons for this, but the one that is not always obvious to people is that when the child process finishes it's work and closes the socket the socket will stay open and connected if the parent process still has an open handle to it.