If you had both the IP's and they were both routable, it should be possible. I'm not sure though if the IP's given out by the 3G network are routable. If you are using WIFI, the IP you get is almost certainly not routable. Assuming you have a routable IP though, it should be possible.
The trick is just discovering the IP of the other device. The best way to do that would probably be to have each device register its IP with a web service when your app starts up, then query that service to find the IPs of other devices.
It might get a bit tricky managing "unregistering" IPs when the app is closed or when an IP changes though, but it should be doable.
ETA:
If you have a server that they can both connect to, you should be able to implement a solution using that server (i.e. without the phones actually directly connecting to each other). This would avoid the issue of having routable IPs for the phones.
As for how to bridge the sockets together - your server program would basicly work like an old-school phone switchboard operator. You'd have some kind of registration & discovery protocol built into the server software that would allow a phone to register itself with the server, query a list of the other phones currently connected to the server, indicate which phone it wants to talk to, and accept connections from other phones.
Once it knows who wants to talk to whom, your server software would connect the two together by reading from one phone's socket and writing that data to the other phone's socket.