tags:

views:

940

answers:

7

Hi,

I'm writing a TCP server that needs to know which interface each connection arrived from. I cannot use the address/subnet to deduce which interface was used, since there might be interfaces with the same address/subnet values. It's Linux based, and there's no need for the code to be portable.

All I could find were functions to get all interfaces, or a single interface by index. I could not find any way to get the interface associated with an accepted TCP socket.

Any ideas? Something I've missed?

EDIT: To reiterate, IP addresses are not unique in my case. Neither the destination addresses (the server itself) nor the source addresses (the clients). Yes, this is a very extreme IP scheme.

A: 

Look at the destination address.

Each interface is normally bound to a unique address. If multiple interfaces are bonded together, it probably doesn't matter which one it used.

The only exception to this, is when using ipv6 anycast, but even then, you wouldn't normally have multiple interfaces on the same host with the same ip.

JimB
In this case, the interfaces are not bound to unique addresses. They are not bonded either. They are connected to different networks, although they have overlapping/identical IPs.
Leeor Aharon
If you have a tcp connection, you must have an ip address. You don't have to bind a socket to a particular address, but your interface needs one
JimB
Yes, the connection has an IP address, but it's not unique. It's not unique for that interface, and it's not unique for that network. I really need a way to get the interface from the socket itself.
Leeor Aharon
It seems you have an unusual case. I'll leave my answer as is, as it's logical for most people's cases, and @Aiden answered with what I was going to add.
JimB
A: 

I think using getsockname() after accept()ing the incoming connection might be what you're after. The two functions getsockname() and getpeername() get the local and remote addresses respectively that a socket is bound to. Both should be valid for a fully connected TCP socket.

Edit: Whilst this seems to be true for OpenBSD according to the man page, the Linux man page differs considerably and so getsockname() after accept() on Linux is almost certainly unuseful. Teaches me for using my memory instead of checking everything. sigh

Wuggy
+2  A: 

The kernel routing table decides which interface to send a packet out on, hence the ability to bond devices. A cursory glance through "Linux Socket Programming, Warren W. Gay" suggests that specifying an interface is bad, and that due to the dynamics of the kernel (firewall, forwarding) it is more complex.

I would suggest altering your IP scheme such that the IP information tells you your interface(s) through looking up in the same way ifconfig does, otherwise you are shooting yourself in the foot design wise.

1) Get the IP information from the TCP session 2) Lookup which interface(s) this could be valid for

I will keep looking in the kernel API though. You shouldn't need to know this, the abstraction is there for a multitude of good reasons.

Extra Thought Pondering on this, it seems that if both interfaces use the same IP then there must be a client address range routing difference (otherwise both interfaces would be used). Your server could examine the routing table based on the client IP

Aiden Bell
The client IPs can overlap as well. The difference that allows these IPs to overlap is the VLAN id. As far as I know, that's inaccessible as well.
Leeor Aharon
you may have found the one situation where you can't use circumstantial evidence to determine interface. I don't think there will be a way, as the interface is kind of abstract from applications higher than the kernel. You could maybe modify your kernel to provide some data through /proc/tcp_sessions_to_devs or something but that is VERY - erm - desperate?
Aiden Bell
Another thought. Trying looking into the interface sniffing libraries that Wireshark uses (libpcap) and look at how the source of iftop enumerates sessions on a device. Might be interesting.
Aiden Bell
+2  A: 

In general, you shouldn't need to know what interface the packets are going to be sent/received on; that's the kernel's routing table's job. It's difficult to find out the interface for a socket because there really is no direct association. The routing of packets can change within the socket's lifetime based on routing information.

For datagram (UDP) sockets, you may be able to use getsockopt(s, IPPROTO_IP, IP_PKTINFO, ...); see getsockopt(2) and ip(7).

For stream (TCP) sockets, one option might be to open multiple listening sockets, one for each interface on the system, and use setsockopt(s, SOL_SOCKET, SO_BINDTODEVICE, ...) to bind each to one interface; see setsockopt(2) and socket(7).

Kieron
Yes, setting separate socket for each interface would work, but seeing as there are dozens of interfaces and they are brought up and taken down dynamically, it's something I want to avoid.
Leeor Aharon
A: 

Obviously not something I've looked into very deeply, let alone tried, this might be one for the "so crazy it just might work" basket...

If it's really only ever going to be for Linux, you could write a custom netfilter module which tracks the incoming connections and notes which interface they come in on, and writes that information somewhere for your server application to read.

Kieron
A: 

Kieron's suggestion to write a netfilter module is probably one way to try, but I would like to refrain from writing my very first kernel module for this solution.

I've come up with another option of using source NAT and translating the source port of the connection to correlate with the connection source. I can assign port ranges for each network and check it in the server. The only problem is that source NAT in iptables is done in the POSTROUTING chain, and I'm not sure it's used for connections that are accepted by that host, so I might need to use another server.

No easy solutions here, too bad I can't get the interface name/index from the socket...

Leeor Aharon
Another netfilter option is to forward the traffic to additional local interfaces that have been set up with distinct IP adresses.
Bell
A: 

I am adding another answer and a potential solution after looking through the source of Wireshark and iftop which seem to have indirectly similar functionality.

Looks to me that you can use libpcap to sniff on interfaces. Presuming you can identify some unique part of the TCP/IP session then you can track it down to an interface fairly simply using filters and session tracking.

No kernel modules (and it plays nice with threads)

http://www.ex-parrot.com/pdw/iftop/ Some simple source to have a peek at www.tcpdump.org/ for libpcap

I think you will be able to match VLANs using it too.

Also wireshark might be useful for debugging. Hope this helps! It's been on my brain since.

Aiden Bell
Yep, that sounds like another option. Thanks for the effort!
Leeor Aharon