Max number of socket on Linux

Question

It seems that the server is limited at ~32720 sockets... I have tried every known variable change to raise up this limit. But the server stay limited at 32720 opened socket, even if there is still 4Go of free memory and 80% of idle cpu...

Here's the configuration

~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 63931
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 798621
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 2048
cpu time               (seconds, -t) unlimited
max user processes              (-u) 63931
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

net.netfilter.nf_conntrack_max = 999999
net.ipv4.netfilter.ip_conntrack_max = 999999
net.nf_conntrack_max = 999999

Any thoughts ?

Answer 1

On Gnu+Linux, maximum is what you wrote. This number is (probably) stated somewhere in networking standards. I doubt you really need so many sockets. You should optimize the way you are using sockets instead of creating dozens all the time.

Answer 2

Which server are you talking about ? It might be it has a hardcoded max, or runs into other limits (max threads/out of address space etc.)

http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1 has some tuning to needed to achieve a lot of connection, but it doesn't help if the server application limits it in some way or another.

Answer 3

If you're considering an application where you believe you need to open thousands of sockets, you will definitely want to read about The C10k Problem. That page discusses many of the issues you will face as you scale up your number of client connections to a single server.

Answer 4

In net/socket.c the fd is allocated in sock_alloc_fd(), which calls get_unused_fd().

Looking at linux/fs/file.c, the only limit to the number of fd's is sysctl_nr_open, which is limited to

int sysctl_nr_open_max = 1024 * 1024; /* raised later */

/// later...
sysctl_nr_open_max = min((size_t)INT_MAX, ~(size_t)0/sizeof(void *)) &
                         -BITS_PER_LONG;

and can be read using sysctl fs.nr_open which gives 1M by default here. So the fd's are probably not your problem.

edit you then probably checked this as well, but would you care to share the output of

#include <sys/time.h>
#include <sys/resource.h>
int main() {
    struct rlimit limit;
    getrlimit(RLIMIT_NOFILE,&limit);
    printf("cur: %d, max: %d\n",limit.rlim_cur,limit.rlim_max);
}

with us?

Answer 5

If you're dealing with openssl and threads, go check your /proc/sys/vm/max_map_count and try to raise it.