Java Web Server with Jetty - TCP Connections Taking Long

Question

I have an application with fairly high traffic (20K req/min) running on the JVM with a Jetty servlet container on Ubuntu. Below is my Jetty configuration:

10 20 2000 2

When I analyze the network traffic, I realize that sometimes it is taking long to establish TCP connections on the port that Jetty is running. The long connections are varying between 3.0s and 9.0s. The port is configured to accept MAX number of TCP connections. Do you know what might be causing the delay in accepting connections?

Thanks

Answer 1

Unfortunatly you didn't give more technical background.

I assume you face a very high system load, since the connections are established the backlog (parameter for listen system call [ServerSocket]) is high enough.

Accepting connections always involve a switch to kernel mode, this may cause a high system load and can be resolved only by scaling the application (using load balancing ) i.e you need bigger machines or more of them.

Answer 2

I believe you're using a "connection per request" model. If so, try to look into following:

• make persistent connections, where the same requester reuses the connection
• switch to UDP messages
• use a load balancer+cluster on two+ boxes

Kernel/application tuning may help, but at 20K/min => 330 TPS I'd say you have to look at different architectural solution, not just tuning. Establishing a TCP connection is a pretty heavy operation, which involves kernel machinery, and thus context switch.