Linux High-Traffic Tuning: Raise ulimit, Tune sysctl TCP Settings and Fix "nf_conntrack: table full"

When a website or API gets busy, the first errors often have nothing to do with CPU or RAM: Too many open files, nf_conntrack: table full, dropping packet, connections that hang in SYN state or a listen queue that overflows. These are kernel and process limits whose defaults suit a small machine, not a server handling tens of thousands of connections. This guide shows how to find which limit you are hitting and how to raise the file descriptor limit (ulimit), tune the TCP stack with sysctl, size the connection tracking table and configure Nginx to match.

Step 1: Find the Limit You Are Hitting

# Kernel messages about dropped packets or full tables
dmesg -T | grep -iE 'conntrack|too many|possible SYN flooding' | tail

# Socket summary and listen queues (Recv-Q close to Send-Q = backlog is full)
ss -s
ss -lnt

# Listen queue overflows and dropped SYNs since boot
nstat -az | grep -iE 'ListenOverflows|ListenDrops|TCPReqQFull'

# Connection tracking usage vs maximum
cat /proc/sys/net/netfilter/nf_conntrack_count /proc/sys/net/netfilter/nf_conntrack_max 2>/dev/null

# Open file limit of the shell and of a running service (Nginx worker example)
ulimit -n
grep 'Max open files' /proc/$(pgrep -f 'nginx: worker' | head -1)/limits

Step 2: Raise the File Descriptor Limit

Every connection, log file and socket uses a file descriptor. The common default of 1024 per process is far too low for a busy proxy or database. Raise the system-wide maximum, the per-user limits for login sessions, and, most importantly, the limit of the systemd services themselves, because services ignore /etc/security/limits.conf.

# System-wide maximum
cat > /etc/sysctl.d/90-files.conf <<'EOF2'
fs.file-max = 2097152
fs.nr_open = 2097152
EOF2

# Login sessions (takes effect at the next login)
cat > /etc/security/limits.d/90-nofile.conf <<'EOF2'
*    soft nofile 1048576
*    hard nofile 1048576
root soft nofile 1048576
root hard nofile 1048576
EOF2

# A specific service, e.g. Nginx (repeat for mysqld, php-fpm, your app)
mkdir -p /etc/systemd/system/nginx.service.d
cat > /etc/systemd/system/nginx.service.d/limits.conf <<'EOF2'
[Service]
LimitNOFILE=1048576
EOF2
systemctl daemon-reload && systemctl restart nginx

Step 3: Tune the TCP Stack with sysctl

The following values are a sensible starting point for a busy web, proxy or API server. Put them in their own file so you can see and revert your changes easily.

cat > /etc/sysctl.d/91-network-tuning.conf <<'EOF2'
# Longer accept and SYN queues for bursts of new connections
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 16384

# More local ports for outgoing connections (proxies, crawlers, API clients)
net.ipv4.ip_local_port_range = 1024 65000
# Reuse TIME_WAIT sockets for new outgoing connections (safe; tcp_tw_recycle no longer exists)
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15

# Keepalive: detect dead peers after about 10 minutes instead of 2 hours
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 5

# Keep SYN cookies on as a basic protection against SYN floods
net.ipv4.tcp_syncookies = 1

# Bigger socket buffers for fast long-distance transfers
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.ipv4.tcp_rmem = 4096 87380 33554432
net.ipv4.tcp_wmem = 4096 65536 33554432

# BBR congestion control (kernel 4.9+)
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
EOF2

sysctl --system
sysctl net.core.somaxconn net.ipv4.tcp_congestion_control

For more background on BBR, see enabling BBR on Linux. Remember that applications pass their own backlog to listen(); Nginx, for example, needs listen 80 backlog=65535; to use the larger queue.

Step 4: Size the Connection Tracking Table

If you use firewalld, ufw, iptables NAT or Docker, the kernel tracks every connection in the conntrack table. When it fills up, new packets are dropped and the site looks "randomly" unreachable. The default maximum depends on RAM and is often only 65536 or 262144.

# Load the module so the keys exist, and raise the limits
modprobe nf_conntrack
cat > /etc/sysctl.d/92-conntrack.conf <<'EOF2'
net.netfilter.nf_conntrack_max = 1048576
# Forget idle established connections after 2 hours instead of 5 days
net.netfilter.nf_conntrack_tcp_timeout_established = 7200
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30
EOF2
sysctl --system

# Hash table size (about max / 4), applied at module load
echo 'options nf_conntrack hashsize=262144' > /etc/modprobe.d/nf_conntrack.conf
echo 262144 > /sys/module/nf_conntrack/parameters/hashsize

# Load the module at boot so the sysctl values are applied
echo nf_conntrack > /etc/modules-load.d/nf_conntrack.conf

Each conntrack entry uses roughly 300 bytes of kernel memory, so one million entries cost around 300 MB. On a pure web server without NAT you can also avoid tracking entirely for ports 80/443 with a notrack rule, but the larger table is the simpler fix.

Step 5: Match the Nginx Configuration

# /etc/nginx/nginx.conf (main context and events block)
worker_processes auto;
worker_rlimit_nofile 1048576;

events {
    worker_connections 65535;
    multi_accept on;
}

http {
    keepalive_timeout 30;
    keepalive_requests 1000;
    # ...
}

# In each server block, let the listen socket use the bigger backlog
#   listen 80 backlog=65535;
#   listen 443 ssl backlog=65535;

nginx -t && systemctl reload nginx
grep 'Max open files' /proc/$(pgrep -f 'nginx: worker' | head -1)/limits

Step 6: Verify Under Load

Re-run the commands from Step 1 during peak traffic. ListenOverflows should stop increasing, the conntrack count should stay well below the maximum, and the "too many open files" errors should disappear from your application logs. For ongoing monitoring of CPU, memory, disk and bandwidth, see our server monitoring guide.

FAQ

Which value matters most?

For proxies and APIs it is usually the file descriptor limit of the service and the conntrack table; for sites with sudden bursts of visitors it is the listen backlog (somaxconn plus the backlog in Nginx). Step 1 tells you which one you are hitting.

sysctl says "cannot stat /proc/sys/net/netfilter/nf_conntrack_max"

The conntrack module is not loaded yet. Run modprobe nf_conntrack and add it to /etc/modules-load.d/ as shown in Step 4.

Do I still need these settings if I use Cloudflare or a CDN?

A CDN reduces the number of connections that reach your server, but the origin still needs enough file descriptors and backlog for the connections the CDN opens, and for any API or WebSocket traffic that is not cached.

Still stuck after following these steps? Open a support ticket and the IMIDC 24/7 technical team will help. Include the server IP, OS version, the error messages you see and the output of the Step 1 commands so we can pinpoint the issue faster.