Conntrack limit #27

New Issue

Ben Wright · 2025-02-17T15:01:47Z

Ben Wright commented

2025-02-17 15:01:47 +00:00

We are aware of an issue with Conntrack, which is a core feature within the Linux Kernel's networking stack.

Conntrack is an intrinsic part of Netfilter, which is the packet filtering framework within Linux. Conntrack allows the kernel to keep track of all logical network connections or sessions, and thereby relate all the packets which may make up this connection. Each connection tracked by Conntrack requires system memory, storing details such as source and destination IP addresses, port number pairs, protocol types, state, and timeout.

The connection tracking system uses these stored entries to quickly process future packets that are part of the same connection to improve performance. The Conntrack limit is the maximum number of simultaneous connections that the kernel will track. If this limit is reached, new connections will be denied and network traffic will be significantly affected. Default value for maximum size varies based on OS and/or memory installed, typically 64435 or 262140, which is very quickly exceeded by some of our customers.

The initial value of this limit is calculated based on system memory at boot time and would be proportionally set according to the available system resources. This limit has to been manually set to 4194304. This value is supposed to be set on boot, however there are circumstances based around order of execution meaning the value can be set before Conntrack is loaded in some instances.

Our Research and Development team are actively working on a new package to deploy, which will prevent a re-occurance. This package will be installed on every EVX, Network Fabric Router, Managed Linux based hardware, Virtual Machines and container that we have in our estate to constantly check and update the relevant values.

We are aware of an issue with Conntrack, which is a core feature within the Linux Kernel's networking stack. Conntrack is an intrinsic part of Netfilter, which is the packet filtering framework within Linux. Conntrack allows the kernel to keep track of all logical network connections or sessions, and thereby relate all the packets which may make up this connection. Each connection tracked by Conntrack requires system memory, storing details such as source and destination IP addresses, port number pairs, protocol types, state, and timeout. The connection tracking system uses these stored entries to quickly process future packets that are part of the same connection to improve performance. The Conntrack limit is the maximum number of simultaneous connections that the kernel will track. If this limit is reached, new connections will be denied and network traffic will be significantly affected. Default value for maximum size varies based on OS and/or memory installed, typically 64435 or 262140, which is very quickly exceeded by some of our customers. The initial value of this limit is calculated based on system memory at boot time and would be proportionally set according to the available system resources. This limit has to been manually set to 4194304. This value is supposed to be set on boot, however there are circumstances based around order of execution meaning the value can be set before Conntrack is loaded in some instances. Our Research and Development team are actively working on a new package to deploy, which will prevent a re-occurance. This package will be installed on every EVX, Network Fabric Router, Managed Linux based hardware, Virtual Machines and container that we have in our estate to constantly check and update the relevant values.

Brandon Currell added the

Network software

label 2025-02-17 15:41:13 +00:00

Sign in to join this conversation.