Descriptors and Apache

Available Languages: en

+ + +

Warning:

This document has not been fully updated + to take into account changes made in the 2.0 version of the + Apache HTTP Server. Some of the information may still be + relevant, but please use it with care.

+ +

A descriptor, also commonly called a file + handle is an object that a program uses to read or write + an open file, or open network socket, or a variety of other + devices. It is represented by an integer, and you may be + familiar with stdin, stdout, and + stderr which are descriptors 0, 1, and 2 + respectively. Apache needs a descriptor for each log file, plus + one for each network socket that it listens on, plus a handful + of others. Libraries that Apache uses may also require + descriptors. Normal programs don't open up many descriptors at + all, and so there are some latent problems that you may + experience should you start running Apache with many + descriptors (i.e., with many virtual hosts).

+ +

The operating system enforces a limit on the number of + descriptors that a program can have open at a time. There are + typically three limits involved here. One is a kernel + limitation, depending on your operating system you will either + be able to tune the number of descriptors available to higher + numbers (this is frequently called FD_SETSIZE). Or you + may be stuck with a (relatively) low amount. The second limit + is called the hard resource limit, and it is sometimes + set by root in an obscure operating system file, but frequently + is the same as the kernel limit. The third limit is called the + soft resource limit. The soft limit is always less + than or equal to the hard limit. For example, the hard limit + may be 1024, but the soft limit only 64. Any user can raise + their soft limit up to the hard limit. Root can raise the hard + limit up to the system maximum limit. The soft limit is the + actual limit that is used when enforcing the maximum number of + files a process can have open.

+ +

To summarize:

+ +

+ #open files <= soft limit <= hard limit <= kernel limit +

+ + +

You control the hard and soft limits using the + limit (csh) or ulimit (sh) + directives. See the respective man pages for more information. + For example you can probably use ulimit -n + unlimited to raise your soft limit up to the hard limit. + You should include this command in a shell script which starts + your webserver.

+ +

Unfortunately, it's not always this simple. As mentioned + above, you will probably run into some system limitations that + will need to be worked around somehow. Work was done in version + 1.2.1 to improve the situation somewhat. Here is a partial list + of systems and workarounds (assuming you are using 1.2.1 or + later).

+ +

BSDI 2.0
FreeBSD 2.2, BSDI 2.1+
Linux
Solaris through 2.5.1
AIX
SCO OpenServer
Compaq Tru64 UNIX/Digital UNIX/OSF
Others

BSDI 2.0

+ +

Under BSDI 2.0 you can build Apache to support more + descriptors by adding -DFD_SETSIZE=nnn to + EXTRA_CFLAGS (where nnn is the number of + descriptors you wish to support, keep it less than the hard + limit). But it will run into trouble if more than + approximately 240 Listen directives are used. This may be + cured by rebuilding your kernel with a higher + FD_SETSIZE.

+ +

FreeBSD 2.2, BSDI 2.1+

+ +

Similar to the BSDI 2.0 case, you should define + FD_SETSIZE and rebuild. But the extra Listen + limitation doesn't exist.

+ +

Linux

+ +

By default Linux has a kernel maximum of 256 open + descriptors per process. There are several patches available + for the 2.0.x series which raise this to 1024 and beyond, and + you can find them in the "unofficial patches" section of the Linux Information HQ. + None of these patches are perfect, and an entirely different + approach is likely to be taken during the 2.1.x development. + Applying these patches will raise the FD_SETSIZE used to + compile all programs, and unless you rebuild all your + libraries you should avoid running any other program with a + soft descriptor limit above 256. As of this writing the + patches available for increasing the number of descriptors do + not take this into account. On a dedicated webserver you + probably won't run into trouble.

+ +

Solaris through 2.5.1

+ +

Solaris has a kernel hard limit of 1024 (may be lower in + earlier versions). But it has a limitation that files using + the stdio library cannot have a descriptor above 255. Apache + uses the stdio library for the ErrorLog directive. When you + have more than approximately 110 virtual hosts (with an error + log and an access log each) you will need to build Apache + with -DHIGH_SLACK_LINE=256 added to + EXTRA_CFLAGS. You will be limited to + approximately 240 error logs if you do this.

+ +

AIX

+ +

AIX version 3.2?? appears to have a hard limit of 128 + descriptors. End of story. Version 4.1.5 has a hard limit of + 2000.

+ +

SCO OpenServer

+ +

Edit the /etc/conf/cf.d/stune file or use + /etc/conf/cf.d/configure choice 7 (User and + Group configuration) and modify the NOFILES + kernel parameter to a suitably higher value. SCO recommends a + number between 60 and 11000, the default is 110. Relink and + reboot, and the new number of descriptors will be + available.

+ +

Compaq Tru64 UNIX/Digital UNIX/OSF

+ +

Raise open_max_soft and + open_max_hard to 4096 in the proc subsystem. + Do a man on sysconfig, sysconfigdb, and + sysconfigtab.
Raise max-vnodes to a large number which + is greater than the number of apache processes * 4096 + (Setting it to 250,000 should be good for most people). + Do a man on sysconfig, sysconfigdb, and + sysconfigtab.
If you are using Tru64 5.0, 5.0A, or 5.1, define + NO_SLACK to work around a bug in the OS. + CFLAGS="-DNO_SLACK" ./configure

+ +

Others

+ +

If you have details on another operating system, please + submit it through our Bug Report + Page.

+ +

In addition to the problems described above there are + problems with many libraries that Apache uses. The most common + example is the bind DNS resolver library that is used by pretty + much every unix, which fails if it ends up with a descriptor + above 256. We suspect there are other libraries that similar + limitations. So the code as of 1.2.1 takes a defensive stance + and tries to save descriptors less than 16 for use while + processing each request. This is called the low slack + line.

+ +

Note that this shouldn't waste descriptors. If you really + are pushing the limits and Apache can't get a descriptor above + 16 when it wants it, it will settle for one below 16.

+ +

In extreme situations you may want to lower the low slack + line, but you shouldn't ever need to. For example, lowering it + can increase the limits 240 described above under Solaris and + BSDI 2.0. But you'll play a delicate balancing game with the + descriptors needed to serve a request. Should you want to play + this game, the compile time parameter is + LOW_SLACK_LINE and there's a tiny bit of + documentation in the header file httpd.h.

+ +

Finally, if you suspect that all this slack stuff is causing + you problems, you can disable it. Add -DNO_SLACK + to EXTRA_CFLAGS and rebuild. But please report it + to our Bug + Report Page so that we can investigate.

+ +