Modifying Open File Descriptor Limits on OS X

13 Feb Modifying Open File Descriptor Limits on OS X

Overview

In this post we will be talking about file descriptors and increasing limit descriptors in OS X Yosemite. File descriptors are limited by the OS. This causes some applications and services to return errors. By increasing the limit of file descriptors we can reduce the amount of “too many files open” error.

What are File Descriptors?

A file descriptor is used by the kernel to represent an open file that it needs to refer back to. These files are placed into an array and given an integer. The integer in the list are what we call file descriptors. When the kernel goes looking for the particular file it will call on the file descriptor array to find where the file is.

Why do we limit the amount of File Descriptors?

Unix and other similar systems place a limit on how many file descriptors are open at once. The limit is set to improve computer performance and to protect the amount of RAM and CPU resources used to track open files. The limit is set high enough that most programs won’t hit the limit. Some applications will hit the limit if they are opening many files like backup services, file comparison utilities and other programs that need to have many open files.

Adjusting Open File limits with Limit Descriptors

First, check what your current limit is at by running:

launchctl limit maxfiles to get  maxfiles 65536 65536

The last two columns displayed are the soft and hard limits, respectively. In OS X Yosemite, the max limit you can set is 65536. If you set it any higher the OS will use default settings of maxfiles 1024 1024  .

There is a few different ways to set a higher limit for the the file descriptors.

• One Time Modification
  To modify the file descriptors for one time use enter the following command:
  sudo sysctl -w kern.maxfiles=20480
  Note, once the computer reboots the default file descriptors will be set back.

• Modifying the System Default
  To make this change permanent, edit /etc/sysctl.conf and add kern.maxfiles=65536 kern.maxfilesperproc=65536 to the file. You will also have to modify the $HOME/.bash_profile and add ulimit -n 65536 65536

• Configuring LaunchDaemons
  LaunchDaemons are the preferred way to modify the file descriptors when the computer boots on a system-wide basis. To do this you will have to create two different plists.

/Library/LaunchDaemons/limit.maxproc.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxfiles</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxfiles</string>
          <string>65536</string>
          <string>65536</string>
        </array>
      <key>RunAtLoad</key>
        <true/>
      <key>ServiceIPC</key>
        <false/>
    </dict>
  </plist>

/Library/LaunchDaemons/limit.maxfiles.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple/DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxproc</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxproc</string>
          <string>2048</string>
          <string>2048</string>
        </array>
      <key>RunAtLoad</key>
        <true />
      <key>ServiceIPC</key>
        <false />
    </dict>
  </plist>

• Permissions & Privileges
  The launchdeamon plists should already retain theses permission. Check to see if both plist files must be owned by root:wheel with file permissions -rw-r–r– .

• Test Modification
  Now that you have configured your limit descriptors to be higher, restart the computer. Check to see if the computer has retained the setting by the launchctl limit command.

Our need for Limit Descriptors

Radmind is a suite of unix tools that we use to maintain the file system of our OS X computers. Radmind uses a tool called fsdiff  to compare all the files on the hard disk. When fsdiff  would run we would get errors saying that there were too many files open and we would need to increase the limit. The issue became more wide spread when we started running Radmind though a JAMF policy. The policy added an additional layer of abstraction from the OS. We fixed the issue by having two different LaunchDaemons start and set the limit descriptors. We have reduced the problem that we were having with fsdiff  and other radmind suite tools.