Hi,

Some words about my Dovecot setup:

• 10 dovecot servers
• each server has some local mailboxes
• each server has proxying enabled to seamlessly connect to another host if the mailbox is not local

Running Dovecot 1.1.1 on FreeBSD 6.3-RELEASE-p3.

First of all, local mailbox access is fine. Proxying works also as expected.

Now for the actual problem:

Shortly after startup, Dovecot starts throwing errors concerning connection which should be proxied.

Here are some examples from the logs:

Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open
files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143)
failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: accept() failed: Too many open
files Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open
files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143)
failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open
files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143)
failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open
files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143)
failed: Too many open files Aug 6 15:49:11 dovecot: imap-login: socket() failed: Too many open
files [..] Aug 6 16:00:05 dovecot: imap-login: accept() failed: Too many open
files Aug 6 16:00:08 dovecot: imap-login: accept() failed: Too many open
files Aug 6 16:00:11 dovecot: imap-login: accept() failed: Too many open
files Aug 6 16:00:13 dovecot: imap-login: accept() failed: Too many open
files

As from the context it seems to be a file descriptor issue, as such I
have done the following:

• added a 'limits -e -n 32768 -U dovecot' to dovecot startup file to
  ensure it has enough FD's
• additionally, I've checked my kernel file limits, however I always
  set it through loader and sysctl to be as large as 64k

sysctl -a |grep files

kern.maxfiles: 65535 kern.maxfilesperproc: 32768 kern.openfiles: 1632

• also checked the limits of the dovecot user like this:

su -c dovecot root -c 'ulimit -a'

core file size (blocks, -c) unlimited data seg size (kbytes, -d) 1048576 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 32768 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 131072 cpu time (seconds, -t) unlimited max user processes (-u) 5547 virtual memory (kbytes, -v) unlimited

• done some monitoring of file descriptor usage for the dovecot user,
  though somewhat inacurate, like this:

while [ : ]; do date; fstat -u dovecot |wc -l; sleep 1; done

this gave me these results while tailing the logfile:

Wed Aug 6 15:44:12 CEST 2008 585 Wed Aug 6 15:44:13 CEST 2008 579 Wed Aug 6 15:44:14 CEST 2008 582 Wed Aug 6 15:44:15 CEST 2008 582 Wed Aug 6 15:44:16 CEST 2008 586 Wed Aug 6 15:44:17 CEST 2008 585 Wed Aug 6 15:44:18 CEST 2008 582

So it didn't seem to be anywhere near the theoretical upper limit I've
set at 32k.

On the other hand, when restarting Dovecot, it works for a few minutes for at least as long as the FD usage as reported above stays at around
500. I remember from the docs that dovecot-login would require the double amount of file descriptors to run.

Assuming my report is somewhat inaccurate and lags behind, I could
imagine that upon a proxy requests it would effectively peak our at some 500 x
2 FDs. Thus effectively reaching the 1024 FD barrier.

I conclude this from the fact that I currently have some 350 - 450
logins in parallel, most of which can and shell be proxied.

From the error message in the logs I got the impression, that only
proxied connections are affected. I was not able to reproduce the issue with non-proxied logins to local
mailboxes.

Some further steps I've done from the configuration side to no avail:

• disabled imaps/pop3s/ssl
• changed between login_process_per_connection yes/no
• played around with login_max_connections et all (raised/lowered)
  limits
• tried with/without plugins enabled
• tried with SQL-based and passwd-file based userdb/passdb, single and
  mixed

Dovecot config:

# dovecot -n # 1.1.1: /usr/local/etc/dovecot.conf base_dir: /var/run/dovecot/ protocols: imap pop3 listen: *, [::] ssl_disable: yes ssl_cert_file: /usr/local/etc/postfix/tls/server.crt ssl_key_file: /usr/local/etc/postfix/tls/server.key disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_process_per_connection: no login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_processes_count: 16 login_max_processes_count: 64 login_max_connections: 64 max_mail_processes: 256 mail_max_userip_connections: 3 verbose_proctitle: yes mail_privileged_group: mail mail_uid: 1000 mail_gid: 1000 mail_location: maildir:~/Maildir fsync_disable: yes maildir_copy_preserve_filename: yes mail_drop_priv_before_exec: yes mail_executable(default): /mailserver/scripts/dovecot/imap mail_executable(imap): /mailserver/scripts/dovecot/imap mail_executable(pop3): /mailserver/scripts/dovecot/pop3 mail_plugins: expire mail_log mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_log_max_lines_per_sec: 0 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra- mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra- mailbox-sep imap_client_workarounds(pop3): pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace: type: private inbox: yes list: yes subscriptions: yes namespace: type: private separator: . prefix: INBOX. hidden: yes auth default: mechanisms: plain login cram-md5 cache_size: 131072 username_translation: #@/@%@ username_format: %Lu verbose: yes debug: yes debug_passwords: yes worker_max_request_count: 16384 passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: prefetch userdb: driver: passwd-file args: username_format=%n /usr/local/etc/postfix/tables/ deliver_passwd userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vpostfix group: vpostfix plugin: expire: Trash 14 Trash/* 14 Spam 14 Spam/* 14 VirusAlerts 14
VirusAlerts/* 14 Quarantine 14 Quarantine/* 14 expire_dict: proxy::expire dict: expire: mysql:/usr/local/etc/dovecot-expire.conf

Maybe I am missing something here. I'd really, really appreciate some help on this to get it sorted out.

Thank you.

Regards,

Gianpaolo