[Dovecot] Fishing attempt locking up dovecot
I've mentioned this before but only heard from one other person who
has experienced this, but it's becoming a pretty serious issue.
The situation:
A spammer sets a bot on a fishing attempt to gain email addresses,
causing numerous login processes to spawn and suck up all available
resources.
The problem:
Obviously this can act like a dos attack, but the real issue is after
the spammer stops (by virtue of being added to our firewall blacklist,
being caught and shut down by their isp, or otherwise), dovecot
doesn't seem to relinquish the resources, causing "too many files
open" errors for normal usage.
The master process usually hangs around 40-50 files open at any given
time, with about 10,000 logins a day (I use: lsof -p cat /var/run/ dovecot/master.pid
| wc -l), after the attempt is over I always see
the files open shoot up to near 3,000 without it able to go down until
a dovecot restart.
This usually happens about once a month, though we can get unlucky and
have it happen a few days apart. I have some log excerpts below.
Wondering if this is happening through my own fault or at least within
my ability to alleviate the issue. Is there a way to limit the number
of connections from an ip address? Has anyone used login_executable to
first hit their own solution to keep track of, and implement
connection restrictions similar to what I'm aiming for?
Thanks for any insight Patrick
dovecot -n # 1.0.8: /usr/local/mail/dovecot/etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /var/log/dovecot protocols: imap imaps pop3 pop3s ssl_cert_file: /usr/local/mail/ssl/certs/dovecot.pem ssl_key_file: /usr/local/mail/ssl/private/dovecot.pem ssl_cipher_list: ALL:!LOW disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/mail/dovecot/libexec/dovecot/ imap-login login_executable(imap): /usr/local/mail/dovecot/libexec/dovecot/imap- login login_executable(pop3): /usr/local/mail/dovecot/libexec/dovecot/pop3- login first_valid_uid: 500 mail_location: maildir:/Volumes/data/mail/%Ld/%Ln mail_executable(default): /usr/local/mail/dovecot/libexec/dovecot/imap- before mail_executable(imap): /usr/local/mail/dovecot/libexec/dovecot/imap- before mail_executable(pop3): /usr/local/mail/dovecot/libexec/dovecot/pop3- before mail_plugin_dir(default): /usr/local/mail/dovecot/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/mail/dovecot/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/mail/dovecot/lib/dovecot/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain digest-md5 login passdb: driver: sql args: /usr/local/mail/dovecot/etc/dovecot-sql.conf userdb: driver: static args: uid=exim gid=exim plugin: quota: maildir
From the logs, first entries showing the problem:
dovecot: Dec 10 16:17:02 Info: pop3-login: Disconnected:
rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:02 Info: pop3-login: Disconnected:
rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:03 Info: pop3-login: Disconnected:
rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:03 Info: pop3-login: Disconnected:
rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:03 Info: pop3-login: Disconnected:
rip=207.245.39.90, lip=<lip>
A little later on:
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login:
user=<pwitest>, method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login:
user=<tsinternetuser>, method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login: user=<bill>,
method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login: user=<web>,
method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login:
user=<barbara>, method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login: user=<www>,
method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login: user=<user>,
method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login:
user=<nathan>, method=PLAIN, rip=207.245.39.90, lip=<lip>
dovecot: Dec 10 16:17:12 Info: pop3-login: Aborted login:
user=<webmaster>, method=PLAIN, rip=207.245.39.90, lip=<lip>
etc. etc.
On Tue, 11 Dec 2007, Patrick Milvich wrote:
I've mentioned this before but only heard from one other person who has experienced this, but it's becoming a pretty serious issue.
The situation: A spammer sets a bot on a fishing attempt to gain email addresses, causing numerous login processes to spawn and suck up all available resources.
Sounds like you could avoid this problem if you configured fail2ban to block the spammer trying all these logins.
However, the underlying problem with Dovecot would remain, and so it is worth fixing that too!
-- Asheesh.
-- The speed of anything depends on the flow of everything.
On Tue, 2007-12-11 at 15:58 -0700, Patrick Milvich wrote:
Obviously this can act like a dos attack, but the real issue is after
the spammer stops (by virtue of being added to our firewall blacklist,
being caught and shut down by their isp, or otherwise), dovecot
doesn't seem to relinquish the resources, causing "too many files
open" errors for normal usage.
I guess there could be problems after you get "too many open files", but you can configure Dovecot so that it never happens. v1.1 warns about this at startup.
Basically you just need to make sure that when starting dovecot, ulimit -n is a bit larger than max_mail_processes + login_max_processes_count + auth_worker_max_count (if you're using MySQL).
participants (3)
-
Asheesh Laroia
-
Patrick Milvich
-
Timo Sirainen