Re: [Dovecot] Fail2Ban and the Dovecot log
Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line:
http://www.voip-info.org/wiki/view/Fail2Ban+(with+iptables)+And+Asterisk
On Mon, 2009-05-11 at 17:15 -0400, Lou Duchez wrote:
Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line:
If you log via syslog, the timestamp will be at the beginning of line.
Timo Sirainen wrote:
On Mon, 2009-05-11 at 17:15 -0400, Lou Duchez wrote:
Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line:
If you log via syslog, the timestamp will be at the beginning of line.
Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-)
Bill
Bill Landry wrote:
Timo Sirainen wrote:
On Mon, 2009-05-11 at 17:15 -0400, Lou Duchez wrote:
Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line:
If you log via syslog, the timestamp will be at the beginning of line.
Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-)
Bill
Maybe there could be a page in the dovecot wiki about Fail2Ban? A definitive Dovecot / Fail2ban resource would be useful. (If nobody else creates one in a week, perhaps I will. But I have to perfect my Fail2banning first ...)
Thanks, guys, for helping me out!
Maybe there could be a page in the dovecot wiki about Fail2Ban? A definitive Dovecot / Fail2ban resource would be useful. (If nobody else creates one in a week, perhaps I will. But I have to perfect my Fail2banning first ...)
I couldn't figure out how to add new pages to wiki.dovecot.org, but here is what I have come up with for an easy Fail2ban recipe:
Configuring Fail2Ban with Dovecot
Make sure your dovecot "log_path" string is empty, in other words allow syslog to do your Dovecot logging (into the default mail log).
This ensures that the log entries will be in a format Fail2ban can work with.In your Fail2ban "jail.conf" file (most likely /etc/fail2ban/jail.conf), add entries like the following:
[dovecot-pop3] enabled = true filter = dovecot-pop3 action = iptables[name=POP3, port=pop3, protocol=tcp] logpath = /var/log/maillog maxretry = 20 bantime = 1200
[dovecot-imap] enabled = true filter = dovecot-imap action = iptables[name=IMAP, port=imap, protocol=tcp] logpath = /var/log/maillog maxretry = 20 bantime = 1200
This arrangement is designed to trap POP3 and IMAP separately, and also
to allow a high number of errors before temporarily "jailing" a user.
This is to decrease the likelihood that a single user from a single IP
will get all his coworkers (temporarily) banned over an honest mistake
in configuration.
- Create a jail called dovecot-pop3.conf (most likely as /etc/fail2ban/filter.d/dovecot-pop3.conf):
[Definition]
failregex = (?: pop3-login: Authentication failure).*rip=(?P<host>\S*),.* (?: pop3-login: Aborted login).*rip=(?P<host>\S*),.* (?: pop3-login: Disconnected).*rip=(?P<host>\S*),.*
ignoreregex =
- Create a jail called dovecot-imap3.conf (most likely as /etc/fail2ban/filter.d/dovecot-imap.conf):
[Definition]
failregex = (?: imap-login: Authentication failure).*rip=(?P<host>\S*),.* (?: imap-login: Aborted login).*rip=(?P<host>\S*),.* (?: imap-login: Disconnected).*rip=(?P<host>\S*),.*
ignoreregex =
- Restart Fail2ban.
Lou Duchez wrote:
This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration.
I have noticed recent breaking attempts which appear to be a slow coordinated botnet using multiple IPs and trying a combination of SMTP + POP + IMAP (can't remember if it did both of the later or just POP?).
As a result I tried to combine all three into a single test. Actually I did the wrong thing, but if you look through my previous posts you can see someone (Bill?) correct me and post the correct config for this
I would recommend you be aware of this - in my case I was seeing less than a few attempts from a given IP in a 10 min period, but lots of what appeared to be coordinated attempts at the server level. (eg some servers were only trying a few logins per day, but across enough IP addresses this was a fairly rapidly filling the logs)
Good luck
Ed W
Ed W wrote:
Lou Duchez wrote:
This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration.
I have noticed recent breaking attempts which appear to be a slow coordinated botnet using multiple IPs and trying a combination of SMTP
- POP + IMAP (can't remember if it did both of the later or just POP?). As a result I tried to combine all three into a single test. Actually I did the wrong thing, but if you look through my previous posts you can see someone (Bill?) correct me and post the correct config for this
I would recommend you be aware of this - in my case I was seeing less than a few attempts from a given IP in a 10 min period, but lots of what appeared to be coordinated attempts at the server level. (eg some servers were only trying a few logins per day, but across enough IP addresses this was a fairly rapidly filling the logs)
Good luck
Ed W
Thanks for the heads-up! Okay then, perhaps the best solution is to make use of the "ignoreip" setting in jail.conf to protect known IP addresses, something like this:
[sasl-iptables] enabled = true backend = polling filter = sasl action = iptables[name=sasl, port=smtp, protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.456.543.210/28 321.654.123.456 maxretry = 2 findtime=1200 bantime = 1200
[dovecot-pop3] enabled = true filter = dovecot-pop3 action = iptables[name=POP3, port=pop3, protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.456.543.210/28 321.654.123.456 maxretry = 2 findtime=1200 bantime = 1200
[dovecot-imap] enabled = true filter = dovecot-imap action = iptables[name=IMAP, port=imap, protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.456.543.210/28 321.654.123.456 maxretry = 2 findtime=1200 bantime = 1200
Note that SMTP, POP3, and IMAP are all looking at the same log file, they all have the same ban parameters (more aggressive than previously proposed), and they all ignore the same IP ranges (in this case a local subnet, a range of public IPs, and one additional public IP). Then in keeping with this, all three filter files (sasl.conf, dovecot-pop3.conf, and dovecot-imap.conf) would have identical configurations:
[Definition]
failregex = : warning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed (?: pop3-login: Authentication failure).*rip=(?P<host>\S*),.* (?: pop3-login: Aborted login \(auth failed).*rip=(?P<host>\S*),.* (?: pop3-login: Disconnected \(auth failed).*rip=(?P<host>\S*),.* (?: imap-login: Authentication failure).*rip=(?P<host>\S*),.* (?: imap-login: Aborted login \(auth failed).*rip=(?P<host>\S*),.* (?: imap-login: Disconnected \(auth failed).*rip=(?P<host>\S*),.*
ignoreregex =
So any failure at any of the three protocols (SMTP, POP3, IMAP) is considered a "strike" by all three, and they should all ban the same guys at the same time. This is as yet untested, but seems like it should be pretty sound.
Lou Duchez wrote:
So any failure at any of the three protocols (SMTP, POP3, IMAP) is considered a "strike" by all three, and they should all ban the same guys at the same time. This is as yet untested, but seems like it should be pretty sound.
I think you only need one service and you can use the iptables-multi (or something similar) to block all the ports if you get a hit?
Ed W
Ed W wrote:
Lou Duchez wrote:
So any failure at any of the three protocols (SMTP, POP3, IMAP) is considered a "strike" by all three, and they should all ban the same guys at the same time. This is as yet untested, but seems like it should be pretty sound.
I think you only need one service and you can use the iptables-multi (or something similar) to block all the ports if you get a hit?
Ed W
!!!
Just when I think I've achieved ultimate pefection on this, someone comes along with a great idea. Thanks!
So I guess we take out the "sasl-iptables" part of jail.conf and replace it with:
[smtppop3imap] enabled = true filter = smtppop3imap action = iptables-multiport[name=smtppop3imap, port="smtp,pop3,imap", protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.123.123.123/27 234.234.234.234 maxretry = 2 findtime = 1200 bantime = 1200
smtppop3imap.conf is as previously described:
[Definition]
failregex = : warning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
ignoreregex =
Just when I think I've achieved ultimate pefection on this, someone comes along with a great idea. Thanks! ... action = iptables-multiport[name=smtppop3imap, port="smtp,pop3,imap", protocol=tcp]
Can I suggest the name "mail" would summarise the stack of items above?
Did you test this - is it correct?
Cheers
Ed W
Ed W wrote:
Just when I think I've achieved ultimate pefection on this, someone comes along with a great idea. Thanks! ... action = iptables-multiport[name=smtppop3imap, port="smtp,pop3,imap", protocol=tcp]
Can I suggest the name "mail" would summarise the stack of items above?
Did you test this - is it correct?
Totally works! Though the setting of "2" retries is a bit severe; "4" is probably more reasonable.
Lou Duchez wrote:
Ed W wrote:
Lou Duchez wrote:
This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration.
I have noticed recent breaking attempts which appear to be a slow coordinated botnet using multiple IPs and trying a combination of SMTP + POP + IMAP (can't remember if it did both of the later or just POP?). As a result I tried to combine all three into a single test.
Actually I did the wrong thing, but if you look through my previous posts you can see someone (Bill?) correct me and post the correct config for thisI would recommend you be aware of this - in my case I was seeing less than a few attempts from a given IP in a 10 min period, but lots of what appeared to be coordinated attempts at the server level. (eg some servers were only trying a few logins per day, but across enough IP addresses this was a fairly rapidly filling the logs)
Good luck
Ed W
Thanks for the heads-up! Okay then, perhaps the best solution is to make use of the "ignoreip" setting in jail.conf to protect known IP addresses, something like this:
(snip)
Or even smarter: create a single filter file called smtppop3imap.conf, and use that same filter for SMTP, POP3, and IMAP. Here's what the filter would look like:
[Definition]
failregex = : warning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
ignoreregex =
The first regex will cover SMTP authentication errors generated by Postfix. The second regex is for Dovecot and authentication errors with POP3 and IMAP.
Sorry to keep posting iterative improvements; every time I think I'm done, I come up with something better (and perhaps worth sharing).
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 11 May 2009, Bill Landry wrote:
Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-)
Well, I use it to _detect_ I'm parsing the Dovecot logs ;-)
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBSgkx5HWSIuGy1ktrAQKxtgf9EsOByCghSTNLm6bZpg/kddMj+aErCGT+ c7ARtutq58ap3t6aBiolizn/MIBmdYqYgSP25MomeyJGPxWMZbFx0B5Ra1G9sVZV Whh0cLTt6zLmBa9Dq5SCkmuf9njVpXEb0fzWGc4rTpunnlD9e69T5EfWmuUofsV8 1b6HgMUg+2BhGw+edJCYXWh0GHQixzxh/QnFD236wQ8jY7BHXl2niTuQPLcPHmvS NPe61QE4M5F2m4xCUVmg8ODmjm+cRCgVG+8SsCJe4ZHmcVhHZ48cyHkOstTGNHDf uEURelkMmd0Wb9OgDQOWAzcdRuz0u+cP3X7Pn64So/mhF9REMq4yLg== =2zlI -----END PGP SIGNATURE-----
On Mon, 2009-05-11 at 14:48 -0700, Bill Landry wrote:
If you log via syslog, the timestamp will be at the beginning of line.
Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-)
Yeah. I don't know what I was thinking when I made it work like that. v1.2 has more sensible logging now, but I don't want to break anyone's setup by doing the change to v1.1. These patches probably apply to v1.1 too though:
http://hg.dovecot.org/dovecot-1.2/rev/0669bc561763 http://hg.dovecot.org/dovecot-1.2/rev/194b80691980
Yeah. I don't know what I was thinking when I made it work like that.
I know what you were thinking: if dovecot is writing to a log such as "mylogfile.log", and other utilities are also writing to "mylogfile.log", it's good to know which lines are dovecot.
But I am satisfied with using syslog logging; it just should be recorded somewhere that syslog is required for compatibility with Fail2Ban. I tried to edit wiki.dovecot.org with this information, but was too incompetent to figure out how to add a page. If I had to create a page with Fail2Ban instructions, it would look like:
Make sure that /etc/dovecot.conf does not have any “log_path” variable set. We need dovecot.conf to use the default system logging so the log is written in a format that fail2ban can work with.
Create the filter file /etc/fail2ban/filter.d/dovecot-pop3imap.conf:
[Definition] failregex = (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.* ignoreregex =
- Add the following to /etc/fail2ban/jail.conf:
[dovecot-pop3imap] enabled = true filter = dovecot-pop3imap action = iptables-multiport[name=dovecot-pop3imap, port="pop3,imap", protocol=tcp] logpath = /var/log/maillog maxretry = 20 findtime = 1200 bantime = 1200
On Sun, 2009-05-17 at 15:28 -0400, Lou Duchez wrote:
Yeah. I don't know what I was thinking when I made it work like that.
I know what you were thinking: if dovecot is writing to a log such as "mylogfile.log", and other utilities are also writing to "mylogfile.log", it's good to know which lines are dovecot.
I think it's a bit unlikely use case. Also seems like a bad idea to combine logs in that way. :) There is anyway still some kind of a prefix always, such as:
May 17 17:19:52 dovecot: Info: Dovecot v1.2.rc3 starting up May 17 17:19:52 auth(default): Info: passwd-file /usr/local/etc/passwd.imap: Read 4 users
But I am satisfied with using syslog logging; it just should be recorded somewhere that syslog is required for compatibility with Fail2Ban. I tried to edit wiki.dovecot.org with this information, but was too incompetent to figure out how to add a page.
You can just write the page name to the URL and then click "Create this page" link. Anyway I added it: http://wiki.dovecot.org/HowTo/Fail2Ban
Timo Sirainen wrote:
On Mon, 2009-05-11 at 14:48 -0700, Bill Landry wrote:
If you log via syslog, the timestamp will be at the beginning of line.
Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-)
Yeah. I don't know what I was thinking when I made it work like that. v1.2 has more sensible logging now, but I don't want to break anyone's setup by doing the change to v1.1. These patches probably apply to v1.1 too though:
http://hg.dovecot.org/dovecot-1.2/rev/0669bc561763 http://hg.dovecot.org/dovecot-1.2/rev/194b80691980
Thanks Timo, your responsiveness to requests (no matter how mundane at times) is always very much appreciated!
Bill
participants (5)
-
Bill Landry
-
Ed W
-
Lou Duchez
-
Steffen Kaiser
-
Timo Sirainen