September 2013 - dovecot

Re: [Dovecot] Logging passwords on auth failure/dealing with botnets
by Joseph Tam 22 Sep '13

22 Sep '13

"Michael Smith (DF)" writes: > Or another option, is there any good DNS based RBLs for botnet IPs, and > is there any way to tie that in to the dovecot auth system? I've been > looking for botnet rbls, but what I've found so far doesn't seem to > work very well. Most of the IPs that I've had to firewall don't exist > in them. /dev/rob0 writes: > The problem with using XBL, anyway, is that you might have legitimate > logins from listed hosts. Example: a traveler using hotel wifi. We > (TINW) really would need a new DNSBL type (or a special result) for > this sort of abuse. > > It's a nice idea, worth building upon, if someone can fund it (or > find the time to develop it, which really amounts to the same thing.) > Imagine also a Dovecot network of reporters, where brute force > attempts worldwide are reported from Dovecots to the DNSBL, not > merely a one-way tie in. > > I'd also suggest listing SSH brute force attacks in the same DNSBL, > possibly with a different result (127.0.0.$port, so IMAP attackers > list as 127.0.0.143, SSH attackers as 127.0.0.22. Yes, we'd have to > incorporate the third quad for ports > 255, but the general idea is > for result codes to be both machine and human readable as much as > possible.) I use bl.blocklist.de as a DNSRBL for ssh BFD, but I think it also detects BFD for other protocols: http://www.blocklist.de/en/index.html The nice thing about this RBL is that you can also contribute by configuring your Fail2Ban/DenyHost to forward logs to the maintainers, to widen the detection network. I get about a 60% hit on ssh BFD attacks. I also found http://openbl.org but they distribute it as a downloadable file rather than as a DNSRBL. Maybe I can introduce the latter to the former. Joseph Tam <jtam.home(a)gmail.com>

3 2

[Dovecot] Solved: startup failures "net_listen_unix(...) failed: Invalid argument"
by C. Mills 22 Sep '13

22 Sep '13

I ran into this and had a bit of trouble tracking a diagnosis down, so here it is for the archives in case anyone else runs into this. A new installation of dovecot was failing to start, with these messages: $ ./sbin/dovecot Error: service(imap-urlauth): net_listen_unix(.../var/run/dovecot/token-login/imap-urlauth) failed: Invalid argument Error: service(auth): net_listen_unix(.../var/run/dovecot/token-login/tokenlogin) failed: Invalid argument Fatal: Failed to start listeners Some of the other processes seemed to be starting up fine, and they created sockets in the expected dir, but the above two wouldn't. stracing the processes didn't show a lot, but eventually I checked the length of the installation path, and found the above two socket paths were exceeding the maximum path length limit for Unix sockets -- 108 chars on my Linux boxen. I haven't checked SuS or Posix to see what the standards say the actual minimum required size for the maximum path length is -- it might be shorter yet on other systems. Moving the installation to a less deeply nested hierarchy fixed the problem. I hope this saves someone else a headache someday. C. C. -- ----------------------------------------------------------------------- C. Mills -----------------------------------------------------------------------

2 1

[Dovecot] dovecot and PFS
by Emmanuel Dreyfus 22 Sep '13

22 Sep '13

Hi Is there known advices on how to favor PFS with dovecot? In Apache, I use the following directives, with cause all modern browsers to adopt 256 bit PFS ciphers, while keeping backward compatibility with older browsers and avoiding BEAST attack: SSLProtocol all -SSLv2 SSLHonorCipherOrder On SSLCipherSuite ECDHE@STRENGTH:ECDH@STRENGTH:DH@STRENGTH:HIGH:-SSLv3-SHA1:-TLSv10 -SHA1:RC4:!MD5:!DES:!aNULL:!eNULL dovecot does not care about BEAST, since attacker cannot inject trafic. Therefore the cipher list get simplier in dovecot.conf: ssl_cipher_list = ECDHE@STRENGTH:ECDH@STRENGTH:DH@STRENGTH:HIGH:!MD5:!DES:!aNULL :!eNULL But that list is good for browsers. I am not aware of documentation about what ciphers are advertised by various mail client. How can I know if that setting has some success pushing PFS? How can I discover which clients fail to negociate PFS ciphers? -- Emmanuel Dreyfus manu(a)netbsd.org

4 4

[Dovecot] mailbox_list_index, stops showing new mails in mailboxes
by Anton Lundin 22 Sep '13

22 Sep '13

Hello. I'm having some weird problem with (probably) mailbox_list_index, that it doesn't see new mail in mailboxes. I'm using 2.2.4 over imap and ssh/imap, and after a while dovecot stops noticing new mail in some folders. Its always the same 2-3 folders of about 30. I read something about list-index corruption in 2.2.2, and i thought it was that i was running into earlier, but this problem still occurs. This is dovecot 2.2.4, running on Solaris 10 x86_64 to homedirs on zfs, where procmail is used to deliver mail to mailboxes in mbox-format. I've tried to purge all indexes and let dovecot rebuild them, but the problem reoccurs after a couple of days. How can i help to debug this issue? Should i grab dovecot.list.index when its out-of-sync for debugging? //Anton -- Anton Lundin +46702-161604

2 1

[Dovecot] verbose_proctitle cuts dsync to proctitle to "dsyn"
by Sven Hartge 22 Sep '13

22 Sep '13

Hi Timo, little cosmetic bug report: using "verbose_proctitle = yes" shortens the proctitle of dsync to just "dsyn". Example: Note the whitespace instead of the 'c' in the proctitle: root@m-st-01:~# ps auwwwx | grep [d]syn virtmail 18141 13.0 0.1 49916 7952 pts/6 R+ 00:49 0:00 dsyn -v -u xxyyzz7 -R backup maildir:/staging/xxyyzz7/Maildir With "verbose_proctitle = no" the output looks like this: root@m-st-01:~# ps auwwwx | grep [d]syn virtmail 18347 78.6 1.6 127356 68884 pts/6 D+ 00:53 0:02 dsync -v -u aabbcc7 -R backup maildir:/staging/aabbcc7/Maildir Version of dovecot is 2.2.5 from the autobuilder (but rebuild locally): root@m-st-01:~# doveadm config -n # 2.2.5 (ec62ea3c940e): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1 [...] Grüße, Sven. -- Sigmentation fault. Core dumped.

2 1

[Dovecot] Problem with MySQL virtual users and dsync
by Jordan Cook 22 Sep '13

22 Sep '13

Hello, I am using dovecot 2.2.5 on FreeBSD and I am having trouble using dsync with SSH. Everything on the server seems to be working fine (I can login, receive and send emails etc). I am using SQL for both passdb, and userdb and this seems to work fine. The 'doveadm user' command returns all the correct information for the user. I am getting the following errors in my log file: Aug 27 21:34:17 server dovecot: replicator: Error: userdb lookup: Connecting timed out Aug 27 21:34:17 server dovecot: replicator: Error: userdb lookup: Disconnected unexpectedly Aug 27 21:34:20 server last message repeated 173381 times It did seem to do an initial sync, but after that nothing is being synchronised. Has anyone else solved this problem previously or is able to help? Regards, Jordan Cook -- Packet House Limited is a company registered in England and Wales. Registered number: 07602377. Registered office: 40 Preston Avenue, Alfreton, Derbyshire, DE55 7JY This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Packet House Limited. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error.

2 1

[Dovecot] Error: file_dotlock_open() failed with file + Too many open files
by Kavish Karkera 22 Sep '13

22 Sep '13

Hi All, We are getting these below error on our IMAP server. is this because of open file limits.?? We have set it to 4096. [root@vish conf.d]# ulimit -n 4096 Aug 27 16:08:26 imap(kavish.karkera(a)example.com-24685): Error: file_dotlock_open() failed with file /data/quarantine/index//data/quarantine/mailstore/24685/.INBOX.kavish.karkera/dovecot.index.cache: Too many open files Aug 27 16:08:26 imap(kavish.karkera(a)example.com-24685): Error: open(/data/quarantine/index//data/quarantine/mailstore/24685/.INBOX.kavish.karkera/dovecot.index.cache.lock) failed: Too many open files Below is our dovecot conf file. ====================================================================== # 2.2.5: /usr/local/dovecot-2.2.5/etc/dovecot-imap/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 CentOS release 5.5 (Final) auth_socket_path = /usr/local/var/run/dovecot2.2-imap/auth-userdb base_dir = /usr/local/var/run/dovecot2.2-imap/ default_vsz_limit = 1 G first_valid_gid = 99 first_valid_uid = 99 hostname = log_path = /var/log/dovecot2.2-imap.log mail_location = maildir:~:INDEX=/data/quarantine/index/%h:CONTROL=/data/quarantine/index/%h mail_plugins = " virtual notify" namespace { inbox = no location = prefix = separator = . type = private } namespace { hidden = yes inbox = yes list = yes location = virtual:~/virtual:LAYOUT=maildir++:INDEX=~/virtual:CONTROL=~/virtual prefix = virtual. separator = . subscriptions = no type = private } namespace { hidden = yes inbox = no list = no location = maildir:~:INDEX=/data/quarantine/index/%h:CONTROL=/data/quarantine/index/%h:LAYOUT=maildir++ prefix = boxvirtual. separator = . subscriptions = no type = private } passdb { args = /usr/local/dovecot-2.2.5/etc/dovecot-imap/dovecot-sql.conf.ext driver = sql } protocols = imap service auth { unix_listener auth-userdb { group = nobody mode = 0666 user = nobody } } service imap-login { inet_listener imap { port = 20143 } vsz_limit = 1 G } service imap-postlogin { executable = script-login -d /usr/local/bin/dovecot.virtual.sh unix_listener imap-postlogin { mode = 0666 } } service imap { executable = /usr/local/dovecot-2.2.5/libexec/dovecot/imap imap-postlogin vsz_limit = 1 G } ssl = no userdb { args = /usr/local/dovecot-2.2.5/etc/dovecot-imap/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 1000 mail_plugins = " virtual notify" } Regards, Kavish Karkera

2 1

[Dovecot] Logging pre-rewrite user ID for application-specific passwords
by Steven Murdoch 22 Sep '13

22 Sep '13

I have set up Dovecot v2.2.5 with application-specific passwords, along the lines of this blog post (http://www.dgsiegel.net/news/2013_05_21-application_specific_passwords_for_…) My users file looks like: user-foo:{BLF-CRYPT}$2...:42:42::/home/user::allow_nets=127.0.0.1/32 user=user user-bar:{BLF-CRYPT}$2...:42:42::/home/user::user=user user:{BLF-CRYPT}$2...:42:42::/home/user::nologin In this way, user-foo can log in with one password (but only from localhost), user-bar can log in from any host with a different password, but both actually being treated as the same user. When these users log in through, the log entries look like: Aug 26 23:03:01 hostname dovecot: imap-login: Login: user=<user>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=12345, TLS Whereas I would like to see the original user name (prior to the re-write), i.e. user-foo and user-bar so I can keep track of which user names are in use and from where. Is there some way to do this? I couldn't see any of the variables being suitable. There is some information in the debug logs, but I would prefer not to need to leave this on when I don't need them. Thanks in advance. Steven

2 1

[Dovecot] Dovecot never release preallocated space in mdbox
by Stéphane BERTHELOT 22 Sep '13

22 Sep '13

Hello everyone, I am currently evaluating dovecot for our new email production servers (20k+ mailboxes) and found out something strange. I'm using those settings on Dovecot 2.2.4 (x86_64 / Slackware / compiled from sources) mdbox_rotate_size = 128M mdbox_rotate_interval = 1d mdbox_preallocate_space = yes with virtual users and location like : mail_location = mdbox:~/mdbox I don't think the remaining config is relevant but ask me if you need some other parts. Using test accounts for 2 weeks now I've figured that the 128M preallocated space is never 'hole punched" (to use a similar term than "man fallocate" on Linux), even when rotating m.* files. From what I understand those files will never be appended again because of the mdbox_rotate_interval. Then doveadm purge creates new files so old ones would never grow again. Here is an example of a mdbox storage using ls -ls (which shows allocated VS used space) total 4065176 1884 -rw------- 1 mail mail 1926656 Jul 29 10:55 dovecot.map.index 4 -rw------- 1 mail mail 460 Jul 29 11:26 dovecot.map.index.log 48 -rw------- 1 mail mail 44304 Jul 29 10:55 dovecot.map.index.log.2 131072 -rw------- 1 mail mail 133165066 Jul 19 15:31 m.10 131072 -rw------- 1 mail mail 133507393 Jul 19 15:32 m.13 131072 -rw------- 1 mail mail 134155182 Jul 19 15:33 m.14 131072 -rw------- 1 mail mail 134213403 Jul 19 15:30 m.2 131072 -rw------- 1 mail mail 46464 Jul 21 04:30 m.21 131072 -rw------- 1 mail mail 134215030 Jul 19 15:30 m.3 131072 -rw------- 1 mail mail 25852 Jul 25 01:54 m.32 131072 -rw------- 1 mail mail 2360 Jul 26 00:05 m.34 131072 -rw------- 1 mail mail 169073 Jul 27 23:18 m.35 131072 -rw------- 1 mail mail 31624 Jul 27 01:55 m.36 131072 -rw------- 1 mail mail 134216982 Jul 28 04:30 m.37 131076 -rw------- 1 mail mail 134217804 Jul 28 04:30 m.38 131072 -rw------- 1 mail mail 134217341 Jul 28 04:30 m.39 131072 -rw------- 1 mail mail 134213719 Jul 19 15:30 m.4 131072 -rw------- 1 mail mail 29740970 Jul 28 04:30 m.40 131072 -rw------- 1 mail mail 129175917 Jul 28 04:30 m.41 131072 -rw------- 1 mail mail 133174937 Jul 28 04:30 m.42 131072 -rw------- 1 mail mail 633436 Jul 28 04:30 m.43 131072 -rw------- 1 mail mail 3154623 Jul 28 04:30 m.44 131072 -rw------- 1 mail mail 3676879 Jul 28 04:30 m.45 131072 -rw------- 1 mail mail 468158 Jul 28 04:30 m.46 131072 -rw------- 1 mail mail 26964 Jul 28 04:30 m.47 131072 -rw------- 1 mail mail 3574599 Jul 28 04:30 m.48 131072 -rw------- 1 mail mail 3789133 Jul 28 04:30 m.49 131072 -rw------- 1 mail mail 134215016 Jul 19 15:30 m.5 131072 -rw------- 1 mail mail 1280074 Jul 28 04:30 m.50 131076 -rw------- 1 mail mail 635459 Jul 28 22:47 m.51 131072 -rw------- 1 mail mail 1459418 Jul 29 10:55 m.52 131072 -rw------- 1 mail mail 132941013 Jul 29 11:26 m.53 131072 -rw------- 1 mail mail 134213475 Jul 19 15:30 m.7 131072 -rw------- 1 mail mail 132240074 Jul 19 15:31 m.9 There's a lot of "lost" space since preallocated space would only be reclaimed when *all* emails in m.X file have refcount=0 and after a doveadm purge call, if I read well the dovecot docs. On mailboxes patterns with low incoming mail (< 100kb / day) this would waste much space. Of course I can decrease rotate size a lot but it would then produce a lot of files and would certainly become similar performance-wise to sdbox/maildir/... There would certainly be smart to use something similar to "FALLOC_FL_PUNCH_HOLE" on rotation (when doing close() ?) so that when we're sure there won't be anymore data appended to file that the allocated space == used space. I will disable space preallocation for our next tests since it wastes much storage for us ; did you have any feedback on how much it may affect performance ? I found in this ML archives some messages about the implementation but didn't see anyone clearly stating how much better preallocation is. Thanks, best regards, Stephane Berthelot.

4 5

[Dovecot] Deleted mails in dsync backups
by Wolfgang.Friebel＠desy.de 22 Sep '13

22 Sep '13

Hello we are doing regular backups of the mdbox mail folders using dsync -u username backup mdbox:/somedir/BACKUPS/username (still with version 2.1.10, but 2.2.5 is in testing phase) We do observe that the directory size in the BACKUPS directory is growing with respect to the original mail folders. I believe this is due to deleted mails not being purged in the backup. We are doing regular doveadm purge -A cleanups, but this does (of course) not affect the backups and subsequent dsync backup runs seem not to clean up deleted mails in the backup either. Will the behaviour of the dsync backup command deal with deleted mails in the 2.2.5 release? If not, what could we do to get rid of the deleted mails in the backup (or is there another reason for the growth in size) We have e.g. after 9 month of running with 2.1.10 du -sk username 588520 du -sk /somedir/BACKUPS/username 863024 -- Wolfgang Friebel Deutsches Elektronen-Synchrotron DESY

2 1