- dovecot - dovecot.org

[Dovecot] performance with 100k messages per folder
by Florin Andrei 26 Aug '11

26 Aug '11

dovecot-2.0-0.10.beta6.20100630.el6.x86_64 on CentOS 6. Virtual machine with 1 GB of RAM on VMWare. The configuration is more or less stock. Postfix receives then delivers to Dovecot. IMAP with mbox. Only one user account, but shared by several people via webmail (Roundcube webmail in Apache on the same machine). No other MUAs. 100k new messages per month, inbox is rotated monthly into a YYYYMM folder by a cron job. Only one monthly folder so far. Messages are never deleted (but I may start deleting old folders a year or two from now). This is mostly for reading, with occasional messages being forwarded. The email was pretty sluggish when logging in to the webmail interface. I asked the admin to increase the RAM from 0.5 to 1 GB. I changed mbox_very_dirty_syncs to yes. These measures seemed to accelerate it a lot. There's still a 1 sec pause when logging in, during which time the dovecot/imap process is using a lot of CPU. It looks like, as long as I give it enough RAM to keep the folders in memory, the whole thing should be fast enough, which is great. Any other tips-n-tricks to keep the email server speedy and the users happy? Should I worry about mbox_very_dirty_syncs as long as there are no other MUAs? -- Florin Andrei http://florin.myip.org/

2 1

[Dovecot] File Permissions and delivery
by Simon Brereton 26 Aug '11

26 Aug '11

Hi I'm very new to Dovecot (been using Courier for 5 years), but I've been persuaded of the merits of Dovecot and since the server needs upgrading that seems like the perfect time/excuse. On a test server, I set up postfix and installed Dovecot (running 32-bit Debian Squeeze, installed from apt-get). I mirrored the mail store (Maildirs, for historical reasons located under /var/spool/mail/virtual/domain.com/user). Then I ran the courier migration perl script and everything was fine and dandy. However, when I can to do the production migration, things weren't as smooth. The new server is 64-bit (not that I think it makes a difference, but if you're going to help me you should have all the information :) Again, I installed Postfix and Dovecot Took down the old server Mirrored the Maildirs Ran the migration script Restarted everything At this point everything looked like it was ok. Mail was being received and delivered to the Maildirs and the IMAP login was fine. However, I noticed errors in the logs when retreiving mail with the MUA along the lines of: Aug 26 16:59:48 mail dovecot: IMAP(simon(a)lydiard.net): open(/var/spool/mail/virtual/domain.net/simon/cur/1314328966.V801I166601bM7… failed: Permission denied (euid=999(mailsystem) egid=115(mailsystem) missing +r perm: /var/spool/mail/virtual/domain.net/simon/cur/1314328966.V801I166601bM756462.mail.net,S=2461:2,) After messing around with the chown and chmod (even though these were exactly the same as the test server) I finally discovered the issue. mail:~# ls /var/spool/mail/virtual/domain.net/simon/new/ -rwxrwx--- 1 postfix mailsystem 2.5K Aug 26 03:33 1314326000.V801I1666018M803015.mail.net,S=2461:2, -rwxrwx--- 1 postfix mailsystem 2.5K Aug 26 03:36 1314326209.V801I1666019M447273.mail.net,S=2460:2, -rw-rw---- 1 postfix mailsystem 2.5K Aug 26 04:00 1314327630.V801I166601aM308173.mail.net,S=2477:2, -rw------- 1 postfix mailsystem 2.5K Aug 26 04:22 1314328966.V801I166601bM756462.mail.net,S=2461:2, -rw------- 1 postfix mailsystem 1.1K Aug 26 16:28 1314372534.V801I166601cM615258.mail.net,S=1097:2, -rw------- 1 postfix mailsystem 1.1K Aug 26 16:31 1314372685.V801I166601dM264242.mail.net,S=1097:2, Mails are being delivered with 0600 permissions and not 0660 (the mails from courier seem to have all been 0770 as you can see). If I manually change the permission (to 0660) then I can see the mail in the MUA. After thinking for a while it occurred to me that this is covered in the LDA section. But making changes to the config file (either permissions or UID/GID) doesn't seem to make a difference. (Yes, I did restart postfix and dovecot after the changes). Anyway, here is my dovecot -n: mail:~# dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 ext3 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_ca_file: /etc/ssl/keys/ca.crt ssl_cert_file: /etc/ssl/keys/mail.net.crt ssl_key_file: /etc/ssl/private/mail.net.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mailsystem mail_location: maildir:/var/spool/mail/virtual/%d/%n maildir_very_dirty_syncs: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: postmaster_address: postmaster@net mail_plugins: quota log_path: info_log_path: deliver_log_format: msgid=%m: %f: %$ auth default: mechanisms: plain login user: mailsystem verbose: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: static args: uid=999 gid=115 home=/var/spool/mail/virtual/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: mailsystem master: path: /var/run/dovecot/auth-master mode: 432 user: mailsystem group: mailsystem plugin: quota: maildir As you can see, I tried to go 0660 in both client and master. The portion of my master.cf 81 # SPB - Attempt to deliver with Dovecot LDA 82 dovecot unix - n n - - pipe 83 flags=DRhu user=mailsystem argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${nexthop} Is there anything else I should include? I'm pretty sure it's an error on my part. I'm just not clued up enough to know where. My second problem is that I thought I had things back to where they were before I messed with chown and chmod, but now I get this in the logs dovecot: dovecot: Fatal: chdir(/var/spool/mail/virtual/domain.net/simon//) failed: Permission denied (euid=999(mailsystem) egid=115(mailsystem) missing +x perm: /var/spool/mail/virtual) But the ls on that is exactly the same as on the test server: ls /var/spool/mail/virtual/ total 44K drwxrwS--- 11 postfix mailsystem 4.0K Aug 25 23:07 ./ drwxrwsr-x 5 amavis mailsystem 4.0K Oct 19 2009 ../ drwxrws--- 5 mailsystem mailsystem 4.0K Aug 26 02:33 domain.net/ So, now I'm stumped. I hope someone can spot the simple thing I've missed! Thanks. Simon

1 0

Re: [Dovecot] limiting number of incorrect logins per connection
by alex＠ahhyes.net 26 Aug '11

26 Aug '11

Hi, I saw that thread already, however it does not offer any solution that can be applied to dovecot directly. That thread has also been asleep for well over a year. It couldnt be that hard for the author to implement this function. It would only require a few lines of code. ----- Reply message ----- From: "Robert Schetterer" <robert(a)schetterer.org> Date: Fri, Aug 26, 2011 17:59 Subject: [Dovecot] limiting number of incorrect logins per connection To: <dovecot(a)dovecot.org> Am 26.08.2011 09:25, schrieb Alex: > Hi Guys, > > Running Dovecot 2 on my server. It is regularly getting dictionary auth > attacked. What I have noticed is that once connected to a pop3/imap > login session, you can send endless incorrect usernames+passwords > attempts. This is a problem for me... I use fail2ban to try and stop > these script kiddies. The problem is that fail2ban detects the bad > auths, firewalls the IP, however, since it's an "established" session, > the attacker can keep authing away... It's only on a subsequent (new) > connection that the firewalling will take effect. > > Why is there no configuration option such as "max auth attempts per > connection"? This would be useful, so once the limit is reached, the > connection is dropped. > > is there a patch/workaround? > there where equal questions in the past i.e read http://comments.gmane.org/gmane.mail.imap.dovecot/46204 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

1 0

Re: [Dovecot] On IMAP vhost login, only Username being used
by hobie＠rumormillnews.com 26 Aug '11

26 Aug '11

Recapping: I'm working to set up Dovecot 2.0.13 along with some additional software (qmail, vpopmail, squirrelmail). It's working fine with squirrelmail now, but trying to connect over SSL with a Thunderbird variant and using IMAP, authentication fails because Dovecot is trying to match only the front part of the name (the user part), ignoring the domain name part of what it's being sent. So, instead of seeking to match virtual user: joe_blow(a)some_domain.com ...it's trying to match: joe_blow ...and failing, since joe_blow is not a system user but is a virtual host user. With Squirrelmail, connecting from localhost via non-SSL IMAP, the match is handled correctly, no problem. I've run the runtbird.sh script but no light was shed on this by the resulting output, all it said was that authentication was failing. Has anyone else encountered this problem? Any suggestions on how to fix it or where to look for additional info? Thanks kindly. --hobie > I tried the runtbird.sh script, hoping for more info - all it did was show that auth is failing. I don't understand why Dovecot is separating the username from the domain name, and trying to match only on the username. Seems like there must be a config setting that affects this and that I'm overlooking. Thanks ahead for any help with this. > > --hobie > > hobie wrote earlier: > > ===== > > Attempting IMAP SSL login on new installation, using Icedove (Debain Thunderbird variant), login fails. Logs show Dovecot attempting to match username only, not username with domain name, on Vpopmail user, so of course no match. Tried with '@' in full username, also with '%'. What's missing? > > Log shows: > > Aug 24 19:30:48 debian dovecot: auth: Debug: client in: CONT<hidden> Aug 24 19:30:48 debian dovecot: auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth > Aug 24 19:30:48 debian dovecot: auth: Debug: vpopmail(postmaster,[IP redacted]): lookup user=postmaster domain= > > Current config: > > # 2.0.13: /usr/local/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 ext3 > auth_debug = yes > auth_verbose = yes > disable_plaintext_auth = no > first_valid_gid = 89 > first_valid_uid = 89 > mail_debug = yes > mail_location = maildir:/home/vpopmail/domains/%d/%n/Maildir > maildir_very_dirty_syncs = yes > passdb { > driver = vpopmail > } > protocols = imap pop3 > ssl_cert = </etc/ssl/certs/dovecot.pem > ssl_key = </etc/ssl/private/dovecot.pem > userdb { > args = quota_template=quota_rule=*:backend=%q > driver = vpopmail > } > > --hobie > > ===== >

1 0

[Dovecot] Dovecot 2.0.13 How to use POP3 order
by Nick Celebic 26 Aug '11

26 Aug '11

I saw a few commits for 2.0.13 that added MAIL_FETCH_POP3_ORDER, but I haven't been able to find any information on how to use it. I'm migrating from a Merak mail server into dovecot and all the email can be copied directly into a Maildir/cur directory. The filenames are just the timestamps. When I check with dovecot pop3, they seem to be randomly ordered. I'm hoping this provides a solution where it will look at the file list or stat order. If this isn't possible, can someone tell me how I can change the order in which dovecot lists mails in a UIDL in POP3? Thanks, Nick Celebic

2 1

[Dovecot] Catch22: user needs space to fix out of space condition
by Joseph Tam 25 Aug '11

25 Aug '11

A mail user reported that he filled up his INBOX (despite reminders he was approaching his filesystem quota), and furthermore, he could not fix the situation because he couldn't expunge message he marked for deletion. The dovecot logs revealed the cause dovecot: imap(user): Error: open(/var/mail/user.lock) failed: Disc quota exceeded This created an impasse where a user cannot free space because he needs to create a lock file that cannot be created because he needs free space. Is there any way out of this without administrator intervention? Joseph Tam <jtam.home(a)gmail.com>

5 4

[Dovecot] dovecot and maillists problem
by Костырев Александр Алексеевич 25 Aug '11

25 Aug '11

Hello there! I have a little trouble with using dovecot as lmtp and maillists. I use dovecot 2.0.13 with exim 4.72 on Debian 6.0. (All stuff at one host). info of maillists is store in mysql table, for example: +---------+-------------------------------------------------------------------------------------------------+ | name | recipients | +---------+-------------------------------------------------------------------------------------------------+ | Hab_Tax | 2504000067-253601001.253809737993@25.example.org,2721097514-272101001.271300057673@.example.org | +---------+-------------------------------------------------------------------------------------------------+ The problem is: When I send a letter to short maillist (as above) it get the same ID and stored only once in the maildir of one user who is the member of that maillist - that's ok, but When I send a letter to maillist with a large number of recipients it occurred that it's saved more than one, because of the headers added by dovecot lmtp (I've changed host name): diff letter_in_one_user/1314238006.M841940P6916.pink\,S\=12778136\,W\=12953189 letler_in_second_user/new/1314237993.M460165P6916.pink\,S\=12778136\,W\=12953189 3,4c3,4 < by pink.atlas-2.ru (Dovecot) with LMTP id 6UiGJBWuVU4EGwAA1nFjLA < ; Thu, 25 Aug 2011 13:06:46 +1100 --- > by pink.atlas-2.ru (Dovecot) with LMTP id VUeGJBWuVU4EGwAA1nFjLA > ; Thu, 25 Aug 2011 13:06:29 +1100 It's the same letter accept the date and id. What I get in logs is: I see that exim send letters in maillist with one and the same id: ...skipped... 13:06:46 1QwPKe-0001nU-U5 <2502(a)example.org> ... 13:06:47 1QwPKe-0001nU-U5 <2502(a)example.org> ... But in logs of dovecot I see: 13:06:14 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: wUWGJBWuVU4EGwAA1nFjLA: 13:06:18 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: wEWGJBWuVU4EGwAA1nFjLA: 13:06:22 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: JkaGJBWuVU4EGwAA1nFjLA: 13:06:26 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: i0aGJBWuVU4EGwAA1nFjLA: 13:06:30 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: 8EaGJBWuVU4EGwAA1nFjLA: 13:06:34 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: VUeGJBWuVU4EGwAA1nFjLA: 13:06:37 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: ukeGJBWuVU4EGwAA1nFjLA: 13:06:42 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: H0iGJBWuVU4EGwAA1nFjLA: 13:06:46 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: hEiGJBWuVU4EGwAA1nFjLA: 13:06:47 msgid=<E1QwPKe-0001nU-U5(a)pink.example.org>: 6UiGJBWuVU4EGwAA1nFjLA: Also in dovecot log I've noticed that: at first auth is searching info in sql for 200 users then log continues with lmtp's info then auth is searching info in sql for another portion of users and this loop in log continues. Dovecot.conf: http://pastebin.com/ueCBU3bP dovecot.sql: http://pastebin.com/kqCq52mC dovecot.log http://pastebin.com/YiyMWpx1 exim.conf: http://pastebin.com/F4MNQkAx exim.log: http://pastebin.com/wn83TZpX

2 6

[Dovecot] On IMAP vhost login, only Username being used
by hobie＠rumormillnews.com 25 Aug '11

25 Aug '11

Attempting IMAP SSL login on new installation, using Icedove (Debain Thunderbird variant), login fails. Logs show Dovecot attempting to match username only, not username with domain name, on Vpopmail user, so of course no match. Tried with '@' in full username, also with '%'. What's missing? Log shows: Aug 24 19:30:48 debian dovecot: auth: Debug: client in: CONT<hidden> Aug 24 19:30:48 debian dovecot: auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Aug 24 19:30:48 debian dovecot: auth: Debug: vpopmail(postmaster,[IP redacted]): lookup user=postmaster domain= Current config: # 2.0.13: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 ext3 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 mail_debug = yes mail_location = maildir:/home/vpopmail/domains/%d/%n/Maildir maildir_very_dirty_syncs = yes passdb { driver = vpopmail } protocols = imap pop3 ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = quota_template=quota_rule=*:backend=%q driver = vpopmail } --hobie

2 1

[Dovecot] Post-login scripting with virtual users
by Mark Willcox 25 Aug '11

25 Aug '11

I am running Dovecot 2.0.13 on Fedora 15. I have migrated from a bincimap installation using checklocalpwd. All email is in folders owned by a unprivileged user, popuser. The email users are entirely virtual and have no relationship to actual linux users. My userdb calls return the UID and GID of popuser and all goes well retrieving the mail via POP3 and IMAP. I am trying to implement a Post-login script to register the IP address that the authentication came from. Dovecot seems to be trying to execute the script as the email non-user, which does not work. I have specified that the script should be run as popuser, but it still runs as devnull(a)dhzone.com for instance. Everything up top that point seems to be happening happily under popuser. I added a "user = popuser" directive and it had no effect that I could see. I know that dovecot read the directive because I changed it to "user = 555" and dovecot would not run because "555" is not a user. I tried adding a "User = popuser" within the unix_listener but that also had no effect. Any ideas? Thank you! ------------------------------ Here is my dovecot -n: # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.40.3-0.fc15.x86_64 x86_64 Fedora release 15 (Lovelock) auth_debug = yes disable_plaintext_auth = no mail_debug = yes mbox_write_locks = fcntl passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 service imap-postpop { executable = script-login /usr/local/bin/set_postpop unix_listener imap-postpop { } user = popuser } service imap { executable = imap imap-postpop } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } ----------------------------------------------------- The dovecot-sql.conf.ext looks like this: driver = mysql connect = host=localhost dbname=webmail user=xxxx password=xxxx password_query = SELECT id AS user, IF( clear IS NULL, CONCAT('{CRYPT}',crypto), CONCAT('{PLAIN}',clear)) AS password \ FROM users WHERE id = '%u' user_query = SELECT home, uid, gid \ FROM users WHERE id = '%u' ------------------------------- >From the maillog: Aug 24 13:01:57 callisto dovecot: auth: Debug: client out: OK#0111#011user=devnull(a)dhzone.com Aug 24 13:01:57 callisto dovecot: auth: Debug: master in: REQUEST... Aug 24 13:01:57 callisto dovecot: auth: Debug: sql(devnull(a)dhzone.com,75.221.209.232): SELECT home, uid, gid FROM users WHERE id = 'devnull(a)dhzone.com' Aug 24 13:01:57 callisto dovecot: auth: Debug: master out: USER#0114007264257#011devnull@dhzone.com#011home=/var/qmail/popbox/dhzone-com/devnull#011uid=555#011gid=555 Aug 24 13:01:57 callisto dovecot: imap-login: Login: user=<devnull(a)dhzone.com>, method=PLAIN, rip=75.221.209.232, lip=76.76.59.241, mpid=667 Aug 24 13:01:57 callisto dovecot: imap-postpop: Error: script-login: Error: user devnull(a)dhzone.com: Error reading configuration: net_connect_unix(/var/run/dovecot/config) failed: Permission denied Aug 24 13:01:57 callisto dovecot: imap-postpop: Error: script-login: Fatal: Internal error occurred. Refer to server log for more information. Aug 24 13:01:57 callisto dovecot: imap(devnull(a)dhzone.com): Post-login script denied access to user devnull(a)dhzone.com Aug 24 13:01:57 callisto dovecot: log: Error: service(imap-postpop): child 668 returned error 89 (Fatal failure) Aug 24 13:02:03 callisto dovecot: master: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) -- _________________ Mark Willcox Data Helper, Inc.

3 9

[Dovecot] Dovecot rejecting Vpopmail User 89
by hobie＠rumormillnews.com 24 Aug '11

24 Aug '11

I'm working to get Dovecot 2.0.13 working along with qmail, Vpopmail and Squirrelmail on a Debian 6.0.2 system, Dovecot compiled, not from a package. Vpopmail has a widely known assigned user/group ID of 89 and is the owner of all the mail folders. Regardless of value of first_valid_uid (1, 89, other), Dovecot denies Squirrelmail connection, saying it can't allow access to UID 89. "Couldn't drop privileges", it says. I'd be grateful for fixes, suggestions, or a pointer to an existing answer to this problem. Thanks ahead. :) Presently dovecot -n shows: # 2.0.13: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.2 ext3 auth_debug = yes disable_plaintext_auth = no mail_debug = yes mail_location = maildir:/home/vpopmail/domains/%d/%n/Maildir maildir_very_dirty_syncs = yes passdb { driver = vpopmail } protocols = imap pop3 ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } userdb { args = quota_template=quota_rule=*:backend=%q driver = vpopmail } --hobie

3 5