- dovecot - dovecot.org

[Dovecot] Some Doveadm Tools lack proper exit codes
by Thomas Leuxner 13 Feb '12

13 Feb '12

Happy New Year Everyone, and *yes* it's that time of the year to archive old stuff again. Please implement proper error codes to support this (scripting) endeavor. => Good $ doveadm user foo userdb lookup: user foo doesn't exist $ echo $? 2 => Bad $ doveadm acl get -u tlx(a)leuxner.net FOO doveadm(tlx(a)leuxner.net): Error: Can't open mailbox FOO: Mailbox doesn't exist: FOO ID Global Rights $ echo $? 0 $ doveconf -n | head # 2.1.rc1 (056934abd2ef): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 Thanks Thomas

2 3

[Dovecot] v2.0.18 released
by Timo Sirainen 12 Feb '12

12 Feb '12

http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz.sig + DIGEST-MD5 authentication supports authorization id now. Patch by Yubao Liu + Added instance_name setting which is used to prefix Dovecot processes in ps output. - LDA/LMTP: Sending a large mail via submission_host or via LMTP proxy may have caused a hang. - Fixed dbox + mail_attachment_dir + zlib problems. - Login processes weren't logging all intended messages with auth_verbose=yes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - IMAP: CONTEXT search return option wasn't handled at all. - dbox: Various error handling fixes. - snarf plugin: Keep the mailbox locked during snarfing to avoid duplicates.

1 0

[Dovecot] Inherited ACLs for mailboxes created by not-owner
by Michael Stilkerich 12 Feb '12

12 Feb '12

Hi, I have shared one of my mailboxes to another user providing him full rights to that mailbox. $ doveadm acl get -u user1 doc ID Global Rights user=user2 admin create delete expunge insert lookup post read write write-deleted write-seen When that user creates a new mailbox within the shared mailbox, the mailbox is successfully created but with an empty ACL. $ doveadm acl get -u user1 doc/hallo ID Global Rights According to the wiki, it should inherit the ACL from its parent, i.e. have full permissions for user2. Dovecot logs the following errors when I access the parent folder the first time after the other user created the child mailbox: dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: Recent flags state corrupted for mailbox Sent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent This is on dovecot 2.1rc5. -Michael

2 4

[Dovecot] Some Dovecot LDA version sends MDN rather than DSN for overquota.
by Mark Sapiro 12 Feb '12

12 Feb '12

It appears that there is some version of the Dovecot LDA that is sending an RFC 3798 MDN rather than an RFC 3464 DSN to report non-delivery to a user who is over quota. See <https://bugs.launchpad.net/mailman/+bug/693134> for a discussion of this and <https://bugs.launchpad.net/mailman/+bug/693134/+attachment/1773018/+files/A…> for the actual MDN message. The issue is that apparently a user was over quota and Dovecot returned an MDN with a disposition of deleted. This is intended to mean the the mail was delivered to the user but the user deleted it without reading it. If the message was actually not delivered, this should be reported to the MTA so it can return an appropriate SMTP status, or the LDA should send an RFC 3464 DSN with an action of failed. Note that I do not actually use the Dovecot LDA nor do I have any access to the system where the notice in the bug report was received, so I may have incomplete information and this may not be relevant to current versions of Dovecot, but I think it is wrong for Dovecot to report over quota non-delivery via an MDN. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 3

[Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP
by Mark Zealey 12 Feb '12

12 Feb '12

Hi there, I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: 451 4.3.0 <xxx(a)yyy.com> Temporary user lookup failure and in the logs: 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx(a)yyy.com): Disconnected unexpectedly I'm sure this is an obvious issue for someone who's come across it before. The /var/run/dovecot/director-userdb socket is on the system; exactly the same permissions as the standard auth-userdb Here are the relevant bits of my config (I'm setting lmtp to listen on a different port than normal). Backend password database is mysql if that makes any difference. lmtp_proxy = yes service lmtp { inet_listener lmtp_test { port = 2450 } } protocol lmtp { auth_socket_path = director-userdb } Thanks, Mark

3 3

[Dovecot] Lost in configuration
by forumer＠smartmobili.com 12 Feb '12

12 Feb '12

Hi, I am trying to configure dovecot/postfix with virtual users and sasl auth but there are so many tutorials with mistakes and subtle differences that at the end I am lost. So I am running Ubuntu server 11.04 (natty) and when I installed my server I had followed the following guide http://workaround.org/ispmail/etch Then I have migrated to dovecot 2.1 and now I am trying to allow authenticated user to use my smtp server because for the moment I get : Relay access denied Here is my configuration (first is file hierarchy followed by file contents) and I would like to know what I need to do to allow authenticated user send email. In this tutorial http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sp… it seems I have to add some configuration lines inside /etc/pam.d/smtp and add database configuration inside /etc/postfix/sasl/smtpd.conf. But what I find weird is the fact I have to configure some sql queries from smtpd.conf while normally dovecot has already all the information to do this kind of query. So before to keep on modifying my config I would like to be sure I am following the right way. Thanks ########################### /etc/postfix ########################### drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ -rw-r--r-- 1 root root 373 2011-07-24 11:50 dynamicmaps.cf -rw-r--r-- 1 root root 1928 2012-02-12 12:45 main.cf -rw-r--r-- 1 root root 5762 2011-08-03 22:06 master.cf -rw-rw---- 1 root postfix 140 2011-07-24 11:43 mysql-virtual-alias-maps.cf -rw-rw---- 1 root postfix 132 2011-07-24 11:47 mysql-virtual-mailbox-domains.cf -rw-rw---- 1 root postfix 128 2011-07-24 11:47 mysql-virtual-mailbox-maps.cf -rw-r--r-- 1 root root 19509 2011-04-05 06:07 postfix-files -rwxr-xr-x 1 root root 8729 2011-04-05 06:07 postfix-script* -rwxr-xr-x 1 root root 25752 2011-04-05 06:07 post-install* drwxr-xr-x 2 root root 4096 2012-02-12 12:41 sasl/ root@xa-12345:/etc/postfix# ll sasl/ drwxr-xr-x 2 root root 4096 2012-02-12 12:41 ./ drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ../ -rw-r--r-- 1 root root 26 2012-02-12 12:41 smtpd.conf ########################### /etc/dovecot ########################### drwxr-xr-x 3 root root 4096 2011-09-24 11:42 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ drwxr-xr-x 2 root dovecot 4096 2012-02-12 12:03 conf.d/ -rw-r--r-- 1 root dovecot 3693 2011-09-23 15:11 dovecot.conf -rw-r--r-- 1 root dovecot 410 2011-09-23 15:11 dovecot-db.conf.ext -rw-r--r-- 1 root dovecot 782 2011-09-23 15:11 dovecot-dict-sql.conf.ext -rw-r--r-- 1 root dovecot 5508 2011-09-23 15:38 dovecot-sql.conf -rw-r--r-- 1 root dovecot 5348 2011-09-23 15:11 dovecot-sql.conf.ext -rw-r--r-- 1 root dovecot 116 2011-09-23 15:11 README ########################### /etc/default/saslauthd ########################### -rw-r--r-- 1 root root 2043 2012-02-12 11:57 /etc/default/saslauthd ########################### /var/spool/postfix/var/run/ ########################### drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ./ drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ../ drwxr-xr-x 2 root sasl 4096 2012-02-12 12:35 saslauthd/ I have added postfix to the sasl group (don't know if it was necessary but I found this instruction on a website) /etc/group: sasl:x:45:postfix /etc/default/saslauthd: ---------------------- START=yes PWDIR="/var/spool/postfix/var/run/saslauthd" PARAMS="-m ${PWDIR}" PIDFILE="${PWDIR}/saslauthd.pid" DESC="SASL Authentication Daemon" NAME="saslauthd" MECHANISMS="pam" MECH_OPTIONS="" THREADS=5 OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" mkdir -p /var/spool/postfix/var/run/saslauthd chown -R root.sasl /var/spool/postfix/var/run/saslauthd Then I have checked saslauth is running after restart: root@xa-12345:/home/vmail# ps -ef | grep saslauthd root 6123 1 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6124 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6125 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6127 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6128 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 Then when I check after restarting if saslauthd socket is created I don't see anything (don't know if it's normal) root@xa-12345:/home/vmail# ls -lh /var/spool/postfix/var/run/saslauthd total 0 /etc/postfix/sasl/smtpd.conf: ----------------------------- pwcheck_method: saslauthd /etc/postfix/main.cf: --------------------- myorigin = /etc/mailname smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no # TLS parameters smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache myhostname = mail.foobar.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = foobar.com mydestination = xa-12345.dadibox.com, localhost.dadibox.com, localhost relayhost = mynetworks = 127.0.0.0/8 80.190.190.190 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all #virtual domains virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf virtual_mailbox_base = /home/vmail virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf # Dovecot LDA virtual_transport = dovecot dovecot_destination_recipient_limit = 1 debug_peer_level = 2 debug_peer_list = 127.0.0.1 smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination smtpd_sasl_application_name = smtpd broken_sasl_auth_clients = yes /etc/dovecot/conf.d/10-auth.conf: -------------------------------------- disable_plaintext_auth = no !include auth-sql.conf.ext !include auth-static.conf.ext /etc/dovecot/conf.d/auth-static.conf.ext: ---------------------------------------- userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } /etc/dovecot/conf.d/auth-sql.conf.ext: -------------------------------------- passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } /etc/dovecot/dovecot-sql.conf: ------------------------------ driver = mysql connect = host=127.0.0.1 dbname=mailserver user=myuser password=mypassword default_pass_scheme = CRAM-MD5 password_query = SELECT email as user, password FROM view_users WHERE email='%u'; /etc/dovecot/conf.d/10-master.conf: ---------------------------------- service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service lmtp { unix_listener lmtp { } } service imap { } service pop3 { } service auth { unix_listener auth-userdb { mode = 0600 user = vmail #group = } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { mode = 0777 } } service auth-worker { } service dict { unix_listener dict { mode = 0600 #user = group = vmail } }

2 2

[Dovecot] Quota Calculation seems to be wrong when using dsync
by Thomas Leuxner 12 Feb '12

12 Feb '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Recently I encountered odd quota increases when using 'dsync' with the latest and greatest: 2.0.8 (89936539e3b8). Say you got something in place like this: plugin { quota = dict:user::file:%h/mdbox/dovecot-quota quota_rule = *:storage=1GB quota_rule2 = Trash:storage=+10%% } Kick off a manual backup: $ dsync -u user(a)domain.tld backup mdbox:/<path to backup>/mdbox This results in doubling the quota for the backed up user. It doesn't seem to matter whether 'mirror' or 'backup' mode is used, both increase the "source" quota. Doesn't sound right to me. You can only get to the "real" usage by running a recalc manually: $ doveadm quota recalc -u user(a)domain.tld Anyone else seen this? Regards Thomas P.S. Merry Christmas -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) iEYEARECAAYFAk0VtH8ACgkQ+meF/S97aXjUkwCfdZFpUPCOY9UiukT+G68jCHHr mmMAnRHz56BXMzpC84CIYvUn4KHpl8OA =M0AZ -----END PGP SIGNATURE-----

2 9

[Dovecot] Migration to mdbox from Maildir, missing folders!
by Giles Coochey 12 Feb '12

12 Feb '12

So I went and migrated a mailbox from mdbox from maildir format. All seemed ok, but I've found that I now have some missing folders. The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What can be done? (I have a backup of the maildir box). Note though that mdbox is now a live mailbox and has sent and received mails since the migration... -- Message sent via my webmail account.

2 1

[Dovecot] folder hierarchies with migration from cyrus to dovecot2
by Wolfgang Rosenauer 12 Feb '12

12 Feb '12

Hi, I finally migrated my IMAP server from cyrus to dovecot2. I have been using cyrus since many years and have not much experience with dovecot yet though. I've converted my imap server using cyrus2dovecot (several times until I thought it was correct) but there is one thing left which confuses me and I'm not sure what I can or need to do to change it. With Cyrus I used unixhierarchysep: yes and subfolders within Cyrus are always below INBOX. I'm used to that in my IMAP clients and wanted to preserve it with Dovecot. So during migration I finally made sure to create the subfolder again below INBOX wich looks fine in Thunderbird for example. But my mobile devices (Android based with K9) were displaying the folders with Cyrus just with their name w/o the INBOX prefix. (Mobile clients are bad in displaying folder hierarchies so it was convenient to have them looking like they were in parallel to the Inbox.) Now I get the folders displayed as INBOX.whatever and my Android mail clients fail to map them to the special (and localized) folders like Trash. While that seems consistent to my current hierarchy it's not what I wanted. So basically my two questions are: - Can I configure Dovecot in a way that it looks like Cyrus? Having subfolders below Inbox but let clients like those on Android display them besides Inbox? - If that's not possible, can I move the subfolders up one level so they are next to the Inbox? How would I do that? Sorry for the beginner questions. I played around with Dovecot before but using it in production environment disclosed some fine differences. Thanks, Wolfgang

2 1

[Dovecot] [PATCH] Allow verification of user's hash in doveadm-pw
by Spack 12 Feb '12

12 Feb '12

Hi, It may be useful for testing purpose to know if a generated password hash correspond to a given clear password. This can be useful to check if a hash generated by another program can be verified by Dovecot without any errors. This patch adds the ability the verify a password hash using `doveadm pw` via the `-V` option. $ doveadm pw -s SSHA.hex Enter new password: Retype new password: {SSHA.HEX}58b910d947c60b35be3e12b0d9897c1f87dfa450e6d5a75c $ doveadm pw -s SSHA.hex -V{SSHA.HEX}58b910d947c60b35be3e12b0d9897c1f87dfa450e6d5a75c Enter new password: Retype new password: {SSHA.HEX}4fdf801f57870fb624bee60895c2308187837dfb3987ca06 (verified) When the user specify a hash, it is checked in place of the usual verification process. This works with all schemes. Just be sure to escape special characters or use simple quotes. $ doveadm pw -s MD5-CRYPT -V'{MD5-CRYPT}$1$R7thDyN.$E0G0czPE/h4S0Hn/b5oVW1' Enter new password: Retype new password: {MD5-CRYPT}$1$rp8nVSqq$gDAPM5iqv1yhBoDZIWsQ4/ (verified) Lets print a failed check: doveadm pw -s CRYPT -V{CRYPT}/uuE/AAAAAAAA Enter new password: Retype new password: reverse password verification check failed FYI the word used for those test cases was "password". Regards, ___ Jimmy Thrasibule <spack.world(a)gmail.com>

2 1

[Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir.
by Glen Johnson 11 Feb '12

11 Feb '12

I have an IMAP mail server running Postfix, and Dovecot. I have an old Notebook computer which I can view all e-mails. The problem I am having is when I try to use any other computer on the network to view the e-mail I can not see the contents of all the folders. I can see some of the folders and their content but not all. To answer a couple of preliminary questions: dovecot version 2.0.13, postfix version 2.8.3 dovecot.conf ------------------ # 2.0.13: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 4.0.1 i386 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no listen = * log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir/ passdb { driver = passwd } passdb { args = /usr/pkg/etc/dovecot/passwd driver = passwd-file } protocols = imap service auth { user = root } service imap-login { executable = /usr/pkg/libexec/dovecot/imap-login } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } protocol lda { postmaster_address = postmaster(a)denisecir.com }

2 1

[Dovecot] doveadm index
by Daniel L. Miller 10 Feb '12

10 Feb '12

Does "doveadm index -A INBOX" reindex everything? What about users with multiple folders, both subbed & not subbed under INBOX? -- Daniel

3 6

[Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO
by Mildred Ki'Lya 10 Feb '12

10 Feb '12

Hi, I am trying to set up my own mail server (to get off Google) and I have a problem with mail delivery. I'm using Postfix as a SMTP front-end, and use the LMTP server included in dovecot to receive mail from postfix. I have a setup which works with the dovecot LDA, it involves calling dovecot-lda with the -d flag. Postfix computes the username by removing what's after the recipient_delimiter and the domain from the mail address, and gives this username to dovecot. But, I'd prefer using LMTP, as it is possible to reject unknown e-mails without having to duplicate the user database (it is explained here: http://wiki2.dovecot.org/LDA/Postfix ) I set up LMTP so that postfix uses the socket provided by dovecot, and it works. Only I get the following error message in the logs: Feb 10 18:05:53 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 <mildred(a)mildred.fr>: Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 <mildred(a)mildred.fr> User doesn't exist: mildred(a)mildred.fr(in reply to RCPT TO command); from=< shanti(a)sogilis.com> to=<mildred(a)mildred.fr> proto=ESMTP helo=< mail-wi0-f179.google.com> Feb 10 18:06:52 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 <mildred-test(a)mildred.fr>: Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 <mildred-test(a)mildred.fr> User doesn't exist: mildred-test(a)mildred.fr (in reply to RCPT TO command); from=< shanti(a)sogilis.com> to=<mildred-test(a)mildred.fr> proto=ESMTP helo=< mail-wi0-f179.google.com> I followed the explanations here: http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP The problem is that my passwd-like file only contains a user named "mildred". I tried to add a user mildred(a)mildred.fr and mildred%mildred.frbut it doesn't work either. Do you know how I could tell dovecot to guess the username correctly ? Thanks, Mildred -- Mildred Ki'Lya http://mildred.fr

2 1

[Dovecot] dsync deleting too many emails (sdbox)
by Asheesh Laroia 09 Feb '12

09 Feb '12

I'm using dsync to synchronize emails on a laptop where wifi connectivity sometimes fails in the middle of a sync. I have a shell script that runs dsync, and here is one line of it including the output of dsync: + dsync -f -m realmail mirror /home/paulproteus/projects/ssh-attach/run ssh rose.makesad.us dsync dsync-local(paulproteus): Error: dbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Unexpectedly lost uid=337460 dsync-local(paulproteus): Error: msg guid lookup failed: Internal error occurred. Refer to server log for more information. [2012-02-02 11:02:12] dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Rebuilding index dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 130608.broken dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 66159.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 336269.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 dsync-remote(paulproteus): Error: proxy server timed out dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 dsync-local(paulproteus): Warning: Transaction log file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/dovecot.index.log was locked for 1528 seconds That seemed problematic, but not dangerous. Then I ran a fresh sync, and found 3000 (of 60,000) messages had been deleted and expunged from the "realmail" box. I'm guessing this is some bad interaction with sdbox and partial file downloads? I haven't read the code for this, but I would guess the dsync process isn't being atomic about file transfers, so it is leaving half-completed transfers in place, which results in corrupt files when they're next examined. dovecot -n output: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-2-amd64 x86_64 Debian wheezy/sid mail_location = sdbox:~/Maildir/dbox passdb { driver = pam } protocols = " imap" ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } -- Asheesh.

3 3

[Dovecot] Strange behavior from shared namespaces and INBOX, probably a bug
by Christoph Bussenius 09 Feb '12

09 Feb '12

Hi, we have experienced some erratic behavior from Dovecot 2.0.15 if a user's INBOX is shared. Some folders of user1, including the INBOX, have been shared using these IMAP commands: . login user1 XXXXXX . setacl INBOX user2 lrwstiekx . setacl box-a user2 lrwstiekx Now if we use telnet to log in as user2 and select "shared/user1", it will contain the same mails as "shared/user1/INBOX". The really strange thing is that "SELECT"-ing "shared/user1" succeeds only if it is the first command afted logging in. If it not the first command (if e. g. the "LIST" or "SELECT" command has already been used), then dovecot will report that the mailbox does not exist. As a result, user agents like Thunderbird will behave very erratically in this setup. They may show both mailboxes ("shared/user1" and "shared/user1/INBOX") with the same mails, but will not always succeed to download mails for "shared/user1". We were able to reproduce this behavior with a quite simple configuration (see further below), so I doubt that the problem is in the configuration. So, is this a bug in Dovecot? I am not quite sure how exactly it should behave, but it definitely does not seem right this way. Below are the results from telnet: Sending: . login user2 XXXXXX . select shared/user1 Receiving: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 5 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1316698162] UIDs valid * OK [UIDNEXT 6] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. Sending: . fetch 1:2 (internaldate rfc822.size) Receiving: * 1 FETCH (INTERNALDATE "25-Apr-2006 23:09:07 +0200" RFC822.SIZE 5660) * 2 FETCH (INTERNALDATE "25-Apr-2006 23:11:01 +0200" RFC822.SIZE 6612) . OK Fetch completed. Sending: . select shared/user1/INBOX Receiving: * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 5 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1316698162] UIDs valid * OK [UIDNEXT 6] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. Sending: . fetch 1:2 (internaldate rfc822.size) Receiving: (Note that these mails are the same as those above) * 1 FETCH (INTERNALDATE "25-Apr-2006 23:09:07 +0200" RFC822.SIZE 5660) * 2 FETCH (INTERNALDATE "25-Apr-2006 23:11:01 +0200" RFC822.SIZE 6612) . OK Fetch completed. Now we select the first mailbox again: Sending: . select shared/user1 Receiving: . NO Mailbox doesn't exist: shared/user1 * OK [CLOSED] Previous mailbox closed. The same command did not fail earlier. Listing the mailboxes shows the following: Sending: . list "" * Receiving: * LIST (\HasNoChildren) "/" "INBOX" * LIST (\Noselect \HasChildren) "/" "shared/user1" * LIST (\HasNoChildren) "/" "shared/user1/box-a" * LIST (\HasNoChildren) "/" "shared/user1/INBOX" . OK List completed. I can reproduce this behavior using the following setup: Virtual users on an ext4 filesystem in /mail/users/{user1,user2} configured in a passwd file (/usr/local/dovecot/etc/dovecot/users): user1:{plain}XXXXXX:::user1:/mail/users/user1:: user2:{plain}XXXXXX:::user2:/mail/users/user2:: Output of doveconf -n: # 2.0.15: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS disable_plaintext_auth = no mail_gid = vmail mail_plugins = acl mail_uid = vmail namespace { inbox = yes location = maildir:~/Maildir prefix = separator = / } namespace { list = children location = maildir:%%h/Maildir prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/mail/shared-mailboxes } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocol imap { mail_plugins = acl imap_acl } Cheers, Christoph Bußenius -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik TU München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

2 1

[Dovecot] Segfaul probably during dsync
by Matthias Rieber 09 Feb '12

09 Feb '12

Hi, I get core dumps, probably during dsync: Core was generated by `dovecot/imap postlogin'. Program terminated with signal 6, Aborted. [New process 7727] #0 0x00002ba911c04ed5 in raise () from /lib/libc.so.6 #0 0x00002ba911c04ed5 in raise () from /lib/libc.so.6 No symbol table info available. #1 0x00002ba911c063f3 in abort () from /lib/libc.so.6 No symbol table info available. #2 0x00002ba91157d775 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:187 backtrace = 0xcebd838 "/usr/local/lib/dovecot/libdovecot.so.0 [0x2ba91157d762] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x2ba91157d7c6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x2ba91157dbb8] -> /usr/local/l"... #3 0x00002ba91157d7c6 in i_internal_fatal_handler (ctx=0x7fffadba32a0, format=<value optimized out>, args=<value optimized out>) at failures.c:645 status = 0 #4 0x00002ba91157dbb8 in i_panic (format=<value optimized out>) at failures.c:259 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffadba3370, reg_save_area = 0x7fffadba32b0}} #5 0x00002ba9112b4018 in index_search_result_update_flags (result=0xdebe3e0, uids=0x7fffadba3640) at index-search-result.c:87 search_arg = {next = 0xd25b098, type = SEARCH_UIDSET, value = {subargs = 0x0, seqset = {arr = {buffer = 0xcebd640, element_size = 8}, v = 0xcebd640, v_modifiable = 0xcebd640}, str = 0x0, time = 0, size = 0, flags = 0, search_flags = 0, date_type = 0, thread_type = MAIL_THREAD_NONE, keywords = 0x0, modseq = 0x0, search_args = 0x0, search_result = 0x0, mailbox_glob = 0x0}, context = 0x0, hdr_field_name = 0x0, not = 0, match_always = 0, nonmatch_always = 0, result = -1} ret = <value optimized out> __FUNCTION__ = "index_search_result_update_flags" #6 0x00002ba91276ac92 in virtual_sync_backend_box (ctx=0xdebcdb0, bbox=0xdcb8e38) at virtual-sync.c:725 sync_flags = <value optimized out> status = {messages = 121, recent = 0, unseen = 111, uidvalidity = 1225530594, uidnext = 182153, first_unseen_seq = 0, first_recent_uid = 182152, highest_modseq = 0, virtual_size = 0, keywords = 0x0, cache_fields = 0x0, nonpermanent_modseqs = 0} ret = <value optimized out> __FUNCTION__ = "virtual_sync_backend_box" #7 0x00002ba91276b661 in virtual_storage_sync_init (box=0xcf524c0, flags=MAILBOX_SYNC_FLAG_FIX_INCONSISTENT) at virtual-sync.c:1400 sync_ctx = <value optimized out> ret = <value optimized out> #8 0x00002ba911299b71 in mailbox_sync_init (box=0xcf524c0, flags=MAILBOX_SYNC_FLAG_FIX_INCONSISTENT) at mail-storage.c:947 _data_stack_cur_id = 0 ctx = <value optimized out> #9 0x00002ba91129a337 in mailbox_sync (box=0x1e2f, flags=7727) at mail-storage.c:993 ctx = <value optimized out> status = {sync_delayed_expunges = 0} #10 0x0000000000417602 in imap_status_get (cmd=0xcf00410, ns=0xcefccd0, mailbox=0xcebd330 "rapidsolution", items=0x7fffadba3a50, result_r=0x7fffadba39e0, error_r=0x7fffadba3a38) at imap-status.c:85 client = (struct client *) 0xceffca0 box = (struct mailbox *) 0x1e2f error = <value optimized out> ret = <value optimized out> #11 0x000000000040e763 in cmd_status (cmd=0xcf00410) at cmd-status.c:52 client = (struct client *) 0xceffca0 status = MAILBOX_NAME_EXISTS_MAILBOX args = (const struct imap_arg *) 0xcf02c68 list_args = (const struct imap_arg *) 0xcf02da8 items = {mailbox_items = 17, guid = 0} result = {status = {messages = 124, recent = 0, unseen = 0, uidvalidity = 1301953092, uidnext = 1, first_unseen_seq = 0, first_recent_uid = 0, highest_modseq = 0, virtual_size = 48005140525491, keywords = 0x0, cache_fields = 0x80, nonpermanent_modseqs = 0}, mailbox_guid = "{µW\021©+\000\000\200\000\000\000\000\000\000"} ns = <value optimized out> mailbox = 0xcf02d58 "virtual.rapidsolution" storage_name = 0xcebd330 "rapidsolution" error = <value optimized out> #12 0x000000000041055c in client_command_input (cmd=0xcf00410) at imap-client.c:672 client = (struct client *) 0xceffca0 command = <value optimized out> __FUNCTION__ = "client_command_input" #13 0x000000000041060d in client_command_input (cmd=0xcf00410) at imap-client.c:722 client = (struct client *) 0xceffca0 command = (struct command *) 0x6 __FUNCTION__ = "client_command_input" #14 0x0000000000410785 in client_handle_input (client=0xceffca0) at imap-client.c:763 _data_stack_cur_id = 3 ret = 54 remove_io = <value optimized out> handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004111a2 in client_input (client=0xceffca0) at imap-client.c:814 cmd = <value optimized out> output = (struct ostream *) 0xcf00288 bytes = <value optimized out> __FUNCTION__ = "client_input" #16 0x00002ba911588e38 in io_loop_call_io (io=0xcf00320) at ioloop.c:384 ioloop = (struct ioloop *) 0xcec5620 t_id = 2 #17 0x00002ba911589dc2 in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:213 ctx = (struct ioloop_handler_context *) 0xcec5850 event = (const struct epoll_event *) 0xcec58c0 list = (struct io_list *) 0xcf00370 io = (struct io_file *) 0x1e2f tv = {tv_sec = 4, tv_usec = 514891} msecs = <value optimized out> ret = <value optimized out> call = false #18 0x00002ba911588dc8 in io_loop_run (ioloop=0xcec5620) at ioloop.c:405 No locals. #19 0x00002ba911576c93 in master_service_run (service=0xcec54f0, callback=0x1e2f) at master-service.c:478 No locals. #20 0x0000000000419401 in main (argc=2, argv=0xcec5370) at main.c:375 service_flags = <value optimized out> storage_service_flags = <value optimized out> postlogin_socket_path = 0xcebd080 "/usr/local/var/run/dovecot/postlogin" username = 0x0 c = <value optimized out> set_roots = {0x41c680, 0x0} Logfile: Apr 10 13:49:34 shrike dovecot: imap-login: Login: user=<matthias>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14235, secured Apr 10 13:49:34 shrike dovecot: (null)Error: rawlog_open: open() failed for /home/matthias/dovecot.rawlog/20110410-134934-14237.in: Permission denied Apr 10 13:49:43 shrike dovecot: imap(matthias): Panic: file index-search-result.c: line 87 (search_result_update_search): assertion failed: (next_uid == mail->uid) Apr 10 13:49:43 shrike dovecot: imap(matthias): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0 [0x2b4df76ce762] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x2b4df76ce7c6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x2b4df76cebb8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_search_result_update_flags+0x2d8) [0x2b4df7405018] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so [0x2b4df88bbc92] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x871) [0x2b4df88bc661] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x2b4df73eab71] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x2b4df73eb337] -> dovecot/imap(imap_status_get+0x92) [0x417602] -> dovecot/imap(cmd_status+0x163) [0x40e763] -> dovecot/imap [0x41055c] -> dovecot/imap [0x41060d] -> dovecot/imap(client_handle_input+0x45) [0x410785] -> dovecot/imap(client_input+0x62) [0x4111a2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x2b4df Apr 10 13:49:43 shrike dovecot: master: Error: service(imap): child 14235 killed with signal 6 (core dumped) # dovecot --version 2.0.11 (cef23de90ff1) Matthias

2 3

[Dovecot] 2.1.rc1 (8a63f621bd2e): SiS permission issue + crash
by Pascal Volk 09 Feb '12

09 Feb '12

Since changeset 8a63f621bd2e I'm one step closer to the perfect working setup. I'm sharing one GID per domain, all it's alias names and accounts. So I can use: mail_attachment_dir = /srv/mail/.SiS/%{gid} In order to avoid permission trouble I've set fs ACL on the .SiS dir: ,--[ getfacl srv/mail/.SiS ]-- | # file: srv/mail/.SiS | # owner: root | # group: root | user::rwx | group::--x | mask::--x | other::-wx | default:user::rwx | default:group::rwx | default:mask::rwx | default:other::--- `-- OK, lets test the setup: dsync -u test-0(a)example.com mirror maildir:/tmp/Maildir rm -rf Maildir && cp -a Maildir_org Maildir && chown -R 70010:70002 Maildir dsync -vu test-1(a)example.com mirror maildir:/tmp/Maildir dsync(test-1(a)example.com): Info: Drafts: only in dest (guid=b6a53627f3cbe24e33030000850d2fad) dsync(test-1(a)example.com): Info: Sent: only in dest (guid=b7a53627f3cbe24e33030000850d2fad) dsync(test-1(a)example.com): Info: Junk-E-Mail: only in dest (guid=b8a53627f3cbe24e33030000850d2fad) dsync(test-1(a)example.com): Info: Trash: only in dest (guid=b9a53627f3cbe24e33030000850d2fad) dsync(test-1(a)example.com): Info: INBOX: only in dest (guid=baa53627f3cbe24e33030000850d2fad) dsync(test-1(a)example.com): Error: stat(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.4f06409857c627e0) failed: Permission denied dsync(test-1(a)example.com): Error: safe_mkstemp(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.) failed: Permission denied dsync(test-1(a)example.com): Panic: file dsync-worker-local.c: line 1644 (local_worker_save_msg_continue): assertion failed: (ret == -1) dsync(test-1(a)example.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4faf1) [0x7f4db31f4af1] -> /usr/local/lib/dovecot/libdovecot.so.0(default_error_handler+0) [0x7f4db31f4b7d] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4db31f4e01] -> dsync() [0x416af8] -> dsync() [0x416e46] -> dsync(dsync_worker_msg_save+0x82) [0x412cb2] -> dsync() [0x40b7e0] -> dsync() [0x417286] -> dsync() [0x417324] -> dsync(dsync_worker_msg_get+0xa8) [0x412dd4] -> dsync() [0x40bbf3] -> dsync() [0x40bd62] -> dsync() [0x40c106] -> dsync() [0x40c318] -> dsync(dsync_brain_msg_sync_new_msgs+0x1c) [0x40c336] -> dsync(dsync_brain_msg_sync_more+0x1ae) [0x40b15e] -> dsync() [0x409b88] -> dsync(dsync_brain_sync+0x231) [0x40a074] -> dsync() [0x4084fb] -> dsync() [0x408729] -> dsync(dsync_brain_sync+0x1a7) [0x409fea] -> dsync() [0x408238] -> dsync() [0x408388] -> dsync(dsync_brain_sync+0x10b) [0x409f4e] -> dsync(dsync_brain_sync_all+0x24) [0x40a16b] -> dsync(main+0x680) [0x407b2b ] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f4db2e3fead] -> dsync() [0x406e09] Aborted (core dumped) Looks like Dovecot ignores the ACL. I think a mail_attachment_dir_mode setting could help to work around this problem. Dunno what would be a good default value. But I would set mail_attachment_dir_mode = 0770 The attachment files have mode 0660, that's perfect in my setup. Regards, Pascal -- The trapper recommends today: c01dcafe.1134404(a)localdomain.org

2 2

Re: [Dovecot] Crash on mail folder delete
by Daniel L. Miller 09 Feb '12

09 Feb '12

On 1/25/2012 3:43 PM, Daniel L. Miller wrote: > On 1/25/2012 3:42 PM, Timo Sirainen wrote: >> On 26.1.2012, at 1.37, Daniel L. Miller wrote: >> >>> Attempting to delete a folder from within the trash folder using >>> Thunderbird. I see the following in the log: >> Dovecot version? >> > 2.1.rc3. I'm compiling rc5 now... > Error still there on rc5. Jan 25 16:03:47 bubba dovecot: imap(dmiller(a)amfes.com): Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Jan 25 16:03:47 bubba dovecot: imap(dmiller(a)amfes.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3f1ba) [0x7f7c3f0331ba] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3f206) [0x7f7c3f033206] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x1804a) [0x7f7c3f00c04a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x47317) [0x7f7c3f2c0317] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6c71) [0x7f7c3dc48c71] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6d47) [0x7f7c3dc48d47] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(acl_mailbox_allocated+0x9e) [0x7f7c3dc4c61e] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_allocated+0x62) [0x7f7c3f2b4662] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xb2) [0x7f7c3f2b3482] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](cmd_delete+0x72) [0x409972] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](command_exec+0x3d) [0x4109fd] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete]() [0x40f9ce] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete]() [0x40faad] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](client_handle_input+0x135) [0x40fcd5] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](client_input+0x5f) [0x4105ff] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f7c3f03f5d6] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f7c3f04065f] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f7c3f03f578] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f7c3f02c593] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](main+0x2a5) [0x418a55] -> /lib/libc.so.6(__libc_start_main+0xfe) [0x7f7c3ec8fd8e] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete]() [0x408449] Jan 25 16:03:48 bubba dovecot: imap(dmiller(a)amfes.com): Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Jan 25 16:03:48 bubba dovecot: imap(dmiller(a)amfes.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3f1ba) [0x7f9e52e211ba] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3f206) [0x7f9e52e21206] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x1804a) [0x7f9e52dfa04a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x47317) [0x7f9e530ae317] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6c71) [0x7f9e51a36c71] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0x6d47) [0x7f9e51a36d47] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(acl_mailbox_allocated+0x9e) [0x7f9e51a3a61e] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_allocated+0x62) [0x7f9e530a2662] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xb2) [0x7f9e530a1482] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](cmd_delete+0x72) [0x409972] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](command_exec+0x3d) [0x4109fd] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete]() [0x40f9ce] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete]() [0x40faad] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](client_handle_input+0x135) [0x40fcd5] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](client_input+0x5f) [0x4105ff] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f9e52e2d5d6] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f9e52e2e65f] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f9e52e2d578] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f9e52e1a593] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete](main+0x2a5) [0x418a55] -> /lib/libc.so.6(__libc_start_main+0xfe) [0x7f9e52a7dd8e] -> dovecot/imap [dmiller(a)amfes.com 192.168.0.91 delete]() [0x408449] Jan 25 16:03:48 bubba dovecot: imap(dmiller(a)amfes.com): Fatal: master: service(imap): child 3300 killed with signal 6 (core dumps disabled) Jan 25 16:03:48 bubba dovecot: imap(dmiller(a)amfes.com): Fatal: master: service(imap): child 3267 killed with signal 6 (core dumps disabled) -- Daniel

2 1

[Dovecot] BUG(?): Incorrect responses for ACL prohibited actions
by Michael M Slusarz 09 Feb '12

09 Feb '12

In a mailbox that has ACL restrictions on both DELETE and EXPUNGE (i.e. no 'e' or 't' rights), I see the following: 3 UID STORE 3173 (UNCHANGEDSINCE 353) +FLAGS \deleted 3 OK Store completed. (UID 3173 is not flagged \deleted) [...] 6 UID EXPUNGE 1:* 6 OK Expunge completed. (At least 1 UID is flagged \deleted in mailbox) Shouldn't these commands be returning "NO" instead of "OK"? RFC 3501 [6.4.6] for STORE: NO - store error: can't store that data and RFC RFC 3501 [6.4.3] for EXPUNGE: NO - expunge failure: can't expunge (e.g., permission denied) Additionally, RFC 5530 [3] provides the NOPERM response code: NOPERM The access control system (e.g., Access Control List (ACL), see [RFC 4314]) does not permit this user to carry out an operation, such as selecting or creating a mailbox. C: f select "/archive/projects/experiment-iv" S: f NO [NOPERM] Access denied My reading of this is that NOPERM should be returned for ANY ACL prohibited action, not just for selecting or creating a mailbox. Dovecot 2.0.12 does not return NOPERM for DELETE/EXPUNGE actions (at a minimum) that are prohibited. Thoughts? michael

2 3

[Dovecot] [dovecot] Getting duplicates when using snarf plugin with mbox backend
by Jonathan SIegle 09 Feb '12

09 Feb '12

Running dovecot version 2.0.11. To reproduce, open two imap sessions and issue a check command from each at the same time with new mail in the queue. 0 login testuser testpw 1 select inbox -- Deliver mail -- 2 check Thanks, Jonathan

2 1

[Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox
by Peter Mogensen 09 Feb '12

09 Feb '12

Hi, Considering the scenario, where you have some old account with a different POP3 UIDL format and you migrate them to dovecot. So these old UIDLs would be saved to dovecot-uidlist. At some later time you want to introduce a virtual POP3 INBOX like described on: http://wiki.dovecot.org/Plugins/Virtual So you decide to make the new UIDL format "%f" - to make them unique across folders. So far so good. But then you decide to migrate to mdbox with all your old UIDLs. The docs says that saving old UIDLs is only supported in Maildir and that %f is only supported in Maildir. So is this at all possible? Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? /Peter

2 1

[Dovecot] vsz_limit
by Héctor Moreno Blanco 09 Feb '12

09 Feb '12

Hello Timo and dovecot teamwork, We have dovecot 2.0.10 installed with this configuration: # dovecot -n # 2.0.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.18.el5 x86_64 Scientific Linux SL release 5.2 (Boron) auth_cache_size = 500 k auth_cache_ttl = 1000 secs disable_plaintext_auth = no first_valid_uid = 501 lock_method = dotlock mail_fsync = always mail_gid = vmail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota autocreate mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin/autocreate = Spam plugin/autosubscribe = Spam plugin/quota = maildir:_backup:ns= plugin/quota_warning = storage=75%% quota-warning 75 %u plugin/quota_warning2 = storage=90%% quota-warning 90 %u plugin/sieve = %h/.dovecot.sieve plugin/sieve_dir = %h/sieve protocols = imap pop3 sieve service anvil { client_limit = 5000 } service auth { client_limit = 9000 unix_listener auth-master { group = vmail mode = 0700 user = vmail } user = root vsz_limit = 2 G } service imap-login { executable = /usr/local/dovecot/libexec/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 2048 process_min_avail = 20 user = vmail } service imap { executable = /usr/local/dovecot/libexec/dovecot/imap process_limit = 2048 } service managesieve-login { executable = /usr/local/dovecot/libexec/dovecot/managesieve-login inet_listener sieve { address = * port = 4190 } inet_listener sieve_deprecated { address = * port = 12000 } user = vmail } service managesieve { executable = /usr/local/dovecot/libexec/dovecot/managesieve } service pop3-login { executable = /usr/local/dovecot/libexec/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 2048 process_min_avail = 20 user = vmail } service pop3 { executable = /usr/local/dovecot/libexec/dovecot/pop3 process_limit = 2048 } service quota-warning { executable = script /usr/local/dovecot/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0700 user = vmail } } ssl_cert = </etc/ssl/correo_xxxx.crt ssl_key = </etc/ssl/correo_xxxx.pem syslog_facility = local3 userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } valid_chroot_dirs = /buzones/ protocol imap { imap_client_workarounds = mail_plugin_dir = /usr/local/dovecot/lib/dovecot mail_plugins = quota autocreate imap_quota } protocol pop3 { mail_plugin_dir = /usr/local/dovecot/lib/dovecot mail_plugins = quota autocreate pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /usr/local/dovecot/var/run/dovecot/auth-master hostname = buzon.xxxx.es mail_plugin_dir = /usr/local/dovecot/lib/dovecot mail_plugins = quota autocreate sieve postmaster_address = postmaster(a)xxxx.es quota_full_tempfail = no rejection_reason = Su mensaje para <%t> fue rechazado automaticamente por nuestro sistema :%n%r rejection_subject = Rechazado: %s sendmail_path = /usr/sbin/exim } protocol sieve { managesieve_implementation_string = dovecot Pigeonhole managesieve_logout_format = bytes ( in=%i : out=%o ) managesieve_max_line_length = 65536 } The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? Thank you very much in advanced. Kind regards. ________________________________ [cid:image001.gif@01CCE647.65B8FB30] Héctor Moreno Blanco División de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com <http://www.gmv.com/> [cid:image002.gif@01CCE647.65B8FB30]<http://www.gmv.com/b2_gmv> [cid:image003.gif@01CCE647.65B8FB30]<http://www.facebook.com/infoGMV> [cid:image004.gif@01CCE647.65B8FB30]<http://www.twitter.com/infoGMV_es> [cid:image005.gif@01CCE647.65B8FB30]<http://www.youtube.com/infoGMV> ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________

3 5

[Dovecot] fts (lucene): indexing of virtual mailboxes?
by Lutz Preßler 09 Feb '12

09 Feb '12

Hello, (recent 2.1alpha2 variant - my test setup known to Timo). No time to diagnose in depth at the moment, but I just noticed that SEARCHing in virtual mailboxes seems not to create lucene index content of its own but use those of referenced mailboxes? The problem is that no new indexing takes place. Example: with INBOX INBOX.in% all in dovecot-virtual, for a given query I only get matches from those mailboxes searched in previously. Regards, Lutz

2 2

Re: [Dovecot] Performance of Maildir vs sdbox/mdbox
by Lee Standen 09 Feb '12

09 Feb '12

On 18.01.2012 21:54, Timo Sirainen wrote: > On Wed, 2012-01-18 at 20:44 +0800, Lee Standen wrote: > >> I've been desperately trying to find some comparative performance >> information about the different mailbox formats supported by Dovecot >> in >> order to make an assessment on which format is right for our >> environment. > > Unfortunately there aren't really any. Everyone who seems to switch > to > sdbox/mdbox usually also change their hardware at the same time, so > there aren't really any before/after metrics. I've of course some > unrealistic synthetic benchmarks, but I don't think they are very > useful. > > So, I would also be very interested in seeing some before/after > graphs > of disk IO, CPU and memory usage of Maildir -> dbox switch in same > hardware. > > Maildir is anyway definitely worse performance then sdbox or mdbox. > mdbox also uses less NFS operations, but I don't know how much faster > (if any) it is with Netapps. We have bought new hardware for this project too, so we might not be able to help out massively on that front... we do have NFS operations monitored though so we should at least be able to compare that metric since the underlying storage operating system is the same. All NetApp hardware runs their Data ONTAP operating system, so the metrics are assured to be the same :) How about this... are there any tools available (that you know of) to capture real live customer POP3/IMAP traffic and replay it against a separate system? That might be a feasible option for doing a like-for-like comparison in our environment? We could probably get something in place to simulate the load if we can do something like that... >> * All mail storage presented via NFS over 10Gbps Ethernet (Jumbo >> Frames) >> >> * Postfix will feed new email to Dovecot via LMTP >> >> * Dovecot servers have been split based on their role >> >> - Dovecot LDA Servers (running LMTP protocol) >> >> - Dovecot POP/IMAP servers (running POP/IMAP protocols) > > You're going to run into NFS caching troubles with the above split > setup. I don't recommend it. You will see error messages about index > corruption with it, and with dbox it can cause metadata loss. > http://wiki2.dovecot.org/NFS http://wiki2.dovecot.org/Director That might be the one thing (unfortunately) which prevents us from going with the dbox format. I understand the same issue can actually occur on Dovecot Maildir as well, but because Maildir works without these index files, we were willing to just go with it. I will raise it again, but there has been a lot of push back about introducing a single point of failure, even though this is a perceived one. The biggest challenge I have at the moment if I try to sell the dbox format is providing some kind of data on the expected gains from this. If it's only a 10% reduction in NFS operations for the typical user, then it's probably not worth our while. > >> - LDA & POP/IMAP servers are segmented into geographically split >> groups >> (so no server sees every single mailbox) >> >> - Nginx proxy used to terminate customer connections, connections >> are >> redirected to the appropriate geographic servers > > Can the same mailbox still be accessed via multiple geographic > servers? > I've had some plans for doing this kind of access/replication using > dsync.. No, we're using the nginx proxy layer to ensure that if a user in Sydney (for example) tries to access a Perth mailbox, their connection is redirected (by nginx) to the Perth POP/IMAP servers. Postfix configuration is handling the same thing on the LMTP side. The requirement here is for all users to have the same settings regardless of location, but still be able to locate the email servers and data close to the customer. > >> * Apache Lucene indexes will be used to accelerate IMAP search for >> users > > Dovecot's fts-solr or fts-lucene? fts-solr. I've been using Lucene/Solr interchangeably when discussing this project with my peers :) > >> Our closest current live configuration (Qmail SMTP, Courier IMAP, >> Maildir) >> has 600K mailboxes and pushes ~ 35,000 NFS operations per second at >> peak >> >> Some of the things I would like to know: >> >> * Are we likely to see a reduction in IOPS/User by using Maildir >> alone under >> Dovecot? > > If you have webmail type of clients, definitely. For > Outlook/Thunderbird > you should still see improvement, but not necessarily as much. > > You didn't mention POP3. That isn't Dovecot's strong point. Its > performance should be about the same as Courier-POP3, but could be > less > than QMail-POP3. Although if many of your POP3 users keep a lot of > mails > on server it > Our existing systems run with about 21K concurrent IMAP connections at any one point in time, not counting Webmail POP3 runs at about 3600 concurrent connections, but since those are not long lived it's not particularly indicative of customer numbers. Vague recollection is something like 25% IMAP, 55-60% POP3, rest < 20% Webmail. I'd have to go back and check the breakdown again. >> * If someone can give some technical reasoning behind why mdbox does >> less >> IOPS than Maildir? > > Maildir renames files a lot. From new/ -> to cur/ and then every time > message flag changes. That's why sdbox is faster. Why mdbox should be > faster than sdbox is because mdbox puts (or should put) more mail > data > physically closer in disks to make reading it faster. > >> I understand some of the reasons for the mdbox IOPS question, but I >> need >> some more information so we can discuss internally and make a >> decision as to >> whether we're comfortable going with mdbox from day one. We're very >> familiar with Maidlir, and there's just some uneasiness internally >> around >> going to a new mail storage format. > > It's at least safer to first switch to Dovecot+Maildir to make sure > that > any problems you might find aren't related to the mailbox format.. Yep, I'm considering that. The flip side is that it's actually going to be difficult for us to change mail format once we've migrated into this system, but we have an opportunity for (literally) a month long testing phase beginning in Feb/March which will let us test as many possibilities as we can.

4 16

[Dovecot] [PATCH] Bad boundary check in client_find_namespace
by Christoph Bussenius 09 Feb '12

09 Feb '12

Hi, while trying to investigate the bug I reported last week, I found that there is a broken boundary check in client_find_namespace in src/imap/imap-commands-util.c. The code is: /* make sure two hierarchy separators aren't next to each others */ for (p = storage_name+1; *p != '\0'; p++) { if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { client_send_tagline(cmd, "NO Invalid mailbox name."); return NULL; } } The loop iterates over the string storage_name starting from its second byte. However, the string may be "". This is the case if you select the root of a namespace, like in "SELECT shared" or "SELECT shared/user1". In that case, the code will read past the end of the buffer from random memory until it finds a zero byte or a duplicate separator. This would fix it: diff -ru dovecot-2.0.15.orig/src/imap/imap-commands-util.c dovecot-2.0.15/src/imap/imap-commands-util.c --- dovecot-2.0.15.orig/src/imap/imap-commands-util.c 2011-08-02 12:29:37.000000000 +0200 +++ dovecot-2.0.15/src/imap/imap-commands-util.c 2011-09-26 18:33:16.121917759 +0200 @@ -81,7 +81,9 @@ } /* make sure two hierarchy separators aren't next to each others */ - for (p = storage_name+1; *p != '\0'; p++) { + for (p = storage_name; *p != '\0'; p++) { + if (p == storage_name) + continue; if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { client_send_tagline(cmd, "NO Invalid mailbox name."); return NULL; Cheers, Christoph Bußenius -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik TU München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

3 2