August 2013 - dovecot

[Dovecot] constant Log synchronization error's
by Kim Bisgaard 10 Aug '13

10 Aug '13

Hi, I am seeing constant "Log synchronization error"s in my logs. See later for an example. This is dovecot 2.2.4, pigeonhole 0.4.1, and postfix 2.10.1 (with mailbox_command = /usr/libexec/dovecot/deliver) on a Fedora 19 system. I am using packages from Atrpms so there are some patches applied - ask for details. The mailboxes have resently been moved from a i386 system. I have tried to manually do "doveadm force-resync" for all users, but the problem persists :-( Here are an example from a user which does not use sieve: 11:45:48 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21531, TLS, session=<XxDNm3XhdAA+xz17> 11:45:52 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21541, TLS, session=<UewQnHXhAwA+xz17> 11:45:54 Y dovecot: imap(X): Error: mdbox /var/mail/X/mdbox/mailboxes/INBOX/Streams/dbox-Mails: map uidvalidity mismatch (0 vs 1283330954) 11:45:54 Y dovecot: imap(X): Disconnected: Internal error occurred. Refer to server log for more information. [2013-07-14 11:45:54] in=353 out=115680 11:45:54 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/storage/dovecot.map.index 11:45:54 Y dovecot: imap(X): Warning: mdbox /var/mail/X/mdbox/storage: rebuilding indexes 11:45:54 Y dovecot: imap(X): Disconnected: Disconnected in IDLE in=707 out=437806 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=4,offset=156 for /var/mail/X/mdbox/mailboxes/INBOX/Kvitteringer/dbox-Mails/dovecot.index: Ex tension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/INBOX/Kvitteringer/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=9,offset=156 for /var/mail/X/mdbox/mailboxes/INBOX/Streams/dbox-Mails/dovecot.index: Extensi on header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/INBOX/Streams/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=13,offset=156 for /var/mail/X/mdbox/mailboxes/Facebook/dbox-Mails/dovecot.index: Extension h eader update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Facebook/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Junk/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Junk/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Sent/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Sent/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=4,offset=156 for /var/mail/X/mdbox/mailboxes/Drafts/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Drafts/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=4,offset=156 for /var/mail/X/mdbox/mailboxes/Archives/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Archives/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=62.199.61.123, lip=192.168.0.5, TLS, session=<dz05nHXh9wA+xz17> 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=4,offset=156 for /var/mail/X/mdbox/mailboxes/Archives/2012/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Archives/2012/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21548, TLS, session=<bAs5nHXhfgA+xz17> 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap(X): Error: Log synchronization error at seq=4,offset=156 for /var/mail/X/mdbox/mailboxes/Trash/Spam/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:55 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Trash/Spam/dbox-Mails/dovecot.index 11:45:55 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21552, TLS, session=<zCw7nHXh2AA+xz17> 11:45:55 Y dovecot: imap(X): Connection closed: Connection reset by peer in=118 out=111246 11:45:56 Y dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=62.199.61.123, lip=192.168.0.5, TLS, session=<Ov9MnHXhgwA+xz17> 11:45:56 Y dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=62.199.61.123, lip=192.168.0.5, TLS, session=<OK5RnHXhiAA+xz17> 11:45:56 Y dovecot: imap(X): Error: mdbox /var/mail/X/mdbox/mailboxes/INBOX/Streams/dbox-Mails: map uidvalidity mismatch (0 vs 1283330954) 11:45:56 Y dovecot: imap(X): Disconnected: Internal error occurred. Refer to server log for more information. [2013-07-14 11:45:56] in=353 out=115647 11:45:56 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/storage/dovecot.map.index 11:45:56 Y dovecot: imap(X): Warning: mdbox /var/mail/X/mdbox/storage: rebuilding indexes 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/INBOX/Kvitteringer/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/INBOX/Kvitteringer/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=10,offset=156 for /var/mail/X/mdbox/mailboxes/INBOX/Streams/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/INBOX/Streams/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=14,offset=156 for /var/mail/X/mdbox/mailboxes/Facebook/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Facebook/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=6,offset=156 for /var/mail/X/mdbox/mailboxes/Junk/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Junk/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=6,offset=156 for /var/mail/X/mdbox/mailboxes/Sent/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Sent/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Drafts/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Drafts/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Archives/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Archives/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Archives/2012/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Archives/2012/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=6,offset=156 for /var/mail/X/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap(X): Error: Log synchronization error at seq=5,offset=156 for /var/mail/X/mdbox/mailboxes/Trash/Spam/dbox-Mails/dovecot.index: Extension header update points outside header size 11:45:57 Y dovecot: imap(X): Warning: fscking index file /var/mail/X/mdbox/mailboxes/Trash/Spam/dbox-Mails/dovecot.index 11:45:57 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21563, TLS, session=<gs9WnHXhcwA+xz17> 11:45:57 Y dovecot: imap(X): Connection closed: Connection reset by peer in=150 out=111316 11:45:57 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21568, TLS, session=<4y1hnHXhkQA+xz17> 11:46:05 Y dovecot: imap-login: Login: user=<X>, method=PLAIN, rip=62.199.61.123, lip=192.168.0.5, mpid=21574, TLS, session=<+zbNnHXhBAA+xz17> My config is: # dovecot -n # 2.2.4: /etc/dovecot/dovecot.conf # OS: Linux 3.9.9-301.fc19.x86_64 x86_64 Fedora release 19 (Schrödinger’s Cat) ext4 mail_location = mdbox:/var/mail/%u/mdbox mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_subaddress_sep = - } postmaster_address = postmaster(a)alleroedderne.adsl.dk protocols = imap recipient_delimiter = - ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } protocol lda { mail_plugins = " sieve" } Regards, Kim

3 3

[Dovecot] Mailbox Format
by Vijay Rajah 10 Aug '13

10 Aug '13

Hello List, I'm in the process of implementing a new mail server environment from scratch. I was wondering, what mailbox format is recommended? I have been using maildir format for my other projects. I would like to use some format that provides Single Instance Storage. I'm using postfix 2.11 and will use lmtp. I will be using dovecot 2.2.5 (Latest) version. I'm not planning on using NFS (if that matters) and will use dsync to backup mails to another server. Thanks in Advance Vijay Rajah

1 0

Re: [Dovecot] prevent dovecot from modifying maildirs
by Joseph Tam 10 Aug '13

10 Aug '13

> In order to save space I prefer no to have such half-files (mails) , > and it is for this reason that I am asking if there is any way to > prevent this. > > I'm not an expert in field of file systems, so I can be asking very > naive questions... but I appreciate your comments on this topic. Filesystem consistency is different than, for lack of a better term, dovecot consistency. The latter places additional constraints on the content of files, rather than treating them as sequences of disk blocks. I don't think you can ensure the latter without stopping all dovecot processes (and maybe the MTA/LDA as well), which will be much more difficult and disruptive than just taking a snapshot. (I think some snapshot aware application like databases and hypervisors allows the operator to put the application in a state of acquiesence, long enough to take a snapshot, then unblock all processes after the snapshot.) I'd take the suggestion of the previous poster: take snapshots, don't worry, be happy. The small amount of inconsistencies is trivial compared with quickly recovering a mail store after disaster. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: [Dovecot] How to troubleshoot LDA or LMTP?
by Joseph Tam 10 Aug '13

10 Aug '13

John Williams writes: > *What doesn't work* I thought the next step would be to execute > dovecot-lda manually (as is suggested on the wiki and in numerous > newsgroup posts), but I can't figure out how to do this. When I execute > it as root and provide my non-root username via the -u option, it > hangs. I can see that it spawns a child process that executes as the > non-root user (specified with the -u option). So, could someone tell > how to execute dovecot-lda manually please? Maybe use "-d"? I had no trouble invoking dovecot-lda -c config-file -d user < mailfile If it doesn't work for you, I suggest process tracing it. > Also, when does LMTP process messages? When sending mail via SMTP, or > when reading mail via IMAP? I would prefer to use LMTP rather than > LDA. The former. Joseph Tam <jtam.home(a)gmail.com>

4 6

[Dovecot] execvp /usr/local/libexec/dovecot/dovecot-lda: Permission denied
by Edward Carraro 09 Aug '13

09 Aug '13

I'm trying to follow the instructions for configuring Dovecot LMTP with postfix for virtual domains but whenever mail arrives, in /var/log/maillog, I see Sep 23 16:01:20 mybox postfix/pipe[572]: 999C83206C6: to=< user(a)mytest.domain.com>, relay=dovecot, delay=0.08, delays=0.04/0.01/0/0.03, dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: pipe_command: execvp /usr/local/libexec/dovecot/dovecot-lda: Permission denied ) ls -lh for /usr/local/libexec/dovecot/dovecot-lda -rwxr-xr-x 1 root staff 62K Sep 20 15:15 dovecot-lda It will deliver mail when I do the sudo invocation of dovecot-lda, but I thought LMTP was suppose to be an alternative to that postfix/main.cf: virtual_transport = dovecot virtual_mailbox_domains = mytest.domain.com dovecot_destination_recipient_limit = 1 /etc/postfix/master.cf: dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${recipient} /usr/local/etc/dovecot/conf.d/10-master.conf: service auth { unix_listener auth-userdb { mode = 0600 user = vmail #group = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } inet_listener lmtp { port = 24 } user = vmail } Did I miss something in my config?

7 9

[Dovecot] Getting default uid/gid of users via socket for virtual user support for sendmail
by Steffen Kaiser 09 Aug '13

09 Aug '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'm trying to make my life easier with sendmail and virtual users. For that I'd like to pass sendmail the home directory and uid/gid for each user. The user data is storred in a LDAP,& I retrieve it from the auth-userdb socket fine. If I add uid / gid to each user, I get them via that socket as well, but is there a way I get the default uid/gid? I mean, without calling an external program, such as doveadm. There is an "initialize" phase, so the default information is retrieved just once per lifetime of the demon, but each time the sendmail binary is invoked. Kind regards, - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUfp2JF3r2wJMiz2NAQL61Af+MuF1BTwaX+Tm+rOWe+smcVUzRmMYZdtL J08wDGNB2EyQ9a0zXwZleO/X3Jvaqr8HSHaC6oTzliS25yqPT4AYG0VVswZoyDTg hggyS9kVse2R1koLZKPAM1EatTmpTsKfUKDjsB3cxbYsBU5NYMJY+u4YATPTl7ui 46K4YmeL4xkKirRXc7j0fIMxdXttuD9zPq20qJ0p41HK2W0ECVHeQpUahozWY45h DREbwdZBZe9PeZNbfOYTLWnxPi1/FK/AX20QH3y3uxEhgMDfYAUHNwAleSCpack5 JEuVYwTPchxsH6uibk62qIsK/uWO3vZM0Zj18S0BBWDNvPhg0Nz92w== =abqh -----END PGP SIGNATURE-----

2 3

[Dovecot] Override master service settigs with spaces
by Ulrich Zehl 08 Aug '13

08 Aug '13

I'd like to override one setting for a master service in conf.d/10-master.conf. Unfortunately, said setting contains spaces, and I do not know how to escape them properly. Here's what I've tried so far. (Note: This is just the easiest/silliest test case I could come up with; not the actual setting or service I want to overwrite.) conf.d/10-master.conf: service quota-status { executable = quota-status -p postfix -o plugin/quota_status_success=Testing 1 2 3 inet_listener { port = 12340 } } Reaction: $ printf "recipient=test(a)example.org\nsize=0\n\n" | nc 127.0.0.1 12340 action=Testing conf.d/10-master.conf: service quota-status { executable = quota-status -p postfix -o "plugin/quota_status_success=Testing 1 2 3" inet_listener { port = 12340 } } Reaction: $ printf "recipient=test(a)example.org\nsize=0\n\n" | nc 127.0.0.1 12340 action=DUNNO [<-- This is the default setting] conf.d/10-master.conf: service quota-status { executable = quota-status -p postfix -o 'plugin/quota_status_success=Testing 1 2 3' inet_listener { port = 12340 } } Reaction: $ printf "recipient=test(a)example.org\nsize=0\n\n" | nc 127.0.0.1 12340 action=DUNNO [<-- This is the default setting] conf.d/10-master.conf: service quota-status { executable = quota-status -p postfix -o plugin/quota_status_success="Testing 1 2 3" inet_listener { port = 12340 } } Reaction: $ printf "recipient=test(a)example.org\nsize=0\n\n" | nc 127.0.0.1 12340 action="Testing [<-- Note the " after = ] conf.d/10-master.conf: service quota-status { executable = quota-status -p postfix -o plugin/quota_status_success=Testing\ 1\ 2\ 3 inet_listener { port = 12340 } } Reaction: $ printf "recipient=test(a)example.org\nsize=0\n\n" | nc 127.0.0.1 12340 action=Testing\ [<-- Note the \ ] I'm out of ideas now. What's the correct way to quote / escape these options?

3 4

[Dovecot] 2.2.5 fails to compile - SSL_OP_SINGLE_ECDH_USE : Solaris 10 + Sunstudio 12.3
by Tomi Vainio 07 Aug '13

07 Aug '13

Hi, Compile fails in these two files src/lib-ssl-iostream/iostream-openssl-context.c src/login-common/ssl-proxy-openssl.c due to missing SSL_OP_SINGLE_ECDH_USE In these there is only #if !defined(OPENSSL_NO_ECDH) when all the other places it always compares also SSL version #if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10002000L Tomppa

3 4

[Dovecot] keywords leak in cmd_append_handle_args()
by Mike Abbott 07 Aug '13

07 Aug '13

In 2.2.5 and earlier it appears that mailbox_keywords_unref(&keywords) is not called in some return paths from cmd_append_handle_args(). Should it be?

2 1

[Dovecot] Postfix aliases with quota-status service
by Thomas Leuxner 07 Aug '13

07 Aug '13

The latest HG commits seem to have fixed some underlying problems with the 'quota-status' service. Now doing some quick tests I wonder if this can be used with aliases on the Postfix side. Appears the 'check_policy_service' used in the example below will query existing users via Dovecot's Auth Backend: http://sys4.de/en/blog/2013/04/05/dovecot-quota-mit-postfix-abfragen/ http://www.postfix.org/SMTPD_POLICY_README.html This works fine for Dovecot user 'philo' where the mail is rejected upfront before it enters the queue: ==> /var/log/mail.log <== Jul 30 13:41:18 spectre postfix/smtpd[31072]: NOQUEUE: reject: RCPT from host.example.com[1.2.3.4]: 554 5.2.2 <philo(a)trashheap.net>: Recipient address rejected: Quota exceeded (mailbox for use r is full); from=<user(a)example.com> to=<philo(a)trashheap.net> proto=ESMTP helo=<host.example.com> Now user 'gunge' is an alias for 'philo' in Postfix, thus the quota check will not recognize the user, will queue the message and reject it at the MDA stage: ==> /var/log/mail.log <== Jul 30 14:41:02 spectre postfix/lmtp[31460]: 3c4HSD5ZkBzBP: to=<philo(a)trashheap.net>, orig_to=<gunge(a)trashheap.net>, relay=spectre.leuxner.net[private/dovecot-lmtp], delay=1.2, delays=1.1/0.01/0/0.13, dsn=4.2.2, status=deferred (host spectre.leuxner.net[private/dovecot-lmtp] said: 452 4.2.2 <philo(a)trashheap.net> Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) This is probably intended behaviour, just want to make sure that I'm not missing a point here. For now the only fix that comes to my mind to create "quota aware" aliases - is creating 'dummy' users in Dovecot which point to the same mailbox rather than performing aliasing on the Postfix end. Open to suggestions... Regards Thomas

7 15