December 2014 - dovecot

Corrupted index cache after migrating from maildir to mdbox
by Tim 09 Jan '15

09 Jan '15

Hi guys, I'm running dovecot/postfix on my Debian Jessie maschine: # dovecot --version 2.2.13 doveconf -n output: http://p.bastelfreak.de/HsVwS/ today I migrated from maildir(located at /var/vmail) to mdbox with these commands: rsync -av -f"+ */" -f"- *" /var/vmail/ /home/vmail/ rm /home/vmail/*/*/Maildir -rf for i in $(doveadm user \*); do u=$(echo $i | cut -d@ -f1); d=$(echo $i |cut -d@ -f2); dsync -v -u $i mirror mdbox:/home/vmail/$d/$u/mdbox; done Then I changed the mail_location to mdbox:/home/vmail/%d/%n/mdbox and did the following commands: service dovecot restart for i in $(doveadm user \*); do u=$(echo $i | cut -d@ -f1); d=$(echo $i |cut -d@ -f2); dsync -v -u $i mirror maildir:/var/vmail/$d/$u/Mail; done doveadm index -A all INBOX for i in $(doveadm user \*); do doveadm -Dv fts rescan -u $i;done unfortunately I'm getting the following two kinds of error messages for all mail boxes in my mail.log: Error: Corrupted index cache file /home/vmail/%domain/%user/mdbox/mailboxes/%maildir/dbox-Mails/dovecot.index.cache: Broken virtual size for mail UID 2564 and: Error: read(/home/vmail/%domain/%user/mdbox/storage/m.108 ): FETCH BODY[] for mailbox %maildir UID 2564 got too little data: 2417 vs 6319 please let me know if you have any suggestions for fixing this issue or if you need more information. kind regards, Tim

6 19

Quota, doveadm, postfixadmin
by Ricardo 09 Jan '15

09 Jan '15

Hello all and happy holidays. I have Postfix+Dovecot+MySQL+Postfixadmin with CentOS 6.6, a virtual machines for testing, and then pass it to servers in production, my problem is the next: I add 30 MB for domain.local, then cast 10 MB for test1(a)domain.local and 20 MB for test2(a)domain.local, so far so good. Sending an attachment from one account to another, the example of test1(a)domain.local to test2(a)domain.local, test2(a)domain.local receive the email with the attachment. If I write in the shell doveadm quota get -u test2(a)domain.local I get the following: Quota name Type Value Limit % user STORAGE 3391 20000 16 user MESSAGE 3 10000 0 Informs me that I this consuming 16% of the 20 MB assigned to this account, but if I delete the file and again write doveadm quota get -u test2(a)domain.local I release this: Quota name Type Value Limit % user STORAGE 6782 20000 33 user MESSAGE 4 10000 0 Normal serious you continue using 16%, not 33%, when you go into the trash, because that happens from one place to another, right? These statistics the Postfixadmin Dame them so much as writing in the shell doveadm quota get -u test2(a)domain.local. I delete the file in the trash and to again write doveadm quota get -u test2(a)domain.local still see me. Quota name Type Value Limit % user STORAGE 6782 20000 33 user MESSAGE 4 10000 0 But I realize that so that it resets to 0%, I have to close the mail client, if not would follow using 33% and is when is 0% as shown here. Quota name Type Value Limit % user STORAGE 1 20000 0 user MESSAGE 2 10000 0 Is this normal? The client to send emails and exceed quota will having to always close the mail client, so you reset and stays at 0%? Can anyone help me? I have looked online but I can't find information about this. Has happened to someone? Thanks so much. Best regards.

2 1

Crashes with tracebacks
by Timothe Litt 05 Jan '15

05 Jan '15

Crashes, redux. I hope I have provided all the information required for a solution. Many thanks in advance for having a look. I have 71 core files for a user, that all happened in the space of about 6 hours. It appears that mail delivered to 'Junk E-mail' is being accessed. I suspect they're all the same issue. I saw the same syslog entry a while back; did a resync & enabled process dumps. Naturally, it went away -- until this cluster of crashes. File system is ext3. It is NFS mounted by other machines, but only the local machine should be touching the mail directories. The user does not have an interactive login - it's an e-mail only account. This user's IMAP client is AppleMail. The delivery agent is procmail; Junk is detected by spamassassin; clamav is also present. pop3 is configured, but I don't think anyone uses it. Typical syslog (time matches core.20063): Dec 17 18:06:22 overkill dovecot: imap(<user>): Error: Next message unexpectedly corrupted in mbox file /home/<user>/mail/Junk E-mail at 1021601 Dec 17 18:06:22 overkill dovecot: imap(<user>): Panic: file mbox-sync.c: line 152 (mbox_sync_read_next_mail): assertion failed: (sync_ctx->input->v_offset != mail_ctx->mail.from_offset || sync_ctx->input->eof) Dec 17 18:06:22 overkill dovecot: imap(<user>): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0 [0x274ecf] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x274f4d] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x27446b] -> /usr/local/lib/dovecot/libdovecot-storage.so.0 [0x16633e] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mbox_sync+0x6b6) [0x168146] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mbox_storage_sync_init+0x81) [0x169391] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x40) [0x186270] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x3d) [0x18715d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x74) [0x1b6c44] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x5f) [0x1895bf] -> dovecot/imap(imap_status_get+0x7a) [0x806396a] -> dovecot/imap(cmd_status+0x128) [0x80588a8] -> dovecot/imap(command_exec+0x33) [0x805c8e3] -> dovecot/imap [0x805b63f] -> dovecot/imap [0x805b6c3] -> dovecot/imap(client_handl Dec 17 18:06:22 overkill dovecot: imap(<user>): Fatal: master: service(imap): child 20063 killed with signal 6 (core dumped) Tracebacks in attached archive were generated with: > cd ~<user>; for C in core.*; do gdb /usr/local/libexec/dovecot/imap $C > -x gdb -q ; mv gdb.log gdb.$C.log; done where 'gdb' is: > set pagination off > set logging overwrite > set logging file gdb.log > set logging on > bt full > q Note that there are unprintable characters in the log files (binary arguments). I have saved, but not attached the mail file & it's .lock file before doing a doveadm force-resync -u <user> 'Junk E-mail'. Let me know if you require a copy to debug this. I have also saved the core files. Here is the dovecot config (I have replaced the domain with example.net). dovecot -n # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.22.14-72.fc6 i686 Fedora Core release 6 (Zod) first_valid_gid = 4000 first_valid_uid = 4000 hostname = smtp.example.net login_greeting = Dovecot ready. Unauthorized access is prohibited. mail_access_groups = mail mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox "Train As Forgotten" { auto = subscribe } mailbox "Train As Ham" { auto = subscribe } mailbox "Train As Spam" { auto = subscribe } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { driver = pam } service imap-login { inet_listener imap { address = imap.v4.example.net imap.v6.example.net port = 143 } inet_listener imaps { address = imap.v4.example.net imap.v6.example.net port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { address = pop.v4.example.net pop.v6.example.net port = 110 } inet_listener pop3s { address = pop.v4.example.net pop.v6.example.net port = 995 ssl = yes } } ssl_ca = </etc/pki/tls/certs/litts_net_ca-cert-crl.pem ssl_cert = </etc/pki/tls/certs/servers-litts-net.pem ssl_key = </etc/pki/tls/private/servers-litts-net.pmk userdb { driver = passwd } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags mail_plugins = } protocol pop3 { mail_plugins = pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } Finally, here is a list of all the core files showing their creation times. Note they're all the same size. ls -l core* | cut -d ' ' -f5- | sort 536576 Dec 17 18:01 core.20067 536576 Dec 17 18:06 core.20063 536576 Dec 17 18:11 core.20464 536576 Dec 17 18:16 core.20665 536576 Dec 17 18:21 core.20913 536576 Dec 17 18:26 core.21098 536576 Dec 17 18:31 core.21359 536576 Dec 17 18:36 core.21589 536576 Dec 17 18:41 core.21805 536576 Dec 17 18:46 core.22077 536576 Dec 17 18:51 core.22262 536576 Dec 17 18:56 core.22518 536576 Dec 17 19:01 core.22771 536576 Dec 17 19:06 core.22955 536576 Dec 17 19:11 core.23227 536576 Dec 17 19:16 core.23438 536576 Dec 17 19:21 core.23679 536576 Dec 17 19:26 core.23856 536576 Dec 17 19:31 core.24089 536576 Dec 17 19:36 core.24328 536576 Dec 17 19:41 core.24569 536576 Dec 17 19:46 core.24806 536576 Dec 17 19:51 core.24994 536576 Dec 17 19:56 core.25233 536576 Dec 17 20:01 core.25447 536576 Dec 17 20:06 core.25612 536576 Dec 17 20:11 core.25825 536576 Dec 17 20:16 core.26091 536576 Dec 17 20:21 core.26309 536576 Dec 17 20:26 core.26453 536576 Dec 17 20:31 core.26773 536576 Dec 17 20:36 core.27020 536576 Dec 17 20:41 core.27278 536576 Dec 17 20:46 core.27505 536576 Dec 17 20:51 core.27681 536576 Dec 17 20:56 core.27913 536576 Dec 17 21:01 core.28214 536576 Dec 17 21:06 core.28445 536576 Dec 17 21:11 core.28739 536576 Dec 17 21:16 core.28975 536576 Dec 17 21:21 core.29205 536576 Dec 17 21:26 core.29407 536576 Dec 17 21:31 core.29635 536576 Dec 17 21:36 core.29892 536576 Dec 17 21:41 core.30132 536576 Dec 17 21:46 core.30407 536576 Dec 17 21:51 core.30542 536576 Dec 17 21:56 core.30834 536576 Dec 17 22:01 core.31099 536576 Dec 17 22:06 core.31264 536576 Dec 17 22:11 core.31529 536576 Dec 17 22:16 core.31761 536576 Dec 17 22:21 core.31981 536576 Dec 17 22:26 core.32191 536576 Dec 17 22:31 core.32436 536576 Dec 17 22:36 core.32665 536576 Dec 17 22:41 core.449 536576 Dec 17 22:46 core.676 536576 Dec 17 22:51 core.877 536576 Dec 17 22:56 core.1134 536576 Dec 17 23:01 core.1332 536576 Dec 17 23:06 core.1516 536576 Dec 17 23:11 core.1757 536576 Dec 17 23:16 core.1935 536576 Dec 17 23:21 core.2175 536576 Dec 17 23:26 core.2380 536576 Dec 17 23:31 core.2625 536576 Dec 17 23:36 core.2871 536576 Dec 17 23:41 core.3141 536576 Dec 17 23:46 core.3460 536576 Dec 17 23:51 core.3631 -- Timothe Litt ACM Distinguished Engineer -------------------------- This communication may not represent the ACM or my employer's views, if any, on the matters discussed.

2 1

Designing an Object Storage Plugin for a specific platform
by Listas＠Adminlinux 02 Jan '15

02 Jan '15

Hi! I'm planning to develop a OSP (Object Storage Plugin) to support a specific platform of my company. Is there any API documentation or sample code of a simple plugin that can be used as a basis for an implementation? Thanks! -- Thiago Henrique

1 1

Postfix and Dovecot SASL - wiki update
by Dan Langille 01 Jan '15

01 Jan '15

I wanted to mention I updated the Postfix and Dovecot SASL wiki entry yesterday. http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL The entry: submission inet n - - - - smtpd was incorrect. It should be: submission inet n - n - - smtpd By specifying n for chroot, it ensures that DNS lookups can occur. Otherwise, if you are using reject_unknown_recipient_domain the following error will occur: Recipient address rejected: Domain not found Hope that helps. — Dan Langille http://langille.org/

2 2

dovecot 2 low TCP speed (fetching big mail)
by Wojciech Puchar 01 Jan '15

01 Jan '15

i use dovecot 2 under FreeBSD (dovecot2-2.2.15 compiled from ports). tried both with kqueue enabled or not. everything works very fast, EXCEPT fetching big mail. tried multiple clients (thunderbird on windows, alpine on the same server, alpine on other unix server connected by 1Gb/s LAN) and fetching peaks at 1MB/s. NO SSL! could you please point me where to search for a problem. hints: writing over IMAP (eg. writing to Sent folder) works at full speed. Doesn't seem like a problem with unix server, every other service goes at speed 70-100MB/s both directions (like samba, ftp). tried to turn off "kqueue" compile time option, no difference. Please give me a hint at least what to check. Does dovecot use other way to sent data over TCP socket than other programs? thank you.

2 1

dovecot-auth only configuration
by Dan Langille 31 Dec '14

31 Dec '14

I am configuring a dovecot-auth only installation. Postfix will use dovecot for sasl auth. Here is the configuration I’ve settled upon. I welcome suggestions for improving this setup. Thanks. # doveconf -n # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.3-RELEASE-p5 amd64 auth_mechanisms = plain login mail_max_userip_connections = 80 mail_privileged_group = mail passdb { args = scheme=SHA512-CRYPT /var/db/dovecot.users driver = passwd-file } protocols = none service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } userdb { args = /var/db/dovecot.users driver = passwd-file } verbose_proctitle = yes — Dan Langille http://langille.org/

1 0

Authenticating Virtual Users without domain
by Leon Kyneur 31 Dec '14

31 Dec '14

Hi, I'm trying to migrate a large number of users to a new Dovecot cluster. The existing mail system allows a user to authenticate with a bare username if they have connected to the correct local IP on the server. e.g. imap.somedomain.com = 1.1.1.1 imap.anotheromain.com = 2.2.2.2 charlie@somedomain can authnenticate as 'charlie' or 'charlie(a)somedomain.com' as long as he is connected to imap.somedomain.com (1.1.1.1) likewise for bare usernames if they connect to imap.anotherdomain.com. A previous colleague actually achieved this by hacking with the Dovecot source code and writing in a lookup table feature. The code is very old and won't patch cleanly to the latest 2.2.15 source. Another platform we are using (commercial product) also has this feature but we also need to migrate these users to Dovecot. I already have a Dovecot proxy layer for mailbox lookup - so ideally I would like to do this on my Dovecot proxies. I know I can also do this kind of thing if I swapped my dovecot proxy for Perdition, however I don't really want to do that. I've looked into checkpassword scripts and could possibly make something work (albeit ugly) - is this the right direction to take here? Thanks, Leon

2 1

Segmentation fault in pigeonhole lib-sieve
by Orion Poplawski 30 Dec '14

30 Dec '14

The sieve plugin for Thundirbird likes to rapidly compile work in progress sieve scripts to continually give feedback on any errors in the script. This can trigger segmentation faults in lib-sieve with certain pathologically incomplete sieve scripts. One example: #0 tag_comparator_validate (valdtr=0x7f291aa713a0, arg=0x7fff5c3cfa58, cmd=0x7f291aa69360) at sieve-comparators.c:143 143 if ( (*arg)->type != SAAT_STRING ) { Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-12.el7.x86_64 glibc-2.17-55.el7_0.3.x86_64 sssd-client-1.11.2-68.el7_0.6.x86_64 zlib-1.2.7-13.el7.x86_64 (gdb) list 138 *arg = sieve_ast_argument_next(*arg); 139 140 /* Check syntax: 141 * ":comparator" <comparator-name: string> 142 */ 143 if ( (*arg)->type != SAAT_STRING ) { 144 sieve_argument_validate_error(valdtr, *arg, 145 ":comparator tag requires one string argument, but %s was found", 146 sieve_ast_argument_name(*arg) ); 147 return FALSE; (gdb) print arg $1 = (struct sieve_ast_argument **) 0x7fff5c3cfa58 (gdb) print *arg $2 = (struct sieve_ast_argument *) 0x0 So sieve_ast_argument_next() is returning NULL and we're trying to dereference it without checking. Here's a completely naive attempt at a patch: --- dovecot-2.2.15/dovecot-2.2-pigeonhole-0.4.3/src/lib-sieve/sieve-comparators.c.null 2014-01-01 15:46:39.000000000 -0700 +++ dovecot-2.2.15/dovecot-2.2-pigeonhole-0.4.3/src/lib-sieve/sieve-comparators.c 2014-12-29 14:01:00.233436697 -0700 @@ -140,6 +140,11 @@ static bool tag_comparator_validate /* Check syntax: * ":comparator" <comparator-name: string> */ + if ( *arg == NULL ) { + sieve_argument_validate_error(valdtr, *arg, + ":comparator tag requires one string argument, but none was found"); + return FALSE; + } if ( (*arg)->type != SAAT_STRING ) { sieve_argument_validate_error(valdtr, *arg, ":comparator tag requires one string argument, but %s was found", -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane orion(a)nwra.com Boulder, CO 80301 http://www.nwra.com

2 2

PAM issues on OS X Yosemite
by Markus Mayer 30 Dec '14

30 Dec '14

Hi, I have been running dovecot successfully on OS X Mavericks for several months. After upgrading to Yosemite, however, PAM authentication for dovecot is failing. Or rather, creating the PAM session is failing. Either way, I can't get to my e-mail. $ /usr/pkg/sbin/dovecot --version 2.2.15 $ /usr/pkg/sbin/dovecot -n # 2.2.15: /usr/pkg/etc/dovecot/dovecot.conf # OS: Darwin 14.0.0 x86_64 hfs auth_debug = yes auth_verbose = yes mail_location = maildir:/Volumes/Secure/%u/Maildir mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = session=yes dovecot driver = pam } ssl_cert = </usr/pkg/etc/openssl/certs/dovecot.pem ssl_key = </usr/pkg/etc/openssl/private/dovecot.pem userdb { driver = passwd } $ defaults read "/System/Library/CoreServices/SystemVersion" ProductVersion 10.10.1 Dec 30 13:21:47 my.host.name dovecot[49247]: auth: Debug: auth client connected (pid=49289) Dec 30 13:21:51 my.host.name dovecot[49247]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=3bBdlHULNgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 lport=143 rport=52278 resp=<hidden> Dec 30 13:21:51 my.host.name dovecot[49247]: auth-worker(49286): Debug: pam(markus,::1): lookup service=dovecot Dec 30 13:21:51 my.host.name dovecot[49247]: auth-worker(49286): Debug: pam(markus,::1): #1/1 style=1 msg=Password: Dec 30 13:21:51 my.host.name dovecot[49247]: auth-worker(49286): Error: pam(markus,::1): pam_open_session() failed: session failure Dec 30 13:21:53 my.host.name dovecot[49247]: auth: Debug: client passdb out: FAIL 1 user=markus It does successfully verify my password. If I purposefully enter a wrong password the error becomes "pam_authenticate() failed: authentication error (password mismatch?)". So that portion is okay. Do you have any suggestions how I might find out why pam_open_session() is failing? The auth process *is* running as root. I have tried these two PAM configurations. The first one based on Maverick's /etc/pam.d/login and used to work fine on Mavericks. # dovecot: auth account password session auth optional pam_krb5.so use_kcminit auth optional pam_ntlm.so try_first_pass auth optional pam_mount.so try_first_pass auth required pam_opendirectory.so try_first_pass account required pam_nologin.so account required pam_opendirectory.so password required pam_opendirectory.so session required pam_launchd.so session required pam_uwtmp.so session optional pam_mount.so I tried to simplify it by using the one suggested on dovecot's PAM wiki. # dovecot: auth account password session auth required pam_opendirectory.so try_first_pass account required pam_nologin.so account required pam_opendirectory.so password required pam_opendirectory.so On Yosemite, neither works. Or, quite possibly, both configurations are fine and the problem lies elsewhere. Any pointers would be greatly appreciated. In the mean time I'll be using auth-passwdfile, since that works. Thanks, -Markus

1 0