July 2008 - dovecot - dovecot.org

[Dovecot] tcp-wrappers.patch
by Edgar Fuß 09 Aug '08

09 Aug '08

I'd like to see tcp-wrappers.patch getting integrated into dovecot. I ported the original 1.0 patch to 1.1, but would prefer not to have to maintain another local patch. As the name suggests, the patch adds libwrap support to dovecot. We use is to limit access from outside our network to secure (imaps/pop3s) protocols only and to exclude certain internal addresses from accessing dovecot in general. I fully understand Timo's concern of people not reading documentation and then whining that librwapping doesn't work whereas they simply forgot to put hosts.{allow,deny} into the login chroot. Would it be acceptable if either dovecot itself or the init script copies /etc/hosts.{allow,deny} into the chroot (unless it's already there)? Also, dovecot could probably complain/abort/turn off libwrap if these files are missing in the login chroot. The problem mainly occurs with the default (Linux) setting of login_dir = /var/run/dovecot/login; on NetBSD, I use /var/chroot/dovecot-login instead, which doesn't get wiped out at boot. Comments? Suggestions? Oh, yes, managesieve (which uses src/login-common) would need a slight update, too.

3 5

[Dovecot] Dovecot 1.1.1 killed with SIGABRT
by Andreas M. Kirchwitz 08 Aug '08

08 Aug '08

Hi folks, with Dovecot 1.1.1, I've had two "crashes" (SIGABRT) recently. Dovecot 1.1.1 has been compiled from source and is running on Fedora 8 Linux 32 Bit (all patches) with custom OpenSSL 0.9.8h. [ output of "dovecot -n" ] # 1.1.1: /usr/local/dovecot/etc/dovecot.conf ssl_cert_file: /usr/local/dovecot/etc/dovecot.crt ssl_key_file: /usr/local/dovecot/etc/dovecot.key login_dir: /usr/local/dovecot/var/run/dovecot/login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login mail_location: mbox:~/Mail:INBOX=/var/spool/mail/%u auth default: mechanisms: plain login digest-md5 cram-md5 passdb: driver: passwd-file args: /usr/local/dovecot/etc/dovecot.passwd userdb: driver: passwd-file args: /usr/local/dovecot/etc/dovecot.passwd First, the "small" problem with Pine 4.64: Jul 20 03:17:04 linux dovecot: imap-login: Login: user=<user1>, method=CRAM-MD5, rip=138.201.2.8, lip=138.201.2.2, TLS Jul 20 03:17:53 linux dovecot: IMAP(user1): UIDVALIDITY changed (1215137127 -> 1216072777) in mbox file /home/user1/Mail/Trash Jul 20 03:17:53 linux dovecot: Panic: IMAP(user1): file mail-index-transaction.c: line 548 (mail_index_transaction_get_next_uid): assertion failed: (recs[count-1].uid >= next_uid) Jul 20 03:17:53 linux dovecot: IMAP(user1): Raw backtrace: imap [0x80cf2f0] -> imap [0x80cf34a] -> imap [0x80cec5c] -> imap [0x80ad621] -> imap [0x80ae56b] -> imap(mail_cache_get_first_new_seq+0x12) [0x80a1d82] -> imap(mail_cache_field_want_add+0x8d) [0x80a555d] -> imap(index_mail_parse_header_init+0x1cc) [0x80964ac] -> imap(index_mail_cache_parse_init+0x4e) [0x809696e] -> imap(mbox_save_init+0x457) [0x807d4a7] -> imap(mailbox_save_init+0x51) [0x809fd71] -> imap(mail_storage_copy+0xc1) [0x809df31] -> imap(cmd_copy+0x1d2) [0x805b1f2] -> imap(cmd_uid+0x59) [0x805f489] -> imap [0x805fe2c] -> imap [0x805fed5] -> imap [0x8060695] -> imap(client_input+0x5e) [0x80608ae] -> imap(io_loop_handler_run+0x100) [0x80d6c30] -> imap(io_loop_run+0x28) [0x80d5dc8] -> imap(main+0x4a1) [0x80683d1] -> /lib/libc.so.6(__libc_start_main+0xe0) [0x149390] -> imap [0x805a211] Jul 20 03:17:53 linux dovecot: child 19853 (imap) killed with signal 6 The user didn't configure his Pine properly. He used IMAP to access his INBOX, but he forgot to configure this for all other folders as well. Now, Pine modified /home/user1/Mail/Trash directly, and when Dovecot stumbled upon this file it killed itself with signal 6. The problem is not reproducible. After this single failure, Dovecot doesn't seem to have any problems with the folder. However, instead of this panic operation, Dovecot maybe could issue a warning or error response to the client so that the user (hopefully) gets a clue about what's going wrong. Second, the "big" problem with Thunderbird 2.0.0.14: Jul 12 01:04:06 linux dovecot: imap-login: Login: user=<user2>, method=CRAM-MD5, rip=138.201.2.4, lip=138.201.2.2, TLS Jul 12 01:04:45 linux dovecot: Panic: IMAP(user2): file index-sync.c: line 39 (index_mailbox_set_recent_uid): assertion failed: (seq_range_exists(&ibox->recent_flags, uid)) Jul 12 01:04:45 linux dovecot: IMAP(user2): Raw backtrace: imap [0x80cf2f0] -> imap [0x80cf34a] -> imap [0x80cec5c] -> imap [0x809ccba] -> imap [0x8080d10] -> imap(mbox_sync+0x522) [0x80817d2] -> imap [0x807a334] -> imap(index_transaction_commit+0x4e) [0x809d99e] -> imap(cmd_copy+0x35f) [0x805b37f] -> imap(cmd_uid+0x59) [0x805f489] -> imap [0x805fe2c] -> imap [0x805fed5] -> imap [0x8060695] -> imap(client_input+0x5e) [0x80608ae] -> imap(io_loop_handler_run+0x100) [0x80d6c30] -> imap(io_loop_run+0x28) [0x80d5dc8] -> imap(main+0x4a1) [0x80683d1] -> /lib/libc.so.6(__libc_start_main+0xe0) [0x149390] -> imap [0x805a211] Jul 12 01:04:45 linux dovecot: child 25672 (imap) killed with signal 6 [happens over and over in an endless loop until Thunderbird is killed] This has been a nasty one. Different user, different mail client. INBOX with 350 messages in total, 3 of them new. Started Thunderbird, index page is displayed fine, then clicked on one of the 3 new messages to read it. For no apparent reason, Dovecot dies with signal 6. Thunderbird reconnected automatically, and then Dovecot dies again. Nice endless loop ... Stopping and restarting the complete Dovecot subsystem with the help of the init.d script didn't help. Finally the crashes stopped after removing the cache files in /home/fubar/Mail/.imap/INBOX. It's obvious that the cache files got corrupted somehow. There was plenty of disk space free. And the cache files have been freshly created from scratch with Dovecot 1.1.0. Other folders were not affected. Except Dovecot, nobody ever touches these cache files. Well, the good news is, it stopped crashing after removing the cache files. The bad news is, without the cache files (which I removed) the problem cannot be reproduced. So, there's nothing more to offer than these two backtraces. Both crashes are not related to each other and happened to different users with different clients. Maybe the backtraces and my descriptions help to find what's going wrong or how to make Dovecot behave more robustly. If there's any additional information needed, please let me know. Sorry that I didn't save the corrupted cache files. Greetings, Andreas

3 6

[Dovecot] 1.1.1 (maildir_uidlist_sync_update): assertion failed: (uidlist->initial_hdr_read)
by Adam McDougall 06 Aug '08

06 Aug '08

Version: 1.1.1 OS: FreeBSD 7.0-STABLE amd64 I have only seen this happen twice so far. I do not know what the two users were doing to cause it. Jun 26 17:23:26 hill dovecot: IMAP(userA): Invalid transaction log size (32812 vs 32920): /home/userA/Maildir/dovecot/private/indexes/.Deleted Messages/dovecot.index.log Jun 26 17:23:26 hill dovecot: IMAP(userA): Invalid transaction log size (32812 vs 32920): /home/userA/Maildir/dovecot/private/indexes/.Deleted Messages/dovecot.index.log Jun 26 17:23:26 hill dovecot: Panic: IMAP(userA): file maildir-uidlist.c: line 1207 (maildir_uidlist_sync_update): assertion failed: (uidlist->initial_hdr_read) Jun 26 17:23:26 hill dovecot: child 28120 (imap) killed with signal 6 Jul 11 13:41:46 hill dovecot: IMAP(userB): Invalid transaction log size (92984 vs 93076): /home/userB/Maildir/dovecot/private/indexes/.Old Backup.Personal/dovecot.index.log Jul 11 13:41:46 hill dovecot: IMAP(userB): Invalid transaction log size (92984 vs 93076): /home/userB/Maildir/dovecot/private/indexes/.Old Backup.Personal/dovecot.index.log Jul 11 13:41:46 hill dovecot: Panic: IMAP(userB): file maildir-uidlist.c: line 1207 (maildir_uidlist_sync_update): assertion failed: (uidlist->initial_hdr_read) Jul 11 13:41:46 hill kernel: pid 85434 (imap), uid 20128: exited on signal 6 (core dumped) hill# gdb /usr/local/libexec/dovecot/imap imap.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `imap'. Program terminated with signal 6, Aborted. Reading symbols from /usr/local/lib/libiconv.so.3...done. Loaded symbols for /usr/local/lib/libiconv.so.3 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/dovecot/imap/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/lib/dovecot/imap/lib01_acl_plugin.so Reading symbols from /usr/local/lib/dovecot/imap/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/lib/dovecot/imap/lib10_quota_plugin.so Reading symbols from /usr/lib/librpcsvc.so.4...done. Loaded symbols for /usr/lib/librpcsvc.so.4 Reading symbols from /usr/local/lib/dovecot/imap/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/local/lib/dovecot/imap/lib11_imap_quota_plugin.so Reading symbols from /usr/local/lib/dovecot/imap/lib20_fts_plugin.so...done. Loaded symbols for /usr/local/lib/dovecot/imap/lib20_fts_plugin.so Reading symbols from /usr/local/lib/dovecot/imap/lib20_listescape_plugin.so...done. Loaded symbols for /usr/local/lib/dovecot/imap/lib20_listescape_plugin.so Reading symbols from /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/local/lib/dovecot/imap/lib21_fts_squat_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000409d7e7c in kill () from /lib/libc.so.7 (gdb) bt full #0 0x00000000409d7e7c in kill () from /lib/libc.so.7 No symbol table info available. #1 0x00000000409d6ceb in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000004b5385 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:152 backtrace = 0x3 <Address 0x3 out of bounds> #3 0x00000000004b610e in i_internal_fatal_handler (type=Could not find the frame base for "i_internal_fatal_handler". ) at failures.c:418 No locals. #4 0x00000000004b564a in i_panic (format=Could not find the frame base for "i_panic". ) at failures.c:193 args = Could not find the frame base for "i_panic". (gdb) hill# file /usr/local/libexec/dovecot/imap /usr/local/libexec/dovecot/imap: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), for FreeBSD 7.0 (700101), dynamically linked (uses shared libs), FreeBSD-style, not stripped

2 2

[Dovecot] Expire plugin with Mysql
by Eric 05 Aug '08

05 Aug '08

Hey guys, got a question about the expire plugin. According to the wiki, the expire plugin "keeps an internal database (Berkeley DB or SQL)". I was wondering what fields in the mysql table are needed for expire to work. I assume it is setup is similar to the quota dict setup, so was just wondering what fields are required (i.e: select_field, where_field, username_field, for quota). Thanks

5 10

[Dovecot] FTS/squat search indexes built when?
by Jay Levitt 05 Aug '08

05 Aug '08

I know that a given search index doesn't get built the first time until you do an actual search on it from an IMAP client. I don't much care about that; I'm the only user, and I don't create new folders too often. At worst, I could create a script to go search every folder once. But what I'm not clear about: Does it then get updated on every delivery by LDA? Or does the index also need to get updated when I do a search, assuming there's been at least one message delivered since the last search? Jay Levitt

5 11

[Dovecot] v1.1.2 released
by Timo Sirainen 04 Aug '08

04 Aug '08

http://dovecot.org/releases/1.1/dovecot-1.1.2.tar.gz http://dovecot.org/releases/1.1/dovecot-1.1.2.tar.gz.sig Development of new features in this release and the upcoming multi-master replication are sponsored by Directi (www.directi.com). Lucene indexing is currently deprecated in favor of the new Solr indexing. I'm even considering removing the Lucene C++ library support, so if you're interested in keeping it send me a mail. Note that the current fts-lucene is somewhat broken as well as non-optimal. Anyone interested in using zlib plugin with Maildir should read http://wiki.dovecot.org/Plugins/Zlib. I'm hoping to get the first v1.2 betas out in a couple of weeks and an eventual v1.2.0 release in a couple of months. + Added full text search indexing support for Apache Lucene Solr server: http://wiki.dovecot.org/Plugins/FTS/Solr + IMAP SORT: Added X-SCORE sort key for use with Solr searches. + zlib plugin supports now bzip2 also. + quota: All backends now take noenforcing parameter. + Maildir: Add ,S=<size> to maildir filename whenever quota plugin is loaded, even when not using Maildir++ quota. + deliver: Allow lda section to override plugin settings. + deliver: Giving a -m <namespace prefix> parameter now silently saves the mail to INBOX. This is useful for e.g. -m INBOX/${extension} + Added a new maildirlock utility for write-locking Dovecot Maildir. + dict-sql: Support non-MySQL databases by assuming they implement the "INSERT .. ON DUPLICATE KEY" using an INSERT trigger. - SORT: Fixed several crashes/errors with sort indexing. - IMAP: BODYSTRUCTURE is finally RFC 3501 compliant. Earlier versions didn't include Content-Location support. - IMAP: Fixed bugs with listing INBOX. - Maildir: maildirfolder file wasn't created when dovecot-shared file existed on the root directory - deliver didn't expand %variables in namespace location settings. - zlib: Copying non-compressed messages resulted in empty mails (except when hardlink-copying between maildirs). - mbox-snarf plugin was somewhat broken - deliver + Maildir: If uidlist couldn't be locked while saving, we might have assert-crashed - mbox: Fixed an assert-crash with \Recent flag handling

5 8

[Dovecot] bugzilla or other similar bug tracking systems
by Diego Liziero 04 Aug '08

04 Aug '08

I was wondering if it could be useful to use such tools to keep track of users bugs. I find somehow harder to search the mailing list if a bug is known, if it's being worked on, if it needs more feedback, in witch release it has been eventually solved, and so on. Any thoughts? Regards, Diego.

3 3

Re: [Dovecot] how to store flags \Seen into read-only mailbox?
by seriv＠parkheights.dyndns.org 04 Aug '08

04 Aug '08

This problem is apparently about changeset http://hg.dovecot.org/dovecot-1.1/rev/eec74a6ad62e. This change makes dovecot behaviour different from what is described in wiki, about sharing and making private /Seen flags, see http://wiki.dovecot.org/SharedMailboxes?highlight=(flags are shared or private) When I've tried to revert it, first time with my client it worked perfectly. But then clients were not able to open any mail folders, and imap processes were dying. -- Sergey Ivanov. ----- seriv(a)parkheights.dyndns.org wrote: > Hi, > I'm trying to use dovecot for storing mailing lists and read-only > access for users. > The OS is Solaris, authentication for all readers of these lists are > from Ldap through PAM, and one local user "listuser" will receive all > mail and store them into it's folders (Maildirs). These maildirs are > readable (read-only) for all others and they are shared by setting > --- > mail_location: > maildir:/home/vmail/listuser/Maildir:CONTROL=/home/vmail/%u/Maildir/control:INDEX=/home/vmail/%u/Maildir/index > --- > Mail became readable because in each mail folder to be shared there is > dovecot-shared with mode 064, and all users included into his group. > > But this is the problem: how to have flag \Seen stored in read-only > mailbox? As recorded by rawlog, it is ignored now: > --- > bash-3.00# cat 20080724-103219-10616.in > 3 namespace > 4 lsub "" "*" > 5 list "" "INBOX" > 6 list "" "Trash" > 7 create "Trash" > 8 select "OmniTI.FreeLotto.Ops" > 9 UID fetch 1:* (FLAGS) > 10 uid store 1:3 +Flags (\Seen) > 11 IDLE > DONE > 12 close > 13 logout > bash-3.00# cat 20080724-103219-10616.out > * OK [RAWLOG TIMESTAMP] 2008-07-24 10:32:19 > * NAMESPACE (("" ".")) NIL NIL > 3 OK Namespace completed. > * LSUB () "." "OmniTI.FreeLotto.ODS" > * LSUB () "." "OmniTI.FreeLotto.Ops" > 4 OK Lsub completed. > * LIST (\HasNoChildren) "." "INBOX" > 5 OK List completed. > 6 OK List completed. > 7 NO Internal error occurred. Refer to server log for more > information. [2008-07-24 10:32:19] > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) > * OK [PERMANENTFLAGS ()] Read-only mailbox. > * 3 EXISTS > * 0 RECENT > * OK [UNSEEN 1] First unseen. > * OK [UIDVALIDITY 1216906857] UIDs valid > * OK [UIDNEXT 4] Predicted next UID > 8 OK [READ-ONLY] Select completed. > * 1 FETCH (UID 1 FLAGS ()) > * 2 FETCH (UID 2 FLAGS ()) > * 3 FETCH (UID 3 FLAGS ()) > 9 OK Fetch completed. > 10 OK Store ignored with read-only mailbox. > + idling > 11 OK Idle completed. > 12 OK Close completed. > * BYE Logging out > 13 OK Logout completed. > --- > > -- > Sergey ivanov.

2 1

[Dovecot] plus addressing bug in dovecot 1.1.x?
by Jon Fullmer 04 Aug '08

04 Aug '08

I had recently upgraded from Dovecot 1.0.14 to 1.1.0, and then to 1.1.1. Ever since the upgrade to 1.1.0, I noticed that "plus addressing" has failed. In case I'm using a foreign phrase, this is the concept of sending an e-mail to "user+spam(a)domain.com", and having dovecot (dovecot LDA, specifically), deliver this to the "spam" submailbox to "user(a)domain.com". I've been using this successfully since 1.0.3. When I upgraded to 1.1.0, I noticed that e-mails were no longer being delivered to the submailboxes as they had been (though they would arrive in the central INBOX). I checked the "dovecot-deliver.log", and I noticed that, ever since I had brought up 1.1.0, I would see: save failed to spam: Unknown namespace ("spam" is the submailbox) After the upgrade to 1.1.1, I still would see these messages (and behavior). I have since installed 1.0.15. The plus addressing now works again, as it had been. With all of these "upgrades" and "downgrades", the config file has remained exactly the same. There have been no other changes to any components on the host system (i.e., new OS-es, new postfix, etc.). Thoughts? Is this a known issue? - Jon

4 7

[Dovecot] nfs_flush errors
by Hans Wunsch 04 Aug '08

04 Aug '08

I'm looking for some help with an nfs error that is filling our logs. I get a constant stream of these messages: Jul 8 10:47:08 servername dovecot: [ID 107833 mail.error] IMAP(username): nfs_flush_file_handle_cache_dir: rmdir(/var/mail) failed: Device busy I have two dovecot IMAP servers. All mailboxes are NFS mounted. The INBOX is /var/mail and the rest are in /home/<user>/mail. It looks like work was done on this back in January: http://www.dovecot.org/list/dovecot-cvs/2008-January/010638.html And again in February: http://hg.dovecot.org/dovecot-1.1/rev/258477a3f1e7 Was a there another fix that I missed or is anybody else is still seeing the same issue? Thanks, Hans

3 4