- dovecot - dovecot.org

[Dovecot] Rebuilding indexes fails on inconsistent mdbox
by Milan Holzäpfel 27 Nov '12

27 Nov '12

Hello all, I have a problem with an incosistent mdbox: Oct 24 10:43:23 two dovecot: imap-login: Login: user=<...>, method=PLAIN, rip=..., lip=..., mpid=4977, TLS Oct 24 10:43:23 two dovecot: imap(listen(a)mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen(a)mjh.name): Error: mdbox map .../mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=638 map_uid=809891 Oct 24 10:43:23 two dovecot: imap(listen(a)mjh.name): Disconnected: Internal error occurred. Refer to server log for more information. [2012-10-24 10:43:23] bytes=115/53726 Oct 24 10:43:23 two dovecot: imap(listen(a)mjh.name): Warning: mdbox .../mdbox/storage: Inconsistency in map index (467,31960 != 467,36768) Oct 24 10:43:23 two dovecot: imap(listen(a)mjh.name): Warning: mdbox .../mdbox/storage: rebuilding indexes Oct 24 10:45:19 two dovecot: imap(listen(a)mjh.name): Panic: file mdbox-storage-rebuild.c: line 773 (rebuild_update_refcounts): assertion failed: (map_uid < msgs[i]->map_uid) Oct 24 10:45:19 two dovecot: imap(listen(a)mjh.name): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x380ca) [0x7f99cf35b0ca] -> /usr/lib/dovecot/libdovecot.so.0(+0x3810e) [0x7f99cf35b10e] -> /usr/li b/dovecot/libdovecot.so.0(i_fatal+0) [0x7f99cf334a67] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10a5) [0x7f99cf5f42d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_s torage_rebuild+0x24) [0x7f99cf5f4414] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x53565) [0x7f99cf5f4565] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7f99cf5c8caa] -> /usr/lib/dovec ot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7f99cf5c8cf3] -> dovecot/imap(client_destroy+0x109) [0x7f99cfaa69e9] -> dovecot/imap(client_input+0xaa) [0x7f99cfaa6dba] -> /usr/lib/dovecot/libdovecot.so.0(i o_loop_call_io+0x48) [0x7f99cf366c98] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f99cf367d27] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f99cf366c28] -> /usr/lib/dovecot /libdovecot.so.0(master_service_run+0x13) [0x7f99cf354e33] -> dovecot/imap(main+0x304) [0x7f99cfa9e554] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f99cef8576d] -> dovecot/imap(+0x95e5) [0 x7f99cfa9e5e5] Oct 24 10:45:19 two dovecot: master: Error: service(imap): child 4977 killed with signal 6 (core dumps disabled) I use: Dovecot 2.0.19-0ubuntu1 Ubuntu 12.04, x86-64, Kernel 3.2.0-32-generic local XFS filesystem for the mdbox The problem appeared out of nowhere. Many messages been continously delivered to this mailbox on this installation since May 2012, and the mdbox was only accessed with deliver and imap/pop3 from dovecot. About four hours after the problem initially appeared, I did a hard reset of the system because it was unresponsive. The error message is exactly the same before and after the hard reset. The problem is triggered by both IMAP access and dovecot deliver access. The whole mdbox is 6.6 GiB large and I guess that it contains about 300k-600k messages. It's an archive of public mailing lists, so I could give access to the files. Can anybody say something about this? May the mdbox be repaired? Regards, Milan Holzäpfel -- Milan Holzäpfel <listen(a)mjh.name>

5 15

[Dovecot] Dovecot IMAP/POP3 auto creating maildir
by Dominic Malolepszy 27 Nov '12

27 Nov '12

Hi I have observed that after deleting a mailbox and removing the user from the userdb, immediately accessing the mail account via POP3/IMAP causes Dovecot to auto create an empty mailbox, because the userdb/passdb details are still cached for 10minutes. Is there any option to tell Dovecot POP3/IMAP not to auto create the mailbox if it does not exist? Alternatively and less desirably is there a mechanism for telling Dovecot to expire a userdb/passdb cached entry? Running Dovecot 2.1.9 Dominic

2 1

[Dovecot] memory allocation issues
by Kelsey Cummings 27 Nov '12

27 Nov '12

One of our dovecot backend servers ran into a problem with it's auth process a few days ago. This doesn't appear to be the error logged when dovecot hits its internal limit so I'm not sure what is going on here. auth: Error: malloc: 58012: Cannot allocate memory auth: Error: Unable to allocate memory for mutexes from the region auth: Error: PANIC: Cannot allocate memory auth: passwd(test,1.1.1.1,<8HTlNHzNIQBAjhKC>): unknown user pop3: Error: Authenticated user not found from userdb, auth lookup +id=2509111297 (client-pid=4781 client-id=1) pop3-login: Internal login failure (pid=4781 id=1) (internal failure, 1 +succesful auths): user=<test>... There was at least 10+GB free RAM on the server and no indication of a system level issue at the same time. The server is running 2.1.9. There were about 3,200 active sessions, with something like 12 new sessions/sec. The other identical servers are/were handling virtually identical load with the same service uptime and haven't had any issues so far. (Crash happened 7 days ago.) -- Kelsey Cummings - kgc(a)corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407

2 4

[Dovecot] fs_flush_file_handle_cache_dir: rmdir(/var/mail) AGAIN
by Ken Mandelberg 27 Nov '12

27 Nov '12

We are running Dovecot 2.0.12. It runs on only one server and /var/mail is local where all the INBOX's are stored. Users other IMAP folders are NFS mounted on the server. We see frequent Error: nfs_flush_file_handle_cache_dir: rmdir(/var/mail) failed: Device busy I know this error message has been around a long time. Should we still be seeing it on 2.0.12 with a local /var/mail. Perhaps the message is misleading and really complaining about folders other than INBOX that are NFS mounted. Our config has dovecot.conf:mail_nfs_index = no dovecot.conf:mail_nfs_storage = yes is that correct given the above? Also, does the flush error messages have any real consequences? My understanding is that the rmdir is meant to fail, just to force a flush.

2 1

[Dovecot] Filesystem quotas
by Andreas Kasenides 27 Nov '12

27 Nov '12

I could not determine if when using filesystem quotas (http://wiki2.dovecot.org/Quota/FS) Dovecot will use the reported quota and limits without the need of configuring them via "quota_rules". Is this possible somehow? Such a feature would determine the quota limits automatically from the filesystem quotas thereby allowing to make use of them in quota warnings. Dovecot already knows the quota settings and limits. Here is evidence from the logs > Nov 21 19:24:22 iolaos dovecot: imap(user): Debug: box=/Mail/ank > mount=/newmail match=yes > Nov 21 19:24:22 iolaos dovecot: imap(user): Debug: quota-fs: > host=xxxx.xxxx.ucy.xx.cy, path=/mail, uid=211, bytes > Nov 21 19:24:22 iolaos dovecot: imap(user): Debug: quota-fs: uid=211, > value=2779115520, limit=8388608000 I am using (testing) version 2.2alpha1. Thankx. -- Andreas Kasenides Senior IT Officer University of Cyprus, Dept. of Computer Science Tel: 22892714, Fax: 22892701

2 1

[Dovecot] Plugin help, number of messages in mailbox
by Richard Platel 27 Nov '12

27 Nov '12

Hi, We use Dovecot for IMAP and POP (but not LDA), we want to do something when a user has an INBOX that becomes empty, or becomes not empty (set a flag in memcached, but that's not really important). I'm writing a plugin (for Dovecot 2.1.7). On mailbox_open() I can use mailbox_get_status() to get a count of messages in the mailbox, and then decrement this in expunge() or increment it in mailbox save_finish() (for IMAP APPEND or COPY commands). However in expunge() and mailbox_save_finish, even after calling the super function, mailbox_get_status doesn't update the number of messages in the mailbox. This is a problem if (for example) there are concurrent POP sessions. Two POP sessions could get all the messages in INBOX, one could logout, calling expunge a few times, eventually causing my plugin to note that the inbox is empty, then our LDA could deliver a message, mark the INBOX not empty, then the other POP session could log out, call expunge and cause my plugin to mark the INBOX empty, when it's not. So in summation: how can a plugin be notified of changes to a mailbox, and then accurately get the real number of messages in that mailbox?

2 1

[Dovecot] Folder layout after Cyrus to Dovecot migration
by Keith Edmunds 27 Nov '12

27 Nov '12

# dovecot --version 1.2.15 We're trying to migrate a server from Cyrus to Dovecot, and we're mostly there. On Cyrus, the folder layout in the MUA is: INBOX subfolder subfolder ... Sent Drafts Queue Trash Users cannot create sibling folders to the Inbox, only subfolders (and sub-subfolders, etc). We've replicated that with: -------------------------------------------------------------------------------- mail_location = maildir:/shared/imapmail/users/%u:INBOX=/shared/imapmail/users/%u:LAYOUT=fs namespace private { separator = / prefix = "INBOX/" inbox = yes } -------------------------------------------------------------------------------- ...however, the Sent/Drafts/Queue/Trash folders are now subfolders of Inbox. Cyrus seems to default to the above layout (at least, I can't find anywhere that defines it). There are no special settings in the MUA to achieve the abovev. Is there a way of replicating the above within Dovecot? We can probably go to V2 if we have to, but 1.2.15 would be preferable. Thanks, Keith

3 5

[Dovecot] cannot update mailbox - unable to lock for exclusive access
by 1st WebDesigns 27 Nov '12

27 Nov '12

Hi there, We're using Dovecot version 1.0.7 and Postfix, and since upgrading our Linux box we're getting this in the maillog: Nov 8 07:49:11 server1 postfix/local[27901]: 04B8E7081DA: to=<xyz(a)xyz.com>, orig_to=<abc(a)abc.com>, relay=local, delay=19, delays=0.07/0/0/19, dsn=4.2.0, status=deferred (cannot update mailbox /var/spool/mail/abc for user abc. unable to lock for exclusive access: Resource temporarily unavailable) Postfix is currently set to: mailbox_delivery_lock = fcntl, dotlock Dovecot has this: mbox_read_locks = fcntl mbox_write_locks = fcntl dotlock I've scoured the web and tried all kinds of different locking mechanisms and combinations to no avail. The mail is eventually delivered but on a busy day this can take several hours. In the evening it generally doesn't happen, which leads me to believe it occurs during the day when people have their mail clients open. However, this particular issue doesn't affect every user that has their mail client open, only some! We currently have the mailboxes in mbox format - yes I know we should upgrade to maildir which we will eventually, however we've been using mbox for the last 10 years and this has only become an issue since upgrading. What I have discovered this morning is a Dovecot connection that is open for 6 minutes before disconnection. During this 6 minutes the above problem occurs (new mail cannot get an exclusive lock). The same user will have connected and disconnected several times before and after, with a connection taking no more than a few seconds. But it seems sometimes the connection is taking longer than it should and I suspect the problem lies here. Please can anyone help! Thank you, G

3 15

[Dovecot] sieve + dkim
by Николай Клименко 26 Nov '12

26 Nov '12

message sent from the sieve-vacation does not contain dkim signature how to configure sieve to send messages via smtp transport with signing?

3 4

[Dovecot] IMAP proxy - can it detect parodying to itself?
by Graham Leggett 26 Nov '12

26 Nov '12

Hi all, I have some IMAP servers fronted with separate perdition processes, and it would be ideal if I could collapse this down to having dovecot do both the IMAP proxying and the IMAP serving at the same time on the same IP addresses. One of the fields in my LDAP entries contains the canonical name of the server that hosts their mailbox, and if I follow the manual at http://wiki2.dovecot.org/PasswordDatabase/ExtraFields#LDAP I could add the host field to enable proxying. My question is whether dovecot has the ability to notice whether dovecot is being asked to proxy to itself, in other words the value of "host" is the current dovecot server, and when this happens, ignore the proxy and just be a straight IMAP server, because the user has connected to the right box already. Is this possible? Regards, Graham --

2 2