December 2011 - dovecot

[Dovecot] compressed mboxes very slow
by kjonca＠o2.pl 29 Jan '12

29 Jan '12

I have some archive mails in gzipped mboxes. I could use them with dovecot 1.x without problems. But recently I have installed dovecot 2.0.12, and they are slow. very slow. Creating index files takes about 10 minutes for ~20M file with 560 messages for bzipped mbox, for gzipped is little better but still unusable :( Stracing dovecot process shows that every ~ 20 messages it rereads complete mbox file. Am I doing something wrong? KJ -- http://modnebzdury.wordpress.com/2009/10/01/niewiarygodny-list-prof-majewsk…

4 13

[Dovecot] v2.x services documentation
by Timo Sirainen 18 Jan '12

18 Jan '12

If you've ever wanted to know everything about the service {} blocks, this should be quite helpful: http://wiki2.dovecot.org/Services

3 4

[Dovecot] Dovecot crashes totally
by Gordon Grubert 18 Jan '12

18 Jan '12

Hello, our dovecot server crashes totally without any really useful log messages. The error log can be found in the attachment. The only way to get dovecot running again is a complete system restart. Dovecot version: 2:2.0.15-0~auto+5 (2.0.15 (6b7242ead6ed)) Configuration : see attachment OS : Debian Squeeze amd64 Dovecot source : http://xi.rename-it.nl/debian/ \ stable-auto/dovecot-2.0 main This problem has already occurred with the version 2.0.13 where the log says as few as the current logs :-( Best regards, Gordon -- Leiter AG Technische Infrastruktur und Basisdienste Universitaetsrechenzentrum (URZ) E.-M.-Arndt-Universitaet Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86-1456 Fax. +49 3834 86-1401

3 3

[Dovecot] dsync panic
by Jürgen Obermann 10 Jan '12

10 Jan '12

Hello, when I try to convert from mbox to mdbox with dsync with one user it always panics: # /opt/local/bin/dsync -v -u userxy backup ssh root@minerva1 /opt/local/bin/dsync -v -u userxy dsync-local(userxy): Info: Papierkorb: uidnext changed: 161 != 1 dsync-local(userxy): Info: INBOX: uidnext changed: 9139 != 9105 dsync-local(userxy): Info: Trash: uidnext changed: 3924 != 3915 dsync-local(userxy): Info: Drafts: uidnext changed: 1563 != 1 dsync-local(userxy): Info: Gesendet: uidnext changed: 369 != 1 dsync-local(userxy): Info: Sent: uidnext changed: 1955 != 1 dsync-remote(userxy): Panic: Trying to allocate 2147483648 bytes dsync-remote(userxy): Error: Raw backtrace: 0xff14bc30 -> 0xff1658c0 -> 0xff15744c -> 0xff157520 -> 0xff159d90 -> 0xff156540 -> 0xff15cf5c -> 0xff156540 -> 0xff157100 -> 0xff16d158 -> 0xff16cfe8 -> 0xff26e350 -> 0x251c4 -> 0x2547c -> 0x21110 -> 0x20494 -> 0x1f230 -> 0x1f3d4 -> 0xff15e46c -> 0xff15f738 -> 0xff15e530 -> 0xff140ec8 -> 0x1525c -> 0x14608 Abort dsync-local(userxy): Error: read() from worker server failed: EOF dsync-local(userxy): Error: Unexpected reply from server: My local configuration is: # /opt/local/bin/doveconf -n # 2.0.16: /opt/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4v auth_verbose = yes disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = imap.hrz.uni-giessen.de localhost mail_fsync = always mail_location = mbox:~/Mail:INBOX=/var/mail/%u mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = mail_log notify zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { inbox = yes location = prefix = separator = / type = private } namespace { hidden = yes list = no location = prefix = Mail/ separator = / subscriptions = yes type = private } passdb { driver = pam } passdb { args = /opt/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Trash autocreate2 = caughtspam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = caughtspam autosubscribe3 = Sent autosubscribe4 = Drafts mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster(a)hrz.uni-giessen.de quota_full_tempfail = yes sendmail_path = /usr/lib/sendmail service auth { client_limit = 11120 } service imap-login { process_min_avail = 16 service_count = 0 vsz_limit = 640 M } service imap { process_limit = 4096 vsz_limit = 1 G } ssl_cert = </etc/ssl/imap_hrz.crt ssl_key = </etc/ssl/imap_hrz.key syslog_facility = local0 userdb { driver = passwd } version_ignore = yes protocol lda { info_log_path = log_path = mail_plugins = mail_log notify zlib sieve } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags mail_max_userip_connections = 20 mail_plugins = mail_log notify zlib autocreate imap_zlib } protocol lmtp { mail_plugins = mail_log notify zlib sieve } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xv%08Xu } On the remote side the configuration is # 2.0.16: /opt/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4u zfs auth_debug = yes auth_debug_passwords = yes auth_verbose = yes disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = minerva1.hrz.uni-giessen.de mail_debug = yes mail_fsync = always mail_location = mdbox:/sanpool/mail%h/mdbox mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = mail_log notify quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 days mdbox_rotate_size = 16 M mmap_disable = yes namespace { inbox = yes location = prefix = separator = / type = private } namespace { hidden = yes list = no location = prefix = Mail/ separator = / subscriptions = yes type = private } namespace { location = maildir:/var/mail/public:CONTROL=/sanpool/mail/%u/Maildir/public:INDEX=/sanpool/mail/%u/Maildir/public prefix = Public/ separator = / subscriptions = no type = public } passdb { driver = pam } passdb { args = /opt/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Trash autocreate2 = caughtspam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = caughtspam autosubscribe3 = Sent autosubscribe4 = Drafts fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = fs:Home-Verzeichnis:noenforcing zlib_save = gz zlib_save_level = 3 } postmaster_address = postmaster(a)hrz.uni-giessen.de quota_full_tempfail = yes sendmail_path = /usr/lib/sendmail service auth { client_limit = 11120 } service imap-login { process_min_avail = 16 service_count = 0 vsz_limit = 640 M } service imap { process_limit = 4096 vsz_limit = 1 G } ssl_cert = </etc/ssl/server.crt ssl_key = </etc/ssl/server.key syslog_facility = local0 userdb { driver = passwd } version_ignore = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags mail_max_userip_connections = 20 mail_plugins = mail_log notify quota zlib autocreate fts fts_squat imap_quota imap_zlib } protocol lda { info_log_path = log_path = mail_plugins = mail_log notify quota zlib sieve } protocol lmtp { mail_plugins = mail_log notify quota zlib sieve } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xv%08Xu } Thank you, Jürgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universität Gießen Heinrich-Buff-Ring 44 Tel. 0641-9913054

2 2

[Dovecot] Building 2.1.rc1 with cluence, but without libstemmer
by Jan Mikkelsen 08 Jan '12

08 Jan '12

Hi, I'm testing out 2.1.rc1, building against clucene 2.3.3.4. Without libstemmber, the build fails because it attempts to include SnowballAnalyzer.h which is not part of "core" clucene. Simple patch below to make dovecot 2.1.rc1 build with clucene but without libstemmer (or textcat). Looking at libstemmer itself, there are multiple versions. The version in clucene is put into libclucene-contribs-lib.so, but that version is older than libstemmer_c, which generates libstemmer.a, tested for in the dovecot 2.1.rc1 configure script. The dovecot code uses SnowballAnalyzer.h as a C++ wrapper around libstemmer. What versions of libstemmer, clucene and libtextcat is dovecot 2.1.rc1 intended to be used against? Is it worth bringing the wrapper class around libstemmer into dovecot to avoid funny dependencies against clucene and so that the -lstemmer test in configure is enough? For reference, I'm building on FreeBSD 9.0-RC3, system gcc. Regards, Jan Mikkelsen. diff -u -d -r dovecot-2.1.rc1/src/plugins/fts-lucene/lucene-wrapper.cc dovecot-2.1.rc1.patched/src/plugins/fts-lucene/lucene-wrapper.cc --- dovecot-2.1.rc1/src/plugins/fts-lucene/lucene-wrapper.cc 2011-11-09 08:45:26.000000000 +1100 +++ dovecot-2.1.rc1.patched/src/plugins/fts-lucene/lucene-wrapper.cc 2011-12-14 23:36:54.319796535 +1100 @@ -23,7 +23,9 @@ #include <CLucene.h> #include <CLucene/util/CLStreams.h> #include <CLucene/search/MultiPhraseQuery.h> +#if defined(HAVE_LUCENE_STEMMER) || defined(HAVE_LUCENE_TEXTCAT) #include "SnowballAnalyzer.h" +#endif /* Lucene's default is 10000. Use it here also.. */ #define MAX_TERMS_PER_DOCUMENT 10000

3 2

[Dovecot] doveadm + dsync merging
by Timo Sirainen 06 Jan '12

06 Jan '12

doveadm already supports some nice things, such as being able to remotely launch a doveadm command via TCP socket. It also supports executing a command for all users or to some specific users using a wildcard. dsync could use these features, so I merged dsync and doveadm into same binary for v2.1. I'll still install "dsync" symlink pointing to "doveadm", and running that way it should be fully backwards compatible with the old dsync binary and its parameters. I'm mainly now wondering about the command naming for running dsync via doveadm. Any suggestions? a) Use "doveadm dsync" prefix, and otherwise keep the names same: dsync mirror -> doveadm dsync mirror dsync backup -> doveadm dsync backup dsync server -> doveadm dsync server (for running dsync remotely via ssh/etc.) b) Don't have the dsync prefix: dsync mirror -> doveadm mirror dsync backup -> doveadm backup dsync server -> doveadm dsync-server (could be hidden from the doveadm commands list) c) Either a) or b), but rename "mirror" to "sync" or "dsync" or "replicate"? d) Something else?

5 5

[Dovecot] Maildir migration and uids
by David Jonas 03 Jan '12

03 Jan '12

I'm in the process of migrating a large number of maildirs to a 3rd party dovecot server (from a dovecot server). Tests have shown that using imap to sync the accounts doesn't preserve the uidl for pop3 access. My current attempt is to convert the maildir to mbox and add an X-UIDL header in the process. Run a second dovecot that serves the converted mbox. But dovecot's docs say, "None of these headers are sent to IMAP/POP3 clients when they read the mail". Is there any way to sync these maildirs to the new server and maintain the uids? The real goal is keep customers email clients happy when they are pointed at the new server. Am I just wishing?

2 2

[Dovecot] Compressing existing maildirs
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

I've just enabled zlib for our users, and am looking at how to compress the existing files. The routine for doing this at http://wiki2.dovecot.org/Plugins/Zlib seems a bit complicated. What do you think about simply doing: find /var/vmail -type f -name "*,S=*" -mtime +1 -exec gzip -S Z -6 '{}' + I.e. find all maildir-files: - with size in the name ("*,S=*") - modified before I enabled zlib plugin - compress them - add the Z suffix - keep timestamps (gzip does that by default) It's of course racy without the maildirlock, but are there any other problems with this approach ? -jf

4 14

[Dovecot] lmtp-postlogin ?
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

We have last-login tracking for imap and pop, and I intend to use this for deciding which users to backup daily. But, it would also be nice to backup users who has only received messages, but not logged in lately.. So is it possible to implement "last-login" tracking for lmtp ? I naively tried copying the settings from imap, but it didn't work: service lmtp-postlogin { executable = script-login /usr/local/sbin/lmtp-postlogin.sh unix_listener lmtp-postlogin { } } service lmtp { executable = lmtp lmtp-postlogin <snip> -jf

2 3

[Dovecot] dsync / separator / namespace config-problem
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

Guess I don't quite get namespaces, but we have the following namespace configured: $ doveconf namespace namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = subscriptions = yes type = private } But when I try running dsync it complains: $ sudo -u atmail dsync -u janfrode(a)tanso.net mirror mdbox:/tmp/janfrodeboks dsync(atmail): Error: user janfrode(a)tanso.net: Initialization failed: namespace configuration error: list=yes requires prefix=INBOX. to end with separator dsync(atmail): Fatal: User init failed I can fix this by adding "separator = ." to the namespace and dsync is happy, but is changing this now safe, or will this affect existing clients? -jf

2 5