April 2012 - dovecot - dovecot.org

[Dovecot] doveadm user -f index
by Micah Anderson 04 Apr '12

04 Apr '12

I've configured my mail_location to have a different location for performance reasons so they aren't in the same location as the mail_location. The 'doveadm user -f home' is useful to find where a user's home directory is for various scripting purposes, but I can't seem to find a way to determine the location of the user's indexes. I can do something with the output of dovecot -a to find the mail_location and then look for a configured INDEX, but then I don't have a good way of translating the %d/%1n/%n type string formatters into their values for a user. thanks for any suggestions! micah --

2 1

[Dovecot] Dovecot and scalable database storage
by list＠airstreamcomm.net 04 Apr '12

04 Apr '12

I saw some interesting mails from TImo back in 2009 talking about the idea of using something like Cassandra db or similar as a storage platform for both email and index/logs. I was wondering if this has been discussed since then, and if there are any plans to support something like this in the future? I have been playing with Cassandra and found that their RackAwareStrategy gives you the ability to replicate writes to as many nodes as you would like, but more importantly what nodes and one of those nodes could be defined by what rack it lives in or what data center it lives in. This means multiple sites high available storage clusters, seemingly a system that dovecot could benefit from in terms of performance and redundancy and simplicity. Any takers?

2 1

[Dovecot] ldap userdb warning in v2.1.1
by Nikolaos Milas 04 Apr '12

04 Apr '12

Hi, I've upgraded from 2.0.13 to 2.1.1 and when I started the service, I got the following warning: Mar 21 10:07:23 imapserver dovecot: master: Dovecot v2.1.1 starting up (core dumps disabled) Mar 21 10:08:17 imapserverdovecot: auth: Warning: ldap: Ignoring changed user_attrs in /etc/dovecot/dovecot-passdb-ldap.conf, because userdb ldap not used. (If this is intentional, set userdb_warning_disable=yes) I didn't see such warnings in 2.0.13. I guess I should/could remove the "user_attrs" line from dovecot-passdb-ldap.conf because it's not needed? (I could also set "userdb_warning_disable=yes" as advised, but I'm trying to figure out what's the real cause of the warning.) The config follows below. Thanks, Nick ============================================================= protocols = imap pop3 mail_location = maildir:~/Maildir/ mail_gid = 502 mail_uid = 502 auth_mechanisms = plain login auth_username_format = %Lu auth_verbose = yes disable_plaintext_auth = no mail_plugins = quota protocol imap { imap_client_workarounds = "delay-newmail " mail_plugins = quota imap_quota } protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master info_log_path = log_path = mail_plugins = quota postmaster_address = sysadmin(a)example.com sendmail_path = /usr/lib/sendmail } userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap } plugin { quota = maildir:User quota quota_rule = *:storage=4G quota_rule2 = Trash:storage=+3%% quota_warning = storage=75%% quota-warning 75 %u quota_warning2 = storage=90%% quota-warning 90 %u } service quota-warning { executable = script /opt/mail1.sh user = vmail unix_listener quota-warning { user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root } service imap-login { service_count = 1 vsz_limit = 64 M } service pop3-login { service_count = 1 vsz_limit = 64 M } ssl_ca = </etc/pki/tls/certs/chain-650.pem ssl_cert = </etc/pki/tls/certs/cert-650.pem ssl_key = </etc/pki/tls/private/key-650.pem syslog_facility = local1 ============================================================= and dovecot-usrdb-ldap.conf is identical to dovecot-passdb-ldap.conf: ============================================================= hosts = localhost tls = no base = ou=people, dc=example, dc=com scope = onelevel ldap_version = 3 dn = uid=authenticate,ou=System,dc=example,dc=com dnpass = secret auth_bind = yes user_filter = (uid=%u) pass_filter = (uid=%u) pass_attrs = uid=user,userPassword=password auth_bind_userdn = uid=%u,ou=people,dc=example,dc=com user_attrs = roomNumber=quota_rule=*:bytes=%$,uid=home=/home/vmail/%u iterate_filter = (objectClass=*) =============================================================

2 2

[Dovecot] dovecot 2.0.19 Panic: file mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion failed: (offset < (uint16_t)-1)
by Jim Lawson 04 Apr '12

04 Apr '12

Had a user who couldn't access his INBOX: > Mar 21 09:21:17 penguina dovecot: imap([USER]): Panic: file > mail-index-sync-ext.c: line 209 (sync_ext_reorder): assertion fai > led: (offset < (uint16_t)-1) > Mar 21 09:21:17 penguina dovecot: imap([USER]): Error: Raw backtrace: > /usr/lib/dovecot/libdovecot.so.0 [0x342683c660] -> /usr > /lib/dovecot/libdovecot.so.0 [0x342683c6b6] -> > /usr/lib/dovecot/libdovecot.so.0 [0x342683bb73] -> > /usr/lib/dovecot/libdovecot > -storage.so.0 [0x3426c966a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x240) > [0x3426c979c0] -> / > usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x401) > [0x3426c99151] -> /usr/lib/dovecot/libdovecot-storage.s > o.0(mail_index_sync_map+0x245) [0x3426c99c55] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0x71b) > [0x3426c8afbb > ] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x3426c85d8b] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_open+0x1c > e) [0x3426c8617e] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_open+0xb5) > [0x3426c4d865] -> /usr/lib/dov > ecot/libdovecot-storage.so.0 [0x3426c75eab] -> > /usr/lib/dovecot/libdovecot-storage.so.0 [0x3426c31006] -> > dovecot/imap [hdtod > d 10.245.30.58 SELECT](cmd_ Stack trace made it look like it was the INBOX, so I deleted the index files for his INBOX and everything was OK. doveconf -n: > # OS: Linux 2.6.18-274.18.1.el5 x86_64 Red Hat Enterprise Linux Server > release 5.8 (Tikanga) > auth_gssapi_hostname = penguina.uvm.edu > auth_krb5_keytab = /etc/krb5.keytab.dovecot > auth_master_user_separator = * > auth_mechanisms = plain login gssapi > base_dir = /var/run/dovecot/ > default_process_limit = 250 > first_valid_uid = 50 > lock_method = flock > login_trusted_networks = [REDACTED] > mail_location = mbox:~/mail:INBOX=/var/spool/mail/%1u/%1.1u/%u > mail_max_lock_timeout = 30 secs > mail_max_userip_connections = 100 > mbox_read_locks = flock > mbox_write_locks = flock > mmap_disable = yes > namespace { > inbox = yes > location = > prefix = > separator = / > type = private > } > namespace { > hidden = yes > list = no > location = > prefix = mail/ > separator = / > type = private > } > namespace { > hidden = yes > list = no > location = > prefix = ~/mail/ > separator = / > type = private > } > namespace { > hidden = yes > list = no > location = > prefix = ~%u/mail/ > separator = / > type = private > } > passdb { > args = /etc/dovecot/passwd.masterusers > driver = passwd-file > master = yes > } > passdb { > driver = pam > } > service imap { > process_limit = 4096 > } > service lmtp { > client_limit = 1 > inet_listener lmtp { > port = 24 > } > } > ssl_cert = <[REDACTED] > ssl_key = < [REDACTED] > userdb { > driver = passwd > } > verbose_proctitle = yes Any questions/suggestions welcome. Jim

2 3

[Dovecot] dovecot-auth restaring and caching
by Angel L. Mateo 04 Apr '12

04 Apr '12

Hello, I'm trying to configure dovecot (2.0.13) to cache user and pass dbs. This a mail server whose purpose is only to deliver messages through dovecot lda. My users are in a ldap server. So I have configure auth_cache_size (with 20MB) and auth_cache_ttl (with 1 day). I have checked that caching is being done, and it is. If a send a message to a user, dovecot looks for it in my ldap server. If then I send another, then it uses cache information. The problem I'm having is that if I have no activity in the server, dovecot stops its auth process and when another message is received, it restarted it, but with an empty cache. This is the auth log for the first message: Mar 22 10:29:41 lynx10 dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 22 10:29:41 lynx10 dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_ldap.so Mar 22 10:29:41 lynx10 dovecot: auth: Debug: master in: USER#0111#011amateo#011service=lda Mar 22 10:29:41 lynx10 dovecot: auth: Debug: prefetch(amateo): passdb didn't return userdb entries, trying the next userdb Mar 22 10:29:41 lynx10 dovecot: auth: Debug: userdb-cache(amateo): miss Mar 22 10:29:41 lynx10 dovecot: auth: Debug: ldap(amateo): user search: <search base and filter> fields=irisMailbox,homeDirectory,uidNumber,gidNumber Mar 22 10:29:41 lynx10 dovecot: auth: Debug: ldap(amateo): result: uidNumber(uid)=XXXXX gidNumber(gid)=XXX homeDirectory(home)=XXXXXXXXXX Mar 22 10:29:41 lynx10 dovecot: auth: Debug: master out: USER#0111#011amateo#011uid=XXXXXX#011gid=XXX#011home=XXXXXXXXXXXX And this is the second one, just after a few minutes: Mar 22 10:41:03 lynx10 dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 22 10:41:03 lynx10 dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_ldap.so Mar 22 10:41:03 lynx10 dovecot: auth: Debug: master in: USER#0111#011amateo#011service=lda Mar 22 10:41:03 lynx10 dovecot: auth: Debug: prefetch(amateo): passdb didn't return userdb entries, trying the next userdb Mar 22 10:41:03 lynx10 dovecot: auth: Debug: userdb-cache(amateo): miss Mar 22 10:41:03 lynx10 dovecot: auth: Debug: ldap(amateo): user search: <search base and filter> fields=irisMailbox,homeDirectory,uidNumber,gidNumber Mar 22 10:41:03 lynx10 dovecot: auth: Debug: ldap(amateo): result: uidNumber(uid)=XXXXX gidNumber(gid)=XXX homeDirectory(home)=XXXXXXXXXXX Mar 22 10:41:03 lynx10 dovecot: auth: Debug: master out: USER#0111#011amateo#011uid=XXXXX#011gid=XXX#011home=XXXXXXXXXX This is my configuration: root@lynx10:/etc/dovecot/conf.d# doveconf -n # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-4-amd64 x86_64 Ubuntu 10.04.4 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_verbose = yes hostname = lynx10 passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 } postmaster_address = postmaster(a)um.es protocols = " imap lmtp pop3" service auth { unix_listener auth-userdb { mode = 0666 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } Any idea? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información _o) y las Comunicaciones Aplicadas (ATICA) / \\ http://www.um.es/atica _(___V Tfo: 868887590 Fax: 868888337

2 3

[Dovecot] File/folder permission issues in 2.1.3
by Christian Rößner 04 Apr '12

04 Apr '12

Hi, I figured out that Dovecot does not honer secondary groups with auth/auth-worker (??), if doing LDAP/TLS stuff. I had to use file system acls to add the user "vmail" to /etc/ssl/private and to the corresponding key file: doveconf -n # 2.1.3: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-generic-pae i686 Ubuntu 10.04.4 LTS auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes hostname = mail.roessner-net.de lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_access_groups = vmail mail_gid = vmail mail_location = mdbox:~/mdbox mail_plugins = autocreate quota acl fts fts_solr zlib mail_log notify mail_privileged_group = mail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = children location = mdbox:%%h/mdbox prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox "Deleted Messages" { special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox junkmail { special_use = \Junk } prefix = separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/var/mail/virtual/shared-mailboxes.db autocreate = Trash autocreate2 = Sent autocreate3 = Drafts autocreate4 = junkmail autosubscribe = Trash autosubscribe2 = Sent autosubscribe3 = Drafts autosubscribe4 = junkmail fts = solr fts_solr = break-imap-search url=http://localhost:8080/solr/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_rule = *:storage=300M:messages=20000 quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve service auth-worker { unix_listener auth-worker { user = vmail } user = vmail } service auth { unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = ::1 port = 24 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl_ca = </ca/psw_net/SSL123_CA_Bundle.pem ssl_cert = </ca/psw_net/mail_roessner-net_de.crt ssl_key = </ca/psw_net/mail_roessner-net_de.key userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = autocreate quota acl fts fts_solr zlib mail_log notify sieve } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags mail_max_userip_connections = 50 mail_plugins = autocreate quota acl fts fts_solr zlib mail_log notify imap_quota imap_acl imap_zlib } Normally, mail is placed under /var/mail/virtual as user vmail, group vmail. Is there something wrong with my config that prevents switching to secondary groups? /etc/dovecot/dovecot-ldap.conf.ext: uris = ldap://ldap0.roessner-net.de/ ldap://db.roessner-net.de/ sasl_bind = yes sasl_mech = EXTERNAL tls = yes tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt tls_cert_file = /etc/ssl/certs/mx0.roessner-net.de.pem tls_key_file = /etc/ssl/private/mx0.roessner-net.de.key.pem tls_require_cert = hard base = ou=people,ou=it,dc=roessner-net,dc=de user_attrs = rnsMSQuota=quota_rule=*:storage=%$,rnsMSMailboxHome=home user_filter = (&(objectClass=rnsMSDovecotAccount)(rnsMSRecipientAddress=%u)) pass_attrs = rnsMSDeliverToAddress=user,userPassword=password pass_filter = (&(objectClass=rnsMSDovecotAccount)(rnsMSRecipientAddress=%u)(rnsMSEnableDovecot=TRUE)) iterate_attrs = rnsMSDovecotUser=user iterate_filter = (objectClass=rnsMSDovecotAccount) default_pass_scheme = CRYPT Thanks in advance. -Christian --- Roessner-Network-Solutions Bachelor of Science Informatik Nahrungsberg 81, 35390 Gießen F: +49 641 5879091, M: +49 176 93118939 USt-IdNr.: DE225643613 http://www.roessner-network-solutions.com

2 1

Re: [Dovecot] Many messages clustered around the same date.saved value
by Joseph Tam 04 Apr '12

04 Apr '12

Timo Sirainen <tss(a)iki.fi> wrote: >>> Which mailbox format? With Maildir the date.saved is taken from >>> dovecot.index.cache file, and in some cases that might get dropped. If >>> it does, then it fallbacks to using the file's ctime. >> >> mbox. > > Ah, with mbox there isn't any usable fallback for date.saved. If it's > not in dovecot.index.cache, the current time is used. I'm a little confused as to why it needed a fallback. In other words, why wasn't date.saved put into the index as soon as the IMAP operation copied it into "Trash"? If this data isn't set at that time, when does it get instantiated? When I actually ask for it? Joseph Tam <jtam.home(a)gmail.com>

2 1

[Dovecot] Is it possible to migrating mail to dovecot using imapc?
by Dr Josef Karthauser 04 Apr '12

04 Apr '12

I'm excited to hear that dovecot now supports imap as a mailbox type. I've got a mailbox over on another imap server, which I want to migrate to dovecot. I was wondering whether I could use 'doveadm backup' to setup a two way synchronisation process between the old imap mailbox and the new dovecot mailbox. That way I can do the migration, and can coordinate the mail clients changing where they pick up their email from later. But, I can't find much documentation on imapc, other than it exists. Is this kind of move possible now? Would someone mind giving me some hints as to how to make it work please? Thanks :), Joe

2 9

[Dovecot] dovecot 2.1 breaks FTS + pre-auth?
by dm-list-email-dovecot＠scs.stanford.edu 04 Apr '12

04 Apr '12

Hi. I use dovecot in the simplest possible way, as an IMAP server in pre-auth mode over ssh or just locally over a unix-domain socket (e.g., with offlineimap, which runs much faster using dovecot for the local message store). Ideally I would like to avoid running any extra daemons or setting up anything as root. Until recently, this has worked fine by just setting the CONFIG_FILE environment variable to something in my home directory. Here is my configuration: $ export CONFIG_FILE=$HOME/etc/dovecot.conf $ dovecot -n # 2.1.3: /home/dm/etc/dovecot.conf # OS: Linux 3.2.13-1-ARCH x86_64 mail_location = maildir:~/Mail/inbox mail_plugins = " fts fts_squat" plugin { fts = squat fts_squat = partial=4 full=10 } doveconf: Error: ssl enabled, but ssl_cert not set doveconf: Fatal: Error in configuration file /home/dm/etc/dovecot.conf: ssl enabled, but ssl_cert not set Full text search used to work just fine with this configuration, and still does on a machine I have running dovecot 2.0.13. However, on the machine with 2.1, I get errors about /var/run/dovecot/index not existing. $ printf "a select INBOX\nb search text xyzzy\nc logout\n" \ | /usr/lib/dovecot/imap * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in as dm imap(dm): Error: net_connect_unix(/var/run/dovecot/indexer) failed: No such file or directory ... Needless to say, no dovecot.index.search or dovecot.index.search.uids file is created after this error. While I can't write /var/run/dovecot, this is not a permission issue. For example, adding base_dir=/home/dm (my home directory) to the configuration file yields the same error for /home/dm/indexer. I'm guessing something has changed where imap requires an indexer daemon and doesn't launch it in pre-auth mode any more, but I can't find anything about this in the documentation. In short, if anyone can tell me how to use FTS in conjunction with pre-auth mode or point me to a working example, I would appreciate it.

2 1

[Dovecot] 2.1.3: doveadm mailbox delete lost the -s option
by Micah Anderson 04 Apr '12

04 Apr '12

Looks like the recently added -s (for unsubscribing) option to doveadm mailbox delete went missing: root@vireo# /usr/bin/doveadm mailbox delete -u micahtest -s restored delete: invalid option -- 's' doveadm mailbox delete [-u <user>|-A] [-S <socket_path>] [-s] <mailbox> [...] although the option still is shown as valid :) micah --

2 1