November 2012 - dovecot

[Dovecot] doveadm with multiple commands
by Timo Sirainen 25 Mar '13

25 Mar '13

This has been in my TODO for a while and today I implemented it (but didn't dare to test it yet :) So the problem has been that you want to run for example: doveadm expunge -A mailbox Trash savedbefore 7d doveadm purge -A This is annoying because it scans through the users' mailboxes twice, wasting disk IO and CPU. So how to make it do the scan only once? The syntax I thought of today would be: doveadm multi [-A | -u wildcards] <separator string> <comand 1> [<separator string> <command 2> [...]] So the first two commands could be for example: doveadm multi -A : expunge mailbox Trash savedbefore 7d : purge The ':' character is the separator. Any other string could be used as well, for example "?SEP?" or whatever. Thoughts? Any better name for the command than "multi"?

6 13

[Dovecot] virtual folder - crash while searching
by Matthias Rieber 10 Mar '13

10 Mar '13

Hi, I've upgraded from dovecot 2.0 to 2.1. When I perform a header search in a virtual folder dovecot crashes. Here's the stacktrace: http://pastebin.com/urSDDzku Regards, Matthias

4 11

[Dovecot] POLL: v2.2 to allow one mail over quota?
by Timo Sirainen 12 Feb '13

12 Feb '13

Currently if user is 1MB under quota and someone tries to deliver mail that is over 1MB, Dovecot rejects the mail. But smaller mails aren't rejected probably for days. So user might not even realize that they didn't receive one of the mails. Also having a user "almost over quota" is a rather strange state I think. So what do you think about v2.2 allowing delivery of one last mail even if it brings the user over quota? Except add a limit that if the message size is as much as the user's entire quota limit it wouldn't be added (or 50% or ..?). Also IMAP wouldn't allow this, since user would get an error anyway. I could make this also optional, but if nobody really wants to keep the old behavior there's really no point in adding the option.

20 27

[Dovecot] question on "doveadm user" and pass/userdb
by Gábor Lénárt 22 Jan '13

22 Jan '13

Hi, I'm about getting to know dovecot in details, I am about the command "doveadm user". The man page shows this example: userdb: jane uid : 8001 gid : 8001 home : /home/jane mail : sdbox:~/sdbox plugins : sieve quota_rule: *:storage=150M It seems to be nice, however I never got this only: ~# doveadm user lgb@office userdb: lgb@office That's all. The dovecot test configuration works otherwise via IMAP/POP3/LMTP nicely. What I guess that the lack of extra information (which would be needed by some scripts of mine, eg for getting user's home easily from shell scripts, as it's hash based, etc) is caused because I don't pass home (and/or other settings) back from userdb and/or passdb. However I do have these in global scope: mail_home = /mailstorage/%Ld/%Ln (yes, this is not hashed at all at the momement but it will be soon) mail_location = maildir:~/Maildir mail_uid = vmail mail_gid = vmail (... and also some global quota stuffs configured in plugin section) Now, I am a bit confused: even with passdb/userdb does not return with user home or GID/UID, dovecot knows these in case of pop3/imap/lmtp access, since these information can be "composed" from those settings. But then, wouldn't it possible so "doveadm user" shows those as well, like lmtp/pop3/imap can resolve those too? Honestly, it's a bit redundant to put needed information into user and pass attrs in every dbs (I have some) when dovecot knows those otherwise ... If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb query tool nothing more), is there any tool which works in this config, displaying extra information (at least user's home), or should I create some script which just repeats the functionality of dovecot's configured mail_home resolution? Thanks.

3 5

[Dovecot] corrupt mdbox, force-resync segfaults
by Metro Domain Admin 03 Jan '13

03 Jan '13

I could use some help with a corrupt mdbox. doveadm force-resync is crashing (see below), but I really need just to get this account functioning. What's my next step, as far as deleting index files? These were the earliest errors I could find: Nov 28 09:40:21 macy dovecot[6615]: imap(cory(a)metro-email.com): Error: Corrupted index cache file /opt/mail/metro-email.com/cory/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: field index too large (66 >= 30)Nov 28 09:40:21 macy dovecot[6615]: imap(cory(a)metro-email.com): Error: mdbox map /opt/mail/metro-email.com/cory/mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=1233 map_uid=1708 As for the segfaults, I am getting this regardless of which mailbox I specify: # /opt/dovecot/bin/doveadm -Dv force-resync -u cory INBOX doveadm(root): Debug: Loading modules from directory: /opt/dovecot/lib/dovecot doveadm(root): Debug: Module loaded: /opt/dovecot/lib/dovecot/lib15_notify_plugin.so doveadm(root): Debug: Module loaded: /opt/dovecot/lib/dovecot/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /opt/dovecot/lib/dovecot/lib20_mail_log_plugin.so doveadm(root): Debug: Module loaded: /opt/dovecot/lib/dovecot/lib21_fts_squat_plugin.so doveadm(root): Debug: Loading modules from directory: /opt/dovecot-2.1.10/lib/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: dlopen(/opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so, 10): Symbol not found: _acl_user_module Referenced from: /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so Expected in: flat namespace in /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: dlopen(/opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so, 10): Symbol not found: _expire_set_deinit Referenced from: /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so Expected in: flat namespace in /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: dlopen(/opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so, 10): Symbol not fo und: _quota_user_module Referenced from: /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so Expected in: flat namespace in /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: dlopen(/opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so, 10): Symbol not found: _i_stream_create_deflate Referenced from: /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so Expected in: flat namespace in /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /opt/dovecot-2.1.10/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm(cory): Debug: auth input: cory(a)domain.com uid=302 gid=6 home=/opt/mail/domain.com/cory doveadm(cory): Debug: changed username to cory(a)domain.com doveadm(cory): Debug: Effective uid=302, gid=6, home=/opt/mail/domain.com/cory doveadm(cory): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/opt/mail/domain.com/cory/mdbox doveadm(cory): Debug: fs: root=/opt/mail/domain.com/cory/mdbox, index=, control=, inbox=, alt= doveadm(cory): Debug: Namespace : Using permissions from /opt/mail/domain.com/cory/mdbox: mode=0700 gid=-1 doveadm(cory): Warning: mdbox /opt/mail/domain.com/cory/mdbox/storage: rebuilding indexes doveadm(cory): Error: mdbox /opt/mail/domain.com/cory/mdbox/storage: Duplicate GUID c3bbb2234d93af505873010097ac0257 in m.1:428519 and m.1:441340 doveadm(cory): Error: mdbox /opt/mail/domain.com/cory/mdbox/storage: Duplicate GUID c3bbb2234d93af505873010097ac0257 in m.1:428519 and m.1:453628 [...] doveadm(cory): Error: mdbox /opt/mail/domain.com/cory/mdbox/storage: Duplicate GUID d45f7f111894af506273010097ac0257 in m.83:8565306 and m.83:8594879 Segmentation fault: 11 macy:postfix root# /opt/dovecot/bin/doveconf -n # 2.1.10: /opt/dovecot-2.1.10/etc/dovecot/dovecot.conf doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Darwin 11.4.2 x86_64 auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_username_format = %n(a)domain.com auth_verbose = yes auth_verbose_passwords = plain default_internal_user = _dovecot default_login_user = _dovenull default_process_limit = 500 disable_plaintext_auth = no first_valid_uid = 300 instance_name = dovecot-2.1.10 listen = 192.168.20.123 mail_debug = yes mail_location = mdbox:%h/mdbox mail_plugin_dir = /opt/dovecot/lib/dovecot mail_plugins = fts fts_squat mail_log notify mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = prefix = separator = / } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from subject } protocols = imap lmtp service aggregator { fifo_listener replication-notify-fifo { mode = 00 } } service auth { unix_listener /opt/local/var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = admin } } service director { fifo_listener login/proxy-notify { mode = 00 } } service imap { executable = imap postlogin } service lmtp { unix_listener /opt/local/var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_cert = </apposite/etc/ssl/mail_domain_com/mail_domain_com.crt ssl_key = </apposite/etc/ssl/mail_domain_com/mail_domain_com.key userdb { args = username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 16 }

2 2

[Dovecot] backtrace from postgres dict
by Steffen Kaiser 20 Dec '12

20 Dec '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, since I enabled ACLs I get following backtrace. It looks like that it happens if the ACLs had been changed only. Also, it looks like that it always happens 1 minute after last "dict: Info: pgsql(localhost): Connected to database dovecot", no matter if the user is logged in (and idle) or logged off. Postgres logs: "could not receive data from client: Connection reset by peer" and "unexpected EOF on client connection". Is there some TCP keep alive issue? Postgres is using the "system default" for TCP keep alive. I'm currently testing ACLs via telnet, so the activity with the ACL dict is pretty low, hence, an idle timeout would make much sense. # ../../sbin/dovecot --version 2.1.10 (a4f02f6c0d32) dict connect string: connect = host=localhost dbname=db user=user 2012-11-28 10:03:39 dict: Info: pgsql(localhost): Connected to database dovecot 2012-11-28 10:03:39 dict: Info: pgsql(localhost): Connected to database dovecot 2012-11-28 10:03:39 dict: Info: pgsql(localhost): Connected to database dovecot 2012-11-28 10:03:58 IMAP(user) [22922]: Info: Disconnected: Logged out in=23 out=763 2012-11-28 10:04:41 dict: Panic: file driver-pgsql.c: line 84 (driver_pgsql_set_state): assertion failed: (state == SQL_DB_STATE_BUSY || db->cur_result == NULL) 2012-11-28 10:04:41 dict: Error: Raw backtrace: /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x4857a) [0x7f91f511157a] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(+0x485c6) [0x7f91f51115c6] -> /usr/local/dovecot-2.1.10/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f91f50e4eaf] -> dovecot2.1/dict() [0x40aa36] -> dovecot2.1/dict() [0x40aa91] -> dovecot2.1/dict() [0x40bed3] -> dovecot2.1/dict() [0x409504] - -> dovecot2.1/dict(sql_db_cache_deinit+0x20) [0x408a60] -> dovecot2.1/dict(main+0x169) [0x405a59] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f91f4b5cc8d] -> dovecot2.1/dict() [0x404bb9] 2012-11-28 10:04:41 dict: Fatal: master: service(dict): child 22923 killed with signal 6 (core not dumped) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBULXbiGoxLS8a3A9mAQI9Ywf/YItPO4fFT70HcuwCkDXgx0Sn7CEyXu2j +1VzOu3R/TMs1rcxYJIvWiei6Nk3wXywzQl84POadJn61Yf2NuT80nMxwBsBbgio e+0oRE9JNVSjtvX5l7kehEdUDQjqRRKyrhWG9KDlpjWidGsYh7pQg0rDtp3UwG57 HuSBMiBAZ9t9sC2DXjn9wT9vweH3gMIhc/K5U0BLF5JG/K8UAKc5QBUgezjIYzvn KP90TJ4RoZ2vnPzmWfeIsjffRBvvC6UUgWmU02e2H9xJxzF32Nix0s8YWi7G91sH Ei057HW5NsKNDtp6B5fYk+xCpC6ucQiwFSr/ZG+uZCAdtTd2qmf60g== =tQYK -----END PGP SIGNATURE-----

1 2

[Dovecot] Maildir hardlinks with sieve
by Benedetto Vassallo 19 Dec '12

19 Dec '12

Hi all, First of all, excuse me for my previous posts (Maildir hardlinks). Today I realized the following issue. Maildir hardlinks works OK on dovecot 2.1.10 without sieve. If set sieve plugin in dovecot-lmtp hardlinks stop working. Is there a way to have both sieve and hardlinks working? Thank you -- Benedetto Vassallo Sistema Informativo di Ateneo Settore Gestione Reti Hardware e Software U.O.B. Sviluppo e manutenzione dei sistemi Università degli studi di Palermo Phone: +3909123860056 Fax: +390916529124 ------------------------------------------------------------------------- This message was sent using the University of Palermo web mail interface.

3 6

[Dovecot] Dovecot director doveadm with switch "-A" error
by Ramon Frontera 08 Dec '12

08 Dec '12

Hi, we have a problem with our director proxy configuration. When we run on proxy server the doveadm command with -A switch, fails with the error: # doveadm -D quota get -A doveadm(user1): Debug: auth input: user=user1 proxy starttls=any-cert doveadm(user1): Error: Proxy is missing destination host doveadm: Error: Failed to iterate through some users If we execute the command with a single user, director assigns host and goes well, Example: # doveadm -D quota get -u user1 doveadm(user1): Debug: auth input: user=user1 proxy starttls=any-cert host=xx.xx.xx.xx proxy_refresh=450 Quota name Type Value Limit % User quota STORAGE 159401 2621440 6 User quota MESSAGE 341 - 0 We have tried different configurations and all goes wrong with this "-A" switch, Timo please, could you tell us if we have a missconfiguration or its a possible director bug ?? We really want execute the expire plugin on ours directors and them redirect to the backends servers of every user. thanks in advance. Our doveconf -n: # dovecot -n # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.14.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_cache_size = 3 k auth_cache_ttl = 15 mins auth_debug = yes auth_verbose = yes base_dir = /var/run/dovecot/ default_client_limit = 30000 default_process_limit = 5000 dict { expire = mysql:/etc/dovecot/dovecot-dict-expire.conf.ext } director_doveadm_port = 990 director_mail_servers = mailserver1 mailserver2 director_servers = proxy1 doveadm_proxy_port = 24245 lmtp_proxy = yes mail_debug = yes mail_plugins = expire quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap-users.conf.ext driver = ldap } plugin { expire = Trash expire2 = Trash/* expire_dict = proxy::expire } postmaster_address = postmaster<at>company protocols = imap pop3 lmtp sieve service auth { client_limit = 27048 unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = dovecot mode = 0660 } } service director { fifo_listener login/proxy-notify { mode = 0600 user = $default_login_user } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0660 } unix_listener login/director { mode = 0666 } } service doveadm { executable = doveadm-server director inet_listener doveadm-server { port = 24245 } } service imap-login { executable = imap-login director inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service lmtp { client_limit = 1 inet_listener lmtp { port = 30025 } } service managesieve-login { executable = managesieve-login director } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert = </etc/pki/tls/certs/imap.crt ssl_key = </etc/pki/tls/private/imap.uib.es_privatekey.pem syslog_facility = local1 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap-users.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { auth_socket_path = director-userdb mail_plugins = expire quota sieve quota passdb { args = /etc/dovecot/dovecot-ldap-pass.conf.lmtp driver = ldap } } protocol doveadm { auth_socket_path = director-userdb } protocol imap { mail_max_userip_connections = 20 mail_plugins = expire quota } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } local ip/24/24 { doveadm_password = supersecretpassword } -- ------------------------------------------ Ramon Frontera Gallardo Universitat de les Illes Balears

2 9

[Dovecot] Cannot STORE \Seen flag on some mails
by Frerich Raabe 07 Dec '12

07 Dec '12

Hi, I'm running Dovecot 1.2.17 on FreeBSD (exact output of 'dovecot -n' is atttached to this mail). The machine is serving a public mailinglist archive which is read-only; all mail arriving for the archive is marked as \Seen using Sieve script. This setup works well most of the time, but I noticed that for *some* mails, the \Seen flag doesn't seem to be stored. Right now I have 31255 mails in one of my folders and I can't seem to mark five of them as \Seen - the others work just fine. I first suspected a client issue so I did a little IMAP session by hand: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready. . LOGIN "xxxxxx" "yyyyyy" . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk] Logged in . SELECT "Lists/Archive/squish" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk) * OK [PERMANENTFLAGS ()] Read-only mailbox. * 31250 EXISTS * 0 RECENT * OK [UNSEEN 27126] First unseen. * OK [UIDVALIDITY 1350573750] UIDs valid * OK [UIDNEXT 31265] Predicted next UID * OK [HIGHESTMODSEQ 9512] Highest . OK [READ-ONLY] Select completed. . SEARCH UNSEEN * SEARCH 27126 27127 28484 29835 29838 . OK Search completed (0.000 secs). . STORE 27126 FLAGS \SEEN . OK Store completed. . SEARCH UNSEEN * SEARCH 27126 27127 28484 29835 29838 . OK Search completed (0.000 secs). . LOGOUT * BYE Logging out . OK Logout completed. Note how the first 'SEARCH UNSEEN' command shows that '27126' is unseen, the subsequent 'STORE' command succeeds - but then 'SEARCH UNSEEN' still shows 27126 as unseen! I have all four logging levels being piped to /var/log/maillog (I verified this by running dovecot --log-error) but the file does not show any problems. I checked the file permissions of the Maildir directories, and it all looks dandy to me. Does anybody have some suggestions how to debug this further, or what the reason for this may be? -- Frerich Raabe - raabe(a)froglogic.com www.froglogic.com - Multi-Platform GUI Testing

2 4

[Dovecot] When are search indexes updated?
by David Abrahams 06 Dec '12

06 Dec '12

I noticed that occasionally searching in my huge archive mailbox can be really slow, so I tried doveadm index on it and it seemed to do a lot of work, which seemed strange given, for example, that dovecot-lda says it keeps Dovecot index files up-to-date. Then I thought, "maybe these are different files than the search indices." If so, that's not entirely clear from the docs and Wiki. So, questions: * When are search indexes updated? * Are they updated incrementally? * If not, why not? * If so, why would a mailbox's index drift out-of-date, as mine had? BTW, I'm using the clucene search backend. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

6 16