September 2013 - dovecot

[Dovecot] Expunged message reappeared, giving a new UID
by Simon Fraser 23 Sep '13

23 Sep '13

I am running dovecot 2.2.2 with tcp based replication, and experiencing some duplicated emails. `doveconf -n` output is below. I have narrowed it down to the following scenario: An email arrives, and is successfully replicated to both nodes. It is in INBOX/new/ at this point on both servers. Connect with a mail client, and delete the message - without delayed expunge. So, for example, mutt (press 'd' then '$' to sync the mailbox), or Evolution set to immediately delete. The following log entry appears on the server the client is connected to: Jul 26 10:40:55 intmail3a dovecot: imap(srf): Warning: /mail/spool/s/srf/mail/INBOX/dovecot-uidlist: Duplicate file entry at line 5: 1374831638.M830754P11591.intmail3a,S=1289,W=1321 (uid 733481 -> 733482) - retrying by re-reading from beginning Jul 26 10:40:55 intmail3a dovecot: imap(srf): Warning: Maildir /mail/spool/s/srf/mail/INBOX: Expunged message reappeared, giving a new UID (old uid=733481, file=1374831638.M830754P11591.intmail3a,S=1289,W=1321) The email reappears in the mailbox, although this time it is in INBOX/new on one node and INBOX/cur on the other. As best I can tell it's always in 'new' on the server the client was connected to and in 'cur' on the other. If any other mailbox operation occurs before the delete (Marking the message in question as 'read', receiving a new message into the mailbox in a different lmtp session, marking as read or deleting & expunging some other message) then this prevents the duplication from happening. What can I do to prevent this happening? Is there a synchronisation interval I can adjust, perhaps? Although the message does get synchronised initially. Thanks in advance, Simon. # 2.2.2: /mail/etc/dovecot/dovecot.conf # OS: Linux 3.5.0-23-generic x86_64 Ubuntu 12.04.2 LTS disable_plaintext_auth = no doveadm_password = secret first_valid_uid = 100 lda_mailbox_autocreate = yes mail_gid = dovecot mail_home = /mail/spool/%1n/%n mail_location = maildir:~/mail:INBOX=~/mail/INBOX mail_plugins = notify replication mail_uid = dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = /mail/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_replica = tcps:intmail3b:12345 sieve_dir = ~/sieve } protocols = imap lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0600 user = dovecot } unix_listener replication-notify { mode = 0600 user = dovecot } } service config { unix_listener config { user = dovecot } } service doveadm { inet_listener { port = 12345 ssl = yes } user = dovecot } service replicator { process_min_avail = 1 } ssl_cert = </mail/etc/certs/mail-cert.pem ssl_client_ca_dir = /etc/ssl/certs ssl_key = </mail/etc/certs/mail-key.pem userdb { args = /mail/etc/dovecot/dovecot-ldap.userdb.conf.ext driver = ldap } protocol lmtp { mail_plugins = notify replication sieve } protocol lda { mail_plugins = notify replication sieve } protocol imap { mail_max_userip_connections = 30 } -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

8 30

[Dovecot] Dsync error: Failed to set attribute vendor/vendor.dovecot/pvt/sieve/default
by Aleksey Tsvetkov 23 Sep '13

23 Sep '13

Hi, Introduction: There are two domains, for example (aaa.com, bbb.com) In the aaa.com domain two users (bob(a)aaa.com, alex(a)aaa.com) In the bbb.com domain there are no users. In time full sync replication (replication_full_sync_interval) operations in logs appear errors: dovecot: dsync-local(bob(a)aaa.com): Error: Mailbox INBOX: Failed to set attribute vendor/vendor.dovecot/pvt/sieve/default: Invalid value for default sieve attribute dovecot: dsync-remote(bob(a)aaa.com): Error: Mailbox INBOX: Failed to set attribute vendor/vendor.dovecot/pvt/sieve/default: Invalid value for default sieve attribute One more error: dovecot: auth-worker(46263): sql(bob(a)bbb.com): unknown user dovecot: auth-worker(46263): sql(bob(a)bbb.com): Unknown user dovecot: auth-worker(46263): sql(bob): unknown user dovecot: auth-worker(46263): sql(bob): Unknown user dovecot: auth-worker(46263): sql(alex(a)aaa.com): unknown user dovecot: auth-worker(46263): sql(alex(a)aaa.com): Unknown user dovecot: auth-worker(46263): sql(alex): unknown user dovecot: auth-worker(46263): sql(alex): Unknown user Command: doveadm user '*' bob(a)aaa.com alex(a)aaa.com dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql dovecot --version 2.2.5 Somebody faced this problem? -- Best regards, Aleksey Tsvetkov System Administrator Company Grand Vision tel. +7(495)933-39-79, ext. 184

4 4

[Dovecot] mdbox index directories: bug?
by Holecz Endre 22 Sep '13

22 Sep '13

Hi, I'm using mdbox now (Dovecot 2.2.4), with indexes on a separate disk. Works fine, but the directories for indexes are also present in the storage folder, and are empty. Userdb (sql) returns this: home /var/vmail/domains/somedomain/someuser mail mdbox:~/mdbox:INDEX=/mnt/spool/dovecot/index/somedomain/s/someuser And the created directory structure is: /var/vmail/domains/somedomain/someuser /mdbox /mailboxes /INBOX /dbox-Mails /storage /mnt/spool/dovecot/index/somedomain/s/someuser /mailboxes /INBOX /storage /var/vmail/domains/somedomain/someuser/mdbox/mailboxes/* is always empty. Am I doing something wrong, or is this by design? :) Regards, Endre

3 3

[Dovecot] doveadm sync only creates directories, don't sync messages?
by Matthew Kaufman 22 Sep '13

22 Sep '13

Running Dovecot 2.1.7. I'm trying to migrate from an environment with mbox files to a maildir configuration. I remove all vestiges of my maildir over in /var/mail I put my old mailboxes in ./Mail and my old inbox in ./matthew. I issue this command: doveadm sync -u matthew(a)matthew.at mbox:./Mail:INBOX=./matthew Initially it gave me errors about separator, so I set the separator in the inbox namespace to "/" explicitly. Now that I've done that, and re-run it, over in the right place /var/mail, it creates "matthew" and a subdirectory "Mail". That's it. Doesn't take long to do it, either. But it doesn't copy any messages at all. I must be doing something wrong... but what? Matthew Kaufman matthew(a)eeph.com

1 0

[Dovecot] pop3c migration?
by Gedalya 22 Sep '13

22 Sep '13

Hi, I'm running dsync migrations using imapc and the source IMAP server is just too slow. It has taken 8 hours to migrate a mailbox with 47,000 messages. It seems most of the mailboxes are never accessed with IMAP, so it would be just as good to migrate them via POP3 which should work faster in my case. Is this actually meant to work? doveadm -D -o pop3c_user=xxxxxx(a)example.com -o 'pop3c_password=*******' -o pop3c_host=120.121.122.123 backup -u yyyyy(a)example.com -R pop3c: gives: doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_listescape_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_init (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(yyyyy(a)example.com): Debug: Added userdb setting: mail=maildir:/stor/real/domains/example.com/yy/yyyyy/Maildir doveadm(yyyyy(a)example.com): Debug: Added userdb setting: plugin/quota_rule=*:storage=5120M doveadm(yyyyy(a)example.com): Debug: Effective uid=1001, gid=1001, home=/stor/real/domains/example.com/yy/yyyyy doveadm(yyyyy(a)example.com): Debug: Namespace : type=private, prefix=POP3-MIGRATION-NS/, sep=, inbox=no, hidden=yes, list=no, subscriptions=yes location=pop3c: doveadm(yyyyy(a)example.com): Debug: fs: root=, index=, control=, inbox=, alt= doveadm(yyyyy(a)example.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/stor/real/domains/example.com/yy/yyyyy/Maildir doveadm(yyyyy(a)example.com): Debug: maildir++: root=/stor/real/domains/example.com/yy/yyyyy/Maildir, index=, control=, inbox=/stor/real/domains/example.com/yy/yyyyy/Maildir, alt= dsync(yyyyy(a)example.com): Debug: Effective uid=1001, gid=1001, home=/stor/real/domains/example.com/yy/yyyyy dsync(yyyyy(a)example.com): Debug: Namespace : type=private, prefix=POP3-MIGRATION-NS/, sep=, inbox=no, hidden=yes, list=no, subscriptions=yes location=pop3c: dsync(yyyyy(a)example.com): Debug: fs: root=, index=, control=, inbox=, alt= dsync(yyyyy(a)example.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=pop3c: dsync(yyyyy(a)example.com): Debug: fs: root=, index=, control=, inbox=, alt= Segmentation fault remembering something about that... : # mkdir /tmp/pop3c # chown vmail:vmail /tmp/pop3c/ # doveadm -D -o pop3c_user=xxxxxx(a)example.com -o 'pop3c_password=*******' -o pop3c_host=120.121.122.123 backup -u yyyyy(a)example.com -R pop3c:/tmp/pop3c doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_listescape_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_init (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(yyyyy(a)example.com): Debug: Added userdb setting: mail=maildir:/stor/real/domains/example.com/yy/yyyyy/Maildir doveadm(yyyyy(a)example.com): Debug: Added userdb setting: plugin/quota_rule=*:storage=5120M doveadm(yyyyy(a)example.com): Debug: Effective uid=1001, gid=1001, home=/stor/real/domains/example.com/yy/yyyyy doveadm(yyyyy(a)example.com): Debug: Namespace : type=private, prefix=POP3-MIGRATION-NS/, sep=, inbox=no, hidden=yes, list=no, subscriptions=yes location=pop3c: doveadm(yyyyy(a)example.com): Debug: fs: root=, index=, control=, inbox=, alt= doveadm(yyyyy(a)example.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/stor/real/domains/example.com/yy/yyyyy/Maildir doveadm(yyyyy(a)example.com): Debug: maildir++: root=/stor/real/domains/example.com/yy/yyyyy/Maildir, index=, control=, inbox=/stor/real/domains/example.com/yy/yyyyy/Maildir, alt= dsync(yyyyy(a)example.com): Debug: Effective uid=1001, gid=1001, home=/stor/real/domains/example.com/yy/yyyyy dsync(yyyyy(a)example.com): Debug: Namespace : type=private, prefix=POP3-MIGRATION-NS/, sep=, inbox=no, hidden=yes, list=no, subscriptions=yes location=pop3c: dsync(yyyyy(a)example.com): Debug: fs: root=, index=, control=, inbox=, alt= dsync(yyyyy(a)example.com): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=pop3c:/tmp/pop3c dsync(yyyyy(a)example.com): Debug: fs: root=/tmp/pop3c/., index=/tmp/pop3c, control=, inbox=, alt= dsync(yyyyy(a)example.com): Debug: Namespace : Using permissions from /tmp/pop3c/.: mode=0755 gid=-1 dsync(yyyyy(a)example.com): Debug: Namespace : Using permissions from /stor/real/domains/example.com/yy/yyyyy/Maildir: mode=0700 gid=-1 dsync(yyyyy(a)example.com): Debug: Namespace : /tmp/pop3c/./INBOX doesn't exist yet, using default permissions dsync(yyyyy(a)example.com): Debug: Namespace : Using permissions from /tmp/pop3c/.: mode=0755 gid=-1 dsync(yyyyy(a)example.com): Debug: pop3c(120.121.122.123): Looking up IP address dsync(yyyyy(a)example.com): Debug: pop3c(120.121.122.123): Connecting to 120.121.122.123:110 dsync(yyyyy(a)example.com): Debug: pop3c(120.121.122.123): Authenticating as xxxxxx(a)example.com dsync(yyyyy(a)example.com): Panic: file mail-storage.c: line 1311 (mailbox_get_metadata): assertion failed: ((items & MAILBOX_METADATA_GUID) == 0 || !guid_128_is_empty(metadata_r->guid)) dsync(yyyyy(a)example.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x45caa) [0xb761ccaa] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x47) [0xb761cdc7] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb761d647] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_metadata+0x120) [0xb76f3040] -> doveadm() [0x80740bc] -> doveadm(dsync_worker_mailbox_iter_next+0x21) [0x8071c11] -> doveadm() [0x806c1f7] -> doveadm(dsync_brain_sync+0x6c0) [0x806b890] -> doveadm(dsync_brain_sync_all+0x18) [0x806c4f8] -> doveadm() [0x806961b] -> doveadm() [0x8069bbf] -> doveadm() [0x80541ec] -> doveadm(doveadm_mail_try_run+0x257) [0x8054ee7] -> doveadm(main+0x391) [0x805d0c1] -> /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb748ae46] -> doveadm() [0x8054001] Aborted Now I'm lost... Am I in the right direction? Thanks, Gedalya # dovecot -n # 2.1.17: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-686-pae i686 Debian 7.1 auth_default_realm = example.com auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain default_client_limit = 8096 dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imapc_host = mail01.example.com mail_gid = vmail mail_plugins = listescape mail_prefetch_count = 50 mail_uid = vmail namespace { hidden = yes list = no location = pop3c: prefix = POP3-MIGRATION-NS/ } namespace inbox { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash expire2 = Trash.* expire3 = Junk expire_dict = proxy::expire pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX sieve = ~/.dovecot.sieve sieve_before = /var/local/lib/dovecot/sieve/global/fileinto-spam.sieve sieve_dir = ~/sieve } pop3c_host = 120.121.122.123 postmaster_address = postmaster(a)example.com service auth-worker { user = $default_internal_user } service auth { client_limit = 20480 } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { process_min_avail = 4 service_count = 0 vsz_limit = 128 M } service imap { process_limit = 10240 } service lmtp { inet_listener lmtp { address = 0.0.0.0 port = 7025 } } service pop3-login { process_min_avail = 4 service_count = 0 vsz_limit = 128 M } service pop3 { process_limit = 4096 } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol doveadm { mail_plugins = listescape pop3_migration } protocol lmtp { mail_plugins = listescape sieve } protocol imap { mail_max_userip_connections = 25 mail_plugins = listescape imap_quota ssl_cert = </etc/dovecot/imap.example.com.crt ssl_key = </etc/dovecot/imap.example.com.key } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = listescape pop3_fast_size_lookups = yes pop3_no_flag_updates = yes pop3_reuse_xuidl = no pop3_save_uidl = yes pop3_uidl_format = %08Xu%08Xv ssl_cert = </etc/dovecot/pop.example.com.crt ssl_key = </etc/dovecot/pop.example.com.key } /etc/dovecot/dovecot-sql.conf.ext : driver = mysql connect = ..... default_pass_scheme = PLAIN password_query = SELECT password, \ '/stor/real/domains/%d/%2n/%n' AS userdb_home, \ 'maildir:/stor/real/domains/%d/%2n/%n/Maildir' as userdb_mail, \ concat('*:storage=', quota_mb, 'M') as userdb_quota_rule, \ 'vmail' AS userdb_uid, 'vmail' AS userdb_gid \ FROM email WHERE userid = '%n' AND domain = '%d' and deleted=0 user_query = SELECT '/stor/real/domains/%d/%2n/%n' AS home, \ 'maildir:/stor/real/domains/%d/%2n/%n/Maildir' as mail, \ concat('*:storage=', quota_mb, 'M') as quota_rule, \ 'vmail' AS uid, 'vmail' AS gid \ FROM email WHERE userid = '%n' AND domain = '%d' and deleted=0 iterate_query = SELECT userid as user, domain FROM email where deleted=0

2 2

[Dovecot] Attachment file larger than expected
by Angel L. Mateo 22 Sep '13

22 Sep '13

Hello, I'm using dovecot 2.1.16 with SiS configured as: mail_attachment_dir = <mypath> mail_attachment_min_size = 128 k mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} I'm having a few errors like: Sep 18 15:46:32 myotis51 dovecot: imap(igd): Error: Attachment file /mail/users/attachments/16/b0/16b0b2023a903db14d7f7bcdb1cc04c3f67a4fcb-b32c0a2aa6ae39527c870000a9061514 larger than expected (235866) They are really few and users haven't reported any problem, but the errors are there. In this case, actual file size of the attachment is: amateo_adm@myotis51:~$ sudo stat /mail/users/attachments/16/b0/16b0b2023a903db14d7f7bcdb1cc04c3f67a4fcb-b32c0a2aa6ae39527c870000a9061514 Fichero: «/mail/users/attachments/16/b0/16b0b2023a903db14d7f7bcdb1cc04c3f67a4fcb-b32c0a2aa6ae39527c870000a9061514» Tamaño: 174476 Bloques: 344 Bloque E/S: 4096 fichero regular Dispositivo: fc00h/64512d Nodo-i: 152334473 Enlaces: 2 Acceso: (0600/-rw-------) Uid: ( 2012/ vmail) Gid: ( 204/ vmail) Acceso: 2013-09-18 15:46:20.540728654 +0200 Modificación: 2013-09-18 15:46:20.556728948 +0200 Cambio: 2013-09-18 15:46:20.556728949 +0200 Creación: - and file is identified as: amateo_adm@myotis51:~$ sudo file /mail/users/attachments/16/b0/16b0b2023a903db14d7f7bcdb1cc04c3f67a4fcb-b32c0a2aa6ae39527c870000a9061514 /mail/users/attachments/16/b0/16b0b2023a903db14d7f7bcdb1cc04c3f67a4fcb-b32c0a2aa6ae39527c870000a9061514: MIME entity, UTF-8 Unicode text, with very long lines, with CRLF line terminators Any idea? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

2 1

[Dovecot] problem with dovecot replication over tcp
by nik600 22 Sep '13

22 Sep '13

Dear all i'm following the wiki at this page http://wiki2.dovecot.org/Replication but i'm getting this errors: #doveadm -v sync tcp:dst_hostname #doveadm(root): Fatal: Failed to start dsync-server command: 75 If i look at the dst dovecot.log file i get: Sep 16 12:42:18 auth: Error: passdb(root,xx): Auth client doesn't have permissions to do a PASS lookup: /var/run/dovecot/auth-userdb mode=0666, but not owned by UID 1002(vmail) Sep 16 12:42:18 doveadm(xx,root): Error: user root: Auth PASS lookup failed Sep 16 12:42:18 doveadm(xx,root): Debug: auth input: Sep 16 12:42:18 doveadm(xx,root): Error: dsync-server: /var/run/dovecot/auth-userdb: passdb lookup failed (to see if user is proxied, because doveadm_proxy_port is set) I've also tried to perform a #chown vmail /var/run/dovecot/auth-userdb But the error is the same. If i do the #doveadm user * I get a list of users loaded from the database i've added these line to my dovecot-sql.conf iterate_query = SELECT username from mailbox where active=1 Any idea? Thanks -- /*************/ nik600 http://www.kumbe.it

2 1

[Dovecot] DH Parameter
by Dimi - 22 Sep '13

22 Sep '13

Hi! Is there any possibility to let dovecot serve >1024 Bit DH Parameters at SSL/TLS-connections? Is it possible to replace /var/lib/dovecot/ssl-parameters.ssl with DH-parameter generated by openssl? If not: Are there any plans to implement that? Thank you!

2 1

[Dovecot] Error: BUG: Worker sent reply with id 393, expected 394
by 刘莹莹 22 Sep '13

22 Sep '13

hi dovecot: when i login 100 users with imap protocol ,Someusers appeared a BUG in maillog like : Sep 10 09:05:22 auth(default): Error: BUG: Worker sent reply with id 393, expected 394 Sep 10 09:05:22 auth(default): Error: worker-server(q82(a)t.com,10.12.80.3): Aborted: Worker is buggy Sep 10 09:05:22 auth(default): Error: BUG: Worker sent reply with id 31, expected 32 Sep 10 09:05:22 auth(default): Error: worker-server(q95(a)t.com,10.12.80.3): Aborted: Worker is buggy Sep 10 09:05:22 auth(default): Error: BUG: Worker sent reply with id 32, expected 33 Sep 10 09:05:22 auth(default): Error: worker-server(q99(a)t.com,10.12.80.3): Aborted: Worker is buggy Sep 10 09:05:22 auth(default): Error: BUG: Worker sent reply with id 29, expected 30 Sep 10 09:05:22 auth(default): Error: worker-server(q22(a)t.com,10.12.80.3): Aborted: Worker is buggy Sep 10 09:05:22 imap-login: Info: Internal login failure (auth failed, 1 attempts): user=<q3(a)t.com>, method=PLAIN, rip=10.12.80.3, lip=10.12.80.6

2 1

[Dovecot] Dovecot 2.0, 2.1 and 2.2.5 core dump when Quota Plugin (FS) is enabled
by Harald Strack 22 Sep '13

22 Sep '13

Hi, we are actually deploying a new IMAP-Server, targeting dovecot 2.1.17. Unfortunately, our tests with imaptest did not succeed, were ending in core dumps. We did this test: ./imaptest host=example.com port=143 user=strack pass=secret disconnect_quit no_pipelining mbox=dovecot-crlf That's what we get in dovecot's log: Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22810, session=<EurP9wPmVgCBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22811, session=<mSXT9wPmWgCBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22812, session=<1GbR9wPmWACBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22813, session=<JKbQ9wPmVwCBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22814, session=<USLS9wPmWQCBu/Qf> Sep 10 11:17:04 imap(strack): Panic: file mbox-storage.c: line 712 (mbox_transaction_unlock): assertion failed: (mbox->box.transaction_count > 0 || mbox->mbox_lock_type == F_UNLCK) Sep 10 11:17:04 imap(strack): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0() [0x3c7824660a] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3c78246656] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3c78219eaa] -> /usr/lib64/dovecot/libdovecot-storage.so.0() [0x3c786600c0] -> /usr/lib/dovecot/lib10_quota_plugin.so(+0xb7e7) [0x7f113b5427e7] -> /usr/lib/dovecot/lib10_quota_plugin.so(+0xb218) [0x7f113b542218] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mbox_sync+0x88) [0x3c7865fea8] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mbox_transaction_save_commit_post+0x27) [0x3c78657c07] -> /usr/lib64/dovecot/libdovecot-storage.so.0() [0x3c786a5313] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9f) [0x3c786b3cdf] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a) [0x3c786a4f5a] -> /usr/lib64/dovecot/libdovecot-storage.so.0() [0x3c78660121] -> /usr/lib/dovecot/lib10_quota_plugin.so(+0xb8af) [0x7f113b5428af] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3e) [0x3c7867aa9e] -> dovecot/imap() [0x408b3a] -> dovecot/imap() [0x4088e4] -> dovecot/imap() [0x408d3c] -> dovecot/imap(cmd_append+0x139) [0x408f99] -> dovecot/imap(command_exec+0x3d) [0x41155d] -> dovecot/imap() [0x41046e] -> dovecot/imap() [0x41055a] -> dovecot/imap(client_handle_input+0x135) [0x410785] -> dovecot/imap(client_input+0x5f) [0x4110af] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x3c78252c16] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x3c78253c9f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x3c78252bb8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x3c7823e083] -> dovecot/imap(main+0x29d) [0x4195bd] Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22826, session=<KyHU9wPmWwCBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22827, session=<kxTY9wPmXACBu/Qf> Sep 10 11:17:04 imap-login: Info: Login: user=<strack>, method=PLAIN, rip=10.10.10.31, lip=10.10.10.21, mpid=22828, session=<CXXg9wPmXQCBu/Qf> Sep 10 11:17:05 imap(strack): Fatal: master: service(imap): child 22806 killed with signal 6 (core dumped) Sep 10 11:17:05 imap(strack): Info: Connection closed in=49 out=3832 Sep 10 11:17:05 imap(strack): Info: Connection closed in=49 out=3832 That's the backtrace of the coredump: Loaded symbols for /lib64/libnss_files.so.2 Reading symbols from /lib64/libnss_sss.so.2...Reading symbols from /usr/lib/debug/lib64/libnss_sss.so.2.debug...done. done. Loaded symbols for /lib64/libnss_sss.so.2 Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcc_s.so.1 Core was generated by `dovecot/imap'. Program terminated with signal 6, Aborted. #0 0x0000003bc02328a5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6_4.2.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.4.x86_64 libcom_err-1.41.12-14.el6_4.2.x86_64 libgcc-4.4.7-3.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 openssl-1.0.0-27.el6_4.2.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x0000003bc02328a5 in raise () from /lib64/libc.so.6 #1 0x0000003bc0234085 in abort () from /lib64/libc.so.6 #2 0x0000003c78246618 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:191 #3 0x0000003c78246656 in i_internal_fatal_handler (ctx=0x7fff1c87efa0, format=<value optimized out>, args=<value optimized out>) at failures.c:649 #4 0x0000003c78219eaa in i_panic (format=0x2c1e <Address 0x2c1e out of bounds>) at failures.c:263 #5 0x0000003c786600c0 in mbox_transaction_unlock (box=0x1a51270, lock_id=<value optimized out>) at mbox-storage.c:711 #6 0x00007f95c08c77e7 in quota_mailbox_transaction_rollback (ctx=0x1a589e0) at quota-storage.c:142 #7 0x00007f95c08c7218 in quota_mailbox_sync_notify (box=0x1a51270, uid=0, sync_type=0) at quota-storage.c:301 #8 0x0000003c7865fea8 in mbox_sync (mbox=0x1a51270, flags=<value optimized out>) at mbox-sync.c:2006 #9 0x0000003c78657c07 in mbox_transaction_save_commit_post (_ctx=0x1a53e20, result=<value optimized out>) at mbox-save.c:807 #10 0x0000003c786a5313 in index_transaction_index_commit (index_trans=0x1a5a530, result_r=0x7fff1c87f190) at index-transaction.c:50 #11 0x0000003c786b3cdf in mail_index_transaction_commit_full (_t=0x7fff1c87f1b8, result_r=0x7fff1c87f190) at mail-index-transaction.c:251 #12 0x0000003c786a4f5a in index_transaction_commit (t=<value optimized out>, changes_r=<value optimized out>) at index-transaction.c:132 #13 0x0000003c78660121 in mbox_transaction_commit (t=<value optimized out>, changes_r=<value optimized out>) at mbox-storage.c:729 #14 0x00007f95c08c78af in quota_mailbox_transaction_commit (ctx=0x1a5a390, changes_r=0x7fff1c87f260) at quota-storage.c:124 #15 0x0000003c7867aa9e in mailbox_transaction_commit_get_changes (_t=<value optimized out>, changes_r=0x7fff1c87f260) at mail-storage.c:1515 #16 0x0000000000408b3a in cmd_append_continue_parsing (cmd=0x1a4ea50) at cmd-append.c:284 #17 0x00000000004088e4 in cmd_append_continue_message (cmd=0x1a4ea50) at cmd-append.c:466 #18 0x0000000000408d3c in cmd_append_continue_parsing (cmd=0x1a4ea50) at cmd-append.c:399 #19 0x0000000000408f99 in cmd_append (cmd=0x1a4ea50) at cmd-append.c:519 #20 0x000000000041155d in command_exec (cmd=0x1a4ea50) at imap-commands.c:148 #21 0x000000000041046e in client_command_input (cmd=0x1a4ea50) at imap-client.c:684 #22 0x000000000041055a in client_command_input (cmd=0x1a4ea50) at imap-client.c:735 #23 0x0000000000410785 in client_handle_next_command (client=0x1a4e240) at imap-client.c:776 #24 client_handle_input (client=0x1a4e240) at imap-client.c:788 #25 0x00000000004110af in client_input (client=0x1a4e240) at imap-client.c:827 #26 0x0000003c78252c16 in io_loop_call_io (io=0x1a4cb70) at ioloop.c:379 #27 0x0000003c78253c9f in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:213 #28 0x0000003c78252bb8 in io_loop_run (ioloop=0x1a35670) at ioloop.c:398 #29 0x0000003c7823e083 in master_service_run (service=0x1a35520, callback=<value optimized out>) at master-service.c:544 #30 0x00000000004195bd in main (argc=1, argv=0x1a35370) at main.c:389 That's the full backtrace of the coredump: #0 0x0000003bc02328a5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6_4.2.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.4.x86_64 libcom_err-1.41.12-14.el6_4.2.x86_64 libgcc-4.4.7-3.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 openssl-1.0.0-27.el6_4.2.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt full #0 0x0000003bc02328a5 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x0000003bc0234085 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x0000003c78246618 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:191 backtrace = 0x1a2d3a8 "/usr/lib64/dovecot/libdovecot.so.0() [0x3c7824660a] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3c78246656] -> /usr/lib64/dovecot/libdovecot.so.0() [0x3c78219eaa] -> /usr/lib64/dovecot/libdovecot-stora"... #3 0x0000003c78246656 in i_internal_fatal_handler (ctx=0x7fff1c87efa0, format=<value optimized out>, args=<value optimized out>) at failures.c:649 status = 0 #4 0x0000003c78219eaa in i_panic (format=0x2c1e <Address 0x2c1e out of bounds>) at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fff1c87f070, reg_save_area = 0x7fff1c87efb0}} #5 0x0000003c786600c0 in mbox_transaction_unlock (box=0x1a51270, lock_id=<value optimized out>) at mbox-storage.c:711 mbox = 0x1a51270 __FUNCTION__ = "mbox_transaction_unlock" #6 0x00007f95c08c77e7 in quota_mailbox_transaction_rollback (ctx=0x1a589e0) at quota-storage.c:142 qbox = 0x1a51858 qt = 0x1a66a90 #7 0x00007f95c08c7218 in quota_mailbox_sync_notify (box=0x1a51270, uid=0, sync_type=0) at quota-storage.c:301 qbox = 0x1a51858 uids = <value optimized out> sizep = <value optimized out> i = <value optimized out> count = <value optimized out> size = <value optimized out> #8 0x0000003c7865fea8 in mbox_sync (mbox=0x1a51270, flags=<value optimized out>) at mbox-sync.c:2006 lock_id = 11 ---Type <return> to continue, or q <return> to quit--- ret = 0 __FUNCTION__ = "mbox_sync" #9 0x0000003c78657c07 in mbox_transaction_save_commit_post (_ctx=0x1a53e20, result=<value optimized out>) at mbox-save.c:807 ctx = 0x1a53e20 __FUNCTION__ = "mbox_transaction_save_commit_post" #10 0x0000003c786a5313 in index_transaction_index_commit (index_trans=0x1a5a530, result_r=0x7fff1c87f190) at index-transaction.c:50 t = 0x1a5a390 ret = 0 __FUNCTION__ = "index_transaction_index_commit" #11 0x0000003c786b3cdf in mail_index_transaction_commit_full (_t=0x7fff1c87f1b8, result_r=0x7fff1c87f190) at mail-index-transaction.c:251 t = 0x1a5a530 index = 0x1a4ccc0 index_undeleted = <value optimized out> #12 0x0000003c786a4f5a in index_transaction_commit (t=<value optimized out>, changes_r=<value optimized out>) at index-transaction.c:132 box = 0x1a51270 itrans = 0x0 result = {log_file_seq = 17, log_file_offset = 6032, commit_size = 196, ignored_modseq_changes = 0} ret = <value optimized out> #13 0x0000003c78660121 in mbox_transaction_commit (t=<value optimized out>, changes_r=<value optimized out>) at mbox-storage.c:729 mt = <value optimized out> box = 0x1a51270 lock_id = 11 ret = <value optimized out> #14 0x00007f95c08c78af in quota_mailbox_transaction_commit (ctx=0x1a5a390, changes_r=0x7fff1c87f260) ---Type <return> to continue, or q <return> to quit--- at quota-storage.c:124 qbox = 0x1a51858 qt = 0x1a5a1e0 #15 0x0000003c7867aa9e in mailbox_transaction_commit_get_changes (_t=<value optimized out>, changes_r=0x7fff1c87f260) at mail-storage.c:1515 _data_stack_cur_id = 4 t = 0x1a5a390 ret = <value optimized out> #16 0x0000000000408b3a in cmd_append_continue_parsing (cmd=0x1a4ea50) at cmd-append.c:284 sync_flags = <value optimized out> imap_flags = <value optimized out> changes = {pool = 0x1a61fe0, uid_validity = 1268835246, saved_uids = {arr = {buffer = 0x1a62000, element_size = 8}, v = 0x1a62000, v_modifiable = 0x1a62000}, ignored_modseq_changes = 0, changed = true, no_read_perm = false} msg = <value optimized out> client = 0x1a4e240 ctx = 0x1a4eb48 args = 0x1a59a98 flags_list = <value optimized out> flags = <value optimized out> keywords_list = <value optimized out> keywords = <value optimized out> internal_date_str = <value optimized out> msg = <value optimized out> internal_date = <value optimized out> ret = <value optimized out> timezone_offset = <value optimized out> save_count = <value optimized out> nonsync = <value optimized out> ---Type <return> to continue, or q <return> to quit--- fatal = <value optimized out> __FUNCTION__ = "cmd_append_continue_parsing" #17 0x00000000004088e4 in cmd_append_continue_message (cmd=0x1a4ea50) at cmd-append.c:466 all_written = true client = 0x1a4e240 ctx = 0x1a4eb48 size = 0 ret = <value optimized out> #18 0x0000000000408d3c in cmd_append_continue_parsing (cmd=0x1a4ea50) at cmd-append.c:399 client = 0x1a4e240 ctx = 0x1a4eb48 args = 0x1a59a98 flags_list = 0x0 flags = 0 keywords_list = 0x3c782426aa keywords = 0x0 internal_date_str = <value optimized out> msg = <value optimized out> internal_date = -1 ret = 0 timezone_offset = 0 save_count = <value optimized out> nonsync = true fatal = <value optimized out> __FUNCTION__ = "cmd_append_continue_parsing" #19 0x0000000000408f99 in cmd_append (cmd=0x1a4ea50) at cmd-append.c:519 client = 0x1a4e240 ctx = 0x1a4eb48 mailbox = 0x1a38910 "INBOX" ---Type <return> to continue, or q <return> to quit--- #20 0x000000000041155d in command_exec (cmd=0x1a4ea50) at imap-commands.c:148 hook = 0x1a36cd0 ret = <value optimized out> #21 0x000000000041046e in client_command_input (cmd=0x1a4ea50) at imap-client.c:684 client = 0x1a4e240 command = <value optimized out> __FUNCTION__ = "client_command_input" #22 0x000000000041055a in client_command_input (cmd=0x1a4ea50) at imap-client.c:735 client = 0x1a4e240 command = <value optimized out> __FUNCTION__ = "client_command_input" #23 0x0000000000410785 in client_handle_next_command (client=0x1a4e240) at imap-client.c:776 size = 1320 #24 client_handle_input (client=0x1a4e240) at imap-client.c:788 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #25 0x00000000004110af in client_input (client=0x1a4e240) at imap-client.c:827 cmd = <value optimized out> output = 0x1a4ca88 bytes = 1320 __FUNCTION__ = "client_input" #26 0x0000003c78252c16 in io_loop_call_io (io=0x1a4cb70) at ioloop.c:379 ioloop = 0x1a35670 t_id = 2 #27 0x0000003c78253c9f in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:213 ctx = 0x1a359e0 ---Type <return> to continue, or q <return> to quit--- events = <value optimized out> event = 0x1a35a50 list = 0x1a4cbc0 io = <value optimized out> tv = {tv_sec = 1799, tv_usec = 999319} msecs = <value optimized out> ret = 1 i = <value optimized out> call = <value optimized out> #28 0x0000003c78252bb8 in io_loop_run (ioloop=0x1a35670) at ioloop.c:398 No locals. #29 0x0000003c7823e083 in master_service_run (service=0x1a35520, callback=<value optimized out>) at master-service.c:544 No locals. #30 0x00000000004195bd in main (argc=1, argv=0x1a35370) at main.c:389 set_roots = {0x41c460, 0x0} login_set = {auth_socket_path = 0x1a2d078 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x419710 <login_client_connected>, failure_callback = 0x418f30 <login_client_failed>} service_flags = <value optimized out> storage_service_flags = <value optimized out> username = <value optimized out> c = <value optimized out> That's our configuration: # 2.1.17: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-358.18.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.4 (Santiago) xfs auth_debug = yes auth_verbose = yes debug_log_path = /var/log/dovecot/dovecot-debug.log default_client_limit = 3000 default_process_limit = 8192 disable_plaintext_auth = no log_path = /var/log/dovecot/dovecot.log login_greeting = Server ready. mail_debug = yes mail_location = mbox:/var/spool/fhmrz_imap/% u/dovecot-home:LAYOUT=maildir++:INBOX=/var/spool/mail/% u:INDEX=/var/opt/index/%u/dovecot-indexes:CONTROL=/var/opt/index/% u/dovecot-control mail_plugin_dir = /usr/lib/dovecot mail_plugins = listescape quota mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size flags quota = fs:User quota:user:noenforcing quota_exceeded_message = Quota exceeded, please go to http://www.timbuktu.edu/over_quota_help for instructions on how to fix this. } protocols = imap pop3 service anvil { client_limit = 16744 } service auth { client_limit = 16800 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 8192 process_min_avail = 8 } service imap { process_limit = 16084 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { process_limit = 16084 } ssl_cert = </etc/pki/tls/certs/example.pem ssl_key = </etc/pki/tls/private/example-key.pem userdb { driver = passwd } protocol imap { mail_max_userip_connections = 30 mail_plugins = listescape quota imap_quota } protocol pop3 { mail_max_userip_connections = 10 pop3_uidl_format = %08Xu%08Xv } The problem occurs with dovecot-2.0.9 and dovecot-2.1.17 and also dovecot-2.2.5! From the stacktrace I could see that the problem origins from the quota plugin. Thus, when I disabled the quota plugin the problem went away. The problem did only occur on a high concurrency level on one mbox/folder, running imaptest with only one user and 10 clients. When we run imaptest in a more real-world fashion with several thousand of users and clients=100, the error did not occur. Because of this I am not sure how problematic this bug will be in a production environment? What do you think? Any help is greatly appreciated! br Harald Strack -- Harald Strack, Dipl.Inf.(FH) IT Development ssystems c/o berliner gazette Danziger Str. 31 10435 Berlin Tel: +49 30 2023 6071 - 1 http://www.ssystems.de

2 1