- dovecot - dovecot.org

Thunderbird not able to edit sieve Message Filters
by Peter Bell 22 Aug '14

22 Aug '14

I'm not sure when this problem first occurred - it is many months since I last editted a sieve filter. I am certainly several versions of Thunderbird and, probably, a few versions of Dovecot on from when I last editted a sieve filter. However, I do know that I have done so in the past. I have just built Dovecot 2.2.13 and Pigeonhole 0.4.3, installed and running on Slackware 14.1 x64. I am running an up-to-date version of Thunderbird (31.0) on Arch Linux x64. The sieve filters are definitely being applied to incoming mail, and working perfectly well. However, when I attempt to edit the Sieve Filters (from Thunderbird Tools->Sieve Message Filters...), the Sieve Filters tab is displayed, but not populated (eg, the Server Sieve Capabilities are not displayed, and there is an endless 'hourglass' displayed in the main mailfilters pane. Does anyone recognise the symptoms and can advise how to resolve the problem, or suggest what logs I should examine/post in order to identify the cause? -- ------------------------------------------------------------------- Peter Bell Mob: +63 (0) 9287 340 343 Tagum City, Philippines. Tel: +63 (0) 84 216 9037 email/msn: peter(a)bellfamily.org.uk Tel: +63 (0) 84 308 0002

5 5

Assertion in istream.c::167 in POP3 mode when maildir content has wrong S value (using 2.2.13)
by Rolf Eike Beer 21 Aug '14

21 Aug '14

Hi, the affected machine is running 2.2.13 on Gentoo x86_64. For me it looks like the old size is still in some struct or whereever even after the file has been renamed. I see that there are many commits following 2.2.13, is that possibly already fixed or do I need to dig in the code? The original problem seems to be that maildrop uses the file size before piping the mail into spamassassin even if the mail is bigger afterwards. Not relevant here, but in case anyone wonders how this happens. Greetings, Eike Error: Cached message size smaller than expected (81118 < 81308) Error: Maildir filename has wrong S value, renamed the file from /var/vpopmail/domains/example.org/user/.maildir/cur/1408452738.16341.mail,S=81118:2, to /var/vpopmail/domains/example.org/user/.maildir/cur/1408452738.16341.mail,S=81308:2, Error: Corrupted index cache file /var/vpopmail/domains/example.org/user/.maildir/dovecot.index.cache: Broken physical size for mail UID 65 Panic: file istream.c: line 167 (i_stream_read): assertion failed: (old_size <= _stream->pos - _stream->skip) Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x6bbef) [0x7f50652a4bef] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6bc4e) [0x7f50652a4c4e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f506525e2be] -> /usr/lib64/dovecot/libdovecot.so.0(i_stream_read+0x214) [0x7f50652ad904] -> /usr/lib64/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f50652adfed] -> /usr/lib64/dovecot/libdovecot.so.0(message_get_body_size+0x9d) [0x7f5065299b3d] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_mail_init_stream+0x200) [0x7f506556f700] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x2dd0f) [0x7f506553cd0f] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_get_stream+0x4b) [0x7f506554915b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x2dae7) [0x7f506553cae7] -> dovecot/pop3(client_create+0x56d) [0x40622d] -> dovecot/pop3() [0x404ea3] -> dovecot/pop3() [0x40508f] -> /usr/lib64/dovecot/libdovecot.so.0(+0x28346) [0x7f5065261346] -> /usr/lib64/dovecot/libdovecot.so.0(+0x28841) [0x7f5065261841] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) [0x7f50652b58de] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7f50652b68b7] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f50652b5969] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f50652b59e8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f50652635a3] -> dovecot/pop3(main+0x233) [0x404bf3] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5064eb6db5] -> dovecot/pop3() [0x404d49]

1 1

About network throughput between Director and backends
by Listas＠Adminlinux 21 Aug '14

21 Aug '14

I plan to migrate my Dovecot IMAP servers to a clustered structure. I'm planning a cluster with high availability and load balancing. I know I need instances of Dovecot Proxy / Director. But these instances can be on another network? There is need for high network throughput between the Director and the backends? For me it would be cheaper to run Director in VM's at Amazon AWS for example. Thanks! -- Thiago Henrique

1 0

doveadm proxy list
by Claus 21 Aug '14

21 Aug '14

Hi, i have configured my dovecot-proxy with "shutdown_clients = no" and "service_count = 1" for a "default_process_limit = 5000" and "default_client_limit = 25000". I don't know if this is a prefered setup for so many users/concurrent sessions but so i can reload my dovecot without disconnecting all sessions. The main problem are pop3 sessions - if i kill them, the mailbox remains locked and the client gets an error. Back to my problem: If i reload dovecot the "doveadm proxy list" shows me the list of the new processes. Is there any possibility to see the stats of the old sessions (user,source,destination) ? greet's Claus PS.: Could someone explain me why i have to set default_client_limit to 5x default_process_limit?

2 2

rebuilding indexes for dovecot pop3
by Arkadiusz Miśkiewicz 21 Aug '14

21 Aug '14

Hi. doveadm -D fetch -u login \ 'imap.envelope imap.bodystructure size.virtual date.received' '*' according to mail archives allows to regenerate indexes and precache specified headers for imap usage. What headers are needed for dovecot pop3 daemon? I have one server where dovecot works for imap only. For pop3 there is different software being used. I want to migrate to dovecot pop3 but I need to precache and regenerate indexes for dovecot pop3. Note, some users used pop3 only, so dovecot imap had no chance to generate any indexes. Thanks, -- Arkadiusz Miśkiewicz, arekm / maven.pl

2 2

Re: Frequent error: unknown namespace Trash
by Michael Ludwig 21 Aug '14

21 Aug '14

Hi Gedalya, I am very sorry for not having replied on-list. That was not my intention! I am not used to this behaviour of mailing lists, normally there is a reply-to field which points to the mailing list mail address. So do I have to use MAILBOX.Trash in 90-quota.conf ?? Or INBOX.Trash ? Cheers, Michael 2014-08-12 19:33 GMT+02:00 Gedalya <gedalya(a)gedalya.net>: > On 08/12/2014 09:28 AM, Michael Ludwig wrote: > >> prefix = INBOX. >> separator = . >> > Maybe that's it? It's therefore MAILBOX.Trash, not just Trash. > > You're supposed to reply on-list! Don't reply to gedalya(a)gedalya.net, > reply to dovecot(a)dovecot.org. If you want you can also cc the message > author, some people do that. > >

3 3

Fatal: lazy_expunge: Unknown namespace: '.EXPUNGED/'
by Philipp Faeustlin 21 Aug '14

21 Aug '14

Hello List, with dovecot version 2.2.13.9, I actually have a problem using lazy_expunge and acl together. Shared/Public Folders work fine with lazy_expunge disabled. If lazy_expunge is enabled and userA got a shared folder form userB, then there is the error shown in subject. My test case to reproduce the error. doveadm mailbox list -u userA@domain --> Output Fine. doveadm acl set -u userB@domain INBOX user=userA@domain lookup read write write-seen write-deleted insert expunge create --> Fine. doveadm mailbox list -u userA@domain --> Some output and then: --> doveadm(userA@domain): Fatal: lazy_expunge: Unknown namespace: '.EXPUNGED/' I think this is a bug in dovecot, can someone of the dovecot developers or someone else confirm this? If it is not please show me the fault in my configuration. In May 2014 there was a thread with the same issue on this mailing list, but with no solution. http://dovecot.org/pipermail/dovecot/2014-May/096355.html Best regards Philipp Fäustlin --------------------------------------------- # 2.2.13.9 (19befd72d9d8): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 Scientific Linux release 6.5 (Carbon) auth_debug = yes auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %Ln auth_verbose = yes listen = * mail_debug = yes mail_gid = vmail mail_location = maildir:~/maildir mail_plugins = " acl lazy_expunge mail_log notify zlib notify replication" mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/maildir:INDEX=%h/Shared/%%u:CONTROL=%h/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } namespace { location = maildir:/vmail/%d/public:INDEXPVT=~/index/%d/public prefix = Groups/ separator = / subscriptions = no type = public } namespace { hidden = yes list = no location = maildir:~/expunged prefix = .EXPUNGED/ separator = / subscriptions = no type = private } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db lazy_expunge = .EXPUNGED/ lazy_expunge_only_last_instance = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:mailhost1.rz.uni-hohenheim.de:12345 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve replication_dsync_parameters = -d -l 30 -U service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 12345 ssl = yes } } service imap-login { process_min_avail = 16 service_count = 0 } service imap-postlogin { executable = script-login /usr/local/bin/postlogin.sh user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 8192 } service lmtp { unix_listener /var/spool/postfix/private/lmtp-dovecot { group = postfix user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { process_min_avail = 16 service_count = 0 } service pop3 { process_limit = 8192 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0660 } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_client_ca_file = /etc/openldap/cacerts/chain.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = " acl lazy_expunge mail_log notify zlib notify replication sieve" } protocol imap { mail_plugins = " acl lazy_expunge mail_log notify zlib notify replication imap_acl imap_zlib" }

1 1

New user mailbox permission
by Gionatan Danti 21 Aug '14

21 Aug '14

Hi all, I have a problem with my Dovecot installation. First, some informations: - OS: CentOS 6.5 x86_64 - Dovecot: dovecot-2.0.9-7.el6.x86_64 Config: mail_location = sdbox:~/dbox user_query = SELECT username, password, 'vmail' as uid, 'vmail' as gid, '/var/vmail/%d/%n' as home, CONCAT('*:storage=', (quota DIV 1000)) as quota_rule FROM mailbox WHERE username='%u' What I want to do is to create a backup user to rsync the mailstore to a remote location. If possible, I don't want to use the root user for this operation. I already added the backup user to "vmail" group. The problem is that when a new user is created, it's base directory permission are 0700 (vmail:vmail). From what I read here (http://wiki2.dovecot.org/SharedMailboxes/Permissions, "Permissions to new /domain/user directories"), simply using the setgid bit is not sufficient because I am not using an absolute path for mail_location. My questions are: 1) all I need to do is to switch to an absolute mail_location? 2) it is possibile to solve my problem _without_ altering mail_location? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 2

How to configure read-only access during a migration?
by Jeffrey Tunison 20 Aug '14

20 Aug '14

Hi. I would appreciate your advice about these TWO PROBLEMS: ======================================================= Find the simplest way to A) configure a Dovecot 2.2.10 IMAP server to access (existing) email users' (mbox-format) mail folders ****in read-only mode.**** B) Do the above while allowing a master_user to access mail on behalf of real users. (I've got master_user access working in read-write mode.) Simplest because while I'm proficient at UNIX systems administration, I'm under time pressure and dovecot has a lot of configuration flexibility. Identifying the right way to do this is proving time- consuming. Namespaces? ACL's? passdb's? I thought it was time to bring in the dovecot wizards. You! GOAL: ======================================================= Without knowing or changing users' passwords, migrate several hundred users of dovecot IMAP to MS Exchange using the commercial product Transend. I'm manging the dovecot side, someone else is doing the Transend/Exchange piece. Migration would occur in batches due to slowness of the Transend side when dealing with users with email collections over a few gigabytes. I have a production IMAP server (dovecot 1.0.13 on 32-bit RHEL 4.9). Let's call it PROD for short. Users INBOXes are stored locally on PROD. Their other mail folders are stored on a second machine and accessed via NFS. Those other folders contain the great majority of the email to be migrated. Let's call the NFS server MIGR for short. On MIGR, I've set up a migration IMAP server (dovecot 2.2.10 on 64-bit RHEL 5.10). MIGR is a dedicated NFS server with newer hardware than PROD. It has an unused Ethernet NIC I can tell Transend to use. Both PROD and MIGR use NIS for authentication. Therefore PAM is consulted for passdb information, normally. No SQL db's used. I set up the second server (MIGR) for several reasons: 1) there are some users mail folders larger than the 32-bit file size limit found on the PROD server. 2) the bulk of the email is stored locally on the MIGR server. 3) the version of dovecot is much newer and therefore more likely to have fewer bugs and more configuration options. 4) I don't want to modify the production environment except where really necessary. Pointing Transend at MIGR instead of PROD seems like it will be faster and better. Telling end-users to use MIGR instead of PROD would allow them to verify that their email came across okay (into MS Exchange) without being able to modify the original copy of email (in Dovecot). In the days and weeks after migration. I may need to re-enable write access on PROD for certain users if the migration does not succeed. Transend has been dying on a number of test users despite tweaking its parameters in consultation with the vendor. "Corrupt" messages seem to be the issue. Users (or sysadmins) might be required to identify and delete the offending messages before we retry their migration. (Which folders and which messages are causing problems will hopefully be discernible in Transend logs.) Ha! If you've read this far, then I'd also say that I welcome any advice on changes to increase performance on MIGR, such as whether trying to minimize indexing as I've done is useful. Current configurations of the two servers follow. =========================================================================== PRODUCTION: =========================================================================== # 1.0.13: /etc/dovecot.conf log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot.log protocols: imaps imap listen: *:143 ssl_listen: *:993 ssl_cert_file: /etc/pki/dovecot/certs/dovecot.pem ssl_key_file: /etc/pki/dovecot/private/dovecot.key login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/imap-login login_processes_count: 20 login_max_processes_count: 500 max_mail_processes: 1524 first_valid_uid: 100 mail_location: mbox:~/Mail:INBOX=/var/mail/%u:INDEX=/var/spool/dovecot-indexes/%u/ mbox_read_locks: dotlock mbox_write_locks: dotlock mbox_dotlock_change_timeout: 60 mbox_min_index_size: 2048 namespace: type: private separator: / inbox: yes namespace: type: private separator: / prefix: Mail/ hidden: yes namespace: type: private separator: / prefix: ~/Mail/ hidden: yes namespace: type: private separator: / prefix: mail/ hidden: yes namespace: type: private separator: / prefix: ~/mail/ hidden: yes auth default: master_user_separator: * verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot.transend master: yes passdb: driver: shadow passdb: driver: pam userdb: driver: passwd =========================================================================== MIGRATION: =========================================================================== # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-348.18.1.el5 x86_64 Scientific Linux release 5.10 (Boron) auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_verbose = yes auth_verbose_passwords = plain first_valid_uid = 100 log_path = /var/log/dovecot.log mail_debug = yes mail_location = mbox:~/Mail:INBOX=/var/spool/concmail/%u:INDEX=MEMORY mbox_min_index_size = 50048 B mbox_write_locks = fcntl namespace four { hidden = yes location = prefix = ~/mail/ separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } namespace one { hidden = yes location = prefix = Mail/ separator = / } namespace three { hidden = yes location = prefix = mail/ separator = / } namespace two { hidden = yes location = prefix = ~/Mail/ separator = / } passdb { args = /etc/dovecot/dovecot.transend driver = passwd-file master = yes } passdb { driver = pam } protocols = imap service imap-login { process_min_avail = 15 service_count = 10 } service imap { process_limit = 2048 } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 1000 }

1 0

LMTP re-reading all messages
by Florent Bautista 20 Aug '14

20 Aug '14

Hi all, I have a little problem with Dovecot 2.1.17 and MailDir accounts. I was using a shared file system (MooseFS), and all was pretty stable. Now I'm switching to CephFS, and Dovecot LMTP have problem to deliver mails. Very often, on large accounts, LMTP processes are running for a few hours before delivering the message. When I strace it, I can see that they are reading (open and stat operations) ALL the messages of the account. I was thinking to an index problem, but I think that after hundreds of delivered mails, indexes are rebuilded and problem should be solved. But not. The same way, If user connects to IMAP, a lot of IMAP processes get stuck on "getquotaroot" command (multiple processes for a single account). The previous FS didn't support mmap, so I disabled it. The new one supports it, so I enabled it, but it does not solve the problem. Do you have an idea of why Dovecot (LMTP or IMAP) spend a lot of time on large accounts ? Thank you a lot. Florent

2 2

antispam plugin crash
by Thomas Witt 20 Aug '14

20 Aug '14

Hi, the dovecot antispam plugin crahses here on a nullpointer dereference in antispam_copy() at mailbox.c The crash tracks back to a call of antispam_classify_copy(asms->box_class, asmb->box_class) At that point in time asmb is set to 0x0, causing the process to crash with SIGSEGV. Regards, Thomas GDB output: Program received signal SIGSEGV, Segmentation fault. antispam_copy (ctx=0x14ed9f0, mail=0x14c6cc0) at mailbox.c:107 107 enum mailbox_copy_type copy_type = (gdb) bt #0 antispam_copy (ctx=0x14ed9f0, mail=0x14c6cc0) at mailbox.c:107 #1 0x00007fdadb627fcd in mailbox_copy () from /usr/lib64/dovecot/libdovecot-storage.so.0 #2 0x00007fdada6cdba0 in act_store_execute () from /usr/lib64/dovecot/libdovecot-sieve.so.0 #3 0x00007fdada6c59d9 in sieve_result_execute () from /usr/lib64/dovecot/libdovecot-sieve.so.0 #4 0x00007fdada6d4a2a in sieve_execute () from /usr/lib64/dovecot/libdovecot-sieve.so.0 #5 0x00007fdada9212cb in lda_sieve_deliver_mail () from /usr/lib64/dovecot/lib90_sieve_plugin.so #6 0x00007fdadb8cae1a in mail_deliver () from /usr/lib64/dovecot/libdovecot-lda.so.0 #7 0x0000000000405ebd in client_input_data_handle () #8 0x00007fdadb34e9bf in io_loop_call_io () from /usr/lib64/dovecot/libdovecot.so.0 #9 0x00007fdadb34f8b7 in io_loop_handler_run_internal () from /usr/lib64/dovecot/libdovecot.so.0 #10 0x00007fdadb34ea29 in io_loop_handler_run () from /usr/lib64/dovecot/libdovecot.so.0 #11 0x00007fdadb34eaa8 in io_loop_run () from /usr/lib64/dovecot/libdovecot.so.0 #12 0x00007fdadb2fe803 in master_service_run () from /usr/lib64/dovecot/libdovecot.so.0 #13 0x0000000000404995 in main () (gdb) p asms $3 = (struct antispam_mailbox *) 0x0

3 9

Panic: file istream.c: line 167 with 2.2.13 + istream.c/.h from dovecot-2-2-dffc6dd2be0d
by Toni Mattila 20 Aug '14

20 Aug '14

Hi, Getting following panics with traces: Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Cached message size smaller than expected (1782 < 1872) Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Maildir filename has wrong S value, renamed the file from /path/cencored/Maildir/cur/1201713258.3136.host.cencored.fi,S=1782:2,S to /path/cencored/Maildir/cur/1201713258.3136.host.cencored.fi,S=1872:2,S Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Corrupted index cache file /path/cencored/Maildir/dovecot.index.cache: Broken physical size for mail UID 5758 Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: read(/path/cencored/Maildir/cur/1201713258.3136.host.cencored.fi,S=1782:2,S) failed: Invalid argument Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: read(/path/cencored/Maildir/cur/1201713258.3136.host.cencored.fi,S=1782:2,S) failed: Invalid argument (uid=5758) Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Cached message size smaller than expected (1781 < 1869) Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Maildir filename has wrong S value, renamed the file from /path/cencored/Maildir/cur/1201767850.29173.host.cencored.fi,S=1781:2,S to /path/cencored/Maildir/cur/1201767850.29173.host.cencored.fi,S=1869:2,S Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Corrupted index cache file /path/cencored/Maildir/dovecot.index.cache: Broken physical size for mail UID 5767 Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: read(/path/cencored/Maildir/cur/1201767850.29173.host.cencored.fi,S=1781:2,S) failed: Invalid argument Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: read(/path/cencored/Maildir/cur/1201767850.29173.host.cencored.fi,S=1781:2,S) failed: Invalid argument (uid=5767) Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Cached message size smaller than expected (9252 < 9346) Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Maildir filename has wrong S value, renamed the file from /path/cencored/Maildir/cur/1205131753.12346.host.cencored.fi,S=9252:2,RS to /path/cencored/Maildir/cur/1205131753.12346.host.cencored.fi,S=9346:2,RS Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Corrupted index cache file /path/cencored/Maildir/dovecot.index.cache: Broken physical size for mail UID 6296 Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Panic: file istream.c: line 167 (i_stream_read): assertion failed: (old_size <= _stream->pos - _stream->skip) Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x575c1f] -> /usr/lib/dovecot/libdovecot.so.0 [0x575c9d] -> /usr/lib/dovecot/libdovecot.so.0 [0x575514] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x13b) [0x57f99b] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x2b) [0x57fbfb] -> /usr/lib/dovecot/libdovecot.so.0 [0x565b5e] -> /usr/lib/dovecot/libdovecot.so.0 [0x567469] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x43) [0x565cd3] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_body+0x2c) [0x565e4c] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x944dd2] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_parts+0x6a) [0x945c7a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_parts+0x16) [0x91ad76] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x94aa7c] -> /usr/lib/dovecot/libdovecot-storage.so.0 [0x94ba89] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblo Jul 6 08:11:52 dovecot: imap(user(a)domain.tld): Fatal: master: service(imap): child 31268 killed with signal 6 (core dumps disabled) These can of course be fixed with maildir-size-fix.pl and deleting dovecot index files, but there are thousands like these. Dovecot 1.2 didn't have any issues with these old maildirs transfered over from courierimap. Best Regards, Toni Mattila

2 9

Dovecot v2.2 FTS is not indexing "text/html" emails...
by Akash 20 Aug '14

20 Aug '14

Hi, I am not sure its intended or a fault in the newest Dovecot versions. I have been using Dovecot v1.2.15 on Debian squeeze and FTS is working as expected. When I search a quoted string "very good", I get 107 results including plain and HTML emails which have this phrase. In order to compare the benefits of lucene over squat, I recently started testing dovecot v2.2.13 on Debian Sid with the same maildir content. But now the same search "very good" yielded just 8 results. I thought it could be some problem with lucene so I tried switching to squat and got 107 results again. After this I deleted the old squat search index files created by v1.2.15 and re-indexed the mail-box by using doveadm index command. Now the same squat search is giving 8 results just as lucene. So I have realized that its not a problem with just lucene but FTS in newer dovecot isn't indexing those emails which have Content-type as text/html. Thus if a mail is like this: Content-Type: text/html <b>He is very good.</b> It isn't shown in search by the squat indexes created using dovecot v2.2.13. I have done further testing on some sample emails which confirmed this behavior. Why is this so? -Regards, Akash

2 2

2.2.13 + hg: Panic: file ioloop.c: line 39 (io_add_file): assertion failed: (fd >= 0)
by Toni Mattila 20 Aug '14

20 Aug '14

Hi, Found this in logfiles: Jul 25 14:12:38 dovecot: pop3(cen(a)so.red): Error: UIDL: File name lookup failed: Message was expunged Jul 25 14:12:38 dovecot: pop3(cen(a)so.red): Disconnected: POP3 UIDLs couldn't be listed top=187/356348, retr=0/0, del=0/1207, size=789346199 Jul 25 14:12:38 dovecot: pop3(cen(a)so.red): Panic: file ioloop.c: line 39 (io_add_file): assertion failed: (fd >= 0) Jul 25 14:12:38 dovecot: pop3(cen(a)so.red): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0xbdfc1f] -> /usr/lib/dovecot/libdovecot.so.0 [0xbdfc9d] -> /usr/lib/dovecot/libdovecot.so.0 [0xbdf514] -> /usr/lib/dovecot/libdovecot.so.0 [0xbf162d] -> /usr/lib/dovecot/libdovecot.so.0(io_add_istream+0x3f) [0xbf272f] -> dovecot/pop3 [0x804d149] -> /usr/lib/dovecot/libdovecot.so.0 [0xbff17a] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0xbf1d88] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xff) [0xbf375f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x1e) [0xbf1e2e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0xbf1eb8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0xb9842d] -> dovecot/pop3(main+0x2a9) [0x804b3b9] -> /lib/i686/nosegneg/libc.so.6(__libc_start_main+0xdc) [0x6f7dec] -> dovecot/pop3 [0x804aeb1] Jul 25 14:12:38 dovecot: pop3(cen(a)so.red): Fatal: master: service(pop3): child 27510 killed with signal 6 (core dumped) (gdb) bt #0 0x00b1a402 in __kernel_vsyscall () #1 0x0070af30 in raise () from /lib/i686/nosegneg/libc.so.6 #2 0x0070c911 in abort () from /lib/i686/nosegneg/libc.so.6 #3 0x00bdfc14 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:193 #4 0x00bdfc9d in i_internal_fatal_handler (ctx=0xbf82cd34, format=0xc0c784 "file %s: line %d (%s): assertion failed: (%s)", args=0xbf82cd54 "\202\232�") at failures.c:657 #5 0x00bdf514 in i_panic (format=0xc0c784 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 #6 0x00bf162d in io_add_file (fd=-1, condition=IO_READ, source_linenum=816, callback=0x804d150 <client_input>, context=0x83ba628) at ioloop.c:40 #7 0x00bf272f in io_add_istream (input=0x83ba960, source_linenum=816, callback=0x804d150 <client_input>, context=0x83ba628) at ioloop.c:86 #8 0x0804d149 in client_output (client=0x83ba628) at pop3-client.c:815 #9 0x00bff17a in stream_send_io (fstream=0x83baa48) at ostream-file.c:468 #10 0x00bf1d88 in io_loop_call_io (io=0x83baaf8) at ioloop.c:441 #11 0x00bf375f in io_loop_handler_run_internal (ioloop=0x83a5498) at ioloop-epoll.c:220 #12 0x00bf1e2e in io_loop_handler_run (ioloop=0x83a5498) at ioloop.c:488 #13 0x00bf1eb8 in io_loop_run (ioloop=0x83a5498) at ioloop.c:465 #14 0x00b9842d in master_service_run (service=0x83a53c0, callback=0x804b4e0 <client_connected>) at master-service.c:566 #15 0x0804b3b9 in main (argc=Cannot access memory at address 0x0 ) at main.c:277 (gdb) Thanks, Toni

2 1

[Patch] mail_log plugin supports remote, local ip addresses
by Vitaliy T 20 Aug '14

20 Aug '14

Hello, In attached file you will find patch for mail_log plugin to output local, remote ip addresses. It may be usefull for somebody. To enable support for dovecot 2.2.x versions change the line 'include "net.h"' instead of 'include "network.h"'. Thank you. P.S. Sorry, for previous empty message. -- With Best Regards, Vitaliy T.

2 1

auth_username_translation and LTMP problem
by Arkadiusz Miśkiewicz 20 Aug '14

20 Aug '14

auth_username_translation seems to be appliet on RCPT TO address of LTMP transport. Why dovecot is doing that? And better question - is there a way to disable auth_username_translation for LTMP but leave enabled for the rest (imap, pop3 etc) ? Background: I'm doing auth_username_translation = @= to allow logins like aaa(a)bbb.pl to be internally translated to aaa=bbb.pl. That works fine. Now my exim delivers mail to dovecot using LTMP and it does translation or it own, so it does: RCPT TO: <aaa=bbb.pl(a)mymbox.pl> Unfortunately looks like dovecot it doing translation one more time and looking in user database for aaa=bbb.pl=mymbox.pl where such user doesn't exist. Only aaa=bbb.pl user exists. Thanks, -- Arkadiusz Miśkiewicz, arekm / maven.pl

2 2

Mail Archiving/Lucene Indexes/mbox
by Larry Rosenman 20 Aug '14

20 Aug '14

I have a script (available on demand) that moves all my mail for a given month to a separate NAMESPACE (#ARCHIVE). When I do that, the lucene-indexes folder in the PRIMARY namespace seems to get cleaned out, and NOTHING gets put in the lucene-indexes folder (although it gets created) in the #ARCHIVE namespace. Is this to spec? the archiving is done via doveadm move commands. Thanks! -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 108 Turvey Cove, Hutto, TX 78634-5688

2 3

Segfault with indexer-worker using Tika/Solr
by G H 20 Aug '14

20 Aug '14

Using Dovecot 2.2.13 with Tika 1.5 on Centos 5 64bit Tika is throwing an exception for some reason, which seems to be causing Dovecot's indexer to segfault. I'm using mdbox/gzipped compressed mail and private indexes for the shared folder. I'm importing mail via formail/dovecot-lda: formail -s /usr/libexec/dovecot/dovecot-lda -d admin -m Public/TestSubFolder < /tmp/Inbox.mbox dovecot -n and the GDB backtrace are attached. # find /var/mail/ -maxdepth 2 /var/mail/_public /var/mail/_public/dovecot.mailbox.log /var/mail/_public/dovecot-uidvalidity /var/mail/_public/dovecot-uidvalidity.53ecade1 /var/mail/_public/storage /var/mail/_public/subscriptions /var/mail/_public/mailboxes /var/mail/admin /var/mail/admin/dovecot-uidvalidity.53ecade2 /var/mail/admin/dovecot.mailbox.log /var/mail/admin/public /var/mail/admin/dovecot-uidvalidity /var/mail/admin/storage /var/mail/admin/subscriptions /var/mail/admin/mailboxes

2 1

BUG: Solr FTS engine appends domain name to index records
by G H 20 Aug '14

20 Aug '14

The Solr FTS engine appends domain names to the "user" and "id" fields inside Solr when indexing. However, when an IMAP search is performed, the domain name is not appended, resulting in 0 results, 100% of the time. You can reproduce this by setting up dovecot to have user names like "admin", "james", etc (as opposed to admin(a)example.com) Then import some mail (ie: "dovecot-lda -d admin < /tmp/mymail.txt"). Now attempt to search for that mail and it will not be returned as a result. You can see in the Solr GUI (http://xxxx:8983/solr/) that record is indexed as: { "uid": 1, "box": "7dff0b0d4f84df53f25300003d823887", "user": "admin(a)example.com", "id": "1/7dff0b0d4f84df53f25300003d823887/admin(a)example.com", "_version_": 1475511406809841700 } However, the IMAP search is sent to Solr performed with user:"admin" instead of user:"admin(a)example.com". Either the Solr FTS plugin needs to not append the domain name or the search does need to do so.

2 2

Trouble getting public folders working
by Bjarne Mark Kondrup 20 Aug '14

20 Aug '14

Hi i have trouble getting my public folders working properly. I have a few issues atm with them. First off i'm using version 2.0.9 as it came with CentOS and would greatly prefer to keep this version. Here is my dovecot.conf file # dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.23.3.el6.centos.plus.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no first_valid_uid = 101 last_valid_uid = 101 login_greeting = Dovecot ready. CEGO webmail server mail_debug = yes mail_gid = vmail mail_location = maildir:/home/vmail/domains/%d/%n mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl namespace { inbox = yes location = prefix = separator = / type = private } namespace { location = maildir:/home/vmail/shared:LAYOUT=fs:INDEX=/home/vmail/shared/ prefix = shared/ separator = / subscriptions = no type = public } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { sieve = /home/vmail/domains/%d/%n/dovecot.sieve sieve_dir = /home/vmail/domains/%d/%n/sieve_scripts sieve_global_dir = /etc/dovecot/sieve/global/ } protocols = imap lmtp sieve service auth { unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = no ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } valid_chroot_dirs = /home/vmail/domains verbose_ssl = yes protocol lmtp { log_path = /var/log/dovecot-deliver.log mail_plugins = quota sieve sieve acl postmaster_address = postmaster(a)cegotest.dk } protocol sieve { info_log_path = /var/log/dovecot-sieve.log log_path = /var/log/dovecot-sieve-errors.log managesieve_implementation_string = dovecot managesieve_max_line_length = 65536 } And my dovecot-ldap file hosts = localhost dn = cn=dovecot,dc=cegotest,dc=dk dnpass = password auth_bind = yes ldap_version = 3 base = o=hosting,dc=cegotest,dc=dk deref = never scope = subtree user_filter = (&(objectClass=JammMailAccount)(mail=%u)(accountActive=TRUE)(delete=FALSE)) pass_attrs = mail,userPassword pass_filter = (&(objectClass=JammMailAccount)(mail=%u)(accountActive=TRUE)(delete=FALSE)) default_pass_scheme = CRYPT Here is my current issues. 1. I cannot get shared folders to work with pr. user settings. What i want is a public folder that all my users can see but each user marks mail as read individually. Atm no matter how i set it up if user1 marks a mail as read, then user2 will also see that mail as read :( 2. ACL to control the permissions. Right now all users can move mail and remove mail from the shared folders. I want people (except a few specific users) to be read only on this public folders. 3. Need to know if i'm creating the folders correctly. Right now i'm under my shared folder directory creating shared_folder and 3 subfolders beneath it (cur, new and tmp). After this it shows up in eg. roundcube and i can subscribe to it. I have read all the documentation and tried all the solutions i can. Mostly moving the index and/or config files to seperate dirs and from my understanding that should give each user their own index file marking what mails have been read ect. But this doesn't happen. Hope you guys are able to help me out a little here. -Bjarne

2 1

Huge difference between the lucene index size created by v2.1 and v2.2
by Akash 20 Aug '14

20 Aug '14

Hi everyone, While examining dovecot versions v2.1 and v2.2 for their lucene search performances, I have noticed a huge difference in the index sizes created by them. Both versions were compiled on same system, against same libclucene, same configure options and were used with same dovecot.conf during run-time. I deleted the lucene-indexes folder and dovecot* files in the Maildir prior to indexing with both versions. The tests were performed on a untouched mail archive folder containing 300000 mails without any dovecot* files in it: root@server:/home/admin/mails/.Archive# ls -l total 78640 drwx------ 2 2500 2500 14098432 Aug 17 22:16 cur drwx------ 2 2500 2500 12435456 Jul 30 09:46 new drwx------ 2 2500 2500 4096 Aug 2 13:02 tmp The command used was: doveadm -v index -u admin Archive After search indexing on v2.1 resulted in: root@server:/home/admin/mails# ls -lh lucene-indexes total 390M -rw------- 1 2500 2500 390M Aug 18 07:03 _25.cfs -rw------- 1 2500 2500 20 Aug 18 07:03 segments.gen -rw------- 1 2500 2500 46 Aug 18 07:03 segments_4d Whereas dovecot v2.2 resulted in: root@server:/home/admin/mails# ls -lh lucene-indexes total 1.5G -rw------- 1 2500 2500 1.5G Aug 18 06:41 _5g.cfs -rw------- 1 2500 2500 20 Aug 18 06:41 segments.gen -rw------- 1 2500 2500 46 Aug 18 06:41 segments_az 390M vs 1.5G. That is a huge difference in size. Why is that? Thanks in advance. -Regards, Akash

2 5

Problems with dovecot 2.2.13 and monit
by Hanno Böck 20 Aug '14

20 Aug '14

Hello, When I upgraded my servers to dovecot 2.2.13 the monitoring tool monit started to send out warnings that it couldn't reach my imap/pop3 servers through ssl any more. The same problem didn't happen on non-ssl-connections. According to people on the monit list this is likely a dovecot issue: https://lists.gnu.org/archive/html/monit-general/2014-06/msg00031.html Let me quote: > the root cause of the error is, that dovecot 2.2.13 closes the > connection if SSL is used in response to LOGOUT command instead of > sending usual response. When no SSL is enabled, dovecot responses to > LOGOUT command normally. [...] > According to RFC 3501 (http://tools.ietf.org/html/rfc3501) LOGOUT is > any-state command, where the server MUST send response before closing > the connection: http://tools.ietf.org/html/rfc3501#section-3.4 > > => the problem is caused by dovecot 2.2.13 bug ... its behaviour is > inconsistent (LOGOUT in non-authenticated state works per RFC > requirement if no SSL is used and doesn't conform to RFC if SSL is > used). It is possible that the problem is related to their DoS-attack > modification, which has most probably unexpected side-effect. Maybe this is related to the DDoS-protection measures that have been added in dovecot 2.2.13. Would apprechiate if someone could have a look. cu, -- Hanno Böck http://hboeck.de/ mail/jabber: hanno(a)hboeck.de GPG: BBB51E42

3 3

Designing a Dovecot Replicate cluster
by Minhas Listas 19 Aug '14

19 Aug '14

Hi all! Do I need one Dovecot Proxy/Director for each Dovecot Replicate Cluster ? Isn't possible to put a Dovecot Proxy/Director on the border of multiple clusters ? Thanks! -- Thiago Henrique www.adminlinux.com.br

1 0

Health monitoring of backend servers
by Minhas Listas 19 Aug '14

19 Aug '14

Does Dovecot Director health monitoring of backend servers ? Or is poolmon (http://www.dovecot.org/list/dovecot/2010-August/051946.html) the best option for this? Maybe Ldirectord? Thanks! -- Thiago Henrique www.adminlinux.com.br

2 4

[SOLVED] Re: Trying Dovecot Replication with dsync
by Minhas Listas 19 Aug '14

19 Aug '14

> > 1) > Aug 14 15:39:34 ceph1 dovecot: imap(account=domaintest.com): Error: > open(/var/run/dovecot/replication-notify-fifo) failed: Permission denied > > What is the correct perms for /var/run/dovecot/replication-notify-fifo ? > Currently: > # ll /var/run/dovecot/replication-notify-fifo > prw------- 1 root root 0 Aug 14 15:57 > /var/run/dovecot/replication-notify-fifo| > > Solved with: # chown dovemail:dovemail /var/run/dovecot/replication-notify-fifo dovemail is the user for imap processes. > 2) > Aug 14 16:02:34 ceph1 dovecot: dsync-server(account=domaintest.com): > Error: Couldn't create lock */var/lib/imap/account=domaintest.com > <http://domaintest.com>/*.dovecot-sync.lock: No such file or directory > Aug 14 16:02:34 ceph1 dovecot: dsync-local(account=domaintest.com): > Error: Couldn't create lock */var/lib/imap/account=domaintest.com > <http://domaintest.com>/*.dovecot-sync.lock: No such file or directory > > In my production cluster, it is automatically created by dovecot when the > user performs the first login: > */var/lib/imap/*<account> for INDEX > */var/spool/imap/*<account> for Mdbox > This was fixed with correct permission on /var/run/dovecot/replication-notify-fifo Thanks! -- Thiago Henrique www.adminlinux.com.br

2 1