- dovecot - dovecot.org

TLS config check
by SW 07 Feb '15

07 Feb '15

Hi All First the essentials: dovecot --version: 2.2.15 /usr/local/etc/dovecot/conf.d/10-ssl.conf: ssl = required ssl_cert = </usr/local/openssl/certs/mail.domain.com.chained.dovecot.ecdsa.crt ssl_key = </usr/local/openssl/certs/mail.domain.com.ecdsa.key ssl_protocols = !SSLv2 !SSLv3 ssl_cipher_list = HIGH:EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:ECDHE-RSA-AES256-SHA:+DHE-RSA-AES256-SHA:!AES256-SHA256:!AES256-GCM-SHA384:!CAMELLIA256-SHA:!AES128:!CAMELLIA128:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!SEED:+AES256-SHA ssl_prefer_server_ciphers = yes I would really appreciate it if someone could tell me if my config is super secure? I run the following email clients: K9 on Android 4.4.2 Thunderbird 31.4 Outlook 2010 I'm interested to know if the config I have is secure and that my cipher list is acceptable. I'm also keen to hear thoughts on my config in respect of Forward Secrecy and the SSLv3/POODLE attack. Thanks!

5 9

Re: Interface for managing users?
by Thomas Moulia 06 Feb '15

06 Feb '15

Hello, > It's actually the way around: pick any user management and configure > Dovecot to use its database. > Depending on what you want, you need a system, that also removes the > user's home dirs and/or creates them from a skeleton. Makes sense, and is nicely simple. > After doing a change in the user database, make sure to run > doveadm auth cache flush > to force Dovecot to pick up the changes, before you fire up the next test. Ah, good to know -- that would've bit me. Thanks for the help, Steffen! -Thomas

1 0

doveadm quota get doesn't recalculate quota (Maildir backend)
by Teodor Milkov 06 Feb '15

06 Feb '15

Hello, I've noticed that "doveadm quota get" doesn't recalculate quota if maildirsize file is missing for some reason. It just says quota is unknown, usage is 0. In contrast, vpopmail does regenerate maildirsize if it is missing. Reading the Maildir++ specification there is this part: ...If maildirsize does not exist, or if its size is at least 5120 bytes, recalculate it using the procedure defined above, and use the recalculated numbers... One seemingly legitimate reason for maildirsize to be missing is when vpopmail mailbox is set to NOQUOTA (i.e. no limits). My workaround to this is to use a fallback quota: quota_rule = ?:messages=10000000 That is, if there's no backend limit detected (missing maildirsize) then fallback to 10 mln message count quota. In this case doveadm quota get _does_ recalculate usage and even creates maildirsize file. So, my questions are: 1. Isn't it still better to calculate and show usage even if maildirsize is missing? Because calling quota recalc and then quota get is racy - someone else may change configuration or usage etc. between our two calls (forks). 2. If there's good reason against implicit recalculation, then maybe quota get output should be more specific that neither quota nor usage are known? Instead of usage=0 show -1 or "-" or "n/a" or "unkown"?

1 0

Resubmission after N days
by Thomas Güttler 06 Feb '15

06 Feb '15

Hi, I would like to implement a 43Folder system[1] with dovecot and a mail user agent. Use case: - I have a new mail in my inbox. I read it and see that I can't handle it now. I want to handle this mail in 5 days. - Now I want to have some sort of resubmission: the mail should be moved to a different location for these 5 days. - After 5 days the mail should be moved to my inbox again. How can this be done with Dovecot/IMAP? I have programming experience, but I am new to Dovecot/IMAP. I would like solve this with Dovecot only (without a database or web server). Does the IMAP protocol over methods which could help? [1] 43Folder https://en.wikipedia.org/wiki/Tickler_file -- http://www.thomas-guettler.de/

7 16

dsync and smooth conversion of Maildir to mdbox
by Sven Strickroth 06 Feb '15

06 Feb '15

Hi, I was experimenting with dsync for Maildir to mdbox conversion. Two isses arised there: * "subscriptions" only contained subscriptions of that namespace, all others were lost - Is there any configuration missing on my side? * "dovecot-acl-list" and "dovecot-acl" shares/permissions were not synced; UPDATE: if I also add "acl" to "mail_plugins" when running dsync (using -o mail_plugins="acl zlib") it works, however, all documentation, e.g. the wiki2, only mention to add acl to lmtp/lda and imap services. - Is there any problem adding acl to "global" mail_plugins? Regarding shared namespace: I cannot convert all mailboxes at once and have to do it mailbox after mailbox. The setting "location = maildir:%%h/Maildir" won't work as some mailboxes are already converted and others aren't. Is there a way to cope with that? "doveconf -n" outout: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16-0.bpo.3-amd64 x86_64 Debian 7.8 dict { acldict = mysql:/etc/dovecot/dovecot-dict-user_shares.conf quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf } mail_gid = 8 mail_location = maildir:%h/Maildir mail_plugins = quota zlib mail_privileged_group = mail mail_shared_explicit_inbox = yes mail_uid = 999 namespace { list = children location = maildir:%%h/Maildir prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/79p-sql.conf driver = sql } plugin { acl = vfile acl_anyone = allow acl_shared_dict = proxy::acldict quota = dict:user::proxy::quotadict quota_grace = 0 quota_rule2 = Trash:bytes=+100M } protocols = " imap lmtp" service auth { unix_listener auth-userdb { group = mail mode = 0660 user = vmail } } service dict { unix_listener dict { group = mail mode = 0660 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0600 user = postfix } user = vmail } userdb { driver = prefetch } userdb { args = /etc/dovecot/79p-sql.conf driver = sql } -- Best regards, Sven Strickroth PGP key id F5A9D4C4 @ any key-server

1 0

Interface for managing users?
by Thomas Moulia 06 Feb '15

06 Feb '15

Hey all, I'd like to use dovecot as an testing/benchmarking IMAP server for switchboard (https://github.com/thusfresh/switchboard) As part of the tests, I need to add and destroy dovecot users. Is there an interface or tool for this sort of user management at a higher-level than editing the user store, e.g. passwd-file, sql, ...? Also, more generally, are there any tools for using dovecot to test against? Thanks! -Thomas

2 1

dovecot.index.log in Maildir/cur
by George Sexton 06 Feb '15

06 Feb '15

I'm seeing two bogus messages appearing my Maildir/cur directory. They're dovecot.index.log and dovecot-uidlist. -rw------- 1 gsexton users 51 Feb 4 09:04 Maildir/cur/dovecot-uidlist:2,S -rw------- 1 gsexton users 244 Feb 4 09:04 Maildir/cur/dovecot.index.log These files are only appearing in the Maildir/cur directory, and not in any other directory that mail is delivered to (e.g. mail list directories). I'm using the current version of Mozilla Thunderbird as my mail client. Mail is delivered to Maildir/ by procmail. I was originally using Courier, but changed to dovecot. I did run the conversion program on all the mail boxes. Other than these If anyone could help me understand what I'm doing wrong, I'd appreciate it. I've researched the mail list and read the wiki articles, but I'm just not seeing anything that looks helpful. Here's my config info: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.7-7-default x86_64 openSUSE 13.2 (x86_64) -> File System: btrfs mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl = required ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem ssl_key_password = ************ ssl_protocols = !SSLv2 !SSLv3 userdb { driver = passwd } protocol imap { mail_max_userip_connections = 20 } -- George Sexton *MH Software, Inc.* Voice: 303 438 9585 http://www.mhsoftware.com

2 4

Re: Re: auth: Warning: DNS lookup took 1.550 s
by Joseph Tam 06 Feb '15

06 Feb '15

ML mail writes: > My fault here, I should have precised that the DNS query it tries to > resolve is simply the DNS name of my mailbox server > (mailboxserver.domain.com) So domain.com is hosted locally on DNS > servers on that very same network as the dovecot servers are located. > Furthermore all the resolvers I use in /etc/resolv.conf are also local > DNS resolvers on that same LAN. All the DNS servers (authoritative and > recursive) are using PowerDNS. That's why I was really puzzled to see > that it can take up to 1.5 seconds to query my own local DNS servers > which are in no way overloaded. That really doesn't change my answer, but since you control the entire DNS chain of query and response, you can further debug it at the server end via logs or network tracing. And you should still run a comprehesive DNS trace (e.g. http://dns.squish.net/) since a misconfiguration could still cause your DNS resolver to go on wild goose chases. In either case, it's probably not Dovecot related. If you want to triage it, just enter static entries into your proxy's /etc/hosts file. Or use bare IPs in your configurations. Joseph Tam <jtam.home(a)gmail.com>

1 0

Block folder names in IMAP
by André Peters 05 Feb '15

05 Feb '15

Hi, I just merged to Maildir "/" as separator with a fs layout. So I am afraid of running into problems when someone creates a folder named "new", "cur", etc. ... I know I could work this around, but isn't there a way to block folders by their name from being created? Thank you! -- André Peters

1 0

Bug when using METADATA and ACLs
by Sergio Talens-Oliag 05 Feb '15

05 Feb '15

Hi, I'm trying to use the current version of dovecot as the IMAP server for Kolab and things seem to be working more or less OK, but while working on it I've found a dovecot bug that can probably be fixed quickly by someone familiar with the code. I've found the problem using shared folders, but it is related to ACLs and METADATA. The problem appears when a user only has the 'l' (lookup) permission on a folder and later tries to get METADATA from all his or her folders; a sample session to ilustrate the problem: A1 GETACL "foo" * ACL foo admin akxeilprwtscd -admin "" sto(a)iti.es lrwstipekxacd A1 OK Getacl completed. A2 SETACL "foo" sto(a)iti.es l A2 OK Setacl complete. A3 GETACL "foo" * ACL foo admin akxeilprwtscd -admin "" sto(a)iti.es al A3 OK Getacl completed. A4 GETMETADATA "foo" (/private/vendor/kolab/folder-type /shared/vendor/kolab/folder-type) A4 NO [NOPERM] Permission denied A5 GETMETADATA "*" (/private/vendor/kolab/folder-type /shared/vendor/kolab/folder-type) <CORE DUMPED> The dovecot.log messages are: Feb 05 13:59:12 imap(sto(a)iti.es): Panic: file imap-client.c: line 636 (client_command_free): assertion failed: (client->output_cmd_lock == NULL) Feb 05 13:59:12 imap(sto(a)iti.es): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x756cf) [0x7f55ab3da6cf] -> /usr/lib/dovecot/libdovecot.so.0(+0x7572e) [0x7f55ab3da72e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f55ab38c575] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0x18c9e) [0x7f55abba5c9e] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0x18e9a) [0x7f55abba5e9a] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0x18ef4) [0x7f55abba5ef4] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](client_handle_input+0x115) [0x7f55abba61e5] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](client_input+0x75) [0x7f55abba65b5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f55ab3ec51b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f55ab3ed59b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f55ab3ec5a9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f55ab3ec628] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f55ab391a43] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](main+0x2b7) [0x7f55abb9a717] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f55aaff7ead] -> dovecot/imap [sto(a)iti.es 127.0.0.1 GETMETADATA](+0xd889) [0x7f55abb9a889] Feb 05 13:59:12 imap(sto(a)iti.es): Fatal: master: service(imap): child 32404 killed with signal 6 (core dumped) And the gdb output is (my binary has no debug symbols, so I guess it is not really useful): (gdb) bt full #0 0x00007f55ab00b165 in raise () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x00007f55ab00e3e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #2 0x00007f55ab3da6c5 in ?? () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f55ab3da72e in ?? () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00007f55ab38c575 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00007f55abba5c9e in client_command_free () No symbol table info available. #6 0x00007f55abba5e9a in ?? () No symbol table info available. #7 0x00007f55abba5ef4 in ?? () No symbol table info available. #8 0x00007f55abba61e5 in client_handle_input () No symbol table info available. #9 0x00007f55abba65b5 in client_input () No symbol table info available. #10 0x00007f55ab3ec51b in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #11 0x00007f55ab3ed59b in io_loop_handler_run_internal () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #12 0x00007f55ab3ec5a9 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #13 0x00007f55ab3ec628 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #14 0x00007f55ab391a43 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #15 0x00007f55abb9a717 in main () No symbol table info available. If someone can look into it I will be grateful; of course if more info is needed I'm available to help as much as I can. Thanks in advance, Sergio. -- Sergio Talens-Oliag <sto(a)iti.es> <http://www.iti.es/> Key fingerprint = FF77 A16B 9D09 FC7B 6656 CFAD 261D E19A 578A 36F2

1 0

auth: Warning: DNS lookup took 1.550 s
by ML mail 05 Feb '15

05 Feb '15

Hello, I am running a dovecot and proxy server on two different virtual machines and on the dovecot proxy server I see around 5-6 times per day the following warning: Feb 03 16:15:12 auth: Warning: proxy(email(a)domain.com,xxx.xxx.xxx.xxx,<ABC123456789>): DNS lookup for mailboxserver.domain.com took 1.550 s I do not really understand how from time to time DNS queries are slow, I tried replicate this issue using dig to resolve the same DNS entry and it was always very fast. Is there any way I can debug better this issue? or is this nothing to worry about really? Regards ML

4 5

lazy_expunge + mdbox
by fernando.figaro＠uol.com.br 05 Feb '15

05 Feb '15

Hi, I'm studying an issue related to mdbox and lazy_expunge. If it is active in the imap configuration, when you delete a message (copy to new folder + expunge old folder), a copy is also sent to the expunge namespace. I realize that is something related to mdbox and lazy_expunge_only_last_instance and found almost no references except this one (http://hg.dovecot.org/dovecot-2.2/file/24d6708cea76/TODO) lazy_expunge_only_last_instance=yes + mdbox doesn't work, because refcounts don't update immediately One other suggestion I found was to disable 'move' capability. Does anyone has other news regarding this issue ? Thanks, Fernando

1 0

Re: auth: Warning: DNS lookup took 1.550 s
by Joseph Tam 05 Feb '15

05 Feb '15

ML mail writes: > I am not running a local DNS cache on the server. As suggested using a > local DNS cache would simply fix this issue but I am more interested to > know what is generating these slow DNS queries... As another poster has commented, a slow DNS response may not be anything related to your setup, but caused by upstream DNS service. Slow DNS response or timeouts is not uncommon, and if you're only getting 6 per day, you should count yourself lucky. If it's the same FDQN that's being delayed, you can maybe run an online DNS diagnostic -- they're may be some bum delegation or unsync'd data you can report, but it will probably not be worth your time to chase this to ground unless it's causing your client to get hung. Joseph Tam <jtam.home(a)gmail.com>

1 0

Hitting wall at 2048 IMAP connections
by Ron Cleven 04 Feb '15

04 Feb '15

We are gradually rolling out Dovecot (IMAP only, no POP3) to our customer base. We are replicating between a pair of CentOS 7 boxes. All has been working wonderfully. However, to be sure our rollout continues to go smoothly, we put together a simple benchmark client program to fire up X persistent IMAP connections (using hundreds of mailboxes) that login, list the folders, select the INBOX, periodically FETCH an email from the INBOX, and otherwise sit in an IDLE loop. As long as we keep the number of concurrent IMAP connections under 2k, everything works fine. 2,000 processes generate a minimal load on the server. However, 2048 is a "hard limit" for number of IMAP processes on a box the way things are configured right now. The number (2048) is a bit too magical to be anything but a misconfiguration of either some kernel limit or a dovecot configuration limit. I have been unable to figure out where else to look. I have pasted in my dovecot config below along with the kernel setting for max_user_instances. Hoping someone can tell me what stupid mistake I have made or what else to check. Just an aside, we front-end this with a separate set of proxy servers that provide an SSL front-end, hence, no SSL in the dovecot config. /proc/sys/fs/inotify/max_user_instances = 4096 # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-123.13.1.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Cor e) auth_master_user_separator = * auth_mechanisms = plain login default_internal_user = sysadm default_login_user = sysadm default_vsz_limit = 512 M disable_plaintext_auth = no doveadm_password = ************** doveadm_port = 12345 imap_max_line_length = 128 k listen = * mail_gid = 2001 mail_location = maildir:~/Maildir mail_plugins = notify replication mail_uid = 2001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=PLAIN /z/imap/usr/master-users driver = passwd-file master = yes pass = yes } passdb { args = scheme=PLAIN username_format=%u /z/imap/usr/users driver = passwd-file } plugin { mail_replica = tcp:imap0s sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M } protocols = imap pop3 sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = sysadm } unix_listener replication-notify { mode = 0666 user = sysadm } } service auth { unix_listener auth-userdb { mode = 0777 } } service doveadm { inet_listener { port = 12345 } user = sysadm } service imap { process_limit = 4096 } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 4096 } service replicator { process_min_avail = 1 } ssl = no userdb { args = username_format=%u /z/imap/usr/users driver = passwd-file } protocol lda { mail_plugins = sieve notify replication } protocol imap { mail_max_userip_connections = 40 } protocol sieve { managesieve_max_line_length = 65536 }

5 4

Corruption of index files
by Oliver Welter 04 Feb '15

04 Feb '15

Hi All, after upgrading my mail server (dovecot 1.1.7 -> 2.2.13) I get tons of messages about corrupted index files in the syslog ("Error: Corrupted transaction log" and "Warning: fscking index file .. dovecot.index". I tried flock and even dotlock, but the problems persist. The system is a VPS with an overlay filesystem which did work flawlessly with the old dovecot install and is not reporting any errors in the host machine. From time to time I get warnings about an exceeded inotify limit for the dovecot user - might this be a problem? Anybody has an idea what might be the cause those problems and how to track them down. Oliver -- Protect your environment - close windows and adopt a penguin!

2 4

Upstart script for Poolmon
by Listas＠Adminlinux 03 Feb '15

03 Feb '15

Hi My servers are Ubuntu 14.04 and I'm needing to make a Upstart script for Poolmon. Does someone already made this and could to share with us ? Thanks! -- Thiago henrique

1 1

Master user without pass=yes error
by Thomas HUMMEL 03 Feb '15

03 Feb '15

Hello Timo, I just tried the master user feature with a very simple setup (Dovecot v2.2.15) : !include auth-master.conf.ext -> passwd-file passdb !include auth-ldap.conf.ext -> ldap passdb (userdb prefetched) without auth_bind=yes without pass=yes I get this userdb lookup error : dovecot: auth: passwd-file(masteruser,157.99.64.42,master,<4Pgesh0OygCdY0Aq>): Master user logging in as normaluser dovecot: auth: Error: prefetch(normaluser,157.99.64.42,<4Pgesh0OygCdY0Aq>): userdb lookup not possible with only userdb prefetch dovecot: imap: Error: Internal auth failure (client-pid=10449 client-id=1) dovecot: imap-login: Internal login failure (pid=10449 id=1) (internal failure, 1 successful auths): user=<normaluser>, method=PLAIN, rip=157.99.64.42, lip=157.99.64.81, mpid=10570, TLS, session=<4Pgesh0OygCdY0Aq> with pass=yes, it works. Feb 2 17:51:24 langres dovecot: auth: passwd-file(masteruser,157.99.64.42,master,<YmjAwx0O0gCdY0Aq>): Master user logging in as normaluser Feb 2 17:51:24 langres dovecot: imap-login: Login: user=<normaluser>, method=PLAIN, rip=157.99.64.42, lip=157.99.64.81, mpid=11647, TLS, session=<YmjAwx0O0gCdY0Aq> I dont quite understand why because the documentation states that 'pass=yes' "means that Dovecot verifies that the login user really exists before allowing the master user to log in. Without the setting if a nonexistent login username is given,[...]" Here, 'normaluser' exists in the ldap passdb so, even with pass=no, I'm not supposed to be in the 'nonesxistent login username' case. Can you help ? thanks. -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Groupe Exploitation et Infrastructure

1 2

quote strings passed to sql
by Juan Bernhard 03 Feb '15

03 Feb '15

Hello list. I'm thinking to migrate the hole user db from system users to mysql. I already did it in a test environment, but something is annoying my OCD... I don't quote the variables username and password sent to the mysql server. I know, the mysql user that dovecot uses only has select rights, but it stills bother me, because its possible to do an useless sql code injection. Is there a way to quote that? Something like exim's quote_mysql? Saludos, Juan.

3 4

Re: Uniqueness of dovecot mailbox-guids
by Christian Binder Stadt Freilassing 02 Feb '15

02 Feb '15

Can no one give some adivse on this issue? Thanks Christian

1 0

LDAP: "unused" & "uid missing"
by Leander Schäfer 02 Feb '15

02 Feb '15

Does someone know what this means? It looks to me like it got the UID, yet it complains ... ? Also, why does it complain about attributes being unused, yet I use prefetch, so they will be used?! Log: [...] dovecot: auth: Debug: ldap(test(a)mydomain.tld,192.168.10.233,<Rl2CF+UNhgDAqArp>): result: mailStorageDirectory=/var/vmail mailUidNumber=2110 mailGidNumber=2110 mailQuotaStorage=10240 mailQuotaMessages=20; mailQuotaMessages,mailStorageDirectory,mailUidNumber,mailQuotaStorage,mailGidNumber unused dovecot: auth: Debug: ldap(test(a)mydomain.tld,192.168.10.233,<Rl2CF+UNhgDAqArp>): result: mailStorageDirectory=/var/vmail mailUidNumber=2110 mailGidNumber=2110 mailQuotaStorage=10240 mailQuotaMessages=20; uid missing [...] As mentioned, I use prefetch, so this is how my dovecot/dovecot-ldap.conf.ext is looking: [...] #user_attrs = mailStorageDirectory=home=%$/%u, mailStorageDirectory=mail=maildir:%$/%u/maildir, mailUidNumber=uid, mailGidNumber=gid, mailQuotaStorage=quota_rule=*:storage=%$, mailQuotaMessages=quota_rule2=*:messages=%$ pass_attrs = uid=user, userPassword=password, mailStorageDirectory=userdb_home=%$/%u, mailStorageDirectory=userdb_mail=maildir:%$/%u/maildir, mailUidNumber=userdb_uid, mailGidNumber=userdb_gid, mailQuotaStorage=userdb_quota_rule=*:storage=%$, mailQuotaMessages=userdb_quota_rule2=*:messages=%$ [...] Thanks

2 1

Sieve permissions issue following update
by David Gessel 02 Feb '15

02 Feb '15

I recently updated dovecot and my sieve filters stopped working. Checking the logs I see: Dec 9 00:09:59 mailhost dovecot: lda(gessel(a)domain.com): Error: sieve: binary save: failed to create temporary file: open(/usr/local/etc/dovecot/sieve/10-move-spam.svbin.mailhost.domain.com.114.) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +w perm: /usr/local/etc/dovecot/sieve, we're not in group 6(mail), dir owned by 143:6 mode=0775) Dec 9 00:09:59 mailhost dovecot: lda(gessel(a)domain.com): Error: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/usr/local/etc/dovecot/sieve/10-move-spam.sieve' need to be pre-compiled using the sievec tool However this fairly clear advice on the failure seems to be contradicted by: # id vmail uid=5000(vmail) gid=5000(vmail) groups=5000(vmail),6(mail) ? dovecot-pigeonhole-0.4.6 = up-to-date with index dovecot2-2.2.15_1 = up-to-date with index uname -a FreeBSD host.domain.com 9.3-RELEASE FreeBSD 9.3-RELEASE #0 r268932: Mon Jul 21 15:51:38 PDT 2014 gessel@host1.domain.com:/usr/obj/usr/src/sys/BARCELONA-13-08 amd64

8 22

managesieve & checkpassword authdb interface
by Peter Hodur 02 Feb '15

02 Feb '15

Hello, does anybody knows what managesieve addon sets in SERVICE env when autentificating to checkpassword? thanks Pete

2 1

Domain quota
by Evgeny Basov 02 Feb '15

02 Feb '15

Hello. I'm tried to use domain quota: plugin { quota = dict:Domain quota:%d:redis:host=127.0.0.1:prefix=domain/ quota_rule = *:storage=0 quota2 = dict:User quota::redis:host=127.0.0.1:prefix=user/ quota2_rule = *:storage=0 quota3_rule = Trash:storage=+5%% } but got next problem with it. Init state: # doveadm quota get -u "testacc(a)m.tld" Quota name Type Value Limit % Domain quota STORAGE 27070 204800 13 Domain quota MESSAGE 21 - 0 User quota STORAGE 27068 - 0 User quota MESSAGE 18 - 0 # doveadm quota get -u "testacc2(a)m.tld" Quota name Type Value Limit % Domain quota STORAGE 27070 204800 13 Domain quota MESSAGE 21 - 0 User quota STORAGE 2 - 0 User quota MESSAGE 3 - 0 Make recalc: # doveadm quota recalc -u "testacc2(a)m.tld" and got: # doveadm quota get -u "testacc2(a)m.tld" Quota name Type Value Limit % Domain quota STORAGE 2 204800 0 Domain quota MESSAGE 3 - 0 User quota STORAGE 2 - 0 User quota MESSAGE 3 - 0 # doveadm quota get -u "testacc(a)m.tld" Quota name Type Value Limit % Domain quota STORAGE 2 204800 0 Domain quota MESSAGE 3 - 0 User quota STORAGE 27068 - 0 User quota MESSAGE 18 - 0 All values of all domain quota are equals. Execute recalc one more time: # doveadm quota recalc -u "testacc(a)m.tld" # doveadm quota get -u "testacc(a)m.tld" Quota name Type Value Limit % Domain quota STORAGE 27068 204800 13 Domain quota MESSAGE 18 - 0 User quota STORAGE 27068 - 0 User quota MESSAGE 18 - 0 # doveadm quota get -u "testacc2(a)m.tld" Quota name Type Value Limit % Domain quota STORAGE 27068 204800 13 Domain quota MESSAGE 18 - 0 User quota STORAGE 2 - 0 User quota MESSAGE 3 - 0 How can I get fast realtime changed domain quota? Used dovecot 2.2.13.

2 1

auth: Error: auth worker: Aborted request: Lookup timed out
by ML mail 01 Feb '15

01 Feb '15

Hello, I was checking the errors on my mailbox server using "doveadm log errors" and found that around 2-3 times a day I get the following error: Jan 29 22:14:43 auth: Error: auth worker: Aborted request: Lookup timed out Any ideas if this is a bad sign? For example some timeout on my side which is too low or is it just an aborted connection from the side of the IMAP client? The configuration on my Dovecot mailbox server can be found here: http://article.gmane.org/gmane.mail.imap.dovecot/80823 It is a Debian 7 box with standard debian packages. Best regards ML

4 6

[Bug ?] IMAP keywords are not preserved during mailbox conversion
by Alexander Moisseev 01 Feb '15

01 Feb '15

IMAP keywords (Thunderbird Tags) are not preserved during Maildir to mdbox conversion. For mailbox conversion I am using the command: # doveadm -o mail_location=maildir:~/Maildir sync -u user(a)example.com mdbox:~/mdbox After conversion all Thunderbird Tags are lost. Is it a bug or expected behavior? -- Alexander

1 0

LDAP Quota Limits & Warnings
by Leander Schäfer 30 Jan '15

30 Jan '15

I found this on the dovecot wiki: The following limit names are supported: * *storage*: Quota limit in kilobytes, 0 means unlimited. * *bytes*: Quota limit in bytes, 0 means unlimited. * *messages*: Quota limit in number of messages, 0 means unlimited. This probably isn't very useful. * *backend*: Quota backend-specific limit configuration. * *ignore*: Don't include the specified mailbox in quota at all. As I understood it, all 5 limit categories are valid to use with maildir format while only "storage" and "messages" are valid to use with dmbox format, right? Also the "ignore" limit may appear more than one time for a user - like Trash:ignore and Spam:ignore, right? So I assumed following should work - and it does - except for "ignore" - even though LDAP does not hold multiple values for it - only a single one for each limit: user_attrs = [...] mailQuotaStorage=quota_rule=*:storage=%$, mailQuotaMessages=quota_rule2=*:messages=%$, mailQuotaIgnore=quota_rule3=*:ignore=%$ pass_attrs = [...] mailQuotaStorage=userdb_quota_rule=*:storage=%$, mailQuotaMessages=userdb_quota_rule2=*:messages=%$, mailQuotaIgnore=userdb_quota_rule3=*:ignore=%$ And unfortunately /var/log/maillog tells me: dovecot: imap(abuse(a)netocean.local): Error: user abuse(a)netocean.local: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:ignore=Trash: Unknown rule limit name: ignore Yet /var/log/debug.log sounds a bit more promising ... but then again not really ... dovecot: imap: Debug: Added userdb setting: mail=maildir:/usr/local/www/default/mail/user@mydomain.tld dovecot: imap: Debug: Added userdb setting: plugin/quota_rule=*:storage=1048576 dovecot: imap: Debug: Added userdb setting: plugin/quota_rule2=*:messages=10 dovecot: imap: Debug: Added userdb setting: plugin/quota_rule3=*:ignore=Trash dovecot: imap(abuse(a)netocean.local): Debug: Effective uid=2110, gid=2110, home=/var/vmail/user(a)mydomain.tld/mail dovecot: imap(abuse(a)netocean.local): Debug: Quota root: name=User quota backend=maildir args= dovecot: imap(abuse(a)netocean.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 dovecot: imap(abuse(a)netocean.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=10 dovecot: imap(abuse(a)netocean.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=10 Additionally, if one of the two working quotas (storage & messages) is exceeded than Dovecot sends out a warning from this setting: plugin { quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u # What message to send to IMAP clients (and SMTP senders) when quota is exceeded? quota_exceeded_message = Quota exceeded, please go to http://MyDomain.TLD for instructions on how to fix this. } No my question is how to provide two different warnings. One for storage like: Sorry, you're mail storage is full. Please delete some. And a second for messages like: Sorry, you've too many messages. Please delete some. Thanks & Best Regards Leander

1 0

Troubleshooting doveadm for Solr Configuration
by Kevin Laurie 30 Jan '15

30 Jan '15

Hey guys, I am trying to pinpoint where the problem with my dovecot-FTS installation . I am trying to rule out between the Dovecot and the Solr Module. I think if the output below looks OK then the dovecot is fine, and will focus more on the Solr. Appreciate if someone could check and confirm. Thanks root@mail:/etc/dovecot/conf.d# doveadm -D -v fts rescan -u admin(a)email.net doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so doveadm(admin(a)email.net): Debug: auth input: admin(a)email.net master_user= admin(a)email.net home=/var/vmail/vmail1/ email.net/t/h/a/admin-2014.10.25.07.15.15/ quota_rule=*:bytes=26214400000 doveadm(admin(a)email.net): Debug: Added userdb setting: plugin/master_user= admin(a)email.net doveadm(admin(a)email.net): Debug: Added userdb setting: plugin/quota_rule=*:bytes=26214400000 doveadm(admin(a)email.net): Debug: Effective uid=2000, gid=2000, home=/var/vmail/vmail1/email.net/t/h/a/thai-2014.10.25.07.15.15/ doveadm(admin(a)email.net): Debug: Quota root: name=user backend=dict args=:proxy::quotadict doveadm(admin(a)email.net): Debug: Quota rule: root=user mailbox=* bytes=26214400000 messages=0 doveadm(admin(a)email.net): Debug: Quota warning: bytes=22282240000 (85%) messages=0 reverse=no command=quota-warning 85 admin(a)email.net doveadm(admin(a)email.net): Debug: Quota warning: bytes=23592960000 (90%) messages=0 reverse=no command=quota-warning 90 admin(a)email.net doveadm(admin(a)email.net): Debug: Quota warning: bytes=24903680000 (95%) messages=0 reverse=no command=quota-warning 95 admin(a)email.net doveadm(admin(a)email.net): Debug: Quota grace: root=user bytes=2621440000 (10%) doveadm(admin(a)email.net): Debug: dict quota: user=admin(a)email.net, uri=proxy::quotadict, noenforcing=0 doveadm(admin(a)email.net): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir://var/vmail/vmail1/ email.net/t/h/a/thai-admin.10.25.07.15.15//Maildir/:INDEX=//var/vmail/vmail… doveadm(admin(a)email.net): Debug: maildir++: root=//var/vmail/vmail1/ email.net/t/h/a/admin-2014.10.25.07.15.15//Maildir, index=, indexpvt=, control=, inbox=//var/vmail/vmail1/ sicl.net/t/h/a/thai-2014.10.25.07.15.15//Maildir, alt= doveadm(admin(a)email.net): Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/Shared/%u doveadm(admin(a)email.net): Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= doveadm(admin(a)email.net): Debug: fts: Indexes disabled for namespace 'Shared/%u/'

2 1

Re: Uniqueness of dovecot mailbox-guids
by Christian Binder Stadt Freilassing 30 Jan '15

30 Jan '15

If mailbox guids should be unique systemwide, would it be safe to delete all 'dovecot-uidlist' files of the user's mailboxes and then do a doveadm force-resync after mailbox migration? Thanks! Christian ----------------ursprüngliche Nachricht----------------- Von: Christian Binder Stadt Freilassing [christian.binder(a)freilassing.de ] An: "dovecot dovecot.org" [dovecot(a)dovecot.org ] Datum: Thu, 29 Jan 2015 09:24:12 +0100 ------------------------------------------------- > Sorry if this is a duplicate message. I'm new in this mailing list and not sure if > the first message really got posted ... > > Dovecot version: 2.2.9 > > We're using doveadm backup to migrate user mailboxes from cyrus to dovecot. I > defined a master user for the old cyrus mailserver in imapc backend to get access > to all the mailboxes. Now I realized that after migration every user's "Inbox" or > "Trash" on the new dovecot server has the same mailbox-guid, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox dd9456... > trash 5549cc... > > Is this a wanted behaviour? If i do not migrate with doveadm backup and just > manually create mailboxes, the mailbox-guids are globally unique, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox c543fe... > trash t49ccc... > > Thanks! > Christian >

1 0

Re: Uniqueness of dovecot mailbox-guids
by Christian Binder Stadt Freilassing 30 Jan '15

30 Jan '15

Update: seems to me that the old cyrus mailserver already uses these non-unique mailbox guids. Is it necessary to have them unique over all private namespaces? Thanks Christian ----------------ursprüngliche Nachricht----------------- Von:Christian Binder Stadt Freilassing [christian.binder(a)freilassing.de ] An:"dovecot dovecot.org" [dovecot(a)dovecot.org ] Datum:Thu, 29 Jan 2015 09:24:12 +0100 ------------------------------------------------- > Sorry if this is a duplicate message. I'm new in this mailing list and not sure if > the first message really got posted ... > > Dovecot version:2.2.9 > > We're using doveadm backup to migrate user mailboxes from cyrus to dovecot. I > defined a master user for the old cyrus mailserver in imapc backend to get access > to all the mailboxes. Now I realized that after migration every user's "Inbox" or > "Trash" on the new dovecot server has the same mailbox-guid, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox dd9456... > trash 5549cc... > > Is this a wanted behaviour? If i do not migrate with doveadm backup and just > manually create mailboxes, the mailbox-guids are globally unique, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox c543fe... > trash t49ccc... > > Thanks! > Christian >

1 0

Info on shared folders
by Bobber 30 Jan '15

30 Jan '15

I'm interested in setting up shared folders in a small office environment.Â Can a folder be created and shared with other users from an IMAP client such as Thunderbird? -- *Bob Wooldridge* Blog: http://kc0dxf.net/blog/

2 1

Re: mdbox: Broken virtual size for mail UID ...
by Dennis Schridde 29 Jan '15

29 Jan '15

Hello everyone! I originally asked Timo for help in this case, but I understand he is very busy these days, so I am now posting my questions to this mailinglist. On Sunday 28 Dec 2014 10:02:08 Timo Sirainen wrote: > On 28 Dec 2014, at 07:06, Dennis Schridde <devurandom(a)gmx.net> wrote: > > My harddrives crashed, and now I get "Broken virtual size for mail UID …" > > messages in the dovecot logs from some mdbox folders. I assume the logged > > emails are destroyed forever, but I would like to restore as much as > > possible of them and keep them in their corrupted / incomplete state. How > > do I do that? > > > > I guess what I need is some kind of "doveadm fsck" command, that tries its > > best to limit the damage to a minimum: Inform me about the broken emails > > and then ask me whether they should be stored in truncated form, be > > deleted, etc. Is there such a thing? > > http://dovecot.org/tools/mdbox-recover.pl Thank you! >From what I read in the Wiki and how I understand the script, I should proceed like this: #!/bin/bash broken_storage=$PWD/storage recovered_storage=$PWD/storage-recovered cd ${recovered_storage} n=1 for f in ${broken_storage}/m.* ; do mdbox-recover.pl ${f} for f in msg.* ; do mv ${f} m.${n} $((n++)) done done mv ${broken_storage} storage-backup mv ${recovered_storage} storage exit 0 Afterwards dovecot will now automatically regenerate the dovecot.map.index files from the now bare m.* files, and as the dovecot.index files in the actual mailboxes reference the GUID, and the messages in the m.* files have the GUID embedded, nothing will be lost. Is that correct? To create proper assessment of the damage, I need to know a bit more: 1. In the script on line 43, the comment says: # end of metadata block missing, finish the previous mail That seems to be wrong, since idx2!=-1 and state==STATE_BODY on that line, while I understand it should be -1 and STATE_META if we were still reading the metadata block. I guess it should be something like: # still in data block, finish the previous mail 2. Line 47 reads: # truncated / broken data? just keep writing to previous file That is the only error which could be detected in that script, right? 3. A message could also be corrupted by loosing its complete data block from \001\003 to \001\002, which cannot be detected in the script, right? 4. The only case to detect this and find other unexpected oddities would be to compare each msg.* in size and offset with the messages referenced in the dovecot.map.index file, correct? How do I do that, if the dovecot.map.index does not contain the GUID of the message? Do I need to fuzzy match the file_id, offset and size with my current values, or is there a more reliable way to do this? 5. Metadata refers to the RFC822 mail headers? Or are they also included in the body, and the mentioned metadata is dovecot-only metadata? If the latter, do they also contain the size of the body and similar information? Best regards, Dennis

1 0

Dovecot antispam
by David Mehler 29 Jan '15

29 Jan '15

Hello, This is not specifically a dovecot issue, but to it's antispam plugin, that being the case if private responses are deemed so please do so. I've got a Postfix, amavisd-new, dovecot (delivery and sieve), setup with spamassassin. It's for the most part working fine, here's the problem, false positives and false negatives. The sieve moves spamassassin-tagged messages in to the Spam folder, sometimes it misses either a false positive or negative. What I'm wanting to do and this is where the dovecot-antispam plugin comes in is automatically retrain messages, a false positive or negative needs to be moved and reclassified accordingly. I've found a lot of google results indicating dspam, when I used this the results were terrible after three months of usage I gave up on it. I've thought about using spamc with dovecot-antispam to handle retraining or crm114, but not much on getting this going. If anyone has any information on this or any other non-dspam setup that would work I'd appreciate knowing about it. Thanks. Dave.

2 3

Re: Indexing Mail faster
by Joseph Tam 29 Jan '15

29 Jan '15

Kevin writes: > Appreciate if you could help with this. I have been trying to address this > "slow search" issue for a while with very limited success(I was trying to > implement FTS also), so I will appreciate if you could support. When I'm stumped, one of the diagnostic tools I use is process tracing. Connect via IMAP, in another window/session process trace the IMAP worker process (Linux: strace -r -otrace.dump {pid})), initiate the search and quit. Then you can look through the trace and see whether it gets hung on a particular operation. Joseph Tam <jtam.home(a)gmail.com>

3 6

Re: Indexing Mail faster
by Kevin Laurie 29 Jan '15

29 Jan '15

Hi Micheal, Here is how it should look:- ~# telnet localhost imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. . login user(a)domain.net password . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY QUOTA] Logged in . select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded \*)] Flags permitted. * 49319 EXISTS * 12 RECENT * OK [UNSEEN 46791] First unseen. * OK [UIDVALIDITY 1414214135] UIDs valid * OK [UIDNEXT 106554] Predicted next UID * OK [NOMODSEQ] No permanent modsequences . OK [READ-WRITE] Select completed (0.128 secs). . search text hello * OK Searched 6% of the mailbox, ETA 2:34 Finally at the end you logout by . logout On Thu, Jan 29, 2015 at 4:39 PM, Kevin Laurie <superinterstellar(a)gmail.com> wrote: > Dear Michael, > You will need to telnet locally to imap. So run the following commands in > your server > > 1. telnet localhost imap > 2. . login email(a)domain.com password > 3. . select inbox > 4. . search text "passport" (This command here will search your inbox and > give you the time needed) > 5. . logout > > Notice the period(.) You need to use it . > > Hope this is clear. > Regards > Kevin > > > On Thu, Jan 29, 2015 at 4:32 PM, M. Koehler <michael.koehler(a)makomi.de> > wrote: > >> Hi Kevin, >> >> I want to test it with my dovecot but I don´t know how to measure it. >> Could you please provide my the configuration/terminal commands to get an >> reliable output? I just have mailboxes up to 4GB in VM on a proxmox server >> (Debian Wheezy at the moment). >> >> regards, >> Michael >> >> >> >> > Am 29.01.2015 um 06:29 schrieb Kevin Laurie < >> superinterstellar(a)gmail.com>: >> > >> > Dear Peter, >> > Noted. Thanks for your input. Appreciate it. >> > At this point my urgent most priority is to get FTS. Waiting 3 mins for >> a >> > body search email is bad(but of course my email box is large) >> > I need to have this sorted out by today as I have been putting this off >> for >> > too long mostly because of lack of troubleshooting documentation >> online(if >> > i do figure this out will create some for guidance) >> > Regards >> > Kevin >> > >> > On Thu, Jan 29, 2015 at 1:18 PM, Peter Hodur <petehodur(a)gmail.com> >> wrote: >> > >> >> >> >> >> >> On Thursday, January 29, 2015, Kevin Laurie < >> superinterstellar(a)gmail.com> >> >> wrote: >> >> >> >>> Dear Peter, >> >>> Oh. Sorry(didnt know you were addressing someone else) my apologies. >> But >> >>> as you can see , I am desperately trying to address this issue. >> >>> >> >> >> >> No problem ;))) i wrote because my search result is good but not so >> >> good as someone wrote here couple of days ago. >> >> >> >> >> >> >> >>> I have 1 disk with 200GB running on a VPS. >> >>> The file system is ext4. >> >>> >> >> >> >> Im not dovecot expert, but i think this is your problem. You need more >> >> iops. If you need performance, VPS may be (if connected to fast SAN) >> but >> >> basicly is not a good choice. >> >> >> >> >> >>> I think I will need to implement FTS to fix this as I will need body >> >>> searches. >> >>> >> >> >> >> FTS could be a solution. You are right. >> >> >> >> >> >>> Have you tried FTS before? >> >>> >> >> >> >> Im sorry, never :( >> >> >> >> But may be someone more skilled than me could answer the main question, >> >> how much dovecot index helps in case imap search agains body. >> >> >> >> Im not sure, but my opinion is that it helps little (may be not at all) >> >> >> >> Ps: may be you use it, but do not forget to setup delivering from mail >> >> server via LDA or LMTP instead of direct to maildirs. This is better >> >> because dovecot updates your indexes at the time of message arrival. >> Later >> >> accesses should be faster. >> >> >> >> Peter >> >> >> >> >> >> >> >>> >> >>> >> >>> Regards >> >>> Kevin >> >>> >> >> >> >>> >> >>> On Thu, Jan 29, 2015 at 12:55 PM, Peter Hodur <petehodur(a)gmail.com> >> >>> wrote: >> >>> >> >>>> Kevin, >> >>>> >> >>>> My message was not addressed to you ;) because my results are NOT so >> >>>> good as someone wrote it here. I can fulltext cca 8k messages in 7-8 >> >>>> seconds. >> >>>> >> >>>> Someone wrote that he has results cca 22k messages in 4 seconds :( >> >>>> >> >>>> Im not sure, but the answer is may be in the index and disk >> subsystem. >> >>>> >> >>>> If and only if dovecot index does not have saved in the index >> keywords >> >>>> from BODY of messages the problem is your/my disk subsystem. >> >>>> >> >>>> Generally, messages from mailing list like this are pretty small. The >> >>>> problem is that dovecot must read all messages. If you have maildir >> its >> >>>> mean open and read many files. >> >>>> >> >>>> But my test search was agains archive of my personal inbox - so not >> onoy >> >>>> small messages like this but also messages with big attachments etc. >> >>>> >> >>>> If dovecot index only headers, than it in case of full text search >> read >> >>>> whole messages - mime parts allows that plain text body can be after >> >>>> attachment etc. >> >>>> >> >>>> And here is it all about iops and throughput. >> >>>> >> >>>> How many disks you have? And what setup? >> >>>> >> >>>> Generally, the only good setup is many smaller disks in RAID 10 >> >>>> (stripped mirrors) and if your fs allows it, with added read cache - >> in >> >>>> case of zfs: l2arc on ssd. >> >>>> >> >>>> >> >>>> >> >>>> Pete >> >>>> >> >>>> >> >>>> >> >>>> On Thursday, January 29, 2015, Kevin Laurie < >> superinterstellar(a)gmail.com> >> >>>> wrote: >> >>>> >> >>>>> Hi Peter, >> >>>>> Sorry, >> >>>>> >> >>>>> I think its maildir >> >>>>> Output of my dovecot -n is listed below:- >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> # 2.2.9: /etc/dovecot/dovecot.conf >> >>>>> # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS >> >>>>> auth_master_user_separator = * >> >>>>> auth_mechanisms = PLAIN LOGIN >> >>>>> dict { >> >>>>> acl = mysql:/etc/dovecot/dovecot- >> >>>>> share-folder.conf >> >>>>> quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf >> >>>>> } >> >>>>> first_valid_uid = 2000 >> >>>>> last_valid_uid = 2000 >> >>>>> listen = * >> >>>>> log_path = /var/log/dovecot.log >> >>>>> mail_debug = yes >> >>>>> mail_gid = 2000 >> >>>>> mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ >> >>>>> mail_plugins = quota fts >> >>>>> mail_uid = 2000 >> >>>>> managesieve_notify_capability = mailto >> >>>>> managesieve_sieve_capability = fileinto reject envelope >> >>>>> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational >> >>>>> regex imap4flags copy include variables body enotify environment >> mailbox >> >>>>> date ihave >> >>>>> namespace { >> >>>>> inbox = yes >> >>>>> location = >> >>>>> mailbox Drafts { >> >>>>> auto = subscribe >> >>>>> special_use = \Drafts >> >>>>> } >> >>>>> mailbox Junk { >> >>>>> auto = subscribe >> >>>>> special_use = \Junk >> >>>>> } >> >>>>> mailbox Sent { >> >>>>> auto = subscribe >> >>>>> special_use = \Sent >> >>>>> } >> >>>>> mailbox "Sent Messages" { >> >>>>> auto = no >> >>>>> special_use = \Sent >> >>>>> } >> >>>>> mailbox Spam { >> >>>>> auto = no >> >>>>> special_use = \Junk >> >>>>> } >> >>>>> mailbox Trash { >> >>>>> auto = subscribe >> >>>>> special_use = \Trash >> >>>>> } >> >>>>> prefix = >> >>>>> separator = / >> >>>>> type = private >> >>>>> } >> >>>>> namespace { >> >>>>> list = children >> >>>>> location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u >> >>>>> prefix = Shared/%%u/ >> >>>>> separator = / >> >>>>> subscriptions = yes >> >>>>> type = shared >> >>>>> } >> >>>>> passdb { >> >>>>> args = /etc/dovecot/dovecot-mysql.conf >> >>>>> driver = sql >> >>>>> } >> >>>>> passdb { >> >>>>> args = /etc/dovecot/dovecot-master-users-password >> >>>>> driver = passwd-file >> >>>>> master = yes >> >>>>> } >> >>>>> plugin { >> >>>>> acl = vfile >> >>>>> acl_shared_dict = proxy::acl >> >>>>> auth_socket_path = /var/run/dovecot/auth-master >> >>>>> quota = dict:user::proxy::quotadict >> >>>>> quota_rule = *:storage=1G >> >>>>> quota_warning = storage=85%% quota-warning 85 %u >> >>>>> quota_warning2 = storage=90%% quota-warning 90 %u >> >>>>> quota_warning3 = storage=95%% quota-warning 95 %u >> >>>>> sieve = /%Lh/sieve/dovecot.sieve >> >>>>> sieve_default = /var/vmail/sieve/dovecot.sieve >> >>>>> sieve_dir = /%Lh/sieve >> >>>>> sieve_global_dir = /var/vmail/sieve >> >>>>> } >> >>>>> protocols = pop3 imap sieve lmtp >> >>>>> service auth { >> >>>>> unix_listener /var/spool/postfix/private/dovecot-auth { >> >>>>> group = postfix >> >>>>> mode = 0666 >> >>>>> user = postfix >> >>>>> } >> >>>>> unix_listener auth-master { >> >>>>> group = vmail >> >>>>> mode = 0666 >> >>>>> user = vmail >> >>>>> } >> >>>>> unix_listener auth-userdb { >> >>>>> group = vmail >> >>>>> mode = 0660 >> >>>>> user = vmail >> >>>>> } >> >>>>> } >> >>>>> service dict { >> >>>>> unix_listener dict { >> >>>>> group = vmail >> >>>>> mode = 0660 >> >>>>> user = vmail >> >>>>> } >> >>>>> } >> >>>>> service imap-login { >> >>>>> process_limit = 500 >> >>>>> service_count = 1 >> >>>>> } >> >>>>> service lmtp { >> >>>>> executable = lmtp -L >> >>>>> inet_listener lmtp { >> >>>>> port = 24 >> >>>>> } >> >>>>> process_min_avail = 5 >> >>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> >>>>> group = postfix >> >>>>> mode = 0600 >> >>>>> user = postfix >> >>>>> } >> >>>>> user = vmail >> >>>>> } >> >>>>> service pop3-login { >> >>>>> service_count = 1 >> >>>>> } >> >>>>> service quota-warning { >> >>>>> executable = script /usr/local/bin/dovecot-quota-warning.sh >> >>>>> unix_listener quota-warning { >> >>>>> group = vmail >> >>>>> mode = 0660 >> >>>>> user = vmail >> >>>>> } >> >>>>> } >> >>>>> ssl = required >> >>>>> ssl_cert = </etc/ssl/certs/sicl.net.crt >> >>>>> ssl_key = </etc/ssl/private/sicl.net.key >> >>>>> userdb { >> >>>>> args = /etc/dovecot/dovecot-mysql.conf >> >>>>> driver = sql >> >>>>> } >> >>>>> protocol lda { >> >>>>> auth_socket_path = /var/run/dovecot/auth-master >> >>>>> lda_mailbox_autocreate = yes >> >>>>> log_path = /var/log/dovecot-sieve.log >> >>>>> mail_plugins = quota fts sieve >> >>>>> postmaster_address = root >> >>>>> } >> >>>>> protocol lmtp { >> >>>>> info_log_path = /var/log/dovecot-lmtp.log >> >>>>> lmtp_save_to_detail_mailbox = yes >> >>>>> mail_plugins = quota sieve >> >>>>> postmaster_address = postmaster >> >>>>> recipient_delimiter = + >> >>>>> } >> >>>>> protocol imap { >> >>>>> imap_client_workarounds = tb-extra-mailbox-sep >> >>>>> mail_plugins = quota fts imap_quota >> >>>>> } >> >>>>> protocol pop3 { >> >>>>> mail_plugins = quota fts >> >>>>> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh >> >>>>> pop3_uidl_format = %08Xu%08Xv >> >>>>> } >> >>>>> >> >>>>> >> >>>>> On Thu, Jan 29, 2015 at 12:37 PM, Kevin Laurie < >> >>>>> superinterstellar(a)gmail.com> wrote: >> >>>>> >> >>>>>> Dear Peter, >> >>>>>> >> >>>>>> My inbox is MDA_external >> >>>>>> Storage: 17GB of 24GB >> >>>>>> >> >>>>>> Subject / From / To is fast but FTS(Full Text Search) for body is >> >>>>>> horrible. I suppose this is where we need Apache Solr. >> >>>>>> >> >>>>>> Do you think my mail storage format is bad? Do I need to change for >> >>>>>> better performance? >> >>>>>> Please advise >> >>>>>> Kevin >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Thu, Jan 29, 2015 at 12:25 PM, Peter Hodur <petehodur(a)gmail.com >> > >> >>>>>> wrote: >> >>>>>> >> >>>>>>>> * Kevin Laurie <superinterstellar(a)gmail.com> 2015.01.24 19:41: >> >>>>>>>> >> >>>>>>>>> Currently the time it takes to search 25,000mails is 4mins. If >> >>>>>>> indexed >> >>>>>>>> how >> >>>>>>>>> much faster are we looking at? >> >>>>>>>> >> >>>>>>>> With a current version of Dovecot a search is pretty fast >> _without_ >> >>>>>>> using >> >>>>>>>> external indexes. I have a view defined (virtual plugin) with >> >>>>>>> around 22.000 >> >>>>>>>> messages in it, and searching the full view only takes 2.5 >> seconds: >> >>>>>>>> >> >>>>>>>> >> >>>>>>> hmmm, could you please tell me more about your setup? What storage >> >>>>>>> format >> >>>>>>> do you use? Maildir or? What is the cumulative size of your >> messages? >> >>>>>>> >> >>>>>>> My results without FTS on ZFS FS with SSD L2ARC are not so good: >> >>>>>>> >> >>>>>>> a4 select INBOX._OLD-OUTLOOK >> >>>>>>> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk >> >>>>>>> JunkRecorded $Forwarded) >> >>>>>>> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft >> $Junk >> >>>>>>> $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. >> >>>>>>> ** 7748 EXISTS* >> >>>>>>> * 0 RECENT >> >>>>>>> * OK [UIDVALIDITY 1421198037] UIDs valid >> >>>>>>> * OK [UIDNEXT 11509] Predicted next UID >> >>>>>>> * OK [HIGHESTMODSEQ 12204] Highest >> >>>>>>> a4 OK [READ-WRITE] Select completed (0.001 secs). >> >>>>>>> >> >>>>>>> a5 search charset utf-8 body "mall" >> >>>>>>> * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 >> 240 >> >>>>>>> 249 >> >>>>>>> 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 >> 385 >> >>>>>>> 389 390 >> >>>>>>> 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 >> 531 >> >>>>>>> 532 533 >> >>>>>>> 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 >> 672 >> >>>>>>> 677 692 >> >>>>>>> 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 >> 821 >> >>>>>>> 833 855 >> >>>>>>> 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 >> 1010 >> >>>>>>> 1011 >> >>>>>>> 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 >> 1137 >> >>>>>>> 1140 >> >>>>>>> 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 >> 1282 >> >>>>>>> 1286 >> >>>>>>> 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 >> 1471 >> >>>>>>> 1494 >> >>>>>>> 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 >> 1606 >> >>>>>>> 1611 >> >>>>>>> 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 >> 1735 >> >>>>>>> 1781 >> >>>>>>> 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 >> 2002 >> >>>>>>> 2006 >> >>>>>>> 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 >> 2208 >> >>>>>>> 2210 >> >>>>>>> 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 >> 2377 >> >>>>>>> 2378 >> >>>>>>> 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 >> 2539 >> >>>>>>> 2551 >> >>>>>>> 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 >> 2754 >> >>>>>>> 2780 >> >>>>>>> 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 >> 2936 >> >>>>>>> 2938 >> >>>>>>> 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 >> 3161 >> >>>>>>> 3178 >> >>>>>>> 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 >> 3386 >> >>>>>>> 3428 >> >>>>>>> 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 >> 3801 >> >>>>>>> 3808 >> >>>>>>> 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 >> 3953 >> >>>>>>> 3954 >> >>>>>>> 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 >> 4059 >> >>>>>>> 4063 >> >>>>>>> 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 >> 4145 >> >>>>>>> 4146 >> >>>>>>> 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 >> 4258 >> >>>>>>> 4259 >> >>>>>>> 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 >> 4502 >> >>>>>>> 4538 >> >>>>>>> 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 >> 4594 >> >>>>>>> 4604 >> >>>>>>> 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 >> 4787 >> >>>>>>> 4788 >> >>>>>>> 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 >> 4901 >> >>>>>>> 4905 >> >>>>>>> 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 >> 5129 >> >>>>>>> 5131 >> >>>>>>> 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 >> 5276 >> >>>>>>> 5277 >> >>>>>>> 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 >> 5391 >> >>>>>>> 5392 >> >>>>>>> 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 >> 5642 >> >>>>>>> 5647 >> >>>>>>> 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 >> 5866 >> >>>>>>> 5867 >> >>>>>>> 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 >> 5958 >> >>>>>>> 5990 >> >>>>>>> 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 >> 6233 >> >>>>>>> 6271 >> >>>>>>> 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 >> 6482 >> >>>>>>> 6492 >> >>>>>>> 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 >> 6852 >> >>>>>>> 6859 >> >>>>>>> 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 >> 7042 >> >>>>>>> 7108 >> >>>>>>> 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 >> 7317 >> >>>>>>> 7322 >> >>>>>>> 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 >> 7417 >> >>>>>>> 7418 >> >>>>>>> 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 >> 7502 >> >>>>>>> 7503 >> >>>>>>> 7558 7588 7589 7628 7685 7695 7699 7703 7723 >> >>>>>>> *a5 OK Search completed (7.846 secs).* >> >>>>>>> >> >>>>>>> >> >>>>>>> Searching against "subject" is pretty fast, few miliseconds ... >> >>>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>> >> >> >

1 0

Indexing Mail faster
by Kevin Laurie 29 Jan '15

29 Jan '15

Hi, I am trying to get faster search results on our webmail client(Roundcube). Besides using Lucene for FTS are there other options? Would having all mails indexed give fast results? Currently the time it takes to search 25,000mails is 4mins. If indexed how much faster are we looking at? Really appreciate if someone could advise about this. Thanks Kevin

8 28

Uniqueness of dovecot mailbox-guids
by Christian Binder Stadt Freilassing 29 Jan '15

29 Jan '15

Sorry if this is a duplicate message. I'm new in this mailing list and not sure if the first message really got posted ... Dovecot version: 2.2.9 We're using doveadm backup to migrate user mailboxes from cyrus to dovecot. I defined a master user for the old cyrus mailserver in imapc backend to get access to all the mailboxes. Now I realized that after migration every user's "Inbox" or "Trash" on the new dovecot server has the same mailbox-guid, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox dd9456... trash 5549cc... Is this a wanted behaviour? If i do not migrate with doveadm backup and just manually create mailboxes, the mailbox-guids are globally unique, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox c543fe... trash t49ccc... Thanks! Christian

1 0

Re: Indexing Mail faster
by Kevin Laurie 29 Jan '15

29 Jan '15

Dear Peter, Noted. Thanks for your input. Appreciate it. At this point my urgent most priority is to get FTS. Waiting 3 mins for a body search email is bad(but of course my email box is large) I need to have this sorted out by today as I have been putting this off for too long mostly because of lack of troubleshooting documentation online(if i do figure this out will create some for guidance) Regards Kevin On Thu, Jan 29, 2015 at 1:18 PM, Peter Hodur <petehodur(a)gmail.com> wrote: > > > On Thursday, January 29, 2015, Kevin Laurie <superinterstellar(a)gmail.com> > wrote: > >> Dear Peter, >> Oh. Sorry(didnt know you were addressing someone else) my apologies. But >> as you can see , I am desperately trying to address this issue. >> > > No problem ;))) i wrote because my search result is good but not so > good as someone wrote here couple of days ago. > > > >> I have 1 disk with 200GB running on a VPS. >> The file system is ext4. >> > > Im not dovecot expert, but i think this is your problem. You need more > iops. If you need performance, VPS may be (if connected to fast SAN) but > basicly is not a good choice. > > >> I think I will need to implement FTS to fix this as I will need body >> searches. >> > > FTS could be a solution. You are right. > > >> Have you tried FTS before? >> > > Im sorry, never :( > > But may be someone more skilled than me could answer the main question, > how much dovecot index helps in case imap search agains body. > > Im not sure, but my opinion is that it helps little (may be not at all) > > Ps: may be you use it, but do not forget to setup delivering from mail > server via LDA or LMTP instead of direct to maildirs. This is better > because dovecot updates your indexes at the time of message arrival. Later > accesses should be faster. > > Peter > > > >> >> >> Regards >> Kevin >> > >> >> On Thu, Jan 29, 2015 at 12:55 PM, Peter Hodur <petehodur(a)gmail.com> >> wrote: >> >>> Kevin, >>> >>> My message was not addressed to you ;) because my results are NOT so >>> good as someone wrote it here. I can fulltext cca 8k messages in 7-8 >>> seconds. >>> >>> Someone wrote that he has results cca 22k messages in 4 seconds :( >>> >>> Im not sure, but the answer is may be in the index and disk subsystem. >>> >>> If and only if dovecot index does not have saved in the index keywords >>> from BODY of messages the problem is your/my disk subsystem. >>> >>> Generally, messages from mailing list like this are pretty small. The >>> problem is that dovecot must read all messages. If you have maildir its >>> mean open and read many files. >>> >>> But my test search was agains archive of my personal inbox - so not onoy >>> small messages like this but also messages with big attachments etc. >>> >>> If dovecot index only headers, than it in case of full text search read >>> whole messages - mime parts allows that plain text body can be after >>> attachment etc. >>> >>> And here is it all about iops and throughput. >>> >>> How many disks you have? And what setup? >>> >>> Generally, the only good setup is many smaller disks in RAID 10 >>> (stripped mirrors) and if your fs allows it, with added read cache - in >>> case of zfs: l2arc on ssd. >>> >>> >>> >>> Pete >>> >>> >>> >>> On Thursday, January 29, 2015, Kevin Laurie <superinterstellar(a)gmail.com> >>> wrote: >>> >>>> Hi Peter, >>>> Sorry, >>>> >>>> I think its maildir >>>> Output of my dovecot -n is listed below:- >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS >>>> auth_master_user_separator = * >>>> auth_mechanisms = PLAIN LOGIN >>>> dict { >>>> acl = mysql:/etc/dovecot/dovecot- >>>> share-folder.conf >>>> quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf >>>> } >>>> first_valid_uid = 2000 >>>> last_valid_uid = 2000 >>>> listen = * >>>> log_path = /var/log/dovecot.log >>>> mail_debug = yes >>>> mail_gid = 2000 >>>> mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ >>>> mail_plugins = quota fts >>>> mail_uid = 2000 >>>> managesieve_notify_capability = mailto >>>> managesieve_sieve_capability = fileinto reject envelope >>>> encoded-character vacation subaddress comparator-i;ascii-numeric relational >>>> regex imap4flags copy include variables body enotify environment mailbox >>>> date ihave >>>> namespace { >>>> inbox = yes >>>> location = >>>> mailbox Drafts { >>>> auto = subscribe >>>> special_use = \Drafts >>>> } >>>> mailbox Junk { >>>> auto = subscribe >>>> special_use = \Junk >>>> } >>>> mailbox Sent { >>>> auto = subscribe >>>> special_use = \Sent >>>> } >>>> mailbox "Sent Messages" { >>>> auto = no >>>> special_use = \Sent >>>> } >>>> mailbox Spam { >>>> auto = no >>>> special_use = \Junk >>>> } >>>> mailbox Trash { >>>> auto = subscribe >>>> special_use = \Trash >>>> } >>>> prefix = >>>> separator = / >>>> type = private >>>> } >>>> namespace { >>>> list = children >>>> location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u >>>> prefix = Shared/%%u/ >>>> separator = / >>>> subscriptions = yes >>>> type = shared >>>> } >>>> passdb { >>>> args = /etc/dovecot/dovecot-mysql.conf >>>> driver = sql >>>> } >>>> passdb { >>>> args = /etc/dovecot/dovecot-master-users-password >>>> driver = passwd-file >>>> master = yes >>>> } >>>> plugin { >>>> acl = vfile >>>> acl_shared_dict = proxy::acl >>>> auth_socket_path = /var/run/dovecot/auth-master >>>> quota = dict:user::proxy::quotadict >>>> quota_rule = *:storage=1G >>>> quota_warning = storage=85%% quota-warning 85 %u >>>> quota_warning2 = storage=90%% quota-warning 90 %u >>>> quota_warning3 = storage=95%% quota-warning 95 %u >>>> sieve = /%Lh/sieve/dovecot.sieve >>>> sieve_default = /var/vmail/sieve/dovecot.sieve >>>> sieve_dir = /%Lh/sieve >>>> sieve_global_dir = /var/vmail/sieve >>>> } >>>> protocols = pop3 imap sieve lmtp >>>> service auth { >>>> unix_listener /var/spool/postfix/private/dovecot-auth { >>>> group = postfix >>>> mode = 0666 >>>> user = postfix >>>> } >>>> unix_listener auth-master { >>>> group = vmail >>>> mode = 0666 >>>> user = vmail >>>> } >>>> unix_listener auth-userdb { >>>> group = vmail >>>> mode = 0660 >>>> user = vmail >>>> } >>>> } >>>> service dict { >>>> unix_listener dict { >>>> group = vmail >>>> mode = 0660 >>>> user = vmail >>>> } >>>> } >>>> service imap-login { >>>> process_limit = 500 >>>> service_count = 1 >>>> } >>>> service lmtp { >>>> executable = lmtp -L >>>> inet_listener lmtp { >>>> port = 24 >>>> } >>>> process_min_avail = 5 >>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>> group = postfix >>>> mode = 0600 >>>> user = postfix >>>> } >>>> user = vmail >>>> } >>>> service pop3-login { >>>> service_count = 1 >>>> } >>>> service quota-warning { >>>> executable = script /usr/local/bin/dovecot-quota-warning.sh >>>> unix_listener quota-warning { >>>> group = vmail >>>> mode = 0660 >>>> user = vmail >>>> } >>>> } >>>> ssl = required >>>> ssl_cert = </etc/ssl/certs/sicl.net.crt >>>> ssl_key = </etc/ssl/private/sicl.net.key >>>> userdb { >>>> args = /etc/dovecot/dovecot-mysql.conf >>>> driver = sql >>>> } >>>> protocol lda { >>>> auth_socket_path = /var/run/dovecot/auth-master >>>> lda_mailbox_autocreate = yes >>>> log_path = /var/log/dovecot-sieve.log >>>> mail_plugins = quota fts sieve >>>> postmaster_address = root >>>> } >>>> protocol lmtp { >>>> info_log_path = /var/log/dovecot-lmtp.log >>>> lmtp_save_to_detail_mailbox = yes >>>> mail_plugins = quota sieve >>>> postmaster_address = postmaster >>>> recipient_delimiter = + >>>> } >>>> protocol imap { >>>> imap_client_workarounds = tb-extra-mailbox-sep >>>> mail_plugins = quota fts imap_quota >>>> } >>>> protocol pop3 { >>>> mail_plugins = quota fts >>>> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh >>>> pop3_uidl_format = %08Xu%08Xv >>>> } >>>> >>>> >>>> On Thu, Jan 29, 2015 at 12:37 PM, Kevin Laurie < >>>> superinterstellar(a)gmail.com> wrote: >>>> >>>>> Dear Peter, >>>>> >>>>> My inbox is MDA_external >>>>> Storage: 17GB of 24GB >>>>> >>>>> Subject / From / To is fast but FTS(Full Text Search) for body is >>>>> horrible. I suppose this is where we need Apache Solr. >>>>> >>>>> Do you think my mail storage format is bad? Do I need to change for >>>>> better performance? >>>>> Please advise >>>>> Kevin >>>>> >>>>> >>>>> >>>>> On Thu, Jan 29, 2015 at 12:25 PM, Peter Hodur <petehodur(a)gmail.com> >>>>> wrote: >>>>> >>>>>> > * Kevin Laurie <superinterstellar(a)gmail.com> 2015.01.24 19:41: >>>>>> > >>>>>> > > Currently the time it takes to search 25,000mails is 4mins. If >>>>>> indexed >>>>>> > how >>>>>> > > much faster are we looking at? >>>>>> > >>>>>> > With a current version of Dovecot a search is pretty fast _without_ >>>>>> using >>>>>> > external indexes. I have a view defined (virtual plugin) with >>>>>> around 22.000 >>>>>> > messages in it, and searching the full view only takes 2.5 seconds: >>>>>> > >>>>>> > >>>>>> hmmm, could you please tell me more about your setup? What storage >>>>>> format >>>>>> do you use? Maildir or? What is the cumulative size of your messages? >>>>>> >>>>>> My results without FTS on ZFS FS with SSD L2ARC are not so good: >>>>>> >>>>>> a4 select INBOX._OLD-OUTLOOK >>>>>> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk >>>>>> JunkRecorded $Forwarded) >>>>>> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk >>>>>> $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. >>>>>> ** 7748 EXISTS* >>>>>> * 0 RECENT >>>>>> * OK [UIDVALIDITY 1421198037] UIDs valid >>>>>> * OK [UIDNEXT 11509] Predicted next UID >>>>>> * OK [HIGHESTMODSEQ 12204] Highest >>>>>> a4 OK [READ-WRITE] Select completed (0.001 secs). >>>>>> >>>>>> a5 search charset utf-8 body "mall" >>>>>> * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 240 >>>>>> 249 >>>>>> 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 385 >>>>>> 389 390 >>>>>> 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 531 >>>>>> 532 533 >>>>>> 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 672 >>>>>> 677 692 >>>>>> 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 821 >>>>>> 833 855 >>>>>> 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 1010 >>>>>> 1011 >>>>>> 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 1137 >>>>>> 1140 >>>>>> 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 1282 >>>>>> 1286 >>>>>> 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 1471 >>>>>> 1494 >>>>>> 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 1606 >>>>>> 1611 >>>>>> 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 1735 >>>>>> 1781 >>>>>> 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 2002 >>>>>> 2006 >>>>>> 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 2208 >>>>>> 2210 >>>>>> 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 2377 >>>>>> 2378 >>>>>> 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 2539 >>>>>> 2551 >>>>>> 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 2754 >>>>>> 2780 >>>>>> 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 2936 >>>>>> 2938 >>>>>> 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 3161 >>>>>> 3178 >>>>>> 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 3386 >>>>>> 3428 >>>>>> 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 3801 >>>>>> 3808 >>>>>> 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 3953 >>>>>> 3954 >>>>>> 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 4059 >>>>>> 4063 >>>>>> 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 4145 >>>>>> 4146 >>>>>> 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 4258 >>>>>> 4259 >>>>>> 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 4502 >>>>>> 4538 >>>>>> 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 4594 >>>>>> 4604 >>>>>> 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 4787 >>>>>> 4788 >>>>>> 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 4901 >>>>>> 4905 >>>>>> 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 5129 >>>>>> 5131 >>>>>> 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 5276 >>>>>> 5277 >>>>>> 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 5391 >>>>>> 5392 >>>>>> 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 5642 >>>>>> 5647 >>>>>> 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 5866 >>>>>> 5867 >>>>>> 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 5958 >>>>>> 5990 >>>>>> 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 6233 >>>>>> 6271 >>>>>> 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 6482 >>>>>> 6492 >>>>>> 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 6852 >>>>>> 6859 >>>>>> 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 7042 >>>>>> 7108 >>>>>> 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 7317 >>>>>> 7322 >>>>>> 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 7417 >>>>>> 7418 >>>>>> 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 7502 >>>>>> 7503 >>>>>> 7558 7588 7589 7628 7685 7695 7699 7703 7723 >>>>>> *a5 OK Search completed (7.846 secs).* >>>>>> >>>>>> >>>>>> Searching against "subject" is pretty fast, few miliseconds ... >>>>>> >>>>> >>>>> >>>> >>

1 0

Dovecot & gethostbyname() vulnerability
by Timo Sirainen 28 Jan '15

28 Jan '15

So just a quick note: Dovecot in general doesn't do DNS lookups, except the ones in configuration files and such. So I don't think there are any Dovecot setups which do DNS lookups for untrusted hostnames. Also, gethostbyname() is used only if IPv6 support isn't compiled into Dovecot. And IPv6 support is enabled by default if the libc is detected to support it, so pretty much all systems use it.

1 0

Uniqueness of dovecot mailbox-guids
by Christian Binder Stadt Freilassing 28 Jan '15

28 Jan '15

Dovecot version: 2.2.9 I'm using doveadm backup to migrate about hundred user mailboxes from cyrus to dovecot. I defined a master user for the old cyrus mailserver in imapc backend to get access to all the mailboxes. Now I realized that after migration every user's "Inbox" or "Trash" on the new dovecot server has the same mailbox-guid, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox dd9456... trash 5549cc... Is this a wanted behaviour? If i do not migrate with doveadm backup and just manually create mailboxes, the mailbox-guids are globally unique, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox c543fe... trash t49ccc... Thanks! Christian

1 0

dovecot migration
by mkawada＠redhat.com 28 Jan '15

28 Jan '15

Dear List, To migrate a dovecot server from dovecot-1.x to dovecot-2.x, then I wonder if the UIDL format "pop3_uidl_format" setting is required in the configuration file for clients not to re-download the same mails from the migrated mail server. Currently, no UIDL format is set in /etc/dovecot file of the one using dovecot-1.x server which means it is commented out as default like below. ]# grep pop3_uidl_format /etc/dovecot.conf #pop3_uidl_format = %08Xu%08Xv <---- Does this mean the "pop3_uidl_format" is in use as "%08Xu%08Xv" by default? I see the same in /etc/dovecot/conf.d/20-imap.conf of the one using dovecot-2.x, and have not been able to find any information saying it's the defualt setting. Here are the questions: 1.Can we say the "pop3_uidl_format = %08Xu%08Xv" is the enabled default setting? Or any ways to check the default? 2.Do they need to specify anything for the "pop3_uidl_format =" field for their migration in this case? Env infomation: dovecot-1.x dovecot-2.x mail_location = mbox Any information/clues would be greatly appreciated. Thanks,

3 4

sendmail not invoking dovecot-lda
by alvin 27 Jan '15

27 Jan '15

hi dovecot mailinglist - Configuration FreeBSD-9.3 sendmail -d0.1 == sendmail-8.14.9 <<-- dovecot --version == dovecot-2.2.15 # =================================================================== # I'm trying to get sendmail to invoke dovecot.m4 ( dovecot-lda ) to # deliver emails to dovecot's virtual users ( /etc/dovecot/passwd ) # or mysql/postgresql virtual users # # # sendmail -bv dovecot.VirtualUser@localhost --> returns "User Unknown" # # cat test.msg.txt | dovecot-lda -d dovecot.VirtualUser ===> works # email is delivered to dovecot.VirtualUser inbox # # =================================================================== - there are tons of howto and comments for dovecot + postfix and very little for dovecot + sendmail ... therefore, it'd be great if we/i can get this dovecot.m4 problem resolved ------------------------------------------------------------------------------ - status of what works and what does NOT work while testing dovecot's LDA ... ------------------------------------------------------------------------------ # # fyi.. recepient user is in the form of user, user@localhost and user(a)domain.com # sendmail -bv user.in/etc/passwd = deliverable sendmail -bv user.in/etc/mail/virtusertable = deliverable to sendmail virtual user -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown dovecot user user.in/etc/passwd == returns UID/GID/home/mailbox ( deliverable ) dovecot user dovecot.user == returns UID/GID/home/mailbox ( deliverable ) # -------------------------------------------------------------------- # dovecot's LDA delivers the test email to the dovecot/mysql recepient # -------------------------------------------------------------------- cat test-email-with-headers.txt | dovecot-lda -d dovecot.user ( email delivered ) # ---------------------------------------------------------------- -->> # mail/mutt/sendmail canNOT deliver to dovecot recepient == FAILS # ---------------------------------------------------------------- -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown 3 echo "testing mail to dovecot" | mail -v -s "testing mail to dovecot" doveccot.user == user unknown == ------------------------------------------------------- - my (relevant to dovecot ) sendmail.mc file looks like: ------------------------------------------------------- dnl # dnl # local mail delivery dnl # define(`PROCMAIL_MAILER_PATH', `/usr/local/bin/procmail')dnl FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl dnl # dnl # I'm trying to get sendmail to use dovecot.m4 to deliver local mail dnl # to dovecot's LDA ( dovecot-lda ) for mysql virtual users dnl # dnl uncomment and use either feature(...dovecot-lda) or mailer(dovecot.m4) dnl dnl FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda',`/usr/local/libexec/dovecot/dovecot-lda -d $u') dnl MAILER(local) MAILER(smtp) dnl dnl MAILER(dovecot)dnl # did NOT work either dnl dnl # trying to use procmail to deliver local mail to system users ( /etc/passwd ) MAILER(procmail) dnl dnl # is the ordering of dovecot.m4 important relative to (local) and (procmail) dnl dnl # trying to use dovecot-lda to deliver email to dovecot's virtual users dnl # defined in /etc/dovecot/password dnl MAILER(dovecot)dnl # see below dnl dnl End of File ------------------------------------------------------- - my (relevant to dovecot ) sendmail.cf file looks like: ------------------------------------------------------- # # lots of deleted ... ??not?? important until Mlocal ---> MAILER(local) # Mlocal, P=/usr/libexec/mail.local, F=lsDFMAw5:/|@qPSXmnz9, S=EnvFromSMTP/HdrFromL, R=EnvToL/ HdrToL, T=DNS/RFC822/SMTP, A=mail.local -l Mprog, P=/bin/sh, F=lsDFMoqeu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, D=$z:/, T=X-Unix/X-Unix/X-Unix, A=sh -c $u ##################################### ### SMTP Mailer specification ### ##################################### # # lots of deleted ... ??not?? important until procmail stuff ---> MAILER(procmail) # ######################*****############## ### PROCMAIL Mailer specification ### ##################*****################## ##### $Id: procmail.m4,v 8.23 2013-11-22 20:51:14 ca Exp $ ##### Mprocmail, P=/usr/local/bin/procmail, F=DFMSPhnu9, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFr omSMTP, T=DNS/RFC822/X-Unix, A=procmail -Y -m $h $f $u ######################*****######################### ### ### ### 10-Jan-15 amo Dovecot.m4 ### ### ### ### dovecot.m4 stuff ### ### ### ##############################*****################# Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, T=DNS/RFC822/X-Unix, A=/usr/local/libexec/dovecot/dovecot-lda -d $u # # End of sendmail.cf ===================================================================== - dovecot has an example dovecot.m4 ... but it doesn't allow sendmail to call dovecot to deliver local mail with dovecot-lda ===================================================================== http://wiki2.dovecot.org/LDA/Sendmail ######################################### ### DOVECOT Mailer specification ### ######################################### # # # dovecot virtual users defined in: /etc/dovecot/password # # # # cp /etc/mail/dovecot.m4 /usr/share/sendmail/cf/mailer # # # # cd /etc/mail && make # # /etc/rc.d/sendmail restart # # # # the dovecot.m4 stanza are in the resulting sendmail.cf # # # ######################################### Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP, T=DNS/RFC822/X-Unix, A=/usr/local/libexec/dovecot/dovecot-lda -d $u # # i've tried using these other options in lieu of the above : # # F=DFMPShnu9 # added "S" option # F=lsDFMAw5:/|@qSPfhn9, # # S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, # # # # tried adding U= between S= and T= lines # U=root:mail U=dovecot:mail U=dovenull:mail # # End of dovecot.m4 http://wiki2.dovecot.org/LDA/Sendmail also states that one could use these 3 statements in lieu of dovecot.m4 # # this replaces sendmail.cf Mlocal P=mail.local with P=dovecot-lda # FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda', \ `/usr/local/libexec/dovecot/dovecot-lda -d $u') dnl MODIFY_MAILER_FLAGS(`LOCAL', `-f') MAILER(procmail) vi /etc/mail/mailertable # # cd /etc/mail && make && /etc/rc.d/sendmail restart # # make and restart sendmail after editing # virtualdomain.example.com vmail:vmail # other options where sendmail still wasn't able to execute dovecot-lda # virtualdomain.example.com dovecot:local # virtualdomain.example.com local:mail # virtualdomain.example.com root:mail ------------------------------------ - my relevant dovecot config options ------------------------------------ # # are there any other major options i'm missing in the dovecot config files # doveconf dovecot -a # # for /etc/passwd passdb { args = driver = pam .. } # # ------------------------------------------------------------------------ # need to get these dovecot's virtual users to be able send/receive emails # ------------------------------------------------------------------------ # sendmail -bv dovecot.VirtualUser@localhost # dovecot.VirtuaUser@localhost... User unknown # passdb { args = /etc/dovecot/passwd # # dovecot.VirtualUser:{PLAIN}password::::: # # end of file driver = passwd-file .. } # passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql .. } ... ... ... userdb { args = driver = passwd .. } # userdb { args = /etc/dovecot/password driver = passwd-file .. } # userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql .. } # # --------------------------------------------------------------------- # -- How do i get sendmail to execute dovecot-lda for incoming emails # -- directed to dovecot virtual users and/or mysql users ?? # --------------------------------------------------------------------- # # end of txt

2 2

How to quote the special wildcard characters '*' and '?'
by David Davidov 27 Jan '15

27 Jan '15

Hello, I am searching for a way to execute doveadm command against mailbox containing a special wildcard character in its name. For example: # doveadm expunge -u 'me?(a)example.com' mailbox 'INBOX.Junk Mail' SAVEDBEFORE 7 The result is: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users with exit code 75 In the mail log: dovecot: auth: Error: Trying to iterate users, but userdbs don't support it To be clear: the question mark is a part of the mailbox's name. Not an attempt to use it as wildcard. Is there a way 1. To quote the special characters or 2. To disable wildcard matcing in the -u parameter regards, -- David Davidov | System Administrator | ICDSoft Ltd.

2 1

"error: number of redirect actions exceeds policy limit"
by Kai Sommer 27 Jan '15

27 Jan '15

Greetings, after testing a lot with my system I have to write to this list to ask for help from the cummunity. My system: * Ubuntu 12.04.5 LTS * Dovecot 2.0.19 * Dovecot config (extract) ––––– 8< ––––– plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_redirects = 10 } ––––– >8–––––– * Sieve filter rules ––––– 8< ––––– require ["copy"]; # rule:[Alles weiterleiten] if true { redirect "mail_1@mydomain"; redirect "mail_2@mydomain"; redirect "mail_3@mydomain"; redirect "mail_4@mydomain"; redirect "mail_5@mydomain"; redirect "mail_6@mydomain"; keep; stop; } ––––– >8–––––– Problem scenario: With the redirect-filter (see above) Dovecot redirects a incoming email to the first address and stops withe the message "error: number of redirect actions exceeds policy limit" (.dovecot.sieve.log). In my mail.log I see the message "Error: sieve: execution of script $PATH/.dovecot.sieve failed, but implicit keep was successful (user logfile $PATH/.dovecot.sieve.log may reveal additional details)". If I reduce the count of recipients to four – which is the Dovecot default number – everything will redireced as expected. I only could find an old problem report here [0] where as the posted solution doesn't work for me (as you can see above). (Also I tried the rule "redirect :copy" that doesn't changed something …) [0] <http://dovecot.org/pipermail/dovecot/2012-August/085038.html> → Is this problem known? – Is there a solution for that problem? Thanks a lot and best regards! – Kai -- Kai Sommer sig: 5E2E FA58 43BD 5599 4068 | pubkey: http://ma.ximi.se/mekey

2 1

Thunderbird: improper command pipelining after EHLO
by Leander Schäfer 27 Jan '15

27 Jan '15

I couldn't find working solutions for this anomalie on the net. What does this mean and does someone know how to fix this? postfix/smtpd[18757]: improper command pipelining after EHLO from unknown[192.168.10.233]: QUIT\r\n Thanks Best Regards, Leander

4 4

imap-login: Fatal: pipe() failed: Too many open files
by Leander Schäfer 26 Jan '15

26 Jan '15

Hi I keep on getting errors and can't connect/login to Dovecot. I did my research but unfortunately without success. It is for sure not ulimit because ulimit is set to unlimited per default already. Still , it complains about "Too many open files" but this is a test system and the service dovecot and postfix have just been started. No one except me is testing on this system. This is very weired ;/ root@WM-01 [~]$ service dovecot stop; service postfix stop ; echo "" > /var/log/maillog ; echo "" > /var/log/slapd.log ; service postfix start ; service dovecot start Stopping dovecot. Waiting for PIDS: 3245. postfix/postfix-script: stopping the Postfix mail system postfix/postfix-script: starting the Postfix mail system Starting dovecot. root@WM-01 [~]$ telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. root@WM-01 [~]$ cat /var/log/mail Jan 26 01:39:12 WM-01 dovecot: imap-login: Fatal: pipe() failed: Too many open files Jan 26 01:39:12 WM-01 dovecot: master: Error: service(imap-login): command startup failed, throttling for 4 secs If I uncoment port = 0 in POP and IMAP then it looks even worse then above. Telnet is not even closed by remote host and three of the "Too many open files" errors are showing up in maillog: root@WM-01 [~]$ telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. ^] telnet> Connection closed. root@WM-01 [~]$ cat /var/log/maillog Jan 26 01:45:19 WM-01 postfix/postfix-script[60560]: starting the Postfix mail system Jan 26 01:45:19 WM-01 postfix/master[60562]: daemon started -- version 1.0, configuration /usr/local/etc/postfix Jan 26 01:45:19 WM-01 dovecot: master: Dovecot v2.2.15 starting up for imap, pop3, lmtp Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: socket(login) failed: Too many open files Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: auth: connect(login) failed: Too many open files Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: net_accept() failed: Too many open files root@WM-01 [~]$ Does anyone have an idea what could be wrong here? I attached the relevant configuration below. I use OpenLDAP as backend. Please let me know if you need more information. Thank you very much. Best Regards, Leander # ========================== dovecot -n ============================== # # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.1-RELEASE amd64 ufs auth_debug = yes auth_debug_passwords = yes auth_failure_delay = 3 secs auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Welcome to Mail Server. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_debug = yes mail_gid = vmail mail_location = maildir:/var/mail/%d/%n mail_plugins = quota expire mail_log notify mail_privileged_group = mail mail_temp_dir = /tmp/Dovecot mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=1G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh postmaster_address = info(a)netocean.de sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service pop3-login { inet_listener pop3s { port = 0 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = </etc/ssl/RootCA/certs/192.168.50.101.pem ssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA ssl_key = </etc/ssl/RootCA/certs/192.168.50.101.key ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_ssl = yes protocol lda { mail_plugins = sieve } protocol imap { mail_max_userip_connections = 3 mail_plugins = imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 3 } # ======================== dovecot/conf.d/auth-ldap.conf.ext =================== # passdb { driver = ldap # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # <doc/wiki/UserDatabase.Prefetch.txt> userdb { driver = prefetch } userdb { driver = ldap args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext # Default fields can be used to specify defaults that LDAP may override #default_fields = home=/home/virtual/%u } # ======================== dovecot/dovecot-ldap.conf.ext =================== # hosts = 127.0.0.1:389 dn = uid=dovecot,ou=systemuser,ou=mail,dc=MyDomain,dc=TLD dnpass = TopSecret tls = yes tls_cert_file = /etc/ssl/RootCA/certs/192.168.50.101.pem tls_key_file = /etc/ssl/RootCA/certs/192.168.50.101.key debug_level = -1 auth_bind = yes ldap_version = 3 base = ou=accounts,ou=mail,dc=MyDomain,dc=TLD scope = subtree user_attrs = mailStorageDirectory=home, mailUidNumber=uid, mailGidNumber=gid, mailQuotaSize=quota_rule=*:bytes=%$, mailQuotaCount=quota_rule2 user_filter = (&(objectClass=mailAccount)(mailAddress=%u)) pass_attrs = mailAddress=user,mailPassword=password pass_filter = (&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=%u)) default_pass_scheme = SSHA

3 8

Re: LDA input validation
by Joseph Tam 26 Jan '15

26 Jan '15

St?phane Cottin writes: >> You're stilling going to lose contents. If dspam fails, the mail >> is dumped, the LDA returns exit code 75, and the MTA will probably >> issue a bounce Email to the sender. > > from dovecot-lda man page : > > 75 A temporary failure. This is returned for almost all failures. See the log file for > details. (EX_TEMPFAIL) > > The mta keep the mail and retry delivery later, which is what I need. Ah, my mistake. But ... > I don't want to deliver email on dspam error, I prefer TEMPFAIL and > future retries, so I have a chance to fix the issue later without > loosing or deliver unprocessed mails ... you only have a limited time to fix it. Depending on the sending MTA's retry schedule, you might have only a few days or less to to deal with it before the sending MTA stops trying. If it's important not to lose mail, then the deliver on failure strategy would be more appropriate. You can even copy the input message that caused dpam to crash to an inspection queue for later perusal. Joseph Tam <jtam.home(a)gmail.com>

2 1

Re: LDA input validation
by Joseph Tam 26 Jan '15

26 Jan '15

St?phane Cottin writes: > dspam already send errors to syslog, the point here is to never loose > email contents. This was a wrong design, i'm now use a wrapper instead > ( see my previous post for details ). You're stilling going to lose contents. If dspam fails, the mail is dumped, the LDA returns exit code 75, and the MTA will probably issue a bounce Email to the sender. If you really don't want the recipient to lose Email, you should buffer the input into a file, run dspam on it, and if the output is not-null, pipe it to dovecot-lda, otherwise pipe the original input. Joseph Tam <jtam.home(a)gmail.com>

4 6

Re: sieve filter not working
by Christian Kivalo 24 Jan '15

24 Jan '15

Am 23. Jänner 2015 22:02:06 MEZ, schrieb Michael Williamson <michael.h.williamson(a)gmail.com> Hello, >> Another question: >> Local mail delivery of received mail from external sources does >indeed >> work? > >Yes. > That is good. >> If so, was that log excerpt in one of the former mails an example of >> such delivery? > >There are dovecot messages in "/var/log/maillog" showing only >imap-login & disconnected. >When I monitor "/var/log/maillog" while sending spam, there are no >messages containing 'dovecot'. I attached that output to a previous >post. It has only messages from postfix and amavis. > Ther should be messages from dovecot when receiving mail at least when dovecot lda has something to do with delivery. >home_mailbox = Maildir/ This tells postfix where to deliver the mails. dovecot is not configured to delivet mails. Take a look at dovecot wiki there are examples of how to incorporate dovecot lda into thendelivery process. >> I believe dovecot has actually nothing to do with delivering your >mails >> so no sieve filters are applied... > >Is it postfix? > yes, seems so >>> Thanks, >>> -Mike - christian

3 3

from Cyrus to Dovecot, seen status on cyrus.index files
by Norberto Bensa 24 Jan '15

24 Jan '15

Hello, Using cyrus2dovecot (https://github.com/a-schild/cyrus2dovecot) I have migrated some test mailboxes from Cyrus (2.4.17) to Dovecot (2.2.9). The problem I'm having is that the seen status is in cyrus.index files, which cyrus2dovecot doesn't understand. Does anyone use cyrus2dovecot for migrations? If so, how do you deal with seen status when it is stored in the .index files? Many thanks in advance, Norberto

4 6

Re: LDA input validation
by Joseph Tam 24 Jan '15

24 Jan '15

St?phane Cottin writes: > A typical .qmail file : > > | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" | /usr/bin/preline -f /usr/lib/dovecot/dovecot-lda -d "$EXT@$USER" -a "$EXT@$USER" -m "$EXT2" > > When dspam segfault or is killed, preline receive an empty content and > only pass to dovecot-lda two headers ( Return-Path and Delivered-To ). > Then dovecot-lda delivers successfully and the original message > contents are lost. Steffen Kaiser rightly points out: > That's the problem of the used Unix pipe and broken by design. The right > member of the pipe (preline) runs independendly of dspam and recieves no > content via pipe. Why should preline or dovecot-lda believe there is an > error? [... and suggests a wrapper script that aborts delivery on empty > dspam output.] I'm not sure if it's appropriate under your circumstances, but you can also pass stderr to stdout as well, and the recipient will get some diagnostic message, which may or may not be helpful. | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" 2>&1 | ... Joseph Tam <jtam.home(a)gmail.com>

2 1