- dovecot - dovecot.org

Advice on once a day message delivery setup
by KT Walrus 14 Jun '16

14 Jun '16

I’m adding once a day mail delivery to my site. Messages are marked by the sender as “overnight” or “once a week” delivery. The way I’m planning on implementing this is to queue messages until midnight in a MySQL database. Each mailbox will be kept in two Dovecot mailstores. The first mailstore will give the users IMAP access to their mailbox. A second mailstore will hold the next day’s new messages. At midnight, a cron job runs to send messages in the MySQL database out to the second mailstore. Then, at 6am, a second cron job will run to sync the two mailstores using doveadm sync. What I am expecting to happen is that during the day, notification messages (from the site) may be delivered to the first mailstore (the one providing IMAP access to the user) but no messages from other users sent during the day will come until after 6am the next day. Each time the user submits a new message, a notification message is sent back to the sender with a link for editing the queued message in the MySQL database and an indication of when it is scheduled for delivery. I am kind of assuming that the morning sync process is fast enough so it can easily complete before noon (in 6 hours) even if I end up having lots of mailboxes on each fully loaded Dovecot server. Is the doveadm sync process reliable and efficient enough for this type of “once a day” morning new message mail delivery? Or, should I just start delivering messages after midnight and not bother with the second mailstore and subsequent sync? Just looking for any advice… I kind of like the idea of modeling my mail service after the US Post Office where the mailman delivers new mail once a day rather than like Twitter/Facebook where messages are posted in real time to encourage users to monitor their boxes throughout the day. Kevin

3 2

doveadm-server protocol change?
by Heiko Schlittermann 13 Jun '16

13 Jun '16

Hi, I'm doing quota checks from a remote machine (the real setup is a bit more complex, if necessary I can explain it in more detail, but I just extracted the bits that are easily reproduceable) # nc backend1 24245 VERSION doveadm-server 1 0 PLAIN agrVMDvHgz0ya2HHzax5svwB2ZHS¹ + heiko quota get But since the backend is upgraded to 2.2.22 it's not possible anymore. The exuse in the log of the backend is: dovecot: doveadm(149.1.1.1¹): Fatal: USER environment is missing and -u option not used Running the `doveadm quota get -u heiko` locally on the backend works as expected. But using the the doveadm-server it doesn't. # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-22-generic x86_64 Ubuntu 16.04 LTS … auth_cache_negative_ttl = 0 auth_cache_ttl = 0 auth_master_user_separator = * base_dir = /run/dovecot/ imap_metadata = yes lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_attribute_dict = file:%h/dovecot-attributes mail_location = maildir:~:INBOX=/volumes/dovecot/inbox/%2.256Nn/%n:INDEX=/volumes/dovecot/cache/%2.256Nn/%n mail_plugins = quota mbox_md5 = all mmap_disable = yes namespace inbox { … passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.passdb.conf.ext driver = ldap } plugin { quota = maildir:User quota quota_grace = 10%% sieve = /volumes/dovecot/sieve/%2.256Nn/%n/.dovecot.sieve sieve_dir = /volumes/dovecot/sieve/%2.256Nn/%n } pop3_uidl_format = %v.%u protocols = " imap lmtp pop3" service auth { extra_groups = ssl-cert } service doveadm { inet_listener { port = 24245 } } service lmtp { inet_listener lmtp { port = 2525 } } ssl = required ssl_cert = </etc/ssl/certs/snakeoil-crt.pem ssl_key = </etc/ssl/private/snakeoil-key.pem userdb { driver = prefetch override_fields = home=/volumes/dovecot/home/%2.256Nn/%n uid=vmail gid=vmail } userdb { args = /etc/dovecot/dovecot-ldap.userdb.conf.ext driver = ldap override_fields = home=/volumes/dovecot/home/%2.256Nn/%n uid=vmail gid=vmail } verbose_proctitle = yes protocol lmtp { mail_plugins = quota sieve quota } protocol lda { mail_plugins = quota sieve } protocol imap { mail_plugins = quota imap_quota } local 149.1.1.0/24¹ { doveadm_password = # hidden, use -P to show it } ¹) It's changed :)

5 13

doveadm fts rescan <namespace> doesn't seem(!) to find <namespace>
by Larry Rosenman 13 Jun '16

13 Jun '16

thebighonker.lerctr.org ~ $ doveadm fts rescan \#ARCHIVE doveadm(ler): Error: Namespace prefix not found: #ARCHIVE thebighonker.lerctr.org ~ $ doveadm fts rescan #ARCHIVE thebighonker.lerctr.org ~ $ doveadm fts rescan inbox doveadm(ler): Error: Namespace prefix not found: inbox thebighonker.lerctr.org ~ $ Doveconf -n attached. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281

1 1

Potential bug report: Cannot use ":args" option via Pigeonhole Sieve pipe plugin
by Zeeshan Muhammad 13 Jun '16

13 Jun '16

Hi all, Following the instructions noted at http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe , I am trying to make use of the Pigeonhole sieve pipe plugin to execute an application with custom arguments via my test sieve script: require ["vnd.dovecot.pipe"]; if address :is "to" "test(a)example.com" { pipe :args [ "first-arg", "second-arg" ] "sieve-pipe-example"; } I am editing my test sieve script via Mozilla Thunderbird v45.1.1 using the Sieve email-client addon (v0.2.3h, see https://github.com/thsmi/sieve#releases and https://addons.mozilla.org/en-US/thunderbird/addon/sieve/ ) When I attempt to save the example sieve script above, I get the following error returned by ManageSieve via Mozilla Thunderbird's Sieve addon: "The script could not be saved: Sieve mail filter: line 3: error: unknown tagged argument ':args' for the pipe command (reported only once at first occurrence). Sieve mail filter: error: validation failed." I have setup sieve-pipe-example as follows in my 90-sieve.conf configuration: service sieve-pipe-script { executable = script /usr/bin/echo user = dovenull # socket name is program-name in Sieve unix_listener sieve-pipe-example { } } I cannot see where I am going wrong. I have combed through the specification at http://hg.rename-it.nl/dovecot-2.2-pigeonhole/raw-file/tip/doc/rfc/spec-bos… as well as the official Dovecot/Pigeonhole wiki-pages and cannot find a solution. If I omit the :args option (and the arguments array), the sieve script successfully gets accepted by ManageSieve. Am I encountering a potential bug or am I doing something silly in my test setup? # Dovecot version: v2.2.24 # Pigeonhole version: v0.4.14 # Operating system: CentOS Linux release 7.2.1511 (Core) # CPU architecture: x86_64 # Dovecot -n output (I have filtered my IP addresses and mail server's FQDN): # 2.2.24 (a82c823): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.14 (099a97c) # OS: Linux 3.10.0-327.18.2.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_default_realm = example.com base_dir = /var/run/dovecot/ default_login_user = vpopmail first_valid_uid = 500 info_log_path = /dev/stderr last_valid_uid = 500 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /dev/stderr mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 20 mail_plugins = " fts fts_lucene" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader vnd.dovecot.pipe namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = vpopmail } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_socket_dir = sieve-execute sieve_extensions = +editheader +vnd.dovecot.pipe sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_extprograms } protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = vchkpw mode = 0666 user = vpopmail } } service imap-login { inet_listener imap { address = 127.0.0.1 port = 143 ssl = no } inet_listener imaps { address = 192.168.1.1 port = 993 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { address = 127.0.0.1 port = 110 ssl = no } inet_listener pop3s { address = 192.168.1.1 port = 995 ssl = yes } } service sieve-pipe-script { executable = script /usr/bin/echo group = dovenull user = dovenull } ssl_cert = </etc/ssl/certs/example.com.crt ssl_key = </etc/ssl/certs/example.com.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = vpopmail } verbose_proctitle = yes protocol lda { hostname = mail.example.com info_log_path = /var/log/dovecot/dovecot-lda.log log_path = /var/log/dovecot/dovecot-lda-errors.log mail_plugins = " fts fts_lucene sieve" postmaster_address = postmaster(a)example.com } protocol imap { mail_plugins = " fts fts_lucene imap_zlib" }

2 2

Marking an entire mailbox read
by ＠lbutlr 13 Jun '16

13 Jun '16

I have an archive mailbox that contains in excess of 100,000 mail messages (stored in maildir) that accidentally got marked as unread. It is too large for my mail client to select all the mail and mark it as read without throttling. Is there a simple way that I can use doveadm or something to simply mark every message in that maildir as read? Or do I just loop through every message and move it to <messaage>+S? find ~/Maildir/.Archive/cur/* -type f -name “1*:2” --exec mv {} {}S \; -- By the way, I think you might be the prettiest girl I've ever seen outside the pages of a really filthy magazine

2 2

sieve vacation script exclude based on sender email address
by Harondel J. Sibble 13 Jun '16

13 Jun '16

I thought I'd asked this question a few years ago but can't seem to find any eveidence of that so here goes. I've been looking at the sieve docs and recipes, done a lot of googling but no joy so far. Using stanard vacation script and that works great, however I want to exclude certain sender email addressess from ever receiving a vacation autoresponse, how do I go about adding that to my existing vacation recipe. I suspect my search terminology is what is causing me not to find anything as I typically am using exclude and similar search terms. -- Harondel J. Sibble Sibble Computer Consulting Ltd. Creating Solutions for the small and medium business computer user. harondel(a)pdscc.com (use pgp keyid 0x3CC3CFCE not 0x3AD5C11D) http://www.pdscc.com Blog: http://www.pdscc.com/blog (604) 739-3709 (voice)

2 3

Unknow Imap Fatal Panic
by Sven Roellig 12 Jun '16

12 Jun '16

Hello, when i sync an Mailbox from old Dovecot to the new Dovecot System the new dovecot Imap create an Fatal/Panic log. Error Log <5Je8ahM17oSwCVUI>: Fatal: master: service(imap): child 4670 killed with signal 6 (core dumps disabled) <ArG9ahM174SwCVUI>: Panic: file mail-index-transaction-update.c: line 803 (mail_index_ext_resize): assertion failed: (record_align != (uint16_t)-1) <ArG9ahM174SwCVUI>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8d04e) [0x7fb23526104e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8d13c) [0x7fb23526113c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb2351ffd1e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_ext_resize+0x3d0) [0x7fb2355c1780] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_update_header+0x133) [0x7fb235545493] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5d5bc) [0x7fb2355455bc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_mailbox_create_indexes+0x10) [0x7fb235545710] -> /usr/lib/dovecot/libdovecot-storage.so.0(dbox_mailbox_create_indexes+0x93) [0x7fb235549f93] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5d87a) [0x7fb23554587a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa06cc) [0x7fb2355886cc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_metadata+0x63) [0x7fb23552d253] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_sync_name+0x2a7) [0x7fb2355892d7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_sync+0x12c) [0x7fb2355899ec] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_refresh_force+0xc5) [0x7fb23558b095] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9f571) [0x7fb235587571] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa074f) [0x7fb23558874f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_metadata+0x63) [0x7fb23552d253] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x3a43e) [0x7fb23552243e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_autoexpunge+0x105) [0x7fb2355226b5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_unref+0x88) [0x7fb235532838] -> dovecot/imap(+0x1b224) [0x7fb235c6a224] -> dovecot/imap(client_input+0xba) [0x7fb235c69d9a] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fb23527508c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7fb2352764f1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fb235275115] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fb2352752b8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fb235206273] -> dovecot/imap(main+0x322) [0x7fb235c5cad2] dovecot -n # 2.3.0.alpha0 (90dd3c6) [XI:2:2.3.0~alpha0-1~auto+111]: /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.0.alpha0 (67dfb5a) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.5 ext4 auth_mechanisms = plain login auth_verbose_passwords = plain dict { expire = mysql:/etc/dovecot/dovecot-dict-expire.conf.ext } imap_hibernate_timeout = 10 secs lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_gid = vmail mail_location = mdbox:/var/vmail/dovecot/mailboxes/%d/%n:INDEX=/var/vmail/dovecot/indexes/%d/%n:ALT=/var/vmail_altstorage/dovecot/mailboxes/%d/%n mail_plugins = quota zlib acl expire mail_uid = vmail namespace inbox { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe } mailbox Trash { auto = subscribe autoexpunge = 10 days special_use = \Trash } mailbox name { special_use = \Drafts \Junk \Sent \Trash \Archive } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/var/vmail/dovecot/shared-mailboxes expire = Trash expire2 = Trash/* expire3 = Spam expire_cache = yes expire_dict = proxy::expire quota = dirsize:User quota quota_exceeded_message = Quota überschritten ..... quota_grace = 10%% quota_over_flag_value = TRUE quota_over_script = quota-warning mismatch %u quota_rule = Trash:storage=+500M quota_rule2 = Spam:storage=+500M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is over quota / Mailbox ist voll quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u quota_warning3 = -storage=100%% quota-warning below %u sieve = /var/vmail/sieve/%d/%n/dovecot.sieve sieve_dir = /var/vmail/sieve/%d/%n/dovecot sieve_global_dir = /var/vmail/sieve/sieve sieve_global_path = /var/vmail/sieve/sieveglobalfilter.sieve zlib_save = lz4 zlib_save_level = 9 } protocols = " imap lmtp" service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0660 } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-hibernate { unix_listener imap-hibernate { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { port = 2003 } unix_listener lmtp { mode = 0666 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 10000 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = users mode = 0666 user = vmail } user = root } ssl_cert = </etc/dovecot/private/....de.pem ssl_key = </etc/dovecot/private/.....pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lda { mail_plugins = quota zlib acl expire sieve } protocol imap { mail_plugins = quota zlib acl expire imap_zlib imap_quota imap_acl } protocol lmtp { mail_plugins = quota zlib acl expire sieve postmaster_address = postmaster(a)blabla.blubb quota_full_tempfail = yes } Sorry for my English Sven

1 0

Scalability of Dovecot in the Cloud
by KT Walrus 12 Jun '16

12 Jun '16

Does anyone have any idea of how many IMAP connections a single cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - non-RAID) can be expected to handle in production. The mailboxes are fairly small (average 5MB total - 50MB max, as I don’t store attachments in Dovecot expect those saved through IMAP in the Sent/Drafts folders) and each user will probably have an average of 2 devices that have the mail clients configured to access each mailbox. Can such a server handle 100,000 mailboxes (200,000 devices/clients)? Or is it more like 10,000? Or, even smaller? I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times the price) or split the mailboxes onto multiple VMs. The VM will also be running LMTP and other Dovecot services (I don’t plan on supporting POP3 at this time). The mailboxes will be sync’d to a backup VM running Dovecot for high availability so has some load from this background activity. LMTP will not be that high a load, I think, since most messages will be delivered by at night. But, clients will have IMAP connections 24/7. Just trying to get an idea of the cost of running a potentially huge/growing mail service in the cloud… I’m going to have to support around a million mailboxes before the site will generate significant revenue to support operations. Kevin

4 4

postfix+dovecot and usernames different to e-mail addresses
by Benjamin 10 Jun '16

10 Jun '16

Hi, I want to replace an old mailserver setup with postfix+dovecot. The problem I'm facing now is, that the old system had usernames different to the e-mail address of the user. For example: E-Mail-Address: foo(a)example.com Username: foo-example.com I configured postfix so that it passes mails to dovecot using the following command (master.cf): dovecot unix - n n - - pipe flags=DRhu user=mail-data:mail-data argv=/usr/lib/dovecot /dovecot-lda -a $(recipient) When dovecot now receives an e-mail from postfix it logs dovecot: lda(mail-data): Error: User initialization failed: Namespace 'INBOX.': Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir dovecot: lda: Fatal: Invalid user settings. Refer to server log for more information. My dovecot-ldap configuration looks quite simple: hosts = 192.168.0.1,192.168.0.2 dn = cn=mailadmin,dc=example,dc=com dnpass = foo auth_bind = yes ldap_version = 3 base = ou=users,dc=example,dc=com user_attrs = mailMessageStore=home user_filter = (&(objectClass=qmailUser)(uid=%u)) pass_filter = (&(objectClass=qmailUser)(uid=%u)) I think dovecot does not know that the username is not the e-mail address, but how can I tell him? Furthermore we have alternative addresses here, so for example there may be an e-mail address bar(a)example.com owned by foo-example.com who has foo(a)example.com as primary address. Dovecot itself (IMAP) works fine, so here it can find the correct Maildir for the user. The difference is, that I connect using the username instead of the e-mail address in this case. # dovecot --version 2.2.22 (fe789d2) # postconf -d | grep mail_version mail_version = 3.1.0 # lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04 LTS Release: 16.04 Codename: xenial Thanks for your help Benjamin

2 5

Easily kick and flush dovecot cache for all dovecot mailboxes in a domain
by Murray T. | Dreamscape 10 Jun '16

10 Jun '16

Hi, We are running a dovecot director cluster. We want to be able to easily kick all mailboxes in a domain off all backend mailstores. We also then want to flush the dovecot cache for all dovecot mailboxes in the domain. Is there a single command we can run on one of the front-end director mail proxies or backend mailstores that will do this instead of having to kick or flush each mailbox in a domain individually on each mailstore. Thanks Murray Trainer Regards, Murray T. Level 3 Project Manager P: +61 8 9422 0894 F: +61 8 9422 0801 www.DreamscapeNetworks.com<http://www.dreamscapenetworks.com/> [dreamscape]<http://www.dreamscapenetworks.com/> DISCLAIMER: This e-mail and/or attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose the contents to other persons or use it for any purpose or store or copy the information in any medium. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Dreamscape Networks. Copyright (c) 2016 www.DreamscapeNetworks.com<http://www.dreamscapenetworks.com/>, All rights reserved.

1 0