- dovecot - dovecot.org

Scalability of Dovecot in the Cloud
by KT Walrus 12 Jun '16

12 Jun '16

Does anyone have any idea of how many IMAP connections a single cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - non-RAID) can be expected to handle in production. The mailboxes are fairly small (average 5MB total - 50MB max, as I don’t store attachments in Dovecot expect those saved through IMAP in the Sent/Drafts folders) and each user will probably have an average of 2 devices that have the mail clients configured to access each mailbox. Can such a server handle 100,000 mailboxes (200,000 devices/clients)? Or is it more like 10,000? Or, even smaller? I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times the price) or split the mailboxes onto multiple VMs. The VM will also be running LMTP and other Dovecot services (I don’t plan on supporting POP3 at this time). The mailboxes will be sync’d to a backup VM running Dovecot for high availability so has some load from this background activity. LMTP will not be that high a load, I think, since most messages will be delivered by at night. But, clients will have IMAP connections 24/7. Just trying to get an idea of the cost of running a potentially huge/growing mail service in the cloud… I’m going to have to support around a million mailboxes before the site will generate significant revenue to support operations. Kevin

4 4

postfix+dovecot and usernames different to e-mail addresses
by Benjamin 10 Jun '16

10 Jun '16

Hi, I want to replace an old mailserver setup with postfix+dovecot. The problem I'm facing now is, that the old system had usernames different to the e-mail address of the user. For example: E-Mail-Address: foo(a)example.com Username: foo-example.com I configured postfix so that it passes mails to dovecot using the following command (master.cf): dovecot unix - n n - - pipe flags=DRhu user=mail-data:mail-data argv=/usr/lib/dovecot /dovecot-lda -a $(recipient) When dovecot now receives an e-mail from postfix it logs dovecot: lda(mail-data): Error: User initialization failed: Namespace 'INBOX.': Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir dovecot: lda: Fatal: Invalid user settings. Refer to server log for more information. My dovecot-ldap configuration looks quite simple: hosts = 192.168.0.1,192.168.0.2 dn = cn=mailadmin,dc=example,dc=com dnpass = foo auth_bind = yes ldap_version = 3 base = ou=users,dc=example,dc=com user_attrs = mailMessageStore=home user_filter = (&(objectClass=qmailUser)(uid=%u)) pass_filter = (&(objectClass=qmailUser)(uid=%u)) I think dovecot does not know that the username is not the e-mail address, but how can I tell him? Furthermore we have alternative addresses here, so for example there may be an e-mail address bar(a)example.com owned by foo-example.com who has foo(a)example.com as primary address. Dovecot itself (IMAP) works fine, so here it can find the correct Maildir for the user. The difference is, that I connect using the username instead of the e-mail address in this case. # dovecot --version 2.2.22 (fe789d2) # postconf -d | grep mail_version mail_version = 3.1.0 # lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04 LTS Release: 16.04 Codename: xenial Thanks for your help Benjamin

2 5

Easily kick and flush dovecot cache for all dovecot mailboxes in a domain
by Murray T. | Dreamscape 10 Jun '16

10 Jun '16

Hi, We are running a dovecot director cluster. We want to be able to easily kick all mailboxes in a domain off all backend mailstores. We also then want to flush the dovecot cache for all dovecot mailboxes in the domain. Is there a single command we can run on one of the front-end director mail proxies or backend mailstores that will do this instead of having to kick or flush each mailbox in a domain individually on each mailstore. Thanks Murray Trainer Regards, Murray T. Level 3 Project Manager P: +61 8 9422 0894 F: +61 8 9422 0801 www.DreamscapeNetworks.com<http://www.dreamscapenetworks.com/> [dreamscape]<http://www.dreamscapenetworks.com/> DISCLAIMER: This e-mail and/or attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose the contents to other persons or use it for any purpose or store or copy the information in any medium. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Dreamscape Networks. Copyright (c) 2016 www.DreamscapeNetworks.com<http://www.dreamscapenetworks.com/>, All rights reserved.

1 0

Re: Advice on once a day message delivery setup
by Joseph Tam 10 Jun '16

10 Jun '16

KT Walrus <kevin(a)my.walr.us> writes: > Just looking for any advice? I kind of like the idea of modeling my > mail service after the US Post Office where the mailman delivers new > mail once a day rather than like Twitter/Facebook where messages are > posted in real time to encourage users to monitor their boxes > throughout the day. You've rediscovered digesting, which many mailing list (including this one), will batch up messages and periodically send them as one large message (using MIME multipart or some other combination technique). The only difference is that your doing it at the receiving end. I wouldn't recommend it for all mail, but it could be useful to aggregate high-volume low-priority mail into managable chunks. There are many ways you could accomplish it, but the most straightforward way is to deliver new mail to sideline mailbox, and have a cron script deliver them to the user's INBOX (possibly digesting them into one message). Joseph Tam <jtam.home(a)gmail.com>

1 0

password expire warning for dovecot users in IMAP/POP login
by mkawada＠redhat.com 09 Jun '16

09 Jun '16

Dear list, Is it possible to give a notification about password exprire warning to users authenticated by OpenLDAP when the users login via dovecot using IMAP or POP? For example, when you ssh to a server and/or run ldapsearch, you can be warned with password expire warning like below: # ssh testuser@localhost testuser@localhost's password: Your password will expire in 31 minute(s). <== Last login: Wed Jun 8 12:22:08 2016 from localhost.localdomain ]$ ldapsearch -LLL -D uid=testuser,ou=People,dc=example,dc=com -w redhat "cn=testuser" -e ppolicy ldap_bind: Success (0) (Password expires in 1808 seconds) <== dn: uid=testuser,ou=People,dc=example,dc=com Does the same can be done for dovecot users authenticated by OpenLDAP in IMAP/POP? Thanks, -- Masaharu Kawada

7 11

userdb for imap proxy
by KT Walrus 08 Jun '16

08 Jun '16

If I’m running only imap-login service in my dovecot imap proxy, do I need to configure userdb or only passdb?

2 2

Fw: Can't delete emails from or write replies to Inbox
by chaouche yacine 08 Jun '16

08 Jun '16

----- Forwarded Message ----- From: chaouche yacine <yacinechaouche(a)yahoo.com> To: Edwin Humphries <edwinh(a)earthcaretech.com.au> Sent: Wednesday, June 8, 2016 10:31 AM Subject: Re: Can't delete emails from or write replies to Inbox We have made a copy of our mail server and we sometimes rsync files between the original server and the copy so that when we need to do maintenance work we unplug the production server and put the copy server in lieu. We later found out that rsync needs to be run with the --no-numeric-ids argument to keep owners and groups intact, otherwise you end up with weired owners and groups. Yassine. ________________________________ From: Edwin Humphries <edwinh(a)earthcaretech.com.au> To: chaouche yacine <yacinechaouche(a)yahoo.com> Sent: Wednesday, June 8, 2016 5:23 AM Subject: Re: Can't delete emails from or write replies to Inbox Chaouche Duh! I reproduced the issue, then found the log file (mail.log, in my case) and found that that user wasn't in the mail group (which is itself a puzzle, because it used to be, but not sure I'm going to spend much time investigating why); added the user to the mail group, restarted dovecot, and hey presto, everything works. Regards, Edwin Humphries Mobile: 0419 233 051 EarthCare Technology (a division of Ironstone Technology Pty Ltd) 79 Barney St, Kiama, NSW, 2533 Web: http://www.earthcaretech.com.au -- This email is intended for the named addressee/s only and may contain confidential or privileged information. If you are not a named addressee please delete it and notify the sender. -- "Sometimes when I consider what tremendous consequences come from little things…I am tempted to think…there are no little things." Bruce Barton "The activist is not the man who says the river is dirty. The activist is the man who cleans up the river." Ross Perot "Our world has enough for each person's need, but not for his greed." Mahatma Gandhi "The best way to predict the future...is to create it." Source unknown "He who knows that enough is enough will always have enough." Lao Tzu "For man two wings are necessary. One wing is physical power and material civilisation; the other is spiritual power and divine civilisation. With one wing only, flight is impossible. Two wings are essential. Therefore, no matter how much material civilisation advances, it cannot attain to perfection except through the uplift of spiritual civilisation." 'Abdu'l-Baha, Promulgation of Universal Peace On 07/06/16 19:20, chaouche yacine wrote: How about some /var/log/dovecot.log lines ? ----- Original Message ----- From: Edwin Humphries <edwinh(a)earthcaretech.com.au> To: Dovecot Mailing List <dovecot(a)dovecot.org> Sent: Tuesday, June 7, 2016 1:22 AM Subject: Can't delete emails from or write replies to Inbox Hi, I'm running Dovecot and Thunderbird, with 2 separate accounts (business and personal). One operates fine, but the other one won't allow me to write replies to the Inbox (so that I can track current threads), and if I move emails to another folder, it copies them instead. New emails do appear in the Inbox. I've checked the permissions on the two Inboxes, and they are the same (except for the owner name). I've tried compacting the problem Inbox, but that hasn't changed things. Edwin

1 0

segfault in IMAP APPEND with compressed maildir
by Roland Rosenfeld 08 Jun '16

08 Jun '16

Hi! After upgrading from Debian wheezy with (self compiled) dovecot 2.2.15 to Debian jessie with (self compiled) 2.2.24, I observe the following segmentation fault in the logs: Jun 7 09:23:09 imap dovecot: imap(user(a)example.com): Error: read(<imap client>) failed: read(size=8003) failed: Connection reset by peer (uid=0, box=trash) Jun 7 09:23:09 imap dovecot: imap(user(a)example.com): Error: zlib.read(/srv/mailstore/user(a)example.com/mail/.trash/tmp/1465283884.M336492P22902.imap): unexpected EOF at 88001 Jun 7 09:23:09 imap dovecot: imap(user(a)example.com): Error: read(zlib(/srv/mailstore/user(a)example.com/mail/.trash/tmp/1465283884.M336492P22902.imap)) failed: read(/srv/mailstore/user(a)example.com/mail/.trash/tmp/1465283884.M336492P22902.imap) failed: zlib.read(/srv/mailstore/user(a)example.com/mail/.trash/tmp/1465283884.M336492P22902.imap): unexpected EOF at 88001 (uid=0, box=trash) Jun 7 09:23:09 imap dovecot: imap(user(a)example.com): Fatal: master: service(imap): child 22902 killed with signal 11 (core dumped) We also observed the same (rare) error in the past on the old system. But on the old system, there were only the first 3 lines without the segmentation fault, so we ignored the issue until now. The problem always happens on IMAP folders where the client writes to, like "trash", "drafts", "sent" and the like. I wasn't able to actively reproduce this issue, but can only observe in the logs that some customers run into this issue from time to time. So all I have is a core dump with the following backtrace: Core was generated by `dovecot/imap'. Program terminated with signal SIGSEGV, Segmentation fault. (gdb) bt full #0 0x00007f57e276f29f in i_stream_default_get_size (stream=0x1fd2790, exact=<optimized out>, size_r=0x7ffed3839718) at istream.c:807 No locals. #1 0x00007f57e17024e4 in zlib_mail_close (_mail=0x1fd4de0) at zlib-plugin.c:170 mail = 0x1fd4de0 zmail = 0x1fd5398 zuser = 0x1fbd040 cache = 0x1fbd050 size = 33201320 #2 0x00007f57e2a2a8b9 in mailbox_save_cancel (_ctx=_ctx@entry=0x1fc4d48) at mail-storage.c:2117 ctx = 0x1fd3dd0 keywords = 0x0 mail = <optimized out> __FUNCTION__ = "mailbox_save_cancel" #3 0x000000000040c759 in cmd_append_finish (ctx=0x1fc4cf0) at cmd-append.c:149 __FUNCTION__ = "cmd_append_finish" #4 0x000000000040c835 in client_input_append (cmd=0x1fc4bc0) at cmd-append.c:89 ctx = <optimized out> client = 0x1fc3fc0 reason = 0x1f9e0b8 "Disconnected in APPEND (1 msgs, 306 secs, 188416/1122858 bytes)" finished = <optimized out> lit_offset = <optimized out> __FUNCTION__ = "client_input_append" #5 0x00007f57e2778dcc in io_loop_call_io (io=0x1fc4ad0) at ioloop.c:564 ioloop = 0x1fa6750 t_id = 2 __FUNCTION__ = "io_loop_call_io" #6 0x00007f57e277a0f1 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x1fa6750) at ioloop-epoll.c:220 ctx = 0x1fa8260 io = <optimized out> tv = {tv_sec = 1799, tv_usec = 997118} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 j = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run_internal" #7 0x00007f57e2778e55 in io_loop_handler_run (ioloop=ioloop@entry=0x1fa6750) at ioloop.c:612 No locals. #8 0x00007f57e2778ff8 in io_loop_run (ioloop=0x1fa6750) at ioloop.c:588 __FUNCTION__ = "io_loop_run" #9 0x00007f57e2713713 in master_service_run (service=0x1fa65f0, callback=callback@entry=0x423a20 <client_connected>) at master-service.c:640 No locals. #10 0x000000000040c427 in main (argc=1, argv=0x1fa6390) at main.c:460 set_roots = {0x42c480 <imap_setting_parser_info>, 0x635440 <lda_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0x1f9e048 "ailed: Connection reset by peer", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x424170 <login_client_connected>, failure_callback = 0x423b30 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 auth_socket_path = 0x42d42e "auth-master" c = <optimized out> This is on a server, which uses compressed maildir on a NFS storage. Here's dovecot -n output: # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.14 (099a97c) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.4 auth_cache_negative_ttl = 5 mins auth_cache_size = 100 M auth_cache_ttl = 15 mins auth_default_realm = example.com auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_gid = 999 mail_location = maildir:~/mail mail_plugins = acl quota zlib mail_uid = 999 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags mmap_disable = yes namespace { list = children location = maildir:%%h/mail:INDEX=~/mail/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl quota = maildir sieve = ~/.dovecot.sieve sieve_dir = ~/mail/sieve sieve_extensions = +imapflags zlib_save = gz } pop3_no_flag_updates = yes pop3_uidl_format = %v.%u protocols = imap pop3 sieve service auth { unix_listener auth-master { group = vmail mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap { process_limit = 4000 } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 100 } service pop3 { process_limit = 1000 } shutdown_clients = no ssl = no syslog_facility = local2 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 10 mail_plugins = acl quota zlib imap_quota imap_acl } protocol pop3 { mail_plugins = acl quota zlib } I hope, that this is all required information to find and solve this issue. Greetings Roland

2 4

Re: [g.danti@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression]
by Gionatan Danti 08 Jun '16

08 Jun '16

Il 26-11-2015 15:15 John R. Dennison ha scritto: > You are strongly encouraged to update that CentOS system. Current is > 6.7 (released some 3 months ago) and dovecot-2.0.9-19. Ouch! I copied outdated information from my old post. My current system _is_ CentOS 6.7 with dovecot dovecot-2.0.9-19.el6_7.2.x86_64 Sorry for the confusion. Still, the problems remain > If you find you need a more modern dovecot version you will find that > dovecot-2.2.19-1 is in the GhettoForge 3rd-party repo, packaged and > maintained by a respected CentOS community member. Information > on this repo can be found by following the links off of: > > http://wiki.centos.org/AdditionalResources/Repositories > The point is that I _don't know_ if more recent Dovecot version have the same problem or not. Moreover, as this is a production machine, I would like to leave a key package as Dovecot as near as possible to the shipping O.S. Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

3 10

Slow auth
by Ranbir 08 Jun '16

08 Jun '16

Hi Everyone, I'm running dovecot on a CentOS 7 box using PLAIN and GSSAPI auth. I need to use both because I have some clients that can't use GSSAPI. I haven't been able to get the userdb working properly without a password file and a userdb file. For example, I have to set the home default and change the username_format. I use FreeIPA and the dovecot server is joined properly to the realm. Authentication works (I don't have any passwords in the userdb file). The problem is authentication is slow. When I use Roundcube, the login takes longer than it should. In fact, every operation (changing folders, opening an email, replying, etc.) is just slow. The web server where Roundcube is running is barely loaded, the dovecot server isn't loaded and the FreeIPA server is basically sitting idle. When I watch /var/log/secure as I login to roundcube, I see this in the logs: http://pastebin.ca/3620032 Why is pam_unix being hit and then pam_sss? I'm thinking the pam_unix failures are the reason why the auth is slow. Here's the output from dovecot -n: http://pastebin.ca/3620029 I'm sure I haven't configured something correctly, hence the performance problems. Any help would be appreciated. -- Ranbir

4 6