[Dovecot] POP3 Performance

[Robert Schetterer]

Am 16.03.2012 12:07, schrieb Mauricio López Riffo:
> Hi,
> 
>     We actually have a mail hosting solutions with aprox. 100 thousand
> of email account, where about 90% of a customers use POP3 like email
> configuration.  About a few mounths (we perfomed a lot of migration
> throught mbox email software to Maildir with dovecot) but i can see that
> the performance is very poor and receive complaint about delays of
> autentications of accounts.
> 
> The solution lives in Metrocluster Netapp storage, filesystem NFS,
> VMware as a virtualization (the mtas are a virtual machines lives in
> netapp too) about 4T of data mails and a 10G network connection (betwen
> mtas and nfs storage)  All account information work in LDAP plataform
> (two servers in replicated mode, no high average or delays detected in
> this servers)
> 
> When the traffic have a peak of 1800 concurrent connections POP3, all of
> service suffer a high load average (about 8 - 20 load average in each
> dovecot) and authenticacion takes about 2 -10 seconds (in low traffic,
> autenticacion takes about 60 miliseconds)
> 
> Also, each dovecot instance lives un MTA server CentOS 5.8 x86_64 with
> 6G RAM (virtual machine) and share's hardware with a exim instance, like
> a MTA relay system (autenticated relay)
> 
> Usage of network have peaks of 80Mbits (all dmz network have 1Gbits of
> bandwith)
> 
> Attach of dovecot -n output:
> 
> # 2.0.18: /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.18-308.1.1.el5 x86_64 CentOS release 5.8 (Final)
> auth_debug_passwords = yes
> auth_default_realm = portalplata.cl
> auth_realms = portalplata.cl
> auth_verbose = yes
> auth_verbose_passwords = plain
> auth_worker_max_count = 100
> base_dir = /var/run/dovecot/
> debug_log_path = /var/log/dovecot.log
> default_process_limit = 200
> default_vsz_limit = 512 M
> disable_plaintext_auth = no
> first_valid_gid = 12
> first_valid_uid = 8
> lock_method = dotlock
> login_greeting = Dovecot mta10
> mail_cache_min_mail_count = 5
> mail_debug = yes
> mail_fsync = always
> mail_full_filesystem_access = yes
> mail_gid = 12
> mail_location = maildir:%h/Maildir:INDEX=/data/cache/indexes/%2d/%1u/%2u/%u
> mail_nfs_storage = yes
> mail_plugins = " quota"
> mail_uid = 8
> maildir_copy_with_hardlinks = no
> passdb {
>   args = /etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> plugin {
>   mail_log_events = delete undelete expunge copy mailbox_delete
> mailbox_rename
>   mail_log_fields = box from subject
>   quota = maildir
> }
> postmaster_address = mail at mail.com
> protocols = imap pop3
> sendmail_path = /usr/lib/sendmail
> service auth {
>   unix_listener auth-userdb {
>     mode = 0600
>     user = exim
>   }
> }
> service imap-login {
>   service_count = 0
> }
> service imap-postlogin {
>   executable = script-login /usr/local/bin/postlogin.sh
>   user = root
> }
> service imap {
>   executable = imap imap-postlogin
> }
> service pop3-login {
>   inet_listener pop3s {
>     port = 995
>     ssl = yes
>   }
>   service_count = 0
> }
> service pop3 {
>   process_limit = 1024
> }
> ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
> ssl_key = </etc/pki/dovecot/private/dovecot.pem
> userdb {
>   args = /etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> verbose_proctitle = yes
> protocol lmtp {
>   mail_plugins = " quota"
> }
> protocol lda {
>   mail_plugins = " quota"
> }
> protocol imap {
>   imap_capability =
>   mail_max_userip_connections = 10
>   mail_plugins = " quota autocreate notify quota imap_quota mail_log"
>   plugin {
>     autocreate = Sent Items
>     autocreate2 = Spam
>     autocreate3 = Drafts
>     autocreate4 = Trash
>     autosubscribe = Sent Items
>     autosubscribe2 = Spam
>     autosubscribe3 = Drafts
>     autosubscribe4 = Trash
>   }
> }
> protocol pop3 {
>   mail_plugins = " quota quota notify mail_log"
>   pop3_reuse_xuidl = yes
>   pop3_save_uidl = yes
>   pop3_uidl_format = %u
> }
> 
> 
> Output of account information in a LDAP:
> 
> # nettester, email.net, MAIL, USERS, cl
> dn: uid=nettester,dc=email.net,o=MAIL,o=USERS,c=cl
> dc: email.net
> mailMessageStore: /export/mdir/3/12/nettester at email.net/Maildir
> uid: nettester
> cn: nettester at email.net
> sn: nettester at email.net
> gidNumber: 12
> homeDirectory: /export/mdir/3/12/nettester at email.net
> mail: nettester at email.net
> uidNumber: 8
> objectClass: mailUser
> objectClass: posixAccount
> objectClass: mailSetting
> loginShell: /bin/false
> description: enable
> service: pop3
> service: imap
> service2: webmail
> mailRate: 200
> mailQuota: 1024M
> deliveryMode: none
> mailReplyText: .
> 
> Any suggestions? All ideas will be have a good receptions ;)
> 
> 
> Pd: Sorry my english
> 

looks like you need to play with some config stuff and do more debug on
your possible bottlenecks, what did you allready played with dovecot to
high performance

guess Timo will help about config settings

after all for short
to read

http://wiki.dovecot.org/Authentication/Caching
http://wiki.dovecot.org/LoginProcess
http://wiki2.dovecot.org/Services

any reasons for that

mail_full_filesystem_access = yes ?

general

nfs may not optimal, but that a long story
also virtual machines have their pros and contras
maildir is not so good in performance
-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria