On 3/16/2012 7:07 AM, Mauricio López Riffo wrote:
Hi,
We actually have a mail hosting solutions with aprox. 100 thousandof email account, where about 90% of a customers use POP3 like email configuration. About a few mounths (we perfomed a lot of migration throught mbox email software to Maildir with dovecot) but i can see that the performance is very poor and receive complaint about delays of autentications of accounts.
The solution lives in Metrocluster Netapp storage, filesystem NFS, VMware as a virtualization (the mtas are a virtual machines lives in netapp too) about 4T of data mails and a 10G network connection (betwen mtas and nfs storage) All account information work in LDAP plataform (two servers in replicated mode, no high average or delays detected in this servers)
When the traffic have a peak of 1800 concurrent connections POP3, all of service suffer a high load average (about 8 - 20 load average in each dovecot) and authenticacion takes about 2 -10 seconds (in low traffic, autenticacion takes about 60 miliseconds)
Also, each dovecot instance lives un MTA server CentOS 5.8 x86_64 with 6G RAM (virtual machine) and share's hardware with a exim instance, like a MTA relay system (autenticated relay)
Usage of network have peaks of 80Mbits (all dmz network have 1Gbits of bandwith)
Attach of dovecot -n output:
# 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.1.1.el5 x86_64 CentOS release 5.8 (Final) auth_debug_passwords = yes auth_default_realm = portalplata.cl auth_realms = portalplata.cl auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 100 base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot.log default_process_limit = 200 default_vsz_limit = 512 M disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 lock_method = dotlock login_greeting = Dovecot mta10 mail_cache_min_mail_count = 5 mail_debug = yes mail_fsync = always mail_full_filesystem_access = yes mail_gid = 12 mail_location = maildir:%h/Maildir:INDEX=/data/cache/indexes/%2d/%1u/%2u/%u mail_nfs_storage = yes mail_plugins = " quota" mail_uid = 8 maildir_copy_with_hardlinks = no passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = box from subject quota = maildir } postmaster_address = mail@mail.com protocols = imap pop3 sendmail_path = /usr/lib/sendmail service auth { unix_listener auth-userdb { mode = 0600 user = exim } } service imap-login { service_count = 0 } service imap-postlogin { executable = script-login /usr/local/bin/postlogin.sh user = root } service imap { executable = imap imap-postlogin } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } service_count = 0 } service pop3 { process_limit = 1024 } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = " quota" } protocol lda { mail_plugins = " quota" } protocol imap { imap_capability = mail_max_userip_connections = 10 mail_plugins = " quota autocreate notify quota imap_quota mail_log" plugin { autocreate = Sent Items autocreate2 = Spam autocreate3 = Drafts autocreate4 = Trash autosubscribe = Sent Items autosubscribe2 = Spam autosubscribe3 = Drafts autosubscribe4 = Trash } } protocol pop3 { mail_plugins = " quota quota notify mail_log" pop3_reuse_xuidl = yes pop3_save_uidl = yes pop3_uidl_format = %u }
Output of account information in a LDAP:
# nettester, email.net, MAIL, USERS, cl dn: uid=nettester,dc=email.net,o=MAIL,o=USERS,c=cl dc: email.net mailMessageStore: /export/mdir/3/12/nettester@email.net/Maildir uid: nettester cn: nettester@email.net sn: nettester@email.net gidNumber: 12 homeDirectory: /export/mdir/3/12/nettester@email.net mail: nettester@email.net uidNumber: 8 objectClass: mailUser objectClass: posixAccount objectClass: mailSetting loginShell: /bin/false description: enable service: pop3 service: imap service2: webmail mailRate: 200 mailQuota: 1024M deliveryMode: none mailReplyText: .
Any suggestions? All ideas will be have a good receptions ;)
Pd: Sorry my english
It doesn't seem to matter what type of hardware you might have, NFS can cause real bottlenecks, even to the point that your machine may report disk errors.
Unfortunately, it's an evil necessity in some shops, but any way to eliminate NFS when large throughput is occurring will definitely help. Make sure you're running the latest version of NFS on all machines since V3 and V4 don't always like each other.
I don't have a solution for it's replacement other than expensive hardware solutions.
steve