- dovecot - dovecot.org

[Dovecot] Proxy IMAP/POP/ManageSieve/SMTP in a large cluster enviroment
by Aliet Santiesteban Sifontes 19 Jul '10

19 Jul '10

Hi to all in the list, we are trying to do some tests lab for a large scale mail system having this requirements: - Scale to maybe 1million users(Only for testing). - Server side filters. - User quotas. - High concurrency. - High performance and High Availability. We plan to test this using RHEL5 and maybe RHEL6. As a storage we are going to use an HP EVA 8400 FC(8 GB/s) We defined this functional roles. 1- Outgoing SMTP gateway servers. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using Postfix Latest version Amavisd-new Clamav Spamassasin maybe some other filters. 2- Incomming SMTP gateway servers. - Load balanced with Piranha LVS using Direct Routing. - RHEL5/6 using Postfix Latest version Amavisd-new Clamav Spamassasin( n servers) maybe some other filters. 3- Webmail Cluster farm. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using RHCS Active/Active. Apache2 PHP5 Horde. 4- Openldap load balanced openldap cluster. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using Openldap. 5- IMAP/POP/ManageSieve/SMTP Proxy or Director. - Load balanced with Piranha LVS using Direct Routing. - ( n servers)RHEL5/6 using Dovecot - Postfix(Director or Proxy). 6- Mail backend. - ( n servers)RHEL5/6 using Dovecot. Now for functional role 6 "Mail Backend" we have some dilemmas. - Recommended scalable filesystem to use for such scenario(Clustered or not). -GFS2?? We have very bad experiences with GFS1 and maildir, GFS2 doesn't seems to improve this also. Using some techniques for session afinity with a backend server seems to help with the locking problems of GFS and the cache. Using GFS many IMAP SMTP servers can in parallel write or read to user mailboxes, if GFS can perform well we prefer this, will have to proxy o use director for the cache problem. - Ext4, Ext3 well tested with many setups but only one backend server can access one ext4 lun, so we have to proxy a user always to a ip backend address or VIP in a case of failure cluster software has to move the server to another node, the cluster service is formed by a IP Address and one or many Luns wich moves from node to node in a case of failure. - Recommended scalable method for partition divition. - Directory Hashing(How it actually works the theory behind this), we see the wiki but we need to understand the theory to balance the directory structure. - Using some criteria ex Lun1(a-h users), Lun2(i-m users).... - Proxy or Director is a must for a Clustered Filsystem or for the solution of a user belonging to one server. - In case we use a Proxy with dovecot we know we can use it with IMAP/POP but are not sure for ManageSieve and more important for SMTP. The question is, how can we proxy for the same user IMAP/POP/ManageSieve and SMTP delivery to mailbox at the same time, can the incoming mail gateways send the email to the proxy servers and dovecot proxy the request for smtp or lmtp appply the same proxy criteria that uses for the protocols IMAP and POP and send the email to the correct backend server and this server do the final delivery?? I mean proxy all the protocols for a user IMAP/POP/SMTP-LMTP?? Any example... Right now we have doubts on this stuff, we are really going to appreciate any help or advice on this... best regards

4 4

[Dovecot] Haha wow, thats freakin cool... mdbox
by Brandon Lamb 19 Jul '10

19 Jul '10

It didnt really click until I had a "ls -ald *" on watch while I moved messages around folders using mdbox... I moved about 200 first to an Archive folder, and scratched my head wondering why i didnt see any new m.* files being created after the move. So then I moved some 5k other messages from other folders to my archive folder... Then it dawned on me, how damn cool, the data never changes, dovecot just updates its index of what folder the message is in?! That is awesome! Also, I ended up answering my own question on the 2m mdbox_rotate_size, if a 10 meg email comes through it will write a 10 meg file, so thats cool too.

4 4

[Dovecot] dsync segmentation fault
by Brandon Lamb 19 Jul '10

19 Jul '10

dsync -u brandon(a)olypen.com mirror mdbox~/mdbox Segmentation fault Took me 3 tries before I noticed the missing colon after mdbox

2 3

[Dovecot] mdbox, mdbox_rotate_size 2m default
by Brandon Lamb 19 Jul '10

19 Jul '10

Anyone else test using different values? Was 2M chosen as a default based on anything specific? Is there any kind of known range where performance starts to go downhill if its greater than x or less than y? Im curious if there would be any obvious reason why I shouldnt set it to 5-10 megs

2 1

[Dovecot] users can not login
by Daminto Lie 19 Jul '10

19 Jul '10

Hi, I was wondering if I could get some help here. Recently, I decided I want to move from Windows XP Pro OS to a Linux OS such as Ubuntu. I am new to Linux OS but willing to learn. I have installed Ubuntu Server 10.04, Apache2, MySQL5, PHPMyAdmin, Postfix, DoveCot and Squirrelmail on my server. I have checked them all and they are all up and running at the moment. Now, what I am trying to do here is to add a couple of users and see whether they can login to squirrelmail. The result is none of those two users are able to login. I went to check the mail.log and the error message is as follows Ubuntu-MailServer dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=<david.keith>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured My silly question is I do not quite understand what the above error means. How do i enable users to login and test to see if they can send and receive emails. Users can login via squirrelmail. Any help would be greatly appreciated.

2 1

[Dovecot] pam needs root with 2.0rc2 on Solaris?
by Laurent Blume 19 Jul '10

19 Jul '10

Hello all, I have a question using PAM to authenticate on Solaris 10, with 2.0rc2. I need to set the user for auth-worker to root, else it will return "Unknown user". Is that expected? I'm using PAM, so why does it need access to /etc/shadow? TIA! Laurent service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. user = root #user = dovecot } Jul 18 09:59:31 tiphaine dovecot: [ID 583609 mail.info] auth: pam(user,127.0.0.1): unknown user

2 1

[Dovecot] how configure dovecot for not case sensitive login
by Wilberth Perez 19 Jul '10

19 Jul '10

Any knows how can say to dovecot that Login = user and Login = User are the same? Thanks.

2 3

[Dovecot] director service and managesieve
by Xavier Pons 19 Jul '10

19 Jul '10

Hi, the director service have implemented any support for managesieve-login?. I get the error: managesieve-login: Error: proxy: host not given: user=<xxx>, method=PLAIN, ... any workaround? thanks Xavier

2 1

[Dovecot] lda with sieve crash
by Michal Hlavinka 19 Jul '10

19 Jul '10

Hi, my colleague found a crash in dovecot, while he was experimenting with configuration for using dovecot with fetchmail. Backtrace can be found here: http://pastebin.com/XQZqYBpH Let me know if you need any other info. Michal # doveconf -n # 20100716 (26936bdd3801+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32 auth_debug = yes auth_verbose = yes info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator- i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest mbox_write_locks = fcntl namespace { hidden = no inbox = yes list = yes location = maildir:~/Maildir prefix = INBOX. separator = . subscriptions = yes type = private } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } verbose_ssl = yes protocol lmtp { mail_plugins = sieve } protocol lda { info_log_path = /var/log/dovecot-deliver.log log_path = /var/log/dovecot-deliver-errors.log mail_plugins = sieve } -------------------------- .dovecot.sieve: require ["fileinto", "regex"]; if header :contains "X-BeenThere" "bugzilla(a)redhat.com" { fileinto "redhat.bugzilla"; } elsif header :contains "List-Id" "scm-commits.lists.fedoraproject.org" { fileinto "fedora.fedora-extras-commits"; } elsif header :contains "List-Id" "package-review.lists.fedoraproject.org" { fileinto "fedora.fedora-package-review"; } elsif header :contains "List-Id" "devel.lists.fedoraproject.org" { fileinto "fedora.fedora-devel-list"; } elsif header :contains "List-Id" "brno-list.redhat.com" { fileinto "redhat.brno-list"; } elsif header :regex "Subject" "^Cron <jkastner@" { fileinto "cron"; } else { # The rest goes into INBOX # default is "implicit keep", we do it explicitly here keep; } -------------------------------

2 2

[Dovecot] Arghh, dovecot-2 and lda
by Brandon Lamb 18 Jul '10

18 Jul '10

I cannot seem to get exim/lda working... In my exim log i just see these 2010-07-18 11:51:08 1OaYOJ-0003Vn-2P == brandon(a)olypen.com R=router_pop T=dovecot_deliver defer (2): No such file or directory: failed to chdir to maildir:~/maildir:INDEX=~/maildir/dovecot-index Now I have all the debugging on as far as i can tell, and exim is obviously being returned some value from dovecot because it knows about the maildir that i return from sql, but yet there is zero being logged in debug or anything else. If I run the dovecot-lda from the command line with the same then it works and i get logs entered command = /usr/local/libexec/dovecot/ -e -k -f "$sender_address" -d "$local_part@$domain" -m "$acl_m1" I am returning a home and mail with userdb, the directory does exist, owned by mail.mail... Im stumped dovecot -n # 2.0.rc2: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian squeeze/sid auth_cache_negative_ttl = 30 s auth_cache_size = 4096 auth_cache_ttl = 30 s auth_debug = yes auth_default_realm = olypen.com auth_mechanisms = plain login auth_username_format = %Lu auth_verbose = yes auth_worker_max_count = 60 base_dir = /var/run/dovecot debug_log_path = /var/log/dovecot/debug.log default_client_limit = 1024 default_internal_user = mail default_login_user = mail default_vsz_limit = 1073741824 deliver_log_format = %s from %f %$ dict { expire = mysql:/etc/dovecot/conf.d/dovecot-dict-expire.conf.ext } disable_plaintext_auth = no dotlock_use_excl = yes first_valid_gid = 8 first_valid_uid = 8 info_log_path = /var/log/dovecot/info.log last_valid_gid = 8 last_valid_uid = 8 log_path = /var/log/dovecot/dovecot.log log_timestamp = %Y-%m-%d %H:%M:%S login_log_format = %s login_log_format_elements = (%u) (%r) mail_access_groups = mail mail_debug = yes mail_fsync = never mail_gid = 8 mail_location = maildir:~/:INDEX=~/dovecot-index mail_log_prefix = %Ls <%u>: mail_max_userip_connections = 4 mail_plugins = quota mail_privileged_group = mail mail_uid = 8 mailbox_idle_check_interval = 60 s maildir_very_dirty_syncs = yes mdbox_rotate_size = 10485760 namespace { hidden = no inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { auth_socket_path = /var/run/dovecot/auth-master autocreate = Trash autocreate2 = Spam autocreate3 = Spam.Delivered autocreate4 = Spam.Removed autocreate5 = Sent autocreate6 = Drafts autosubscribe = Trash autosubscribe2 = Sent expire = Trash 30d expire2 = Trash/* 30d expire3 = Spam 14d expire4 = Spam/Removed 6h expire5 = Spam/Delivered 2d expire6 = Deleted* 30d expire7 = Archive 30d expire_dict = proxy::expire qoute_rule5 = Spam.Delivered:ignore quota = maildir:User quota quota_rule2 = Trash:ignore quota_rule3 = Deleted Items:ignore quota_rule4 = Spam:ignore quota_rule6 = Spam.Removed:ignore quota_warning = storage=95%% /etc/dovecot/quota-warning.sh 95 quota_warning2 = storage=80%% /etc/dovecot/quota-warning.sh 80 trash = /etc/dovecot/conf.d/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 6 } protocols = imap pop3 service auth { executable = /usr/local/libexec/dovecot/auth unix_listener auth-client { group = mail mode = 0666 user = mail } unix_listener auth-master { group = mail mode = 0666 user = mail } unix_listener auth-userdb { group = mail mode = 0666 user = mail } vsz_limit = 536870912 } service imap-login { client_limit = 512 executable = /usr/local/libexec/dovecot/imap-login process_limit = 20 process_min_avail = 4 service_count = 0 user = dovecot vsz_limit = 134217728 } service imap { drop_priv_before_exec = no executable = /usr/local/libexec/dovecot/imap vsz_limit = 536870912 } service pop3-login { client_limit = 512 executable = /usr/local/libexec/dovecot/pop3-login process_limit = 20 process_min_avail = 4 service_count = 0 user = dovecot vsz_limit = 134217728 } service pop3 { drop_priv_before_exec = no executable = /usr/local/libexec/dovecot/pop3 vsz_limit = 536870912 } ssl_cert = </etc/dovecot/ssl/dovecot-cert.pem ssl_key = </etc/dovecot/ssl/dovecot-key.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } valid_chroot_dirs = /mail verbose_proctitle = yes protocol lda { auth_socket_path = /var/run/dovecot/auth-userdb info_log_path = /var/log/dovecot/deliver.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/deliver.log mail_fsync = optimized mail_plugin_dir = /usr/local/lib/dovecot/lda mail_plugins = $mail_plugins postmaster_address = postmaster(a)olypen.com quota_full_tempfail = yes rejection_reason = Your message to <%t> with subject of <%s> was automatically rejected:%n%r rejection_subject = Mailbox Full: %s sendmail_path = /usr/sbin/exim } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep imap_id_log = * imap_logout_format = bytes=%i/%o imap_max_line_length = 65536 mail_max_userip_connections = 20 mail_plugin_dir = /usr/local/lib/dovecot/imap mail_plugins = $mail_plugins imap_quota autocreate } protocol pop3 { mail_max_userip_connections = 2 mail_plugin_dir = /usr/local/lib/dovecot/pop3 mail_plugins = $mail_plugins pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = no pop3_lock_session = no pop3_logout_format = retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = yes pop3_reuse_xuidl = no pop3_uidl_format = %08Xu%08Xv }

2 14