[Dovecot] Some POP3 questions
Hi everyone
Been spending the whole day on the wiki and in the list archives on using dovecot for POP3 mail. I'd like to present my findings and ask for some feedback.
Our current software setup:
- Courier-IMAP running POP3/IMAP
- Postfix MTA
- MySQL backend for user and login info
As for the system layout, it looks like this:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
- Mail directory structure: /mail/e/ex/example.com/user
Usage:
- Currently reaching in excess of 1,200 concurrent connections, traffic roughly 36GB per day
- 90% POP3 (mostly pop+delete), 10% IMAP
- ~170GB of mail on disk
Current problems experienced:
- Massive load averages (>500)
- Memory exhaustion (3GB)
- Server looses network connectivity (related to massive loads)
I've got a new server ready for action, well after mail directories have been rsync'd over. Will have 8GB of memory and much fast discs.
I'm looking at giving Dovecot + Postfix/LDA a go. From my research today it seems that running a POP server without LDA (or without the W=(\d+) filename field) is a futile business. I'm running Gentoo, and the stable release in portage is currently dovecot 1.1.7, with 1.1.14 being available but masked.
Long stories short, will Dovecot with the Postfix/LDA give performance equal than Courier-IMAP or improved (because of the cached message sizes)?
Kind regards
-- Kenneth Kalmer kenneth.kalmer@gmail.com http://opensourcery.co.za @kennethkalmer
On Ter, 2009-05-05 at 15:41 +0200, Kenneth Kalmer wrote:
Hi everyone (....)
Long stories short, will Dovecot with the Postfix/LDA give performance equal than Courier-IMAP or improved (because of the cached message sizes)?
Improved without a doubt. But I advise you to go to the latest available version (masked or not).
-- Jose Celestino SAPO.pt::Systems http://www.sapo.pt --------------------------------------------------------------------- * Progress (n.): The process through which Usenet has evolved from smart people in front of dumb terminals to dumb people in front of smart terminals.
Long stories short, will Dovecot with the Postfix/LDA give performance equal than Courier-IMAP or improved (because of the cached message sizes)?
Another thing you want to look at is the UIDLsetting in dovecot.conf, if it doesn't match the one in Courier, then people with leave mail on server will download all their email again (once) because the UIDL list is different.
Just my 2 cents...goodluck
Sincerely, Wouter van der Schagt
On Tue, May 5, 2009 at 3:57 PM, Wouter van der Schagt wouter@vdschagt.comwrote:
Long stories short, will Dovecot with the Postfix/LDA give performance
equal than Courier-IMAP or improved (because of the cached message sizes)?
Another thing you want to look at is the UIDLsetting in dovecot.conf, if it doesn't match the one in Courier, then people with leave mail on server will download all their email again (once) because the UIDL list is different.
Which, to some extent, is almost what they deserve for storing their mail ;)
Thanks for the pointer, will make a note quickly
Best
-- Kenneth Kalmer kenneth.kalmer@gmail.com http://opensourcery.co.za @kennethkalmer
On May 5, 2009, at 9:41 AM, Kenneth Kalmer wrote:
As for the system layout, it looks like this:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
NFS may be problematic. Dovecot expects a perfectly working NFS setup,
which seems to be a bit rare to find. http://wiki.dovecot.org/NFS
So I'd do some testing before moving all the users.
On Tue, May 5, 2009 at 5:34 PM, Timo Sirainen tss@iki.fi wrote:
On May 5, 2009, at 9:41 AM, Kenneth Kalmer wrote:
As for the system layout, it looks like this:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
NFS may be problematic. Dovecot expects a perfectly working NFS setup, which seems to be a bit rare to find. http://wiki.dovecot.org/NFS
Just for the record, dovecot will be access the mail directly on the disks, it will be the LDA's on the postfix instances coming in over NFS. I'm reading the NFS again to make sure I understand the risks
Thanks
-- Kenneth Kalmer kenneth.kalmer@gmail.com http://opensourcery.co.za @kennethkalmer
On Tue, 2009-05-05 at 23:32 +0200, Kenneth Kalmer wrote:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
NFS may be problematic. Dovecot expects a perfectly working NFS setup, which seems to be a bit rare to find. http://wiki.dovecot.org/NFS
Just for the record, dovecot will be access the mail directly on the disks, it will be the LDA's on the postfix instances coming in over NFS. I'm reading the NFS again to make sure I understand the risks
If the LDA is Dovecot deliver, it's the same problem since it also updates index/control files that cause the problems. But if you're using something else that does nothing but write the maildir files, then it's fine.
On Tue, May 5, 2009 at 11:39 PM, Timo Sirainen tss@iki.fi wrote:
On Tue, 2009-05-05 at 23:32 +0200, Kenneth Kalmer wrote:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
NFS may be problematic. Dovecot expects a perfectly working NFS setup, which seems to be a bit rare to find. http://wiki.dovecot.org/NFS
Just for the record, dovecot will be access the mail directly on the disks, it will be the LDA's on the postfix instances coming in over NFS. I'm reading the NFS again to make sure I understand the risks
If the LDA is Dovecot deliver, it's the same problem since it also updates index/control files that cause the problems. But if you're using something else that does nothing but write the maildir files, then it's fine.
I'm looking to use the Dovecot LDA so the caches are updated and the filenames are correct, to get good POP3 performance.
I'm looking forward to giving Dovecot a go, but I'll rig up a staging environment first.
Thanks for the encouraging responses everyone.
Best
-- Kenneth Kalmer kenneth.kalmer@gmail.com http://opensourcery.co.za @kennethkalmer
On Wed, May 6, 2009 at 5:25 AM, Kenneth Kalmer kenneth.kalmer@gmail.com wrote:
On Tue, May 5, 2009 at 11:39 PM, Timo Sirainen tss@iki.fi wrote:
On Tue, 2009-05-05 at 23:32 +0200, Kenneth Kalmer wrote:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
NFS may be problematic. Dovecot expects a perfectly working NFS setup, which seems to be a bit rare to find. http://wiki.dovecot.org/NFS
Just for the record, dovecot will be access the mail directly on the disks, it will be the LDA's on the postfix instances coming in over NFS. I'm reading the NFS again to make sure I understand the risks
If the LDA is Dovecot deliver, it's the same problem since it also updates index/control files that cause the problems. But if you're using something else that does nothing but write the maildir files, then it's fine.
I'm looking to use the Dovecot LDA so the caches are updated and the filenames are correct, to get good POP3 performance.
I'm looking forward to giving Dovecot a go, but I'll rig up a staging environment first.
Thanks for the encouraging responses everyone.
Best
-- Kenneth Kalmer kenneth.kalmer@gmail.com http://opensourcery.co.za @kennethkalmer
I moved from a 4 gig nfs server with eight mailheads running exim as lda, courier imap/pop3 (over nfs) to an 8 gig nfs server and moved to dovecot imap/pop3 running directly on the nfs server and dropped to four mailheads running exim with dovecot lda.
At first I tried to keep my imap/pop3 services running on the 4 mailheads but over nfs had major weird 500 load spikes randomly, moving these to the nfs server solved all of that. The 4 mailheads still deliver to maildir over nfs using dovecot as lda. I have been pleasantly pleased. Our webmail app in our old setup used to take 10-30 seconds to load and now it is instantaneous. Some of it might be from upgrading hardware, but I definitely had a HUGE noticeable difference and you couldnt convince me to go back.
We did have about a week of sucktastic traffic and my support department hating me because all our pop3 leave-messages-on-server users had to redownload all their mail, but it was worth the headache. Dovecot is just too sweet, with its performance and plugins to ever have to even think about a choice between it or software X.
Just my $0.02
On Wed, May 6, 2009 at 7:23 AM, Brandon Lamb brandonlamb@gmail.com wrote:
On Wed, May 6, 2009 at 5:25 AM, Kenneth Kalmer kenneth.kalmer@gmail.com wrote:
On Tue, May 5, 2009 at 11:39 PM, Timo Sirainen tss@iki.fi wrote:
On Tue, 2009-05-05 at 23:32 +0200, Kenneth Kalmer wrote:
- NFS server which is the same server that houses single Courier-IMAP installation (Gentoo, 3GB memory)
- 2x Postfix instances delivering to NFS
NFS may be problematic. Dovecot expects a perfectly working NFS setup, which seems to be a bit rare to find. http://wiki.dovecot.org/NFS
Just for the record, dovecot will be access the mail directly on the disks, it will be the LDA's on the postfix instances coming in over NFS. I'm reading the NFS again to make sure I understand the risks
If the LDA is Dovecot deliver, it's the same problem since it also updates index/control files that cause the problems. But if you're using something else that does nothing but write the maildir files, then it's fine.
I'm looking to use the Dovecot LDA so the caches are updated and the filenames are correct, to get good POP3 performance.
I'm looking forward to giving Dovecot a go, but I'll rig up a staging environment first.
Thanks for the encouraging responses everyone.
Best
-- Kenneth Kalmer kenneth.kalmer@gmail.com http://opensourcery.co.za @kennethkalmer
I moved from a 4 gig nfs server with eight mailheads running exim as lda, courier imap/pop3 (over nfs) to an 8 gig nfs server and moved to dovecot imap/pop3 running directly on the nfs server and dropped to four mailheads running exim with dovecot lda.
At first I tried to keep my imap/pop3 services running on the 4 mailheads but over nfs had major weird 500 load spikes randomly, moving these to the nfs server solved all of that. The 4 mailheads still deliver to maildir over nfs using dovecot as lda. I have been pleasantly pleased. Our webmail app in our old setup used to take 10-30 seconds to load and now it is instantaneous. Some of it might be from upgrading hardware, but I definitely had a HUGE noticeable difference and you couldnt convince me to go back.
We did have about a week of sucktastic traffic and my support department hating me because all our pop3 leave-messages-on-server users had to redownload all their mail, but it was worth the headache. Dovecot is just too sweet, with its performance and plugins to ever have to even think about a choice between it or software X.
Just my $0.02
Oh yea, and I just recently discovered the virtual mailbox plugin and fell even more in love. I am developing a Gmail clone inhouse for our new webmail (we're an ISP). It is cool plugins like this that make me glad I switched to Dovecot. Being able to consolidate various Deleted, Trash, Deleted Items, Deleted Mails folders all into a single virtual/Trash folder is GREAT.
participants (5)
-
Brandon Lamb
-
Jose Celestino
-
Kenneth Kalmer
-
Timo Sirainen
-
Wouter van der Schagt