[Dovecot] nfs and pop3
Hi,
I'm working with dovecot 1.0.beta2 for pop3 only and have set index=memory, so no indizes are created on the disk. pop3_uidl_format is set to %f. The maildirs are located on an nfs-share. So generally, this setup is very similar to what qmail-popup and qmail-pop3d does. However, everytime I'm switching from qmail-pop3d to dovecot, the system load increases from less than 2 to more than 30 (and later up to 60 and more). The problem is I/O, probably caused by nfs-access. But what is causing this accesses? qmail-pop3d just takes filenames as UIDL, and moves the mails from new to cur, reads them to send them to the client or deletes them. This is working fine. But what is dovecot doing? There must be a lot more things that cause that high I/O load. Indizes are disabled, so they cannot be the cause any longer. Any ideas? I just want to switch to dovecot at some point finally without having qmail-pop3d as backup.
Regards Marten
On Wed, 2006-01-25 at 22:55 +0100, Marten Lehmann wrote:
Hi,
I'm working with dovecot 1.0.beta2 for pop3 only and have set index=memory, so no indizes are created on the disk. pop3_uidl_format is set to %f. The maildirs are located on an nfs-share. So generally, this setup is very similar to what qmail-popup and qmail-pop3d does. However, everytime I'm switching from qmail-pop3d to dovecot, the system load increases from less than 2 to more than 30 (and later up to 60 and more). The problem is I/O, probably caused by nfs-access. But what is causing this accesses? qmail-pop3d just takes filenames as UIDL, and moves the mails from new to cur, reads them to send them to the client or deletes them. This is working fine. But what is dovecot doing? There must be a lot more things that cause that high I/O load. Indizes are disabled, so they cannot be the cause any longer. Any ideas? I just want to switch to dovecot at some point finally without having qmail-pop3d as backup.
The difference is that with Dovecot LIST command is returning the messages' exact size with CR+LF linefeeds. I didn't actually know that this wasn't required with POP3. Now, the simplest fix in your case would be to just make Dovecot return the message's physical size with stat(). But I'm not sure if this is a good default behavior. The physical size isn't used by IMAP, so if the mailbox has both POP3 and IMAP access, it needs to store both physical and virtual sizes to cache file.. Anyway, this patch should help you: Index: src/pop3/client.c =================================================================== RCS file: /var/lib/cvs/dovecot/src/pop3/client.c,v retrieving revision 1.60 diff -u -r1.60 client.c --- src/pop3/client.c 14 Jan 2006 18:48:17 -0000 1.60 +++ src/pop3/client.c 26 Jan 2006 19:54:41 -0000 @@ -85,9 +85,9 @@ buffer_set_used_size(message_sizes_buf, 0); failed = FALSE; - mail = mail_alloc(t, MAIL_FETCH_VIRTUAL_SIZE, NULL); + mail = mail_alloc(t, MAIL_FETCH_PHYSICAL_SIZE, NULL); while (mailbox_search_next(ctx, mail) > 0) { - uoff_t size = mail_get_virtual_size(mail); + uoff_t size = mail_get_physical_size(mail); if (size == (uoff_t)-1) { failed = TRUE;
On Thu, Jan 26, 2006 at 09:56:27PM +0200, Timo Sirainen wrote:
The difference is that with Dovecot LIST command is returning the messages' exact size with CR+LF linefeeds. I didn't actually know that this wasn't required with POP3.
Now, the simplest fix in your case would be to just make Dovecot return the message's physical size with stat(). But I'm not sure if this is a good default behavior. The physical size isn't used by IMAP, so if the mailbox has both POP3 and IMAP access, it needs to store both physical and virtual sizes to cache file..
ah, excellent!
I suppose this would also make it possible to read the S=<size> part of maildir filenames by simply reading the directory, rather than stat()ing all the files?
I expect this would siginificantly reduce disk IO.
grant.
On Fri, 2006-01-27 at 15:28 +1100, grant beattie wrote:
On Thu, Jan 26, 2006 at 09:56:27PM +0200, Timo Sirainen wrote:
The difference is that with Dovecot LIST command is returning the messages' exact size with CR+LF linefeeds. I didn't actually know that this wasn't required with POP3.
Now, the simplest fix in your case would be to just make Dovecot return the message's physical size with stat(). But I'm not sure if this is a good default behavior. The physical size isn't used by IMAP, so if the mailbox has both POP3 and IMAP access, it needs to store both physical and virtual sizes to cache file..
ah, excellent!
I suppose this would also make it possible to read the S=<size> part of maildir filenames by simply reading the directory, rather than stat()ing all the files?
I expect this would siginificantly reduce disk IO.
Yes, it already uses S= from filename if it's there.
On Fri, Jan 27, 2006 at 05:45:27PM +0200, Timo Sirainen wrote:
I suppose this would also make it possible to read the S=<size> part of maildir filenames by simply reading the directory, rather than stat()ing all the files?
I expect this would siginificantly reduce disk IO.
Yes, it already uses S= from filename if it's there.
hmm, that didn't seem to be the case after looking at truss output, but I may have been mistaken. I'll take a closer look tomorrow.
grant.
Hello,
Now, the simplest fix in your case would be to just make Dovecot return the message's physical size with stat(). But I'm not sure if this is a good default behavior.
thanks, this solved the problem as far as I can see (I will finally see on next Monday's business load).
Regards Marten
participants (4)
-
grant beattie
-
Lorens Kockum
-
Marten Lehmann
-
Timo Sirainen