Greetings Patrick, fellow Postfix user, and Postfix author.
Patrick Ben Koetter put forth on 5/8/2010 5:04 PM:
- Stan Hoeppner stan@hardwarefreak.com:
If this is the case, if I switch from having Postfix do local delivery to /var/mail to having Postfix use dovecot LDA, what other changes would I need
http://wiki.dovecot.org/LDA/Postfix
to make? Would I still be able to sort new messages with Tbird filter
Sure. Tbird filter rules are local, client rules run on your computer and not on the mailserver.
I should have worded that question better. What I was trying to ask is that, if using dovecot LDA, if new messages would still be presented to Tbird in the INBOX imap folder. If LDA dropped new messages into a different imap folder then Tbird wouldn't see them in the INBOX, and that's where all my filter rules apply. Looking back this was probably a dumb question. I asked it because I'm totally unfamiliar with LDA. I'll be reading up on it.
rules? What directory would dovecot LDA drop the new messages into? Would
Dovecot deliver will put messages into the same place where Postfix local puts them now.
Got it. That's good to know.
I need to make name space changes? I've never actually done anything
Nope, deliver replaces Postfix local LDA transparently.
Good. Fantastic, actually.
manually with name spaces. I let dovecot figure it out automagically.
Thanks for the interest in my topic. I know it's not a very interesting one since apparently I'm the only one on the planet experiencing this.
Putting 40k+ messages - IIRC you wrote that - into a single mailbox is not unusual allthough I hardly now any postmaster who recommends that.
The total is somewhere around there, and growing daily due to list mail. I'm not sure of your definition of "single mailbox" here. Yes, the messages are all in the same UNIX user account, but the messages are spread across a number of mbox files, not a single file. I have ~12,000 messages in each of two mbox files, ~5,000 in another, a couple with ~3,000, and a number with less than 1,000. Dovecot handles all of these quickly and easily, with low memory consumption. Searching the large mbox files is quick when the squat indexes aren't stale. Mailbox size, individual mbox size, don't seem to be a performance factor. My only performance problem is related to the INBOX, which usually has less than 150 messages in it. Tbird loads those 150-300 headers from the INBOX folder slower than it loads 12,000+ headers from the debian-users folder or spam-l folder.
At this point I'm assuming the problem has to lie with Tbird, as dovecot should be performing the same regardless of folder size or location on disk, *UNLESS* dovecot hasn't already indexed /var/mail/%user when the client connects. I don't know how dovecot handles indexing of /var/mail/%user files, or more importantly, I don't know *when* dovecot indexes those files. If they aren't indexed before the client connects and requests the contents of INBOX, I can see there being a performance drop, although I wouldn't think it would be great enough to cause a ~20 second delay for 125 messages or 60 seconds for ~300 messages.
Having no index on 40k messages means the mail server needs to scan the directory any time a mail client accesses the mailbox. This takes time and might explain the delay you experience.
Whoa. We've had a major disconnect somewhere in the conversation. I'm not lacking indexes on 40k messages. They're all well indexed by dovecot. Some have multiple indexes due to squat. Performance while accessing any of those 40k is fast as lightning. Again, the performance problem I have is _strictly with new messages in the INBOX folder_. More precisely, Tbird pulling the headers for new messages. IFAIK, Tbird only pulls the headers, not the entire message. It only pulls the entire message when you open/access it.
Dovecots ability always to provide an up to date if you use its LDA deliver was one of the major reasons to use Dovecot for me. We had a non-Dovecot IMAP server with a load of 60. It went down to 2 the day we began to use Dovecot with deliver AND access to messages became faster.
I agree. Dovecot rocks. And it's probably not at fault WRT my current performance issue. I'm just asking the question in the off chance that my performance issue WRT INBOX is due to something I may have screwed up in my dovecot config or setup, or maybe...
Timo identified a bug related to processing of mbox files that was to have been fixed in 1.2.11. I don't think my current issue is related. But, I get the feeling that mbox code paths have taken a back seat to maildir and other code paths through the last few release cycles due to the popularity of the latter and the declining popularity of the former. I am one of the users who helped Timo identify that mbox bug merely by observation here. It's always within the realm of possibility that I've run into another less than optimal code path. Like I sad, it's probably a Tbird issue, but it can't hurt to ask here just in case there might be a dovecot issue in play as well. It could be a problem with both code sets, Tbird and Dovecot, that only surfaces when using both together.
It's an odd obscure "mild" performance problem that just shouldn't exist. I can't find it documented anywhere, nothing like it shows up searching Google. I thought it was related to Tbird's GLODA, but after disabled that and scrubbing all the Tbird cache and index files, the problem still exists. Again, only for INBOX, but not for any other imap folders.
It's frustrating trying to solve this, to say the least. :( I don't have enough in depth knowledge of either application to figure this out on my own. It seems the only way to communicate with the Tbird devs is via bug reports. I can't really file a Tbird bug report as I just don't have enough information available to file one that they can do anything with.
-- Stan