[Dovecot] Index question
The day I switch our user to dovecot and it goes into production, there will be a few thousand people all generating their indices for the first time (we have a legacy requirement for procmail, at least for now), so I wondered:
- Is there is anyway to pre-emptively force indexing?
- Is there any value in doing so? That is, will I gain much in terms of making the first time everyone connects to dovecot quicker (not that big initial wait)?
--
Stewart Dean, Unix System Admin, Henderson Computer Resources
Center of Bard College, Annandale-on-Hudson, New York 12504
sdean@bard.edu voice: 845-758-7475, fax: 845-758-7035
- Stewart Dean <sdean@bard.edu>:
The day I switch our user to dovecot and it goes into production, there will be a few thousand people all generating their indices for the first time (we have a legacy requirement for procmail, at least for now), so I wondered:
Simply call dovecot deliver from within procmail Problems solved.
-- Ralf Hildebrandt (Ralf.Hildebrandt@charite.de) plonk@charite.de Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155 http://www.arschkrebs.de In the rather clear UNIX language: the TCPA cartel will be always root on your system(s) and you will it never be again.
On Thu, Jul 05, 2007 at 02:50:40PM +0200, Ralf Hildebrandt wrote:
- Stewart Dean <sdean@bard.edu>:
The day I switch our user to dovecot and it goes into production, there will be a few thousand people all generating their indices for the first time (we have a legacy requirement for procmail, at least for now), so I wondered:
Simply call dovecot deliver from within procmail Problems solved.
That will not help for big archive-type folders which seldom get new e-mail delivered to them. Indexes would still have to be generated for them on first access after the migration so the question was how can this be done in advance?
GeertStewart Dean wrote:
The day I switch our user to dovecot and it goes into production, there will be a few thousand people all generating their indices for the first time (we have a legacy requirement for procmail, at least for now), so I wondered:
- Is there is anyway to pre-emptively force indexing?
- Is there any value in doing so? That is, will I gain much in terms of making the first time everyone connects to dovecot quicker (not that big initial wait)?
I'm faced with a similar situation. I use an custom event looped MTA written in perl. By the time a message is ready to be delivered, it's temp file usually already resides on the physical disk where it will reside.
At this point delivery is done by:
1) link current file to new folder
2) unlink old tempfile nameUsing deliver would introduce huge amounts of additional overhead, since the user is already authenticated, the home directory is known and in fact the file is already on the disk.
What would be great is a stand alone program which could be run given a directory list to reindex. This could be run from MTAs or even better hooked up with an inotify watcher like iwatch, so all directory deliveries are delayed for a few seconds (to collect multiple directory writes) then reindex when a "dirty" directory goes quiet for a few seconds.
-- Like feeling your best ever, all day, every day? Here's how... Your simple secrets are here - http://RadicalHealth.com
David Favor wrote:
I'm faced with a similar situation. I use an custom event looped MTA written in perl. By the time a message is ready to be delivered, it's temp file usually already resides on the physical disk where it will reside.
That's not qpsmtpd is it??? ;-)
What would be great is a stand alone program which could be run given a directory list to reindex.
You could write one yourself; all it would take is some out of band program that connects via IMAP and requests the minimal IMAP command that would cause a reindex (Timo?). Presumably you have access to the username/password combination (since you are able to hardlink files into their mail directory).
John
-- John Peacock Director of Information Research and Technology Rowman & Littlefield Publishing Group 4501 Forbes Boulevard Suite H Lanham, MD 20706 301-459-3366 x.5010 fax 301-429-5748
John Peacock wrote:
David Favor wrote:
I'm faced with a similar situation. I use an custom event looped MTA written in perl. By the time a message is ready to be delivered, it's temp file usually already resides on the physical disk where it will reside.
That's not qpsmtpd is it??? ;-)
I've been using one I wrote sometime ago. I'm also about to start testing qpsmtpd-async.
What would be great is a stand alone program which could be run given a directory list to reindex.
You could write one yourself; all it would take is some out of band program that connects via IMAP and requests the minimal IMAP command that would cause a reindex (Timo?). Presumably you have access to the username/password combination (since you are able to hardlink files into their mail directory).
John
This is what I came up with too and if possible I'd rather have the load put on another program rather than all the overhead of starting up IMAP sessions.
-- Like feeling your best ever, all day, every day? Here's how... Your simple secrets are here - http://RadicalHealth.com
On Thu, 2007-07-05 at 08:42 -0400, Stewart Dean wrote:
The day I switch our user to dovecot and it goes into production, there will be a few thousand people all generating their indices for the first time (we have a legacy requirement for procmail, at least for now), so I wondered:
- Is there is anyway to pre-emptively force indexing?
Not in any easy way currently. http://dovecot.org/list/dovecot/2007-June/023639.html
- Is there any value in doing so? That is, will I gain much in terms of making the first time everyone connects to dovecot quicker (not that big initial wait)?
If you could handle UW-IMAP's load, Dovecot's shouldn't be much bigger even when building initial indexes.
Timo Sirainen wrote:
On Thu, 2007-07-05 at 08:42 -0400, Stewart Dean wrote:
The day I switch our user to dovecot and it goes into production, there will be a few thousand people all generating their indices for the first time (we have a legacy requirement for procmail, at least for now), so I wondered:
- Is there is anyway to pre-emptively force indexing?
Not in any easy way currently. http://dovecot.org/list/dovecot/2007-June/023639.html
- Is there any value in doing so? That is, will I gain much in terms of making the first time everyone connects to dovecot quicker (not that big initial wait)?
If you could handle UW-IMAP's load, Dovecot's shouldn't be much bigger even when building initial indexes.
I think I'll second the original posters request.. I too would really like to be able to pre-emptively force indexing.
It's not so much the load that's the problem but the users initial impression with our brand new Dovecot server. A number of them have even sent in support requests regarding it "The mail server is slow!!!". This is typically users with many many mails in many folders...
(I've told them that it'll be quicker after it's reindex everything but...)
- Peter
participants (7)
-
David Favor
-
Geert Hendrickx
-
John Peacock
-
Peter Eriksson
-
Ralf Hildebrandt
-
Stewart Dean
-
Timo Sirainen