[Dovecot] architecture to handle 1000 messages per second?

[Michael Orlitzky]

Bob Eastbrook wrote:
> On Fri, Jan 1, 2010 at 3:07 PM, Thomas Løcke <thomas.granvej6 at gmail.com> wrote:
> 
>> Hi Bob,
>>
>> Just to make sure I understand you: Some app is sending emails to
>> orders at example.com at the rate of 1000 per second, and another app is
>> fetching email from orders at example.com to process them?
> 
> Hello,
> 
> Actually, thousands of customers would send order emails to a system
> running postifx.  All orders end up in orders at example.com.  Then,
> another app fetches these emails via Dovecot (POP or IMAP), ideally at
> the rate of 1000 per second.
> 
> I think it might be difficult to accomplish this via POP since I think
> Dovecot would have to lock the account while doing the POP downloads.
> This makes it difficult to have more than one app downloading at a
> time.
> 
> Cheers,
> Bob

Don't use POP3 or IMAP; instead deliver the messages to a command. For 
example, GNU Mailman is mailing list software which pipes each incoming 
list message to a Python script. That command can then perform the 
necessary processing.

I don't know what you had in mind, but if the messages can be handled 
independently, it's easy to add new machines. Just duplicate the config 
on another box, and create an MX record for it.


virtual_alias_domains
---------------------
# Each address in example.com must be aliased to a local user.
example.com		IGNORED


virtual_alias_maps
---------------------
# The virtual address "orders at example.com" is mapped to the local user
# of the same name.
orders at example.com	orders


alias_maps
---------------------
# And then the local user's mail is delivered to a script rather than to
# a mailbox.
orders		"|/path/to/your/script"