Hello, I'm finding a way to have a pool of smtp-auth servers in front of my actuals servers to permit relay to our customers. Now users sends emails connecting to the right IP via mail.domain.tld, with authentication. Every server has it's own user database in mysql.
I'd like to have a pool of server (with round robin, ideally with postfix + sasl + dovecot) that authenticate the users with the actuals servers and then pointing mail.domain.tld to this pool and not to one server.
My trouble is how to authenticate users.
Can I use:
http://wiki.dovecot.org/Authentication/MultipleDatabases
to tell Dovecot to authenticate with every MySQL database on the servers "behind"?
If yes, is there a better way to do this considering I'm already using some proxy machines (dovecot) for pop3/imap and so I already have a db with domain.tld -> IP of the server? I also have some machine without a MySQL DB for users (vpopmail + cdb), so the option with MultipleDatabases will be good only for some machines.
Something similar to the proxy mechanism that I already use for pop3/imsp will be good: the proxy connect to the machine behind that do the authentication, I only need a table with domain -> server IP (that I already have).
Thanks for any advice.
On Wed, 2010-09-29 at 17:07 +0200, mailing@securitylabs.it wrote:
Hello, I'm finding a way to have a pool of smtp-auth servers in front of my actuals servers to permit relay to our customers. Now users sends emails connecting to the right IP via mail.domain.tld, with authentication. Every server has it's own user database in mysql.
Can I use:
http://wiki.dovecot.org/Authentication/MultipleDatabases
to tell Dovecot to authenticate with every MySQL database on the servers "behind"?
Yes, but if you have many of them, that's quite a lot of unnecessary SQL lookups. And since they're done one at a time each lookup slows down the authentication.
If yes, is there a better way to do this considering I'm already using some proxy machines (dovecot) for pop3/imap and so I already have a db with domain.tld -> IP of the server? I also have some machine without a MySQL DB for users (vpopmail + cdb), so the option with MultipleDatabases will be good only for some machines.
You could always use checkpassword script as passdb to implement whatever kind of logic you want.
On 01/10/2010 17:23, Timo Sirainen wrote:
On Wed, 2010-09-29 at 17:07 +0200, mailing@securitylabs.it wrote:
Hello, I'm finding a way to have a pool of smtp-auth servers in front of my actuals servers to permit relay to our customers. Now users sends emails connecting to the right IP via mail.domain.tld, with authentication. Every server has it's own user database in mysql.
Can I use:
http://wiki.dovecot.org/Authentication/MultipleDatabases
to tell Dovecot to authenticate with every MySQL database on the servers "behind"? Yes, but if you have many of them, that's quite a lot of unnecessary SQL lookups. And since they're done one at a time each lookup slows down the authentication.
I'm quite possibly missing the point, but you might restructure your application to have:
- one mysql db (replicated if necessary for robustness) with ALL customer data in it
- pool of postfix servers which auth to the remote db (they can be persuaded to cache auth results if you wish)
Optionally you could also adjust your structure to:
- first smtp server to be hit accepts mail
- transport map in sql which then directs that server to forward the mail onto the correct backend server
- remember that your transport map can indicate the use of other ports, eg you can have an optimised postfix installation which is only reachable from the frontend machines, but skips a load of address-rewriting/spam scanning, etc. Configuration is a bit like the re-injection configu when using amavis
Could have completely missed the point, but this way you get a shared pool of frontend machines (so might as well spam/av block on them), which then use a transport map to get the mail internally to the correct mailbox server. You can obviously also use a similar sql query to ask Dovecot frontend servers to proxy connections to the correct backend server...
Does that help at all?
Ed W
Il 01/10/2010 18:55, Ed W ha scritto:
On 01/10/2010 17:23, Timo Sirainen wrote:
I'm quite possibly missing the point, but you might restructure your application to have:
- one mysql db (replicated if necessary for robustness) with ALL customer data in it
- pool of postfix servers which auth to the remote db (they can be persuaded to cache auth results if you wish)
You are completely right, I simply don't like the idea to have one single big db with user data and all troubles that can happen (performance, replication configuration, data corruption...)
Could have completely missed the point, but this way you get a shared pool of frontend machines (so might as well spam/av block on them), which then use a transport map to get the mail internally to the correct mailbox server. You can obviously also use a similar sql query to ask Dovecot frontend servers to proxy connections to the correct backend server...
I already have a pool of frontend server (MX) that do AV/Spam filtering and a transport map to the internal machine, but I don't do any query, I verify the existence of the recipient via recipient address verification before queue with Postfix.
I think - for now - I will try authentication with multiple database with Dovecot, by now I only have 4 internal servers with MySQL, so performance I hope will not be a problem considering that Dovecot cache the results.
I can also implement a procedure that periodically extract only user/password from the internal DB and export them to the smtp auth db servers, but it's not in real time and may be I'll encounter some locking/performance problem during export/import.
On 01/10/2010 18:33, mailing@securitylabs.it wrote:
Il 01/10/2010 18:55, Ed W ha scritto:
On 01/10/2010 17:23, Timo Sirainen wrote:
I'm quite possibly missing the point, but you might restructure your application to have:
- one mysql db (replicated if necessary for robustness) with ALL customer data in it
- pool of postfix servers which auth to the remote db (they can be persuaded to cache auth results if you wish)
You are completely right, I simply don't like the idea to have one single big db with user data and all troubles that can happen (performance, replication configuration, data corruption...)
I think if you consider this for a bit longer you will see that you have near exactly the same issues in the way you have done things? eg if a node fails then you have lost a chunk of your data, hence backups need to be done for each machine (raising complexity), performance is limited per machine, you have no way to fail over to another db (say when doing upgrades)
If instead you start with a single aggregated DB then you have a single
master source to backup and protect. Then you can distribute that
master version as far and wide as you wish, eg if you want to you can
simply backup and restore it onto each server every hour if you want.
Or use one way replication to get it onto the slaves.. Or export it to
some flat file format and have postfix turn that into a map or whatever...
However, my personal favourite would just be a single master DB and appropriate secondary for resiliance. Lookups should likely be quite cheap compared with the load on your backend servers and a single DB server should likely handle a very large incoming mail load?
I think - for now - I will try authentication with multiple database with Dovecot, by now I only have 4 internal servers with MySQL, so performance I hope will not be a problem considering that Dovecot cache the results.
Whatever works, but it seems like you might want to take a step back and look at the complete system and check it's still sane? Seems like you are adding a chunk of complexity in order to have what you believe is a simpler system elsewhere? Why not draw out the complete system of interactions doing it with various permutations of master db, split master/slave, master with slave copy per server and master with exported maps, plus the current solution - see which one is really the simplest overall? My guess would be something like single master/slave turns out to be the winner?
Good luck!
Ed W
participants (3)
-
Ed W
-
mailing＠securitylabs.it
-
Timo Sirainen