[Dovecot] Dovecot MTA

[Gedalya]

On 11/08/2013 08:39 AM, Timo Sirainen wrote:
> On 8.11.2013, at 14.29, Gedalya <gedalya at gedalya.net> wrote:
>
>>> * In normal load don't queue mails, just continue delivering the mail through different processes/services until it succeeds or fails, and only after that return ok/failure to the SMTP client. So there's no (forced) post-queue filtering, everything would normally happen pre-queue. This is required because in Germany (and EU in general?) you aren't allowed to just drop spams after SMTP server has responsed OK to the client, even if you’re 100% sure it’s a spam. So this would also mean that the SMTP DATA replies will come more slowly, which means that the SMTP server must be able to handle a lot more concurrent SMTP connections, which means that in large installations the smtpd process must be able to asynchronously handle multiple SMTP client connections.
>> This is basically what I normally do with exim, and I believe it can be achieved with postfix, so basically your point is a single asynchronous smtpd for multiple connections?
> But can you (easily) configure them so that pre-queue filtering happens normally, except under heavy load it would automatically switch to post-queue filtering to avoid temporarily rejecting mails?

Dunno.. define "easy". I love exim, but some people would say even 
simple things are not easy with it. But I would agree that a lot can be 
achieved if you start designing something with these problems in mind.
We could all really use a flexible way to decide when to do things and 
on which back-end server dependent upon what is available, and general 
load on the system.

>
>> My experience has been that the real problem with SMTP-time decision making is the concurrency of the extremely heavy (e.g.) spamassassin processes, heavy in both memory and CPU, and I/O if you use bayes which you should.
> Yeah. In cloud(-like) environments the idea is also that Antispam/virus instances could be started and stopped on the automatically on the fly as needed.
>
>>> * Dovecot MTA is a new product, which means we can add some requirements to how it's being used, especially related to securely sending emails between servers. It could do a bunch of checks at startup and fail to even start if everything isn't correct. Here are some things I had in mind - not sure if all of these are good ideas or not:
>> They are all good ideas as long as these requirements can be turned off per site :-)
> That was kind of the idea, that some of these couldn’t be turned off :) So the idea being that Dovecot MTA would slowly start making email a secure communication method.

OK that's a pretty aggressively noble idea. I'm in favor. You'd probably 
want to (also) run the tests externally and publicly as a form of 
positive feedback - like a web-based test that grants a domain a 
"dovecot certified" status.

>>> * Configuration: It would take years to implement all of the settings that Postfix has, but I think it's not going to be necessary. In fact I think the number of new settings to dovecot.conf that Dovecot MTA requires would be very minimal. Instead nearly all of the configuration could be done using Sieve scripts. We'd need to implement some new MTA-specific Sieve extensions and a few core features/configurations/databases that the scripts can use, but after that there wouldn't be really any limits to what could be done with them.
>> It comes to mind that you would want a separate master process for this in case one would run it on the same box with dovecot imap. Or at least a way to restart/reconfigure it separately.
> You could run two Dovecot instances if you wanted to. But I have also some plans for making it possible to restart/upgrade Dovecot without losing any existing connections.
>
That would be warmly received.