[Dovecot] SMTP Submission/Proxy server - WAS Re: Released Pigeonhole v0.4.0 for Dovecot v2.2.1.

[Stephan Bosch]

On 5/10/2013 4:02 PM, Charles Marcus wrote:
> On 2013-05-09 5:23 PM, Stephan Bosch <stephan at rename-it.nl> wrote:
>> First of all, it provides a convenient way to add SMTP AUTH support 
>> to any MTA.
>
> Just to make sure I understand this correctly, basically, this means 
> that if someone needs to provide SASL *client* capability on a 
> postfix+dovecot system - ie, so that postfix can relay certain emails 
> to certain destinations through an alternate relay server that 
> requires SASL based SMTP AUTH - they would no longer need cyrus-sasl 
> to accomplish this?

Ehhh.. no :) It implements the server-side SMTP AUTH, so that your MTA 
doesn't have to any more. So the client will authenticate to Dovecot 
rather than to the regular MTA/MSA. But, again, this is a rather trivial 
matter and not the main reason for building this proxy.

>> The LEMONADE profile is rather elaborate and not many clients or 
>> servers support it yet. I'm hoping that by providing a chicken, more 
>> eggs will follow soon.
>
> I like that dovecot is willing to take a chance on being first to 
> support these kinds of enhanced services, but I will say, it is very 
> important that any support for said enhancements be rock-solid.

What do you mean exactly?

>> To provide some sort of solution for the short term, I guess I'll 
>> just add an optional auto-save-to-sent feature.
>
> Sounds great to me, but...
>
> In my opinion, because of the ubiquitous nature of MUAs saving 
> messages to a sent folder, having a reliable and low-impact method for 
> automatically filtering/removing/deleting these duplicates out should 
> be a requirement before this feature is considered ready. It will be a 
> big and immediate problem for any installation that chooses to enable 
> this feature, as virtually all MUAs will be configured to save sent 
> messages to a/the sent folder. It will also be an ongoing problem for 
> all installations (existing and new alike), as users add their 
> accounts to new computers, phones, tablets and other devices/MUAs, 
> totally ignoring the instructions from their providers that they no 
> longer need to enable this feature.

Yes, I agree.

> In fact... after thinking about this some more, I wonder...
>
> Would there be some reasonably reliable way to detect when an MUA is 
> uploading/saving messages to the Sent folder,

Hmm, not sure. Do MUAs normally generate the Message-ID header, or is 
that created by the server? That could be one way to detect the 
duplicates in the Sent folder.

> and if so, could the LEMONADE protocol be leveraged to create/send a 
> 'notification' email to that user based on some kind of system 
> template (hard coded? customizable?), informing them that there is no 
> need to do this, and even including a link to a dovecot wiki page 
> explaining how to disable the 'Save copy to Sent folder' feature in 
> common MUAs?
>
> Then it would be up to individual SysAdmins to keep the wiki updated 
> with sections for any clients they become aware of that aren't already 
> on the page.
>
> Maybe future enhancements could even (try to) detect the MUA client 
> (is this possible to do reliably?), and a direct link to the section 
> of the wiki for that specific client could be provided?

Relying on user action doesn't sound like a very appealing solution to 
me. :)

> Another thing that I know that google is really good at is 
> automatically filtering (I guess they're deleting?) any and all 
> duplicate emails. I have noticed this when copying a message store 
> from one IMAP server to a gmail account. I had cases where the number 
> of messages in certain folders wasn't the same, and upon 
> investigation, noticed that the original/source in fact had some 
> duplicate messages in certain folders.

That is entirely possible.

> So, maybe you could 'kill two birds with one stone' so to speak. and 
> whatever is done to address the duplicate Sent messages could also be 
> leveraged to address duplicate messages in general? Although I guess 
> it is not the same problem, so maybe not...

You mean something like this?

http://hg.rename-it.nl/dovecot-2.2-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt

>> When the submission service has direct access to the user's mail 
>> storage, that is trivial to implement. However, if the submission 
>> service is unprivileged, that will be a little more difficult.
>
> Are you talking about the difference between dovecot accessing mails 
> with one system user, vs accessing mails with the individual users 
> userID?

No, I'd like to be able to run SMTP submission without any direct 
filesystem access privileges, with e.g. one submission process handing 
submissions for many clients/users at the same time. For accessing the 
URLAUTHs there is already a support service in current Dovecot. 
Something similar could be devised for storing messages to Sent folders 
in that case.

Regards,

Stephan.