On 2013-05-09 5:23 PM, Stephan Bosch stephan@rename-it.nl wrote:
On 5/9/2013 6:05 PM, Charles Marcus wrote:
On 2013-05-09 10:35 AM, Stephan Bosch stephan@rename-it.nl wrote:
Currently, I'm building an SMTP submission proxy server.
Can you elaborate on this?
It basically acts as a front-end to your normal MTA. First of all, it provides a convenient way to add SMTP AUTH support to any MTA.
Excellent, thanks Stephan.
Just to make sure I understand this correctly, basically, this means that if someone needs to provide SASL *client* capability on a postfix+dovecot system - ie, so that postfix can relay certain emails to certain destinations through an alternate relay server that requires SASL based SMTP AUTH - they would no longer need cyrus-sasl to accomplish this?
... and auto-save-to-sent, avoiding the overhead of the 'Copy to Sent' behavior we are currently forced to use where a message is first uploaded when it is sent, then again when it is saved to the sent folder?
Depends a bit on what you have in mind. The LEMONADE profile has a forward-without-download scheme for this, using the SMTP BURL extension (https://tools.ietf.org/html/rfc4468) and IMAP CATENATE (https://tools.ietf.org/html/rfc4469) and URLAUTH (https://tools.ietf.org/html/rfc4467). Using CATENATE, the client can combine existing message parts with new text to compose a new message. Using SMTP BURL and IMAP URLAUTH, the SMTP server can access that message directly from the IMAP server without the need for the client to download it first.
Some more direct approach is also possible, e.g. let the submission server store the message in the Sent folder implicitly (as Google apparently does). This has a few problems though, mainly that the mail client will have to be configured correctly not to store an additional copy of its own. Unfortunately, there is no standardized method of signalling this from server to client. Google probably filters out the duplicates, we don't really know. Also, which folder does the user use as Sent folder? We could use the IMAP SPECIAL-USE (https://www.ietf.org/rfc/rfc6154.txt) extension to find out. Anyway, adding support for implicitly storing sent messages in the \Sent folder should be easy enough, but it is not a fool-proof solution. Timo was discussing this a while back on the SMTP mailinglist, but people there weren't too enthusiastic about standardizing a feature like this so far.
Ok, I agree the main problem would be the possibility of duplicate messages, but I would think with the powerful filtering capabilities of sieve, it should be possible (not sure how easy though) to hard code a filter to watch for and filter/remove/delete any duplicate that the MUA uploads.
The LEMONADE profile is rather elaborate and not many clients or servers support it yet. I'm hoping that by providing a chicken, more eggs will follow soon.
I like that dovecot is willing to take a chance on being first to support these kinds of enhanced services, but I will say, it is very important that any support for said enhancements be rock-solid.
To provide some sort of solution for the short term, I guess I'll just add an optional auto-save-to-sent feature.
Sounds great to me, but...
In my opinion, because of the ubiquitous nature of MUAs saving messages to a sent folder, having a reliable and low-impact method for automatically filtering/removing/deleting these duplicates out should be a requirement before this feature is considered ready. It will be a big and immediate problem for any installation that chooses to enable this feature, as virtually all MUAs will be configured to save sent messages to a/the sent folder. It will also be an ongoing problem for all installations (existing and new alike), as users add their accounts to new computers, phones, tablets and other devices/MUAs, totally ignoring the instructions from their providers that they no longer need to enable this feature.
In fact... after thinking about this some more, I wonder...
Would there be some reasonably reliable way to detect when an MUA is uploading/saving messages to the Sent folder, and if so, could the LEMONADE protocol be leveraged to create/send a 'notification' email to that user based on some kind of system template (hard coded? customizable?), informing them that there is no need to do this, and even including a link to a dovecot wiki page explaining how to disable the 'Save copy to Sent folder' feature in common MUAs?
Then it would be up to individual SysAdmins to keep the wiki updated with sections for any clients they become aware of that aren't already on the page.
Maybe future enhancements could even (try to) detect the MUA client (is this possible to do reliably?), and a direct link to the section of the wiki for that specific client could be provided?
Another thing that I know that google is really good at is automatically filtering (I guess they're deleting?) any and all duplicate emails. I have noticed this when copying a message store from one IMAP server to a gmail account. I had cases where the number of messages in certain folders wasn't the same, and upon investigation, noticed that the original/source in fact had some duplicate messages in certain folders.
So, maybe you could 'kill two birds with one stone' so to speak. and whatever is done to address the duplicate Sent messages could also be leveraged to address duplicate messages in general? Although I guess it is not the same problem, so maybe not...
When the submission service has direct access to the user's mail storage, that is trivial to implement. However, if the submission service is unprivileged, that will be a little more difficult.
Are you talking about the difference between dovecot accessing mails with one system user, vs accessing mails with the individual users userID?
Probably, in that case I'll make it use a special support service to perform the actual delivery to the sent folder. Any suggestions are welcome.
Thanks Stephan,
--
Best regards,
Charles