[Dovecot] Saving Sent Messages to Sent Folder
Hi Timo,
There was another thread (it has come up at least a few times in the past few years I've been lurking) on the postfix list about having some kind of automatic 'Save to Sent' option to avoid the users mail client from having to upload messages twice (obviously the only ones of concern are ones with large attachments) - once to send it, and once to save the copy in the sent folder.
I know and understand that doing this is not generally the job of the MTA, but hopefully most people can at least admit that there would be a (potentially huge, in the case of any large company that deals with a lot of large attachments in their email) savings in bandwidth (and user frustration) if a way could be found to do this reliably and at little to no 'cost' in terms of CPU/Disk IO, so I have a question about how this might be done...
Given: Postfix can be configured to use the Dovecot LDA for delivering mail incoming.
My question is simply, why wouldn't it be possible to create a Dovecot LSA ('submission agent'), that could be defined in postfix's master.cf file, which could then be configured to a) pipe the message to postfix, and b) if message is successfully sent, save a copy to the users Sent folder?
Obviously, there would be some caveats - ie, postfix and dovecot would both have to be running on the same box (or at a minimum on the same local network)...
Anyway, this is just something that really bugs me (and has for many, many years) every time I send a message with a large attachment, and have to watch it slowly process the message sending it, then again to save it to the Sent folder, so Ijust wanted to get your take on if this is even remotely feasible...
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, 4 Mar 2010, Charles Marcus wrote:
Given: Postfix can be configured to use the Dovecot LDA for delivering mail incoming.
And that's all what's needed.
Configure your postfix to honor subaddresses, IMHO, it's seperated by "+" in postfix, and pass it as argument of "-m" option to deliver.
Configure your MUA to always BCC to me+Sent@example.net
This config has three advantages over the traditional one:
you transmit the message over the wire just once.
you know that, if to send the message failed, the message is not sent, in opposite to differ from "upload to Sent via IMAP failed" in the second stage.
the message in your Sent mailbox has the queueid. Debugging is much easier.
One disadvantage:
- If the message is accepted, but for what reason ever is discarded by the MTA, the data is lost.
Regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBS4+5L7+Vh58GPL/cAQLHsAgAw7MLxmporZp8siSRSqPBdb7t2cfvJPa7 dg2NNLJAuoo093897U0IT02W1iYhU7rjSPraWsFmOA4YZnyZK6WicEhi8Usmp8kz Yx66v9lHCPcZ6JRkneCVxPhBqiCBMucGo8JUjytQcA4I+QnQpFqInZuFyB3IyJhf /WVneTTQ390gkv653zaYilUZEmeq9ZnrV1Sged/1TSfGyjtLcCwU23gmb4I+kWG7 eaJdb/LyUmzn6d+JZaSB/WJO5kAQ9gxvgzIOeJwqt1MzGWZMW7NklfUEbJLlUdlM DHYTHPJrynkA8T93og2ddDknURg0BCP2YYlX72KGadq88F6qq5Nd4A== =QvyY -----END PGP SIGNATURE-----
On 2010-03-04 8:44 AM, Steffen Kaiser wrote:
Given: Postfix can be configured to use the Dovecot LDA for delivering mail incoming.
And that's all what's needed.
Configure your postfix to honor subaddresses, IMHO, it's seperated by "+" in postfix, and pass it as argument of "-m" option to deliver.
Configure your MUA to always BCC to me+Sent@example.net
This config has three advantages over the traditional one:
- you transmit the message over the wire just once.
Interesting, and yes, apparently this would suffice as a workaround for individual users, but imo the less configuring that needs to be done *in the client* the better - most importantly, there is less chance of problems from a user configuration error. I'd prefer to just uncheck the 'Save a copy' option, and let the save to sent happen totally on the server side.
- you know that, if to send the message failed, the message is not sent, in opposite to differ from "upload to Sent via IMAP failed" in the second stage.
Sorry, I don't understand what you mean here (language problem most likely)...
- the message in your Sent mailbox has the queueid. Debugging is much easier.
I'm guessing if this could be done with some kind of dovecot LSA proxy, that the queue ID could still be saved (and even additional headers added if desired).
One disadvantage:
- If the message is accepted, but for what reason ever is discarded by the MTA, the data is lost.
Why would it be accepted then discarded? Anyway, in such a case the data is lost regardless, right?
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, 4 Mar 2010, Charles Marcus wrote:
Interesting, and yes, apparently this would suffice as a workaround for individual users, but imo the less configuring that needs to be done *in the client* the better - most importantly, there is less chance of problems from a user configuration error. I'd prefer to just uncheck the 'Save a copy' option, and let the save to sent happen totally on the server side.
Configure postfix to add the BCC to all messages in the MSA or all authentificated or however you can identify your users apart others.
- you know that, if to send the message failed, the message is not sent, in opposite to differ from "upload to Sent via IMAP failed" in the second stage.
Sorry, I don't understand what you mean here (language problem most likely)...
Traditionally the MUA first transmits the message first via SMTP to a MTA or MSA, then via IMAP into the Sent folder.
If the first step succeeds, but the second does not, my users are worried that the message has not been sent and try again. The error message, however, states correctly that the message was sent successfully, but could not been uploaded into the sent folder.
One disadvantage:
- If the message is accepted, but for what reason ever is discarded by the MTA, the data is lost.
Why would it be accepted then discarded? Anyway, in such a case the data is lost regardless, right?
Because, traditionally, the message is transmitted via IMAP, too, it is in the Sent folder. Well, a MSA should not discard a message in the first place :-)
Regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBS4/NGL+Vh58GPL/cAQI4WAf9Eha5u6sRNyyPdYl2IuJCwI0/vxDLpWvO KB6EO1YJZe3+mbb38i13twXaBOV0NM5gdemRQ80ptdFpJ/uVc62Tm7FkdpHpzfNI 9W9vLPbuPXr4XNvs8Hy0/LecrQ4U2Qp9kmQTCSEieZXMJPEq5CAGkX2Q1H67rmxH tEt7RuiMnaOF7DERW2wwyE7WPwO3yVFxChzkR6x56fBfkmoaRarxLSGXBh9SS1c3 GKf+9n3z0OUhhJ1pDPC4nPYU8SUBQx2fFBhLxUzntc+JtSzPPvGqs18HJ/UHhmdy Wl0rBTvFSiRGJO4XqJYd8rw2C5O/CwiUTGjv6dFBW3OuFbZ5pyX8JA== =Rbcg -----END PGP SIGNATURE-----
On 2010-03-04 10:09 AM, Steffen Kaiser wrote:
Configure postfix to add the BCC to all messages in the MSA or all authentificated or however you can identify your users apart others.
Right - this is what is being discussed now...
- you know that, if to send the message failed, the message is not sent, in opposite to differ from "upload to Sent via IMAP failed" in the second stage.
Sorry, I don't understand what you mean here (language problem most likely)...
Traditionally the MUA first transmits the message first via SMTP to a MTA or MSA, then via IMAP into the Sent folder.
If the first step succeeds, but the second does not, my users are worried that the message has not been sent and try again. The error message, however, states correctly that the message was sent successfully, but could not been uploaded into the sent folder.
Ahh, got it...
Right, and this is something that has always bugged me about TB (2, and now 3)... the error message is wrong. If it successfully sends the message, but has an error saving to the Sent folder, the error message says 'There was a problem *sending* your message...' - I keep forgetting to go open a bug report, but I'll do that now... thanks for the reminder... :)
--
Best regards,
Charles
On 2010-03-04 8:32 AM, Charles Marcus wrote:
My question is simply, why wouldn't it be possible to create a Dovecot LSA ('submission agent'), that could be defined in postfix's master.cf file,
Oh - and by 'LSA', I didn't mean to suggest you should architect a complete 'smtp listener'. I'm thinking more in terms of a 'proxy', where your service would simply proxy the submission transaction while retaining a copy of the email up until the point where it would be needed to be saved to the users Sent folder - and if the send transaction failed for any reason, the email copy is simply discarded.
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, 4 Mar 2010, Charles Marcus wrote:
Some MUAs have a server-side outbox.
Store anything therein, and the server picks up the messages. To/CC/BCC is taken from the message headers.
You can implement this:
a) via a Dovecot plugin, that triggers some demon.
b) cron job
c) filesystem listener (inotify in Linux)
Regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBS4+8Fr+Vh58GPL/cAQKI6Af/YTzxu/2TPcX98wDIDzZ7k5tXuQ0hCZuk xOkV5we/WB8CqcfsfqiRy1f+3E1+S/2X/nB4q/gZv9/Pfhv8SnqVM+DT3ogfGF0d El7V0O1flqNvnwROft6qWcfMsKWjDPVLy/ug6MY2H1N+askEdZTG1nNnn5uf42l0 STdfWZNL58/ePEEJWZXc2mM3gE500r65J+LLl9afI+gj513Y0T9gI2Al05roT1tA 15JVytU8GNNfLolwPDy07pzJ/6TakXYESg1/5YknATt5Z9OH9Iof6+pPH54Jy4sT Gfn1hdQMtYwakdfalzQ7bxbjxwccqP7SpeUJZORnYOtKga47sHs4eQ== =U+xj -----END PGP SIGNATURE-----
On 2010-03-04 8:56 AM, Steffen Kaiser wrote:
On Thu, 4 Mar 2010, Charles Marcus wrote:
Some MUAs have a server-side outbox.
Yes, but most clients don't directly support them.
I'm talking about something that can be used in the enterprise, without requiring user intervention, aside from a *simple* configuration option (like simply unchecking 'Place a copy in' for Sent messages)...
I know there are probably a lot of ways this *can* be done... but again, I'm talking something server side that will require the bare minimum with respect to client configuration (and also that most clients will support (I think most clients do support the option to *not* save a sent copy?)), as well as minimal server side configuration.
Maybe this is a bad idea... but like I said, this is something that has really bothered me for a long time, and there simply *has* to be a sane way to make this problem go away...
Anyway, thanks for the responses... at least your first suggestion is something I can for myself, but it isn't something I'd like to support in an enterprise - at least not unless/until Thunderbird gets full support for Group Polices (and that would only apply to Windows Domain based customers, and not all are).
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, 4 Mar 2010, Charles Marcus wrote:
Anyway, thanks for the responses... at least your first suggestion is something I can for myself, but it isn't something I'd like to support in an enterprise - at least not unless/until Thunderbird gets full support for Group Polices (and that would only apply to Windows Domain based customers, and not all are).
Are Thunderbirds prefs still plain text?
Using the Login script I've added settings there (well, actually Mozilla Seamonky) once sometime ago. I just added a bunch of lines to the prefs at each login. IMHO, the last definition wins. And the quick launcher starts a bit later, so the changes do have effect in each session.
Regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBS4/O4b+Vh58GPL/cAQLTSAgAtY4o5Ubg8rnye+4WlFGzU76ewFUOD7Cg QfhBUbVE3ojLCAY/WJCO1G+/gTmG8g7XwN/kLvXsq2f5fzyjdQMF0oKFDe6zhClB wgqr1Fu9RgNowYpIQJexo5cXFKAe784pigph2I4nTjN1zwRvPxnzRJk7XTFTquBh BeKti9DB4WWswAZRaGGNXnI2pGkFd2P89ejaeeUH9NkETt45rprk7c6xjAWBVlC9 sLh9L6Rq3OTNtcyTXYHXBjdjiLdZdy+DeBNCBYcyM6YFMqxfFnxQK67oo4/FyhaI R6226hTiup7VEa+rrKSzxOTz4j+QV+s9aCekwq3qlZMcFIN0QzYxMQ== =Em5J -----END PGP SIGNATURE-----
On 2010-03-04 10:16 AM, Steffen Kaiser wrote:
Anyway, thanks for the responses... at least your first suggestion is something I can for myself, but it isn't something I'd like to support in an enterprise - at least not unless/until Thunderbird gets full support for Group Polices (and that would only apply to Windows Domain based customers, and not all are).
Are Thunderbirds prefs still plain text?
Using the Login script I've added settings there (well, actually Mozilla Seamonky) once sometime ago. I just added a bunch of lines to the prefs at each login. IMHO, the last definition wins. And the quick launcher starts a bit later, so the changes do have effect in each session.
Yes - and I do configure it like this, but users can still change them during that session - with mandatory GPO support, they wouldn't be able to.
That's why I'd prefer this to be strictly server-side... and why I'm excited to learn that this may be achievable now, with some reasonably simple config changes... although this will be my first foray into the world of sieve, so will almost definitely be asking questions about that
- and same for quotas...
Time to get my dovecot test server reinstalled (I wiped it for other reasons) and start testing, because I want this to be working for when I convert my main client from Courier to dovecot. I've been trying to get them to do this for a long time, but now they have given me the go ahead to test a few accounts with big mail boxes to see firsthand the performance improvements. They send/receive a lot of messages with large attachments, so being able to disable 'save to sent' and have it 'just work' will be a huge plus.
--
Best regards,
Charles
On Thu, 2010-03-04 at 08:32 -0500, Charles Marcus wrote:
There was another thread (it has come up at least a few times in the past few years I've been lurking) on the postfix list about having some kind of automatic 'Save to Sent' option to avoid the users mail client from having to upload messages twice (obviously the only ones of concern are ones with large attachments) - once to send it, and once to save the copy in the sent folder.
LEMONADE group solved this with IMAP URLAUTH (RFC 4467) and SMTP BURL (RFC 4468) extensions. The idea is basically (copy&pasting from RFCs):
C: RCPT TO:ron@gryffindor.example.com S: 250 2.1.5 ron@gryffindor.example.com OK. C: BURL imap://harry@gryffindor.example.com/outbox ;uidvalidity=1078863300/;uid=25;urlauth=submit+harry :internal:91354a473744909de610943775f92038 LAST S: 250 2.5.0 Ok.
So after receiving BURL command, SMTP server connects to IMAP server and fetches the message:
S: * OK [CAPABILITY IMAP4REV1 URLAUTH] example.com IMAP server
C: a001 LOGIN submitserver secret
S: a001 OK submitserver logged in
C: a002 URLFETCH "imap://joe@example.com/INBOX/;uid=20/
;section=1.2;urlauth=submit+fred:internal
:91354a473744909de610943775f92038"
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;section=1.2
;urlauth=submit+fred:internal
:91354a473744909de610943775f92038" {28}
S: Si vis pacem, para bellum.
S:
S: a002 OK URLFETCH completedNow, the problem is of course that neither Dovecot nor Postfix support these extensions. For Dovecot I was thinking about using METADATA extension's storage for storing the URLAUTH stuff, but METADATA isn't really implemented yet either.
On 2010-03-04 9:32 AM, Timo Sirainen wrote:
LEMONADE group solved this with IMAP URLAUTH (RFC 4467) and SMTP BURL (RFC 4468) extensions. The idea is basically (copy&pasting from RFCs):
C: RCPT TO:ron@gryffindor.example.com S: 250 2.1.5 ron@gryffindor.example.com OK. C: BURL imap://harry@gryffindor.example.com/outbox ;uidvalidity=1078863300/;uid=25;urlauth=submit+harry :internal:91354a473744909de610943775f92038 LAST S: 250 2.5.0 Ok.
So after receiving BURL command, SMTP server connects to IMAP server and fetches the message:
But wouldn't this also require the MUA to support the concept of an 'Outbox'? TB3 currently has partial support for an Outbox, but only for 'sending messages in background', not for letting the *server* pick them up and handle it. Yes, support could be added, but I'd much prefer something purely server side that just works regardless of the MUA. Of course, the MUA would have to have a configurable option to *not* 'save a copy' of messages it sends on the server (I would think most do), and support would also have to be added for postfix, which I have no idea if Wietse would have any desire to do this (I lean toward not).
The thread is progressing on the postfix list though, and it appears a working solution just might be achievable now through the use of sender_bcc_maps and sieve, if you are using postfix, dovecot and the dovecot LDA...
I'll post here the result of that conversation to clarify if this can be done...
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, 4 Mar 2010, Charles Marcus wrote:
'sending messages in background', not for letting the *server* pick them
You could put that mailbox on the IMAP server, probably. There the server can pick them up.
something purely server side that just works regardless of the MUA. Of
That means: SMTP.
Regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBS4/NvL+Vh58GPL/cAQLGMggAhSuVoJnTosS46W1h+p4FB5vCQdsDytpe uSAJF+tzKizMDM9Z9V4WxPYzlU+2933M/HlV/75c8Ijzy08VJmMrthM7WzQo+fcI Oc3NjhtwU+f8uxOImgM/PK/XedfDKOOzYPg4VXoE/STwLiOI60AGq788INxyTyzZ lQRHxclKsvyzFMnRpJU3eZvkSTYjPmAmN3ZiUCiIpZz65unINc70bzvYeufQTPAy vEP8Xfud/LMigvIAY2bX9ddywTXVgVUOhSeuIcarA3/9de/oAGcjVLdc+jNwAuM3 U57Ss2GPghWdH2+LY0Peoyz7U9s7Ntr9UBDc29iotdkHlL/KsAuPJw== =D8MZ -----END PGP SIGNATURE-----
On 2010-03-04 10:11 AM, Steffen Kaiser wrote:
something purely server side that just works regardless of the MUA. Of
That means: SMTP.
Right - which is why I started this thread as exploring the possibility of some kind of 'submission proxy' service that would work with postfix's submission service.
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, 4 Mar 2010, Charles Marcus wrote:
Right - which is why I started this thread as exploring the possibility of some kind of 'submission proxy' service that would work with postfix's submission service.
I do as well.
If I understand sender_bcc_maps correctly, you can configure my first suggestion straight forward.
sender_bcc_maps:
me@example.com me+Sent@example.com
Then pass the detail to deliver's "-m" option, which is the default mailbox the message is filed in.
Sieve could probably limit the detail to some selective ones, if you don't like the subaddressing feature in general.
Regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBS4/Qnr+Vh58GPL/cAQJP4Af9GkBVdsRvrvWYgggpk91T5b2i2BKwP3Lp zLH2tDsgeURkwP55veSyUkUh2PRT/JKTrWV1ft7742CMapp5uEVcpsCDznmCBl0d lQ+Mqws5ZMjtY+9c0fhH0unvWTvzh+NLioDZNE6qc+jK/EoVNx8JjGPPQi5r8F0K EsRHtyfP1ccl6vY89V14cCo2I8qY6NkNwgQA7YWd6mQzEXSpHEI1mLheH0kHWCpM aGxn/c6kMwzsR/gpHDanXLIIH2rNyeupsQPaWDfk3EwCVv+FeFppF/gxZ1sxT6yA duJqg+tu6GVaviDoz0xfqN77af3okK/D1GS/c2naZAj2k0VSswGJeA== =2L/G -----END PGP SIGNATURE-----
On 2010-03-04 10:24 AM, Steffen Kaiser wrote:
If I understand sender_bcc_maps correctly, you can configure my first suggestion straight forward.
sender_bcc_maps:
me@example.com me+Sent@example.com
Then pass the detail to deliver's "-m" option, which is the default mailbox the message is filed in.
Sieve could probably limit the detail to some selective ones, if you don't like the subaddressing feature in general.
I do... :)
The only remaining question is if:
'-o sender_bcc_maps=hash:/etc/postfix/sender_bcc'
Can be added only to the submission service in master.cf. Still waiting on a definitive answer (don't want to break my production postfix server testing this, and I don't have a test server up at the moment).
--
Best regards,
Charles
On Thu, 2010-03-04 at 10:05 -0500, Charles Marcus wrote:
On 2010-03-04 9:32 AM, Timo Sirainen wrote:
LEMONADE group solved this with IMAP URLAUTH (RFC 4467) and SMTP BURL (RFC 4468) extensions. The idea is basically (copy&pasting from RFCs):
C: RCPT TO:ron@gryffindor.example.com S: 250 2.1.5 ron@gryffindor.example.com OK. C: BURL imap://harry@gryffindor.example.com/outbox ;uidvalidity=1078863300/;uid=25;urlauth=submit+harry :internal:91354a473744909de610943775f92038 LAST S: 250 2.5.0 Ok.
So after receiving BURL command, SMTP server connects to IMAP server and fetches the message:
But wouldn't this also require the MUA to support the concept of an 'Outbox'?
MUA would have to support both of those URLAUTH and BURL extensions, so that it can register a temporary URL on the IMAP server, then connect to SMTP server and give that URL to BURL command (instead of sending the mail with DATA command).
So from MUA's point of view it's basically the same as before: save to IMAP and after that send via SMTP.
support would also have to be added for postfix, which I have no idea if Wietse would have any desire to do this (I lean toward not).
Yeah, he probably isn't interested in adding IMAP client support for Postfix, although it could be pretty basic support.
On 2010-03-04 10:47 AM, Timo Sirainen wrote:
MUA would have to support both of those URLAUTH and BURL extensions, so that it can register a temporary URL on the IMAP server, then connect to SMTP server and give that URL to BURL command (instead of sending the mail with DATA command).
So from MUA's point of view it's basically the same as before: save to IMAP and after that send via SMTP.
Ah, ok... well, honestly, this would probably be the 'ideal' solution, but I don't see it happening anytime soon, if ever...
Yeah, he probably isn't interested in adding IMAP client support for Postfix, although it could be pretty basic support.
No worries... apparently this is completely doable using postfix, dovecot, dovecot's LDA, and sieve... not quite as simple as just defining a new 'dovecot LSA proxy', but still doable.
Last ditch effort/comment - maybe a 'dovecot LSA proxy' would be useful for more than just this? But you never answered as to whether or not such a thing is even remotely feasible, much less doable in the real world...
;)
--
Best regards,
Charles
On 2010-03-04 11:44 AM, Charles Marcus wrote:
No worries... apparently this is completely doable using postfix, dovecot, dovecot's LDA, and sieve... not quite as simple as just defining a new 'dovecot LSA proxy', but still doable.
The downside to this is, anyone doing this will have to maintain explicit sender_bcc_maps (unless I get a positive response on the advisability of using a regex in the sender_bcc_maps to avoid the necessity).
--
Best regards,
Charles
On Thu, 2010-03-04 at 11:44 -0500, Charles Marcus wrote:
Last ditch effort/comment - maybe a 'dovecot LSA proxy' would be useful for more than just this? But you never answered as to whether or not such a thing is even remotely feasible, much less doable in the real world...
Sure everything is possible.. :) But it's not really something that I'm interested in developing.
On 04/03/2010 15:47, Timo Sirainen wrote:
On Thu, 2010-03-04 at 10:05 -0500, Charles Marcus wrote:
On 2010-03-04 9:32 AM, Timo Sirainen wrote:
LEMONADE group solved this with IMAP URLAUTH (RFC 4467) and SMTP BURL (RFC 4468) extensions. The idea is basically (copy&pasting from RFCs):
C: RCPT TO:<ron@gryffindor.example.com> S: 250 2.1.5 ron@gryffindor.example.com OK. C: BURL imap://harry@gryffindor.example.com/outbox ;uidvalidity=1078863300/;uid=25;urlauth=submit+harry :internal:91354a473744909de610943775f92038 LAST S: 250 2.5.0 Ok.So after receiving BURL command, SMTP server connects to IMAP server and fetches the message:
But wouldn't this also require the MUA to support the concept of an 'Outbox'?
MUA would have to support both of those URLAUTH and BURL extensions, so that it can register a temporary URL on the IMAP server, then connect to SMTP server and give that URL to BURL command (instead of sending the mail with DATA command).
So from MUA's point of view it's basically the same as before: save to IMAP and after that send via SMTP.
This seems like such a convolution...
Given that the RFC already proposes some changes to the IMAP side then it would seem sensible to get the IMAP server to do the proxy connection to the MTA and deliver. Perhaps a simple case of adding a flag when saving into a folder would mark the message as being required to be sent onwards?
I would guess the reason for the LEMONADE version is that they wanted it to be quite explicit if mail was not immediately able to be sent, rather than creating a bounce (REJECT vs BOUNCE). However, I should imagine a bit more thought about how to implement the IMAP side could allow an extension which handled submission and also gave immediate feedback in the case of obviously non-deliverable mail?
The other pain in the bum about most current clients is that they:
- send the email via smtp
- post the message to the Sent folder
- THEN they DOWNLOAD the message again from the Sent folder!! THIS is craziness!
Ed W
On Fri, 2010-03-05 at 09:59 +0000, Ed W wrote:
MUA would have to support both of those URLAUTH and BURL extensions, so that it can register a temporary URL on the IMAP server, then connect to SMTP server and give that URL to BURL command (instead of sending the mail with DATA command).
So from MUA's point of view it's basically the same as before: save to IMAP and after that send via SMTP.
This seems like such a convolution...
Given that the RFC already proposes some changes to the IMAP side then it would seem sensible to get the IMAP server to do the proxy connection to the MTA and deliver. Perhaps a simple case of adding a flag when saving into a folder would mark the message as being required to be sent onwards?
Well, I'm not very happy about the idea of IMAP server sending messages to SMTP server either.. :)
On 2010-03-05 5:10 AM, Timo Sirainen wrote:
Well, I'm not very happy about the idea of IMAP server sending messages to SMTP server either.. :)
Not to belabor the point, but the dovecot LDA is already talking to the SMTP server in one direction... this is why I proposed something as simple as possible, a 'LSA proxy'... just enough to make sure the message is accepted for delivery, then save the copy to the Sent folder. I imagine a lot more things could be done to, especially if sieve support was available to it.
Of course, I have no idea what 'as simple as possible' means in terms of code requirements, especially since - were you to do it, which you have already said you have no interest in - you would obviously want it to be rock-solid... ;)
That said - could this be done in a plugin? I didn't think so, but maybe I'm wrong?
--
Best regards,
Charles
On Fri, 2010-03-05 at 06:08 -0500, Charles Marcus wrote:
Not to belabor the point, but the dovecot LDA is already talking to the SMTP server in one direction... this is why I proposed something as simple as possible, a 'LSA proxy'... just enough to make sure the message is accepted for delivery, then save the copy to the Sent folder. I imagine a lot more things could be done to, especially if sieve support was available to it.
That said - could this be done in a plugin? I didn't think so, but maybe I'm wrong?
With v2.0 you can do basically everything as external plugin. Even managesieve no longer requires patching Dovecot.
LDA "talks" to SMTP server by calling sendmail binary. That's kind of ugly. v2.0 actually has SMTP client support, so maybe I should change it to use localhost:25 as default instead of using sendmail binary. Anyway, your LSA proxy would be able to use the SMTP client library. Although it would probably be easiest to start with the current LMTP code, because it already supports proxying and saving to local disk, those could be combined and the server could be added to support incoming SMTP clients..
On 2010-03-05 8:43 AM, Timo Sirainen wrote:
With v2.0 you can do basically everything as external plugin. Even managesieve no longer requires patching Dovecot.
LDA "talks" to SMTP server by calling sendmail binary. That's kind of ugly. v2.0 actually has SMTP client support, so maybe I should change it to use localhost:25 as default instead of using sendmail binary. Anyway, your LSA proxy would be able to use the SMTP client library. Although it would probably be easiest to start with the current LMTP code, because it already supports proxying and saving to local disk, those could be combined and the server could be added to support incoming SMTP clients..
Excellent! Thanks for the feedback Timo...
--
Best regards,
Charles
On 05/03/2010 10:10, Timo Sirainen wrote:
On Fri, 2010-03-05 at 09:59 +0000, Ed W wrote:
MUA would have to support both of those URLAUTH and BURL extensions, so that it can register a temporary URL on the IMAP server, then connect to SMTP server and give that URL to BURL command (instead of sending the mail with DATA command).
So from MUA's point of view it's basically the same as before: save to IMAP and after that send via SMTP.
This seems like such a convolution...
Given that the RFC already proposes some changes to the IMAP side then it would seem sensible to get the IMAP server to do the proxy connection to the MTA and deliver. Perhaps a simple case of adding a flag when saving into a folder would mark the message as being required to be sent onwards?
Well, I'm not very happy about the idea of IMAP server sending messages to SMTP server either.. :)
Go on... Why's that..?
Weight of history defines that we do things in certain ways and we sometimes get stuck in a bit of a rut, but if M$ has shown us one thing it's that we should (cautiously) look at how disparate systems can be integrated into a cohesive whole (granted they also showed how you can make an insecure system also, but I think that's an optional problem).
Not a dig at Dovecot, but: many software projects overlook the opportunity to integrate with other systems and become larger than the individual pieces. An example in point would be that I'm sitting here battling with SNMP + Cacti + Nagios trying to get them all to talk to each other... There has to be a reason Groundworks charges so much for selling you a package where this is already done...
Spinning off at a tangent, but I fell in love with (the concept of)
Lotus Notes some 18 years ago. The way I saw it was a massive
distributed multi-master data store + some presentation layers which
could make any database look like whatever you wanted it to look like.
I used it for:
- Email inbox
- Calendar
- Project documentation, discussion and design
- Staff holiday tracking
- Recruitment workflow (track all candidate details, results of interviews, contact correspondence, etc)
- Loads of inhouse custom one off projects
I also used it as an SQL database (with a bit of magic) and built an
application used to handle billions of £s of financing for a UK bank.
The IRA blew up one of the banks offices (which kind of stopped the
server working so well), all the staff simply changed their Notes tel
number to that of a different office and just carried on as though
nothing had happened... No data lost, work carried on
I had naively assumed that IMAP servers would head down the same road... To my eye it's all just unstructured data and I really don't see what's so special about a CalDev server or an SMTP server which makes it anything other than a plugin to "an unstructured data store".
If anyone starts to buy that idea then lift your vision and imagine that we start to see all these just distributed databases, specialist interfaces to query them efficiently and a bunch of protocols to distribute documents between the databases - personally I would then vote we start to shift to some kind of jabber style protocol to connect all these datastores together. Once you head down that road you can imagine perhaps an MMS style storage model where the sender hosts all the mail storage and just sends a short "SMS" note to the recipient to let them know an email is waiting for them. (possibly even has some small positive anti-spam benefit...)
Anyway, back to reality...
So what's the problem with a protocol extension which effectively means "take this message, connect to the server which was pre-configured and fully tested by you earlier, and give it a "MAIL FROM", "RCPT TO", "DATA" and let me know the answer"?
Cheers
Ed W
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Ed,
On 2010-03-05 19:18, Ed W wrote:
Go on... Why's that..?
Weight of history defines that we do things in certain ways and we sometimes get stuck in a bit of a rut, but if M$ has shown us one thing it's that we should (cautiously) look at how disparate systems can be integrated into a cohesive whole (granted they also showed how you can make an insecure system also, but I think that's an optional problem).
Not a dig at Dovecot, but: many software projects overlook the opportunity to integrate with other systems and become larger than the individual pieces. An example in point would be that I'm sitting here battling with SNMP + Cacti + Nagios trying to get them all to talk to each other... There has to be a reason Groundworks charges so much for selling you a package where this is already done...
Spinning off at a tangent, but I fell in love with (the concept of) Lotus Notes some 18 years ago. The way I saw it was a massive distributed multi-master data store + some presentation layers which could make any database look like whatever you wanted it to look like. I used it for:
- Email inbox
- Calendar
- Project documentation, discussion and design
- Staff holiday tracking
- Recruitment workflow (track all candidate details, results of interviews, contact correspondence, etc)
- Loads of inhouse custom one off projects
I also used it as an SQL database (with a bit of magic) and built an application used to handle billions of £s of financing for a UK bank. The IRA blew up one of the banks offices (which kind of stopped the server working so well), all the staff simply changed their Notes tel number to that of a different office and just carried on as though nothing had happened... No data lost, work carried on
I had naively assumed that IMAP servers would head down the same road... To my eye it's all just unstructured data and I really don't see what's so special about a CalDev server or an SMTP server which makes it anything other than a plugin to "an unstructured data store".
If anyone starts to buy that idea then lift your vision and imagine that we start to see all these just distributed databases, specialist interfaces to query them efficiently and a bunch of protocols to distribute documents between the databases - personally I would then vote we start to shift to some kind of jabber style protocol to connect all these datastores together. Once you head down that road you can imagine perhaps an MMS style storage model where the sender hosts all the mail storage and just sends a short "SMS" note to the recipient to let them know an email is waiting for them. (possibly even has some small positive anti-spam benefit...)
... kind of reminds me of Wave. I see a huge potential in Wave, once they get the federation part and some other details right. :)
Patrick.
STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key E883A005 https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/
iEYEARECAAYFAkuUXWAACgkQ7yMg/OiDoAWTiwCfd1ew65Ts//JWcwku1neRzky7 5tkAoI4hxcOmebn/FtQ5U67HRKOb1Oya =pqHA -----END PGP SIGNATURE-----
On 2010-03-05 4:59 AM, Ed W wrote:
Perhaps a simple case of adding a flag when saving into a folder would mark the message as being required to be sent onwards?
Way to error-prone for my taste. What about the luser that accidentally drops 5000 messages in there <shudder>... no thanks... ;)
<snip>
The other pain in the bum about most current clients is that they:
- send the email via smtp
- post the message to the Sent folder
That is what this thread is about...
- THEN they DOWNLOAD the message again from the Sent folder!! THIS is craziness!
Yeah, but I don't see a sane way to handle that one, and the client would definitely have to have direct support for 'it', whatever it turned out to be. I'm only interested in server-side solutions and was just hoping that the LSA proxy idea might be doable and wouldn't be too hard...
Wishing I was a coder (for the 12th time today already)... or had the time to learn...
--
Best regards,
Charles
participants (5)
-
Charles Marcus
-
Ed W
-
Patrick Nagel
-
Steffen Kaiser
-
Timo Sirainen