[Dovecot] Sieve/pigeonhole with Exim and Dovecot LDA
I am trying to configure my Dovecot installation to provide Vacation/Out-of-the-office emails using the Sieve plugin. My setup is a little bit peculiar:
Internet Internet
| ^
V |Provider's POP3 server Provider's SMTP server | ^ V | -------------- getmail | my server | | V | Dovecot LDA ---> Sieve/vacation -----> Exim | V Dovecot my server ---------------
Sorry for the ASCII art above - I thought it would be quicker than trying to explain.
The trouble I'm having is getting the Dovecot LDA to send successfully through the local exim instance out-of-office replies back to the provider's smtp server - when receiving fresh email from the provider (through getmail). Dovecot LDA tries to send the replies - but Exim freezes them because they don't contain the sender data in the format Exim wants it. Exim can either receive sender info:
- On the command line, after the "-f" command line switch (but only when called by root or other users passed under "trusted_users" in exim.conf).
- In the header of the email - in the "From:" field - but only, apparently, if it was called with the "-t" switch. Full exim command line documentation here: http://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command...
I can't figure out what command line options the Dovecot LDA is using when calling exim. I also couldn't find a way to get Dovecot LDA to pass extra options to exim, when trying to send email. The exim log has the following:
2013-05-20 15:35:15 1UeRBB-0001xc-Ar Frozen (message created with -f <>)
I've inspected the frozen message - and it has the correct sender in the "From:" field - but it seems that exim isn't using that, because it wasn't called with the "-t" option.
In dovecot.conf, I have the following for Dovecot LDA and sieve:
protocols = imap sieve
protocol lda { log_path = /var/log/dovecot/dovecot-deliver.log info_log_path = /var/log/dovecot/dovecot-deliver-info.log postmaster_address = admin@mydomain.co.uk hostname = mydomain.co.uk mail_plugins = sieve mail_plugin_dir = /usr/lib/dovecot sendmail_path = /usr/sbin/exim }
service managesieve-login { inet_listener sieve { port = 4190 } }
plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve }
Dovecot LDA is called by getmail using the vmail user - and using the same user it is trying to call exim to deliver the out-of-office replies.
I'm using Dovecot 2.2.1 with pigeonhole 0.4.0.
I can post the rest of dovecot.conf if it would help. I've read through the stuff at dovecot.org - but all the Dovecot LDA and exim info refers to Exim passing email to Dovecot using Dovecot LDA - not Dovecot LDA sending email out using Exim.
On 20/05/13 16:12, Sebastian Arcus wrote:
I am trying to configure my Dovecot installation to provide Vacation/Out-of-the-office emails using the Sieve plugin. My setup is a little bit peculiar:
Internet Internet | ^ V |Provider's POP3 server Provider's SMTP server | ^ V | -------------- getmail | my server | | V | Dovecot LDA ---> Sieve/vacation -----> Exim | V Dovecot my server ---------------
Sorry for the ASCII art above - I thought it would be quicker than trying to explain.
The trouble I'm having is getting the Dovecot LDA to send successfully through the local exim instance out-of-office replies back to the provider's smtp server - when receiving fresh email from the provider (through getmail). Dovecot LDA tries to send the replies - but Exim freezes them because they don't contain the sender data in the format Exim wants it. Exim can either receive sender info:
- On the command line, after the "-f" command line switch (but only when called by root or other users passed under "trusted_users" in exim.conf).
- In the header of the email - in the "From:" field - but only, apparently, if it was called with the "-t" switch. Full exim command line documentation here: http://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command...
I can't figure out what command line options the Dovecot LDA is using when calling exim. I also couldn't find a way to get Dovecot LDA to pass extra options to exim, when trying to send email. The exim log has the following:
2013-05-20 15:35:15 1UeRBB-0001xc-Ar Frozen (message created with -f <>)
I've inspected the frozen message - and it has the correct sender in the "From:" field - but it seems that exim isn't using that, because it wasn't called with the "-t" option.
In dovecot.conf, I have the following for Dovecot LDA and sieve:
protocols = imap sieve
protocol lda { log_path = /var/log/dovecot/dovecot-deliver.log info_log_path = /var/log/dovecot/dovecot-deliver-info.log postmaster_address = admin@mydomain.co.uk hostname = mydomain.co.uk mail_plugins = sieve mail_plugin_dir = /usr/lib/dovecot sendmail_path = /usr/sbin/exim }
service managesieve-login { inet_listener sieve { port = 4190 } }
plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve }
Dovecot LDA is called by getmail using the vmail user - and using the same user it is trying to call exim to deliver the out-of-office replies.
I'm using Dovecot 2.2.1 with pigeonhole 0.4.0.
I can post the rest of dovecot.conf if it would help. I've read through the stuff at dovecot.org - but all the Dovecot LDA and exim info refers to Exim passing email to Dovecot using Dovecot LDA - not Dovecot LDA sending email out using Exim.
I have done a bit more testing, and it seems Dovecot LDA uses the following command options when sending out email through Exim:
exim -i -f <> -- recipient@address.com
The problem with the above is that it sets an empty address for the "Sender" field in the message envelope. The message "From" header is set correctly - but the envelope "Sender" field is empty. As I use exim in smart relay mode, exim can only use the "Sender" field from the envelope to authenticate against the provider's SMTP server (Exim doesn't seem to have any variable expansion for the "From" field in the header to be used during SMTP authentication) - thus the authentication fails and the message can't go away.
On 05/20/2013 12:02 PM, Sebastian Arcus wrote:
Exim doesn't seem to have any variable expansion for the "From" field
If using the From header actually makes sense to you... then see $h_<header name> at http://www.exim.org/exim-html-current/doc/html/spec_html/ch-string_expansion..., you probably want to restrict the usage of this as much as possible. The envelope sender must be empty for bounces and auto-replies, pretty good article here: https://github.com/Exim/exim/wiki/EximAutoReply Later I'll read through your whole message again and maybe I'll come up with something more concrete and detailed..
On 20/05/13 17:12, Gedalya wrote:
On 05/20/2013 12:02 PM, Sebastian Arcus wrote:
Exim doesn't seem to have any variable expansion for the "From" field
If using the From header actually makes sense to you... then see $h_<header name> at http://www.exim.org/exim-html-current/doc/html/spec_html/ch-string_expansion..., you probably want to restrict the usage of this as much as possible. The envelope sender must be empty for bounces and auto-replies, pretty good article here: https://github.com/Exim/exim/wiki/EximAutoReply Later I'll read through your whole message again and maybe I'll come up with something more concrete and detailed..
Thanks for that. I've just tried using $header_from: in my exim authenticator in client mode when talking to the provider's SMTP server in smart relay mode (instead of $sender_address) - but for some strange reason it just won't work. I've poured over the exim logs in debug mode
- and so far I can't make sense of what is happening. I'll try some more to figure it out and get it working.
On 05/20/2013 05:13 PM, Sebastian Arcus wrote:
On 20/05/13 17:12, Gedalya wrote:
On 05/20/2013 12:02 PM, Sebastian Arcus wrote:
Exim doesn't seem to have any variable expansion for the "From" field
If using the From header actually makes sense to you... then see $h_<header name> at http://www.exim.org/exim-html-current/doc/html/spec_html/ch-string_expansion...,
you probably want to restrict the usage of this as much as possible. The envelope sender must be empty for bounces and auto-replies, pretty good article here: https://github.com/Exim/exim/wiki/EximAutoReply Later I'll read through your whole message again and maybe I'll come up with something more concrete and detailed..
Thanks for that. I've just tried using $header_from: in my exim authenticator in client mode when talking to the provider's SMTP server in smart relay mode (instead of $sender_address) - but for some strange reason it just won't work. I've poured over the exim logs in debug mode - and so far I can't make sense of what is happening. I'll try some more to figure it out and get it working.
OK, now I had some more time to look at your situation. We can ask, do you really need the sender? How do you use it? You're trying to authenticate using the sender, do you have the passwords in a lookup file? Perhaps this can be a good idea: set up a special authenticator with: client_condition = ${if match_ip{$sender_host_address}{:@[]}{1}{0}} so that it can only be used for locally submitted messages (this _should_ work, test it), and statically configure it with credentials that would work with your upstream SMTP server? Either way, you shouldn't have an authenticator that would trust the From: header and do something with it, unless the situation is very tightly controlled. You probably want to put more restrictions there to make sure this works only when intended, i.e. dovecot autoreplies.
Now, as for $header_from, first of all, it's "$header_from:", with the colon in the end. Yea, I know. Secondly, I have no idea if it would be available in an authenticator. Consider that an authenticator is not really something that is related to processing an individual message. One thing is for sure, you would need to set connection_max_messages = 1 in the smtp transport which would be handling these messages. I know that that helps to make $sender_address available in the authenticator, try your luck with $h_from: or try to pass that data in somehow, ACL variables or something, let me know how that goes - I'm curious, but if you need further help you should probably ask on the exim-users mailing list (and point me at the thread ;-))
On 20/05/13 23:40, Gedalya wrote:
On 05/20/2013 05:13 PM, Sebastian Arcus wrote:
On 20/05/13 17:12, Gedalya wrote:
On 05/20/2013 12:02 PM, Sebastian Arcus wrote:
Exim doesn't seem to have any variable expansion for the "From" field
If using the From header actually makes sense to you... then see $h_<header name> at http://www.exim.org/exim-html-current/doc/html/spec_html/ch-string_expansion...,
you probably want to restrict the usage of this as much as possible. The envelope sender must be empty for bounces and auto-replies, pretty good article here: https://github.com/Exim/exim/wiki/EximAutoReply Later I'll read through your whole message again and maybe I'll come up with something more concrete and detailed..
Thanks for that. I've just tried using $header_from: in my exim authenticator in client mode when talking to the provider's SMTP server in smart relay mode (instead of $sender_address) - but for some strange reason it just won't work. I've poured over the exim logs in debug mode - and so far I can't make sense of what is happening. I'll try some more to figure it out and get it working.
OK, now I had some more time to look at your situation. We can ask, do you really need the sender? How do you use it? You're trying to authenticate using the sender, do you have the passwords in a lookup file?
Yes - they are in a lookup file.
Perhaps this can be a good idea: set up a special authenticator with: client_condition = ${if match_ip{$sender_host_address}{:@[]}{1}{0}} so that it can only be used for locally submitted messages (this _should_ work, test it), and statically configure it with credentials that would work with your upstream SMTP server?
This is what I'm using at the moment to authenticate against the provider's SMTP server (upstream) in smtp smart relay mode:
fixed_plain_client:
driver = plaintext
public_name = PLAIN
client_send = ^$sender_address^${lookup{$sender_address}
lsearch{/etc/exim/exim-client.passwd}{$value}{fail}}
Either way, you shouldn't have an authenticator that would trust the From: header and do something with it, unless the situation is very tightly controlled. You probably want to put more restrictions there to make sure this works only when intended, i.e. dovecot autoreplies.
I only have internal lan clients connecting to this server - and even if, for any reason which I can't think at the moment - they would want to pass a fake "From:" header - it would be useless without passing the right password that goes with it.
Now, as for $header_from, first of all, it's "$header_from:", with the colon in the end.
Yes - I've tried it with the colon.
Yea, I know.
Secondly, I have no idea if it would be available in an authenticator.
I just tried it again, with debugging on, and I get the following:
212.227.15.163 in hosts_try_auth? yes (matched "auth.smtp.1and1.co.uk") scanning authentication mechanisms SMTP>> AUTH PLAIN ************************************ tls_do_write(bfac815f, 49) SSL_write(SSL, bfac815f, 49) outbytes=49 error=0 waiting for data on socket Calling SSL_read(8109288, bfac855f, 4096) read response data: size=37 SMTP<< 535 no password in decoded response fixed_plain_client authenticator yielded 2 LOG: MAIN fixed_plain_client authenticator failed H=auth.smtp.1and1.co.uk [212.227.15.163] 535 no password in decoded response
I don't think header_from: is available during authentication - or something else is happening which is escaping me right now.
Consider that an authenticator is not really something that is related to processing an individual message. One thing is for sure, you would need to set connection_max_messages = 1 in the smtp transport which would be handling these messages.
That's an interesting one. I've been running several sites for a few years now with exim in smart relay - without connection_max_messages = 1
- and had no problems so far. Maybe it's because only few lan clients are involved - or I've been lucky so far :-)
I know
that that helps to make $sender_address available in the authenticator, try your luck with $h_from: or try to pass that data in somehow, ACL variables or something, let me know how that goes - I'm curious, but if you need further help you should probably ask on the exim-users mailing list (and point me at the thread ;-))
I think I'll have to do that. Thanks again for all the suggestions.
On 05/20/2013 07:37 PM, Sebastian Arcus wrote:
That's an interesting one. I've been running several sites for a few years now with exim in smart relay - without connection_max_messages = 1 - and had no problems so far. Maybe it's because only few lan clients are involved - or I've been lucky so far :-)
The point is that the transport, and then in turn the authenticator are meant to potentially process more than one message in a single connection. What is the meaning of $sender_address or $header_*? The sender of which message? The headers from which message? If you do anything message-specific at this stage, you need to set this so only one message is sent per connection, so that message-specific variables can be meaningful.
On 21/05/13 01:38, Gedalya wrote:
On 05/20/2013 07:37 PM, Sebastian Arcus wrote:
That's an interesting one. I've been running several sites for a few years now with exim in smart relay - without connection_max_messages = 1 - and had no problems so far. Maybe it's because only few lan clients are involved - or I've been lucky so far :-)
The point is that the transport, and then in turn the authenticator are meant to potentially process more than one message in a single connection. What is the meaning of $sender_address or $header_*? The sender of which message? The headers from which message? If you do anything message-specific at this stage, you need to set this so only one message is sent per connection, so that message-specific variables can be meaningful. That makes sense - I was just surprised it hasn't bitten me in the back so far. I'll amend the configs to process one message at a time in the future. I can only assume so far it was using the sender address of the first message to be processed?
On 20/05/13 23:40, Gedalya wrote:
On 05/20/2013 05:13 PM, Sebastian Arcus wrote:
On 20/05/13 17:12, Gedalya wrote:
On 05/20/2013 12:02 PM, Sebastian Arcus wrote:
Exim doesn't seem to have any variable expansion for the "From" field
If using the From header actually makes sense to you... then see $h_<header name> at http://www.exim.org/exim-html-current/doc/html/spec_html/ch-string_expansion...,
you probably want to restrict the usage of this as much as possible. The envelope sender must be empty for bounces and auto-replies, pretty good article here: https://github.com/Exim/exim/wiki/EximAutoReply Later I'll read through your whole message again and maybe I'll come up with something more concrete and detailed..
Thanks for that. I've just tried using $header_from: in my exim authenticator in client mode when talking to the provider's SMTP server in smart relay mode (instead of $sender_address) - but for some strange reason it just won't work. I've poured over the exim logs in debug mode - and so far I can't make sense of what is happening. I'll try some more to figure it out and get it working.
OK, now I had some more time to look at your situation. We can ask, do you really need the sender? How do you use it? You're trying to authenticate using the sender, do you have the passwords in a lookup file? Perhaps this can be a good idea: set up a special authenticator with: client_condition = ${if match_ip{$sender_host_address}{:@[]}{1}{0}} so that it can only be used for locally submitted messages (this _should_ work, test it), and statically configure it with credentials that would work with your upstream SMTP server? Either way, you shouldn't have an authenticator that would trust the From: header and do something with it, unless the situation is very tightly controlled. You probably want to put more restrictions there to make sure this works only when intended, i.e. dovecot autoreplies.
Now, as for $header_from, first of all, it's "$header_from:", with the colon in the end. Yea, I know. Secondly, I have no idea if it would be available in an authenticator. Consider that an authenticator is not really something that is related to processing an individual message. One thing is for sure, you would need to set connection_max_messages = 1 in the smtp transport which would be handling these messages. I know that that helps to make $sender_address available in the authenticator, try your luck with $h_from: or try to pass that data in somehow, ACL variables or something, let me know how that goes - I'm curious, but if you need further help you should probably ask on the exim-users mailing list (and point me at the thread ;-))
Just a follow-up to my previous messages. I've posted to the exim list, and the best option seems to be to run a check in the authenticator, and if the $sender_address is empty, to use a default one instead. Note that this is probably safe in my setup, on a small lan with tightly controlled clients and where exim accepts no outside incoming email - but it might not be safe in some other different arrangements.
Here is my authenticator, in case it helps anybody else (replace "my@default_address.com" with something suitable for your setup):
#modified client auth for sieve ($sender_address missing)
fixed_plain_client:
driver = plaintext
public_name = PLAIN
client_send = ^${if eq{$sender_address}{}{my@default_address.com}
{$sender_address}}^${lookup{${if eq{$sender_address}{}
{my@default_address.com}{$sender_address}}}
lsearch{/etc/exim/exim-client.passwd}{$value}{fail}}
And here is a link to the thread on exim list:
https://lists.exim.org/lurker/message/20130523.094659.1dc9fbe2.en.html
participants (2)
-
Gedalya
-
Sebastian Arcus