problem in setting up proxy
Hello,
We're having difficulty with our updated cluster of dovecot servers accessing the email storage on the NFS mounts. It seems index files get corrupted when 2 backend mailservers access the same account, and from documentation setting up a director proxy in front of the backup servers. I'm trying to just set up a straight proxy first, which the documents say is the first step, and although I can see the connections coming into the server when I try to login via the proxy, the connection times out and there are no logs from dovecot anywhere saying what happened to the connection.
The configs I have set up for this in dovecot are:
dovecot.conf
# Protocols we want to be serving. protocols = imap pop3
#when re-enabling quota enforcement add quota in below: mail_plugins = $mail_plugins mail_log notify
protocol imap { # Space separated list of plugins to load (default is global mail_plugins). #when re-enabling quota enforcement add imap_quota in below: mail_plugins = $mail_plugins }
# A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, ::
# Base directory where to store runtime data. #base_dir = /var/run/dovecot/
# Name of this instance. In multi-instance setup doveadm and other commands # can use -i <instance_name> to select which instance is used (an alternative # to -c <config_path>). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot
# Greeting message for clients. login_greeting = Welcome to easyMail.
shutdown_clients = yes
# Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf
# A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf
service auth { unix_listener auth-master { mode = 0600 user = vmail } }
conf.d/10-auth.conf
## ## Authentication processes ## # Username formatting before it's looked up from databases. You can use # the standard variables here, eg. %Lu would lowercase the username, %n would # drop away the domain if it was given, or "%n-AT-%d" would change the '@' into # "-AT-". This translation is done after auth_username_translation changes. auth_username_format = %Lu
# Space separated list of wanted authentication mechanisms: # plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey # gss-spnego # NOTE: See also disable_plaintext_auth setting. auth_mechanisms = plain login # # Password database is used to verify user's password (and nothing more). # You can have multiple passdbs and userdbs. This is useful if you want to # allow both system users (/etc/passwd) and virtual users to login without # duplicating the system users into virtual database. # # <doc/wiki/PasswordDatabase.txt> # # User database specifies where mails are located and what user/group IDs # own them. For single-UID configuration use "static" userdb. # # <doc/wiki/UserDatabase.txt>
#!include auth-deny.conf.ext #!include auth-master.conf.ext
#!include auth-system.conf.ext #!include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext !include auth-static.conf.ext
conf.d/auth-static.conf.ext
# Static passdb. Included from auth.conf.
# This can be used for situations where Dovecot doesn't need to verify the # username or the password, or if there is a single password for all users: # # - proxy frontend, where the backend verifies the password # - proxy backend, where the frontend already verified the password # - authentication with SSL certificates # - simple testing
passdb static { driver = static args = nopassword=y default_fields = proxy=y host=10.5.10.121 }
#passdb { # driver = static # args = password=test #}
#userdb { # driver = static # args = uid=vmail gid=vmail home=/home/%u #}
conf.d/10-logging.conf
## ## Log destination. ##
# Log file to use for error messages. "syslog" logs to syslog, # /dev/stderr logs to stderr. #log_path = syslog
# Log file to use for informational messages. Defaults to log_path. #info_log_path = # Log file to use for debug messages. Defaults to info_log_path. #debug_log_path =
# Syslog facility to use if you're logging to syslog. Usually if you don't # want to use "mail", you'll use local0..local7. Also other standard # facilities are supported. #syslog_facility = mail
## ## Logging verbosity and debugging. ##
# Log unsuccessful authentication attempts and the reasons why they failed. auth_verbose = yes
# In case of password mismatches, log the attempted password. Valid values are # no, plain and sha1. sha1 can be useful for detecting brute force password # attempts vs. user simply trying the same password over and over again. # You can also truncate the value to n chars by appending ":n" (e.g. sha1:6). #auth_verbose_passwords = no
# Even more verbose logging for debugging purposes. Shows for example SQL # queries. auth_debug = yes
# In case of password mismatches, log the passwords and used scheme so the # problem can be debugged. Enabling this also enables auth_debug. #auth_debug_passwords = no
# Enable mail process debugging. This can help you figure out why Dovecot # isn't finding your mails. mail_debug = yes
# Show protocol level SSL errors. verbose_ssl = yes
# mail_log plugin provides more event logging for mail processes. plugin { # Events to log. Also available: flag_change append #mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Available fields: uid, box, msgid, from, subject, size, vsize, flags # size and vsize are available only for expunge and copy events. #mail_log_fields = uid box msgid size }
I'm basically expecting this to forward the login requests on to 10.5.10.121 when I try to access the email account through the proxy. When I attempt this I am able to see the connections in a tcp dump, but dovecot does not log anything about the attempt. I clearly must be missing something, can you let me know what I need to do or check?
Thank you Ted easyDNS Technologies
Hi!
First of all, can you provide output of 'doveconf -n'? It's much cleaner to read and shows what's really there?
Aki
On 22.1.2019 17.57, Ted wrote:
Hello,
We're having difficulty with our updated cluster of dovecot servers accessing the email storage on the NFS mounts. It seems index files get corrupted when 2 backend mailservers access the same account, and from documentation setting up a director proxy in front of the backup servers. I'm trying to just set up a straight proxy first, which the documents say is the first step, and although I can see the connections coming into the server when I try to login via the proxy, the connection times out and there are no logs from dovecot anywhere saying what happened to the connection.
The configs I have set up for this in dovecot are:
dovecot.conf
# Protocols we want to be serving. protocols = imap pop3
#when re-enabling quota enforcement add quota in below: mail_plugins = $mail_plugins mail_log notify
protocol imap { # Space separated list of plugins to load (default is global mail_plugins). #when re-enabling quota enforcement add imap_quota in below: mail_plugins = $mail_plugins }
# A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, ::
# Base directory where to store runtime data. #base_dir = /var/run/dovecot/
# Name of this instance. In multi-instance setup doveadm and other commands # can use -i <instance_name> to select which instance is used (an alternative # to -c <config_path>). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot
# Greeting message for clients. login_greeting = Welcome to easyMail.
shutdown_clients = yes
# Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf
# A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf
service auth { unix_listener auth-master { mode = 0600 user = vmail } }
conf.d/10-auth.conf
## ## Authentication processes ## # Username formatting before it's looked up from databases. You can use # the standard variables here, eg. %Lu would lowercase the username, %n would # drop away the domain if it was given, or "%n-AT-%d" would change the '@' into # "-AT-". This translation is done after auth_username_translation changes. auth_username_format = %Lu
# Space separated list of wanted authentication mechanisms: # plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey # gss-spnego # NOTE: See also disable_plaintext_auth setting. auth_mechanisms = plain login # # Password database is used to verify user's password (and nothing more). # You can have multiple passdbs and userdbs. This is useful if you want to # allow both system users (/etc/passwd) and virtual users to login without # duplicating the system users into virtual database. # # <doc/wiki/PasswordDatabase.txt> # # User database specifies where mails are located and what user/group IDs # own them. For single-UID configuration use "static" userdb. # # <doc/wiki/UserDatabase.txt>
#!include auth-deny.conf.ext #!include auth-master.conf.ext
#!include auth-system.conf.ext #!include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext !include auth-static.conf.ext
conf.d/auth-static.conf.ext
# Static passdb. Included from auth.conf.
# This can be used for situations where Dovecot doesn't need to verify the # username or the password, or if there is a single password for all users: # # - proxy frontend, where the backend verifies the password # - proxy backend, where the frontend already verified the password # - authentication with SSL certificates # - simple testing
passdb static { driver = static args = nopassword=y default_fields = proxy=y host=10.5.10.121 }
#passdb { # driver = static # args = password=test #}
#userdb { # driver = static # args = uid=vmail gid=vmail home=/home/%u #}
conf.d/10-logging.conf
## ## Log destination. ##
# Log file to use for error messages. "syslog" logs to syslog, # /dev/stderr logs to stderr. #log_path = syslog
# Log file to use for informational messages. Defaults to log_path. #info_log_path = # Log file to use for debug messages. Defaults to info_log_path. #debug_log_path =
# Syslog facility to use if you're logging to syslog. Usually if you don't # want to use "mail", you'll use local0..local7. Also other standard # facilities are supported. #syslog_facility = mail
## ## Logging verbosity and debugging. ##
# Log unsuccessful authentication attempts and the reasons why they failed. auth_verbose = yes
# In case of password mismatches, log the attempted password. Valid values are # no, plain and sha1. sha1 can be useful for detecting brute force password # attempts vs. user simply trying the same password over and over again. # You can also truncate the value to n chars by appending ":n" (e.g. sha1:6). #auth_verbose_passwords = no
# Even more verbose logging for debugging purposes. Shows for example SQL # queries. auth_debug = yes
# In case of password mismatches, log the passwords and used scheme so the # problem can be debugged. Enabling this also enables auth_debug. #auth_debug_passwords = no
# Enable mail process debugging. This can help you figure out why Dovecot # isn't finding your mails. mail_debug = yes
# Show protocol level SSL errors. verbose_ssl = yes
# mail_log plugin provides more event logging for mail processes. plugin { # Events to log. Also available: flag_change append #mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Available fields: uid, box, msgid, from, subject, size, vsize, flags # size and vsize are available only for expunge and copy events. #mail_log_fields = uid box msgid size }
I'm basically expecting this to forward the login requests on to 10.5.10.121 when I try to access the email account through the proxy. When I attempt this I am able to see the connections in a tcp dump, but dovecot does not log anything about the attempt. I clearly must be missing something, can you let me know what I need to do or check?
Thank you Ted easyDNS Technologies
Hello,
Absolutely, thanks, here it is:
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no login_greeting = Welcome to easyMail. mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " mail_log notify" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = nopassword=y default_fields = proxy=y host=10.5.10.121 driver = static name = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap pop3 service auth { unix_listener auth-master { mode = 0600 user = vmail } } ssl = no verbose_proctitle = yes verbose_ssl = yes protocol imap { mail_plugins = " mail_log notify" }
Thank you Ted easyDNS Technologies On 2019-01-23 2:39 a.m., Aki Tuomi wrote:
Hi!
First of all, can you provide output of 'doveconf -n'? It's much cleaner to read and shows what's really there?
Aki
On 22.1.2019 17.57, Ted wrote:
Hello,
We're having difficulty with our updated cluster of dovecot servers accessing the email storage on the NFS mounts. It seems index files get corrupted when 2 backend mailservers access the same account, and from documentation setting up a director proxy in front of the backup servers. I'm trying to just set up a straight proxy first, which the documents say is the first step, and although I can see the connections coming into the server when I try to login via the proxy, the connection times out and there are no logs from dovecot anywhere saying what happened to the connection.
The configs I have set up for this in dovecot are:
dovecot.conf
# Protocols we want to be serving. protocols = imap pop3
#when re-enabling quota enforcement add quota in below: mail_plugins = $mail_plugins mail_log notify
protocol imap { # Space separated list of plugins to load (default is global mail_plugins). #when re-enabling quota enforcement add imap_quota in below: mail_plugins = $mail_plugins }
# A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, ::
# Base directory where to store runtime data. #base_dir = /var/run/dovecot/
# Name of this instance. In multi-instance setup doveadm and other commands # can use -i <instance_name> to select which instance is used (an alternative # to -c <config_path>). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot
# Greeting message for clients. login_greeting = Welcome to easyMail.
shutdown_clients = yes
# Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf
# A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf
service auth { unix_listener auth-master { mode = 0600 user = vmail } }
conf.d/10-auth.conf
## ## Authentication processes ## # Username formatting before it's looked up from databases. You can use # the standard variables here, eg. %Lu would lowercase the username, %n would # drop away the domain if it was given, or "%n-AT-%d" would change the '@' into # "-AT-". This translation is done after auth_username_translation changes. auth_username_format = %Lu
# Space separated list of wanted authentication mechanisms: # plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey # gss-spnego # NOTE: See also disable_plaintext_auth setting. auth_mechanisms = plain login # # Password database is used to verify user's password (and nothing more). # You can have multiple passdbs and userdbs. This is useful if you want to # allow both system users (/etc/passwd) and virtual users to login without # duplicating the system users into virtual database. # # <doc/wiki/PasswordDatabase.txt> # # User database specifies where mails are located and what user/group IDs # own them. For single-UID configuration use "static" userdb. # # <doc/wiki/UserDatabase.txt>
#!include auth-deny.conf.ext #!include auth-master.conf.ext
#!include auth-system.conf.ext #!include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext !include auth-static.conf.ext
conf.d/auth-static.conf.ext
# Static passdb. Included from auth.conf.
# This can be used for situations where Dovecot doesn't need to verify the # username or the password, or if there is a single password for all users: # # - proxy frontend, where the backend verifies the password # - proxy backend, where the frontend already verified the password # - authentication with SSL certificates # - simple testing
passdb static { driver = static args = nopassword=y default_fields = proxy=y host=10.5.10.121 }
#passdb { # driver = static # args = password=test #}
#userdb { # driver = static # args = uid=vmail gid=vmail home=/home/%u #}
conf.d/10-logging.conf
## ## Log destination. ##
# Log file to use for error messages. "syslog" logs to syslog, # /dev/stderr logs to stderr. #log_path = syslog
# Log file to use for informational messages. Defaults to log_path. #info_log_path = # Log file to use for debug messages. Defaults to info_log_path. #debug_log_path =
# Syslog facility to use if you're logging to syslog. Usually if you don't # want to use "mail", you'll use local0..local7. Also other standard # facilities are supported. #syslog_facility = mail
## ## Logging verbosity and debugging. ##
# Log unsuccessful authentication attempts and the reasons why they failed. auth_verbose = yes
# In case of password mismatches, log the attempted password. Valid values are # no, plain and sha1. sha1 can be useful for detecting brute force password # attempts vs. user simply trying the same password over and over again. # You can also truncate the value to n chars by appending ":n" (e.g. sha1:6). #auth_verbose_passwords = no
# Even more verbose logging for debugging purposes. Shows for example SQL # queries. auth_debug = yes
# In case of password mismatches, log the passwords and used scheme so the # problem can be debugged. Enabling this also enables auth_debug. #auth_debug_passwords = no
# Enable mail process debugging. This can help you figure out why Dovecot # isn't finding your mails. mail_debug = yes
# Show protocol level SSL errors. verbose_ssl = yes
# mail_log plugin provides more event logging for mail processes. plugin { # Events to log. Also available: flag_change append #mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Available fields: uid, box, msgid, from, subject, size, vsize, flags # size and vsize are available only for expunge and copy events. #mail_log_fields = uid box msgid size }
I'm basically expecting this to forward the login requests on to 10.5.10.121 when I try to access the email account through the proxy. When I attempt this I am able to see the connections in a tcp dump, but dovecot does not log anything about the attempt. I clearly must be missing something, can you let me know what I need to do or check?
Thank you Ted easyDNS Technologies
On 23 Jan 2019, at 16.33, Ted <ted@easydns.com> wrote:
Hello,
Absolutely, thanks, here it is:
passdb { args = nopassword=y default_fields = proxy=y host=10.5.10.121 driver = static name = static }
at least change this to:
passdb { driver = static name = static args = nopassword=y proxy=y host=10.5.10.121 }
but not entirely sure if there is something else.
Sami
Hello,
Thanks for the advice, I have made that change now but the server is still doing the same fail with no logs thing it was before.
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no login_greeting = Welcome to easyMail. mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " mail_log notify" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = nopassword=y proxy=y host=10.5.10.121 driver = static name = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap pop3 service auth { unix_listener auth-master { mode = 0600 user = vmail } } ssl = no verbose_proctitle = yes verbose_ssl = yes protocol imap { mail_plugins = " mail_log notify" }
Anything else you can see I could try?
Thank you Ted easyDNS Technologies On 2019-01-23 10:12 a.m., Sami Ketola wrote:
On 23 Jan 2019, at 16.33, Ted <ted@easydns.com> wrote:
Hello,
Absolutely, thanks, here it is:
passdb { args = nopassword=y default_fields = proxy=y host=10.5.10.121 driver = static name = static } at least change this to:
passdb { driver = static name = static args = nopassword=y proxy=y host=10.5.10.121 }
but not entirely sure if there is something else.
Sami
Op 23/01/2019 om 17:23 schreef Ted:
Hello,
Thanks for the advice, I have made that change now but the server is still doing the same fail with no logs thing it was before.
You can use sudo doveadm log find to find out where the various
messages are being logged. Maybe you looked in the wrong log file.
Regards,
Stephan.
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 auth_debug = yes auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no login_greeting = Welcome to easyMail. mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " mail_log notify" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = nopassword=y proxy=y host=10.5.10.121 driver = static name = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap pop3 service auth { unix_listener auth-master { mode = 0600 user = vmail } } ssl = no verbose_proctitle = yes verbose_ssl = yes protocol imap { mail_plugins = " mail_log notify" }
Anything else you can see I could try?
Thank you Ted easyDNS Technologies On 2019-01-23 10:12 a.m., Sami Ketola wrote:
On 23 Jan 2019, at 16.33, Ted <ted@easydns.com> wrote:
Hello,
Absolutely, thanks, here it is:
passdb { args = nopassword=y default_fields = proxy=y host=10.5.10.121 driver = static name = static } at least change this to:
passdb { driver = static name = static args = nopassword=y proxy=y host=10.5.10.121 }
but not entirely sure if there is something else.
Sami
Hey,
We found the issue in our side, it turns out the problem was on the loadbalancers and not in dovecot at all. Sorry to have troubled you.
Thank you Ted easyDNS Technologies On 2019-01-23 10:12 a.m., Sami Ketola wrote:
On 23 Jan 2019, at 16.33, Ted <ted@easydns.com> wrote:
Hello,
Absolutely, thanks, here it is:
passdb { args = nopassword=y default_fields = proxy=y host=10.5.10.121 driver = static name = static } at least change this to:
passdb { driver = static name = static args = nopassword=y proxy=y host=10.5.10.121 }
but not entirely sure if there is something else.
Sami
participants (4)
-
Aki Tuomi
-
Sami Ketola
-
Stephan Bosch
-
Ted